BackgroundGenerator

class lib.multithreading.BackgroundGenerator(generator: Callable, prefetch: int = 1, name: str | None = None, args: tuple | None = None, kwargs: dict[str, T.Any] | None = None)

Bases: MultiThread

Run a task in the background background and queue data for consumption

Parameters:

generator (Callable) – The generator to run in the background
prefetch (int) – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1
name (str | None) – The thread name. if None a unique name is constructed of the form {generator.__name__}_N where N is an incrementing integer. Default: None
args (tuple | None) – The argument tuple for generator invocation. Default: None.
kwargs (dict[str, T.Any] | None) – keyword arguments for the generator invocation. Default: None.

Notes

Putting to the internal queue only blocks if put is called while queue has already reached max size. Therefore this means prefetch is actually 1 more than the parameter supplied (N in the queue, one waiting for insertion)

References

https://stackoverflow.com/questions/7323664/

Attributes Summary

`errors`	List of thread error values
`has_error`	`True` if a thread has errored, otherwise `False`
`name`	The name of the thread

Methods Summary

`check_and_raise_error`()	Checks for errors in thread and raises them in caller.
`completed`()	Check if all threads have completed
`is_alive`()	Check if any threads are still alive
`iterator`()	Iterate items out of the queue
`join`()	Join the running threads, catching and re-raising any errors
`start`()	Start all the threads for the given method, args and kwargs

Attributes Documentation

errors

List of thread error values

Type:: list

has_error: True if a thread has errored, otherwise False

name: The name of the thread

Methods Documentation

check_and_raise_error() → None

Checks for errors in thread and raises them in caller.

Raises:: Error – Re-raised error from within the thread
Return type:: None

completed() → bool

Check if all threads have completed

Return type:: True if all threads have completed otherwise False

is_alive() → bool

Check if any threads are still alive

Return type:: True if any threads are alive. False if no threads are alive

iterator() → Generator

Iterate items out of the queue

Yields:: The items from the generator
Return type:: Generator

join() → None

Join the running threads, catching and re-raising any errors

Clear the list of threads for class instance re-use

Return type:: None

start() → None

Start all the threads for the given method, args and kwargs

Return type:: None

iterator() → Generator

Iterate items out of the queue

Yields:: The items from the generator
Return type:: Generator