multithreading module
Multithreading/processing utils for faceswap
- class lib.multithreading.BackgroundGenerator(generator: Callable, prefetch: int = 1, name: str | None = None, args: tuple | None = None, kwargs: dict[str, T.Any] | None = None)
Bases:
MultiThread
Run a task in the background background and queue data for consumption
- Parameters:
generator (iterable) – The generator to run in the background
prefetch – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1
int – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1
optional – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1
name (str, optional) – The thread name. if
None
a unique name is constructed of the form {generator.__name__}_N where N is an incrementing integer. Default:None
args (tuple, Optional) – The argument tuple for generator invocation. Default:
None
.kwargs (dict, Optional) – keyword arguments for the generator invocation. Default:
None
.
Notes
Putting to the internal queue only blocks if put is called while queue has already reached max size. Therefore this means prefetch is actually 1 more than the parameter supplied (N in the queue, one waiting for insertion)
References
https://stackoverflow.com/questions/7323664/
- iterator() Generator
Iterate items out of the queue
- Yields:
Any – The items from the generator
- class lib.multithreading.FSThread(target: Callable | None = None, name: str | None = None, args: tuple = (), kwargs: dict[str, T.Any] | None = None, *, daemon: bool | None = None)
Bases:
Thread
Subclass of thread that passes errors back to parent
- Parameters:
target (callable object, Optional) – The callable object to be invoked by the run() method. If
None
nothing is called. Default:None
name (str, optional) – The thread name. if
None
a unique name is constructed of the form “Thread-N” where N is a small decimal number. Default:None
args (tuple) – The argument tuple for the target invocation. Default: ().
kwargs (dict) – keyword arguments for the target invocation. Default: {}.
- check_and_raise_error() None
Checks for errors in thread and raises them in caller.
- Raises:
Error – Re-raised error from within the thread
- run() None
Runs the target, reraising any errors from within the thread in the caller.
- class lib.multithreading.MultiThread(target: Callable, *args, thread_count: int = 1, name: str | None = None, **kwargs)
Bases:
object
Threading for IO heavy ops. Catches errors in thread and rethrows to parent.
- Parameters:
target (callable object) – The callable object to be invoked by the run() method.
args (tuple) – The argument tuple for the target invocation. Default: ().
thread_count (int, optional) – The number of threads to use. Default: 1
name (str, optional) – The thread name. if
None
a unique name is constructed of the form {target.__name__}_N where N is an incrementing integer. Default:None
kwargs (dict) – keyword arguments for the target invocation. Default: {}.
- check_and_raise_error() None
Checks for errors in thread and raises them in caller.
- Raises:
Error – Re-raised error from within the thread
- completed() bool
Check if all threads have completed
- Return type:
True
if all threads have completed otherwiseFalse
- property errors: list[tuple[type[BaseException], BaseException, traceback] | tuple[Any, Any, Any] | None]
List of thread error values
- Type:
list
- property has_error: bool
True
if a thread has errored, otherwiseFalse
- Type:
bool
- is_alive() bool
Check if any threads are still alive
- Returns:
True
if any threads are alive.False
if no threads are alive- Return type:
bool
- join() None
Join the running threads, catching and re-raising any errors
Clear the list of threads for class instance re-use
- property name: str
The name of the thread
- Type:
str
- start() None
Start all the threads for the given method, args and kwargs
- lib.multithreading.total_cpus()
Return total number of cpus