multithreading module

Multithreading/processing utils for faceswap

class lib.multithreading.BackgroundGenerator(generator: Callable, prefetch: int = 1, name: str | None = None, args: tuple | None = None, kwargs: dict[str, T.Any] | None = None)

Bases: MultiThread

Run a task in the background background and queue data for consumption

Parameters:

  • generator (iterable) – The generator to run in the background

  • prefetch – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1

  • int – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1

  • optional – The number of items to pre-fetch from the generator before blocking (see Notes). Default: 1

  • name (str, optional) – The thread name. if None a unique name is constructed of the form {generator.__name__}_N where N is an incrementing integer. Default: None

  • args (tuple, Optional) – The argument tuple for generator invocation. Default: None.

  • kwargs (dict, Optional) – keyword arguments for the generator invocation. Default: None.

Notes

Putting to the internal queue only blocks if put is called while queue has already reached max size. Therefore this means prefetch is actually 1 more than the parameter supplied (N in the queue, one waiting for insertion)

References

https://stackoverflow.com/questions/7323664/

iterator() → Generator

Iterate items out of the queue

Yields:

Any – The items from the generator

class lib.multithreading.FSThread(target: Callable | None = None, name: str | None = None, args: tuple = (), kwargs: dict[str, T.Any] | None = None, *, daemon: bool | None = None)

Bases: Thread

Subclass of thread that passes errors back to parent

Parameters:

  • target (callable object, Optional) – The callable object to be invoked by the run() method. If None nothing is called. Default: None

  • name (str, optional) – The thread name. if None a unique name is constructed of the form “Thread-N” where N is a small decimal number. Default: None

  • args (tuple) – The argument tuple for the target invocation. Default: ().

  • kwargs (dict) – keyword arguments for the target invocation. Default: {}.

check_and_raise_error() → None

Checks for errors in thread and raises them in caller.

Raises:

Error – Re-raised error from within the thread

run() → None

Runs the target, reraising any errors from within the thread in the caller.

class lib.multithreading.MultiThread(target: Callable, *args, thread_count: int = 1, name: str | None = None, **kwargs)

Bases: object

Threading for IO heavy ops. Catches errors in thread and rethrows to parent.

Parameters:

  • target (callable object) – The callable object to be invoked by the run() method.

  • args (tuple) – The argument tuple for the target invocation. Default: ().

  • thread_count (int, optional) – The number of threads to use. Default: 1

  • name (str, optional) – The thread name. if None a unique name is constructed of the form {target.__name__}_N where N is an incrementing integer. Default: None

  • kwargs (dict) – keyword arguments for the target invocation. Default: {}.

check_and_raise_error() → None

Checks for errors in thread and raises them in caller.

Raises:

Error – Re-raised error from within the thread

completed() → bool

Check if all threads have completed

Return type:

True if all threads have completed otherwise False

property errors: list[tuple[type[BaseException], BaseException, traceback] | tuple[Any, Any, Any] | None]

List of thread error values

Type:

list

property has_error: bool

True if a thread has errored, otherwise False

Type:

bool

is_alive() → bool

Check if any threads are still alive

Returns:

True if any threads are alive. False if no threads are alive

Return type:

bool

join() → None

Join the running threads, catching and re-raising any errors

Clear the list of threads for class instance re-use

property name: str

The name of the thread

Type:

str

start() → None

Start all the threads for the given method, args and kwargs

lib.multithreading.total_cpus()

Return total number of cpus