DataTracker

class lib.infer.profile.DataTracker(size: dataclasses.InitVar[int], max_vram: dataclasses.InitVar[float], face_scaling: int, has_detector: bool)

Bases: object

Stores data from the benchmarking process

Parameters:

size (InitVar[int]) – The number of plugins that data is being tracked for
face_scaling (int) – The amount of scaling to apply to downstream non-detection plugins
has_detector (bool) – True if the first plugin in the pipeline is a detector
max_vram (InitVar[float]) – The maximum amount of total VRAM to allow Cuda to reserve when profiling

Attributes Summary

`batch_sizes`	All batch size combinations that did not OOM
`combos_exhausted`	`True` if we have run out of possible combinations to attempt
`has_oom`	`True` if the last iteration hit an OOM or fell outside our max VRAM threshold

Methods Summary

`add_iterations_row`()	Add a new row to the iterations list
`add_next_batch_sizes`()	Add the next batch size configuration to the batch size array based on the output from the last test
`collect_vram`()	Store the currently allocated and reserved Cuda VRAM stats
`get_samples`([index, adjusted])	Obtain the number of sample processed by each plugin for a certain valid batch size combination
`get_samples_stats`(method[, index, adjusted])	Obtain the average or minimum samples processed for all plugins for a certain batch size combination
`update_iterations`(iterations, matrix_id)	Update the iteration count from a plugin runner in a thread-safe way

Attributes Documentation

batch_sizes: All batch size combinations that did not OOM

combos_exhausted: bool = False: True if we have run out of possible combinations to attempt

has_oom: True if the last iteration hit an OOM or fell outside our max VRAM threshold

Methods Documentation

add_iterations_row() → None

Add a new row to the iterations list

Return type:: None

add_next_batch_sizes() → None

Add the next batch size configuration to the batch size array based on the output from the last test

Return type:: None

collect_vram() → None

Store the currently allocated and reserved Cuda VRAM stats

Return type:: None

get_samples(index: int | None = None, adjusted: bool = False) → npt.NDArray[np.float64]

Obtain the number of sample processed by each plugin for a certain valid batch size combination

Parameters:

index (int | None) – The testing index to obtain the samples for or None for all tests
adjusted (bool) – True to obtain results adjusted for any non-detector scaling. Default: False

Return type:

The number of samples processed by each plugin

get_samples_stats(method: T.Literal['mean', 'min'], index: int | None = None, adjusted: bool = False) → npt.NDArray[np.float64]

Obtain the average or minimum samples processed for all plugins for a certain batch size combination

Parameters:

method (T.Literal['mean', 'min']) – mean to obtain the mean number of samples for all plugins. min to obtain the minimum number of samples processed by a plugin
index (int | None) – The testing index to obtain the average samples for or None for all tests. Default: None
adjusted (bool) – True to obtain results adjusted for any non-detector scaling. Default: False

Return type:

The average number of samples processed by all plugins

update_iterations(iterations: int, matrix_id: int) → None

Update the iteration count from a plugin runner in a thread-safe way

Parameters:

iterations (int) – The iteration count for the plugin
matrix_id (int) – The column id that belongs to the plugin

Return type:

None

add_iterations_row() → None

Add a new row to the iterations list

Return type:: None

add_next_batch_sizes() → None

Add the next batch size configuration to the batch size array based on the output from the last test

Return type:: None

property batch_sizes: npt.NDArray[np.int64]: All batch size combinations that did not OOM

collect_vram() → None

Store the currently allocated and reserved Cuda VRAM stats

Return type:: None

combos_exhausted: bool = False: True if we have run out of possible combinations to attempt

get_samples(index: int | None = None, adjusted: bool = False) → npt.NDArray[np.float64]

Obtain the number of sample processed by each plugin for a certain valid batch size combination

Parameters:

index (int | None) – The testing index to obtain the samples for or None for all tests
adjusted (bool) – True to obtain results adjusted for any non-detector scaling. Default: False

Return type:

The number of samples processed by each plugin

get_samples_stats(method: T.Literal['mean', 'min'], index: int | None = None, adjusted: bool = False) → npt.NDArray[np.float64]

Obtain the average or minimum samples processed for all plugins for a certain batch size combination

Parameters:

method (T.Literal['mean', 'min']) – mean to obtain the mean number of samples for all plugins. min to obtain the minimum number of samples processed by a plugin
index (int | None) – The testing index to obtain the average samples for or None for all tests. Default: None
adjusted (bool) – True to obtain results adjusted for any non-detector scaling. Default: False

Return type:

The average number of samples processed by all plugins

property has_oom: bool: True if the last iteration hit an OOM or fell outside our max VRAM threshold

update_iterations(iterations: int, matrix_id: int) → None

Update the iteration count from a plugin runner in a thread-safe way

Parameters:

iterations (int) – The iteration count for the plugin
matrix_id (int) – The column id that belongs to the plugin

Return type:

None

vram: list[tuple[int, int]] = <dataclasses._MISSING_TYPE object>: list of (max allocated, max reserved) VRAM for each testing phase

vram_limit: float = <dataclasses._MISSING_TYPE object>: The limit that Cuda reserved memory must remain within