CV2DNNAlign

class plugins.extract.align.cv2_dnn.CV2DNNAlign

Bases: ExtractPlugin

CV2 DNN Plugin for face alignment

Attributes Summary

device

The selected device to run torch ops on

Methods Summary

`from_torch`(batch)	Run inference on a PyTorch model.
`load_model`()	Load the CV2 DNN Aligner Model
`load_torch_model`(model, weights_path[, ...])	Load a PyTorch model, apply the weights and pass a warmup batch through
`post_process`(batch)	Override to perform post-processing
`pre_process`(batch)	Format the ROI faces detection boxes for prediction
`process`(batch)	Predict the 68 point landmarks

Attributes Documentation

device: The selected device to run torch ops on

Methods Documentation

from_torch(batch: ndarray) → ndarray

Run inference on a PyTorch model.

This function does not need to be used, however it handles torch backend for better throughput, so it is recommended. Must have used self.load_torch_model to load the Torch model to use this function.

Parameters:: batch (ndarray) – The batch array to feed to the PyTorch model
Return type:: The result from the PyTorch model

load_model() → Net

Load the CV2 DNN Aligner Model

Return type:: The loaded cv2-DNN model

load_torch_model(model: Module, weights_path: str, return_indices: list[int] | None = None) → Module

Load a PyTorch model, apply the weights and pass a warmup batch through

This function does not need to be used, but some default Faceswap optimizations are performed here, so without using this function you will either need to apply them yourself or not have them applied

Parameters:

model (Module) – The Torch model to load
weights_path (str) – Full path to the weights file to load
return_indices (list[int] | None) – If the model outputs multiple items, but you only require some of them, the indices of the required items can be placed here so that when calling from_torch any extra data is not copied from the GPU. Default: None (return all data)

Return type:

The loaded model ready for inference

post_process(batch: ndarray) → ndarray[tuple[Any, ...], dtype[float32]]

Override to perform post-processing

Parameters:

batch (ndarray) – This will be the output from the previous ‘process’ step

Returns:

For detect plugins this must be an (N, M, left, top, right, bottom) bounding boxes for
detected faces scaled to model input size as float32. N is the batch size, M is the number
of detections per batch
For align plugins this must be an (N, 68, 2) float32 array for each (x, y) landmark point
for each face in the batch. co-ordinates should be normalized to 0.0 to 1.0 range
For mask plugins this must be an (N, size, size) float32 image in range 0. - 1.0 for each
face in the batch
For identity plugins this must be an (N, M) float32 identity embedding

Return type:

ndarray[tuple[Any, …], dtype[float32]]

pre_process(batch: ndarray) → ndarray

Format the ROI faces detection boxes for prediction

Parameters:: batch (ndarray) – The batch of face detection bounding boxes as (bs, l, t, r, b)
Return type:: The face detection bounding boxes formatted to take an image patch for prediction

process(batch: ndarray) → ndarray

Predict the 68 point landmarks

Parameters:

feed – The batch to feed into the aligner
batch (ndarray)

Return type:

The predictions from the aligner

load_model() → Net

Load the CV2 DNN Aligner Model

Return type:: The loaded cv2-DNN model

pre_process(batch: ndarray) → ndarray

Format the ROI faces detection boxes for prediction

Parameters:: batch (ndarray) – The batch of face detection bounding boxes as (bs, l, t, r, b)
Return type:: The face detection bounding boxes formatted to take an image patch for prediction

process(batch: ndarray) → ndarray

Predict the 68 point landmarks

Parameters:

feed – The batch to feed into the aligner
batch (ndarray)

Return type:

The predictions from the aligner