BiSeNetFP

class plugins.extract.mask.bisenet_fp.BiSeNetFP

Bases: FacePlugin

Neural network to process face image into a segmentation mask of the face

Attributes Summary

device

The selected device to run torch ops on

Methods Summary

`from_torch`(batch)	Run inference on a PyTorch model.
`load_model`()	Initialize the BiSeNet Face Parsing model.
`load_torch_model`(model, weights_path[, ...])	Load a PyTorch model, apply the weights and pass a warmup batch through
`post_process`(batch)	Process the output from the model
`pre_process`(batch)	Format the detected faces for prediction
`process`(batch)	Get the masks from the model

Attributes Documentation

device: The selected device to run torch ops on

Methods Documentation

from_torch(batch: ndarray) → ndarray

Run inference on a PyTorch model.

This function does not need to be used, however it handles torch backend for better throughput, so it is recommended. Must have used self.load_torch_model to load the Torch model to use this function.

Parameters:: batch (ndarray) – The batch array to feed to the PyTorch model
Return type:: The result from the PyTorch model

load_model() → BiSeNet

Initialize the BiSeNet Face Parsing model.

Return type:: The loaded BiSeNetFP model

load_torch_model(model: Module, weights_path: str, return_indices: list[int] | None = None) → Module

Load a PyTorch model, apply the weights and pass a warmup batch through

This function does not need to be used, but some default Faceswap optimizations are performed here, so without using this function you will either need to apply them yourself or not have them applied

Parameters:

model (Module) – The Torch model to load
weights_path (str) – Full path to the weights file to load
return_indices (list[int] | None) – If the model outputs multiple items, but you only require some of them, the indices of the required items can be placed here so that when calling from_torch any extra data is not copied from the GPU. Default: None (return all data)

Return type:

The loaded model ready for inference

post_process(batch: ndarray) → ndarray

Process the output from the model

Parameters:: batch (ndarray) – The predictions from the masker
Return type:: The final masks

pre_process(batch: ndarray) → ndarray

Format the detected faces for prediction

Parameters:: batch (ndarray) – The batch of aligned faces in the correct format for the model
Return type:: The updated images for feeding the model

process(batch: ndarray) → ndarray

Get the masks from the model

Parameters:: batch (ndarray) – The batch to feed into the masker
Return type:: The predicted masks from the plugin