Mask

class lib.align.aligned_mask.Mask(storage_size: int = 128, storage_centering: CenteringType = 'face')

Bases: object

Face Mask information and convenience methods

Holds a Faceswap mask as generated from plugins.extract.mask and the information required to transform it to its original frame.

Holds convenience methods to handle the warping, storing and retrieval of the mask.

Parameters:

storage_size (int) – The size (in pixels) that the mask should be stored at. Default: 128.
storage_centering (CenteringType) – The centering to store the mask at. One of “legacy”, “face”, “head”. Default: “face”

stored_size: The size, in pixels, of the stored mask across its height and width.

stored_centering

The centering that the mask is stored at. One of “legacy”, “face”, “head”

Type:: CenteringType

Attributes Summary

`affine_matrix`	The affine matrix to transpose the mask to a full frame.
`interpolator`	The cv2 interpolator required to transpose the mask to a full frame.
`mask`	The mask at the size of `stored_size` with any requested blurring, threshold amount and centering applied.
`original_roi`	The original region of interest of the mask in the source frame.
`stored_mask`	The mask at the size of `stored_size` as it is stored (i.e. with no blurring/ centering applied).

Methods Summary

`add`(mask, affine_matrix)	Add a Faceswap mask to this `Mask`.
`from_dict`(mask)	Populates the `Mask` from a dictionary loaded from an alignments file.
`get_full_frame_mask`(width, height)	Return the stored mask in a full size frame of the given dimensions
`replace_mask`(mask)	Replace the existing `_mask` with the given mask.
`set_blur_and_threshold`([blur_kernel, ...])	Set the internal blur kernel and threshold amount for returned masks
`set_dilation`(amount)	Set the internal dilation object for returned masks
`set_sub_crop`(source_offset, target_offset, ...)	Set the internal crop area of the mask to be returned.
`to_dict`([is_png])	Convert the mask to a dictionary for saving to an alignments file
`to_png_meta`()	Convert the mask to a dictionary supported by png itxt headers.

Attributes Documentation

affine_matrix: The affine matrix to transpose the mask to a full frame.

interpolator: The cv2 interpolator required to transpose the mask to a full frame.

mask: The mask at the size of stored_size with any requested blurring, threshold amount and centering applied.

original_roi: The original region of interest of the mask in the source frame.

stored_mask: The mask at the size of stored_size as it is stored (i.e. with no blurring/ centering applied).

Methods Documentation

add(mask: npt.NDArray[np.uint8], affine_matrix: npt.NDArray[np.float32]) → T.Self

Add a Faceswap mask to this Mask.

The mask should be the original output from plugins.extract.mask

Parameters:

mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
affine_matrix (npt.NDArray[np.float32]) – The normalized transformation matrix required to transform the mask from (0, 1) to the original frame.

Return type:

This mask object

from_dict(mask: MaskAlignmentsFile) → Self

Populates the Mask from a dictionary loaded from an alignments file.

Parameters:: mask (MaskAlignmentsFile) – A dictionary stored in an alignments file containing the keys mask, affine_matrix, interpolator, stored_size, stored_centering
Return type:: This loaded Mask object

get_full_frame_mask(width: int, height: int) → ndarray

Return the stored mask in a full size frame of the given dimensions

Parameters:

width (int) – The width of the original frame that the mask was extracted from
height (int) – The height of the original frame that the mask was extracted from

Return type:

The mask affined to the original full frame of the given dimensions

replace_mask(mask: npt.NDArray[np.uint8]) → None

Replace the existing _mask with the given mask.

Parameters:: mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
Return type:: None

set_blur_and_threshold(blur_kernel: int = 0, blur_type: Literal['gaussian', 'normalized'] | None = 'gaussian', blur_passes: int = 1, threshold: int = 0) → None

Set the internal blur kernel and threshold amount for returned masks

Parameters:

blur_kernel (int) – The kernel size, in pixels to apply gaussian blurring to the mask. Set to 0 for no blurring. Should be odd, if an even number is passed in (outside of 0) then it is rounded up to the next odd number. Default: 0
blur_type (Literal['gaussian', 'normalized'] | None) – The blur type to use. gaussian or normalized box filter. Default: gaussian
blur_passes (int) – The number of passed to perform when blurring. Default: 1
threshold (int) – The threshold amount to minimize/maximize mask values to 0 and 100. Percentage value. Default: 0

Return type:

None

set_dilation(amount: float) → None

Set the internal dilation object for returned masks

Parameters:: amount (float) – The amount of erosion/dilation to apply as a percentage of the total mask size. Negative values erode the mask. Positive values dilate the mask
Return type:: None

set_sub_crop(source_offset: np.ndarray, target_offset: np.ndarray, centering: CenteringType, coverage_ratio: float = 1.0, y_offset: float = 0.0) → None

Set the internal crop area of the mask to be returned.

This impacts the returned mask from mask if the requested mask is required for different face centering than what has been stored.

Parameters:

source_offset (np.ndarray) – The (x, y) offset for the mask at its stored centering
target_offset (np.ndarray) – The (x, y) offset for the mask at the requested target centering
centering (CenteringType) – The centering to set the sub crop area for. One of “legacy”, “face”. “head”
coverage_ratio (float) – The coverage ratio to be applied to the target image. None for default (1.0). Default: None
y_offset (float) – Amount to additionally adjust the masks’s offset along the y-axis. Default: 0.0

Return type:

None

to_dict(is_png=False) → MaskAlignmentsFile

Convert the mask to a dictionary for saving to an alignments file

Parameters:

is_png – True if the dictionary is being created for storage in a png header otherwise False. Default: False

Returns:

The Mask for saving to an alignments file. Contains the keys mask,
affine_matrix, interpolator, stored_size, stored_centering

Return type:

MaskAlignmentsFile

to_png_meta() → MaskAlignmentsFile

Convert the mask to a dictionary supported by png itxt headers.

Returns:

The Mask for saving to an alignments file. Contains the keys mask,
affine_matrix, interpolator, stored_size, stored_centering

Return type:

MaskAlignmentsFile

add(mask: npt.NDArray[np.uint8], affine_matrix: npt.NDArray[np.float32]) → T.Self

Add a Faceswap mask to this Mask.

The mask should be the original output from plugins.extract.mask

Parameters:

mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
affine_matrix (npt.NDArray[np.float32]) – The normalized transformation matrix required to transform the mask from (0, 1) to the original frame.

Return type:

This mask object

property affine_matrix: ndarray: The affine matrix to transpose the mask to a full frame.

from_dict(mask: MaskAlignmentsFile) → Self

Populates the Mask from a dictionary loaded from an alignments file.

Parameters:: mask (MaskAlignmentsFile) – A dictionary stored in an alignments file containing the keys mask, affine_matrix, interpolator, stored_size, stored_centering
Return type:: This loaded Mask object

get_full_frame_mask(width: int, height: int) → ndarray

Return the stored mask in a full size frame of the given dimensions

Parameters:

width (int) – The width of the original frame that the mask was extracted from
height (int) – The height of the original frame that the mask was extracted from

Return type:

The mask affined to the original full frame of the given dimensions

property interpolator: int: The cv2 interpolator required to transpose the mask to a full frame.

property mask: ndarray: The mask at the size of stored_size with any requested blurring, threshold amount and centering applied.

property original_roi: ndarray: The original region of interest of the mask in the source frame.

replace_mask(mask: npt.NDArray[np.uint8]) → None

Replace the existing _mask with the given mask.

Parameters:: mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
Return type:: None

set_blur_and_threshold(blur_kernel: int = 0, blur_type: Literal['gaussian', 'normalized'] | None = 'gaussian', blur_passes: int = 1, threshold: int = 0) → None

Set the internal blur kernel and threshold amount for returned masks

Parameters:

blur_kernel (int) – The kernel size, in pixels to apply gaussian blurring to the mask. Set to 0 for no blurring. Should be odd, if an even number is passed in (outside of 0) then it is rounded up to the next odd number. Default: 0
blur_type (Literal['gaussian', 'normalized'] | None) – The blur type to use. gaussian or normalized box filter. Default: gaussian
blur_passes (int) – The number of passed to perform when blurring. Default: 1
threshold (int) – The threshold amount to minimize/maximize mask values to 0 and 100. Percentage value. Default: 0

Return type:

None

set_dilation(amount: float) → None

Set the internal dilation object for returned masks

Parameters:: amount (float) – The amount of erosion/dilation to apply as a percentage of the total mask size. Negative values erode the mask. Positive values dilate the mask
Return type:: None

set_sub_crop(source_offset: np.ndarray, target_offset: np.ndarray, centering: CenteringType, coverage_ratio: float = 1.0, y_offset: float = 0.0) → None

Set the internal crop area of the mask to be returned.

This impacts the returned mask from mask if the requested mask is required for different face centering than what has been stored.

Parameters:

source_offset (np.ndarray) – The (x, y) offset for the mask at its stored centering
target_offset (np.ndarray) – The (x, y) offset for the mask at the requested target centering
centering (CenteringType) – The centering to set the sub crop area for. One of “legacy”, “face”. “head”
coverage_ratio (float) – The coverage ratio to be applied to the target image. None for default (1.0). Default: None
y_offset (float) – Amount to additionally adjust the masks’s offset along the y-axis. Default: 0.0

Return type:

None

property stored_mask: ndarray: The mask at the size of stored_size as it is stored (i.e. with no blurring/ centering applied).

to_dict(is_png=False) → MaskAlignmentsFile

Convert the mask to a dictionary for saving to an alignments file

Parameters:

is_png – True if the dictionary is being created for storage in a png header otherwise False. Default: False

Returns:

The Mask for saving to an alignments file. Contains the keys mask,
affine_matrix, interpolator, stored_size, stored_centering

Return type:

MaskAlignmentsFile

to_png_meta() → MaskAlignmentsFile

Convert the mask to a dictionary supported by png itxt headers.

Returns:

The Mask for saving to an alignments file. Contains the keys mask,
affine_matrix, interpolator, stored_size, stored_centering

Return type:

MaskAlignmentsFile