DetectedFace

Bases: object

Detected face and landmark information

Holds information about a detected face, it’s location in a source image and the face’s 68 point landmarks.

Methods for aligning a face are also callable from here.

Parameters:

image (np.ndarray | None) – Original frame that holds this face. Optional (not required if just storing coordinates). Default: None
left (int | None) – The left most point (in pixels) of the face’s bounding box as discovered in plugins.extract.detect
width (int | None) – The width (in pixels) of the face’s bounding box as discovered in plugins.extract.detect
top (int | None) – The top most point (in pixels) of the face’s bounding box as discovered in plugins.extract.detect
height (int | None) – The height (in pixels) of the face’s bounding box as discovered in plugins.extract.detect
landmarks_xy (np.ndarray | None) – The 68 point landmarks as discovered in plugins.extract.align. Should be an array of 68 (x, y) points of each of the landmark co-ordinates.
mask (dict[str, aligned_mask.Mask] | None) – The generated mask(s) for the face as generated in plugins.extract.mask.
identity (dict[str, np.ndarray] | None)

Attributes Summary

`aligned`	The aligned face connected to this detected face.
`bottom`	Bottom point (in pixels) of face detection bounding box within the parent image
`has_landmarks`	`True` if this object contains landmarks
`identity`	Identity mechanism as key, identity embedding as value
`landmarks_xy`	The frame space 2D landmarks for this detected face.
`right`	Right point (in pixels) of face detection bounding box within the parent image

Methods Summary

`add_identity`(name, embedding)	Add an identity embedding to this detected face.
`add_landmarks_xy`(landmarks)	Add landmarks to the detected face object.
`add_mask`(name, mask, affine_matrix[, ...])	Add a `Mask` to this detected face
`clear_all_identities`()	Remove all stored identity embeddings
`from_alignment`(alignment[, image, with_thumb])	Set the attributes of this class from an alignments file and optionally load the face into the `image` attribute.
`from_png_meta`(alignment)	Set the attributes of this class from alignments stored in a png exif header.
`get_landmark_mask`(area[, dilation, ...])	Obtain a `LandmarksMask` for this face
`get_training_masks`()	Obtain the decompressed combined training masks.
`load_aligned`(image[, size, dtype, ...])	Align a face from a given image.
`store_training_masks`(masks[, delete_masks])	Concatenate and compress the given training masks and store for retrieval.
`to_alignment`()	Return the detected face formatted for an alignments file
`to_png_meta`()	Return the detected face formatted for insertion into a png itxt header.

Attributes Documentation

aligned: The aligned face connected to this detected face.

bottom: Bottom point (in pixels) of face detection bounding box within the parent image

has_landmarks: True if this object contains landmarks

identity: Identity mechanism as key, identity embedding as value

landmarks_xy: The frame space 2D landmarks for this detected face.

right: Right point (in pixels) of face detection bounding box within the parent image

Methods Documentation

add_identity(name: str, embedding: ndarray) → None

Add an identity embedding to this detected face. If an identity already exists for the given name it will be overwritten

Parameters:

name (str) – The name of the mechanism that calculated the identity
embedding (ndarray) – The identity embedding

Return type:

None

add_landmarks_xy(landmarks: ndarray) → None

Add landmarks to the detected face object. If landmarks already exist, they will be overwritten.

Parameters:: landmarks (ndarray) – The 68 point face landmarks to add for the face
Return type:: None

add_mask(name: str, mask: npt.NDArray[np.uint8], affine_matrix: np.ndarray, storage_size: int = 128, storage_centering: CenteringType = 'face') → None

Add a Mask to this detected face

The mask should be the original output from plugins.extract.mask If a mask with this name already exists it will be overwritten by the given mask.

Parameters:

name (str) – The name of the mask as defined by the plugins.extract.mask._base.name parameter.
mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
affine_matrix (np.ndarray) – The transformation matrix required to transform the mask to the original frame.
storage_size (int) – The size the mask is to be stored at. Default: 128
storage_centering (CenteringType) – The centering to store the mask at. One of “legacy”, “face”, “head”. Default: “face”

Return type:

None

clear_all_identities() → None

Remove all stored identity embeddings

Return type:: None

from_alignment(alignment: FileAlignments | PNGAlignments, image: ndarray | None = None, with_thumb: bool = False) → Self

Set the attributes of this class from an alignments file and optionally load the face into the image attribute.

Parameters:

alignment (FileAlignments | PNGAlignments) – The alignment object to obtain the alignments from
image (ndarray | None) – If an image is passed in, then the image attribute will be set to the cropped face based on the passed in bounding box co-ordinates
with_thumb (bool) – Whether to load the jpg thumbnail into the detected face object, if provided. Default: False

Return type:

This DetectedFace object populated by the incoming alignment dict

from_png_meta(alignment: PNGAlignments) → Self

Set the attributes of this class from alignments stored in a png exif header.

Parameters:: alignment (PNGAlignments) – A dictionary entry for a face from alignments stored in a png exif header containing the keys x, w, y, h, landmarks_xy and mask
Return type:: Self

get_landmark_mask(area: T.Literal['eye', 'mouth', 'face', 'face_extended'], dilation: float = 0, blur_kernel: int = 0, blur_type: T.Literal['gaussian', 'normalized'] | None = 'gaussian', blur_passes: int = 1) → npt.NDArray[np.uint8]

Obtain a LandmarksMask for this face

Landmark based masks are generated from Aligned Face landmark points. An aligned face must be loaded. As the data is coming from the already aligned face, no further mask cropping is required.

Parameters:

area (T.Literal['eye', 'mouth', 'face', 'face_extended']) – The type of mask to obtain. face is a full face mask, face_extended is a face mask that extends above the eyebrows. The others are masks for those specific areas
dilation (float) – The amount of dilation to apply to the mask. as a percentage of the mask size. Default: 0
blur_kernel (int) – The kernel size, in pixels to apply gaussian blurring to the mask. Set to 0 for no blurring. Should be odd, if an even number is passed in (outside of 0) then it is rounded up to the next odd number. Default: 0
blur_type (T.Literal['gaussian', 'normalized'] | None) – The blur type to use. gaussian or normalized box filter. Default: gaussian
blur_passes (int) – The number of passed to perform when blurring. Default: 1

Return type:

The generated landmarks mask for the selected area

get_training_masks() → ndarray | None

Obtain the decompressed combined training masks.

Returns:

A 3D array containing the decompressed training masks as uint8 in 0-255 range if
training masks are present otherwise None

Return type:

ndarray | None

load_aligned(image: np.ndarray | None, size: int = 256, dtype: str | None = None, centering: CenteringType = 'head', coverage_ratio: float = 1.0, y_offset: float = 0.0, force: bool = False, is_aligned: bool = False, is_legacy: bool = False) → None

Align a face from a given image.

Aligning a face is a relatively expensive task and is not required for all uses of the DetectedFace object, so call this function explicitly to load an aligned face.

This method plugs into lib.align.AlignedFace to perform face alignment based on this face’s landmarks_xy. If the face has already been aligned, then this function will return having performed no action.

Parameters:

image (np.ndarray | None) – The image that contains the face to be aligned. Default: None
size (int) – The size of the output face in pixels. Default: 256
dtype (str | None) – Optionally set a dtype for the final face to be formatted in. Default: None
centering (Literal["legacy", "face", "head"]) – The type of extracted face that should be loaded. “legacy” places the nose in the center of the image (the original method for aligning). “face” aligns for the nose to be in the center of the face (top to bottom) but the center of the skull for left to right. “head” aligns for the center of the skull (in 3D space) being the center of the extracted image, with the crop holding the full head. Default: “head”
coverage_ratio (float) – The amount of the aligned image to return. A ratio of 1.0 will return the full contents of the aligned image. A ratio of 0.5 will return an image of the given size, but will crop to the central 50%% of the image. Default: 1.0
y_offset (float) – The amount to adjust the aligned face along the y_axis in -1. to 1. range. Default: 0.0
force (bool) – Force an update of the aligned face, even if it is already loaded. Default: False
is_aligned (bool) – Indicates that the image is an aligned face rather than a frame. Default: False
is_legacy (bool) – Only used if is_aligned is True. True indicates that the aligned image being loaded is a legacy extracted face rather than a current head extracted face

Return type:

None

Notes

This method must be executed to get access to the following a lib.align.aligned_face.AlignedFace object

store_training_masks(masks: list[ndarray | None], delete_masks: bool = False) → None

Concatenate and compress the given training masks and store for retrieval.

Parameters:

masks (list[ | None]) – A list of training mask. Must be all be uint-8 3D arrays of the same size in 0-255 range
delete_masks (bool) – True to delete any of the Mask objects owned by this detected face. Use to free up non-required memory usage. Default: False

Return type:

None

to_alignment() → FileAlignments

Return the detected face formatted for an alignments file

Returns:

The alignment dict will be returned with the keys x, w, y, h,
landmarks_xy, mask. The additional key thumb will be provided if the
detected face object contains a thumbnail.

Return type:

FileAlignments

to_png_meta() → PNGAlignments

Return the detected face formatted for insertion into a png itxt header.

Returns:

The alignments dict will be returned with the keys x, w, y, h,
landmarks_xy and mask

Return type:

PNGAlignments

add_identity(name: str, embedding: ndarray) → None

Add an identity embedding to this detected face. If an identity already exists for the given name it will be overwritten

Parameters:

name (str) – The name of the mechanism that calculated the identity
embedding (ndarray) – The identity embedding

Return type:

None

add_landmarks_xy(landmarks: ndarray) → None

Add landmarks to the detected face object. If landmarks already exist, they will be overwritten.

Parameters:: landmarks (ndarray) – The 68 point face landmarks to add for the face
Return type:: None

add_mask(name: str, mask: npt.NDArray[np.uint8], affine_matrix: np.ndarray, storage_size: int = 128, storage_centering: CenteringType = 'face') → None

Add a Mask to this detected face

The mask should be the original output from plugins.extract.mask If a mask with this name already exists it will be overwritten by the given mask.

Parameters:

name (str) – The name of the mask as defined by the plugins.extract.mask._base.name parameter.
mask (npt.NDArray[np.uint8]) – The mask that is to be added as output from plugins.extract.mask as a UINT8 image
affine_matrix (np.ndarray) – The transformation matrix required to transform the mask to the original frame.
storage_size (int) – The size the mask is to be stored at. Default: 128
storage_centering (CenteringType) – The centering to store the mask at. One of “legacy”, “face”, “head”. Default: “face”

Return type:

None

property aligned: AlignedFace: The aligned face connected to this detected face.

property bottom: int: Bottom point (in pixels) of face detection bounding box within the parent image

clear_all_identities() → None

Remove all stored identity embeddings

Return type:: None

from_alignment(alignment: FileAlignments | PNGAlignments, image: ndarray | None = None, with_thumb: bool = False) → Self

Set the attributes of this class from an alignments file and optionally load the face into the image attribute.

Parameters:

alignment (FileAlignments | PNGAlignments) – The alignment object to obtain the alignments from
image (ndarray | None) – If an image is passed in, then the image attribute will be set to the cropped face based on the passed in bounding box co-ordinates
with_thumb (bool) – Whether to load the jpg thumbnail into the detected face object, if provided. Default: False

Return type:

This DetectedFace object populated by the incoming alignment dict

from_png_meta(alignment: PNGAlignments) → Self

Set the attributes of this class from alignments stored in a png exif header.

Parameters:: alignment (PNGAlignments) – A dictionary entry for a face from alignments stored in a png exif header containing the keys x, w, y, h, landmarks_xy and mask
Return type:: Self

get_landmark_mask(area: T.Literal['eye', 'mouth', 'face', 'face_extended'], dilation: float = 0, blur_kernel: int = 0, blur_type: T.Literal['gaussian', 'normalized'] | None = 'gaussian', blur_passes: int = 1) → npt.NDArray[np.uint8]

Obtain a LandmarksMask for this face

Landmark based masks are generated from Aligned Face landmark points. An aligned face must be loaded. As the data is coming from the already aligned face, no further mask cropping is required.

Parameters:

area (T.Literal['eye', 'mouth', 'face', 'face_extended']) – The type of mask to obtain. face is a full face mask, face_extended is a face mask that extends above the eyebrows. The others are masks for those specific areas
dilation (float) – The amount of dilation to apply to the mask. as a percentage of the mask size. Default: 0
blur_kernel (int) – The kernel size, in pixels to apply gaussian blurring to the mask. Set to 0 for no blurring. Should be odd, if an even number is passed in (outside of 0) then it is rounded up to the next odd number. Default: 0
blur_type (T.Literal['gaussian', 'normalized'] | None) – The blur type to use. gaussian or normalized box filter. Default: gaussian
blur_passes (int) – The number of passed to perform when blurring. Default: 1

Return type:

The generated landmarks mask for the selected area

get_training_masks() → ndarray | None

Obtain the decompressed combined training masks.

Returns:

A 3D array containing the decompressed training masks as uint8 in 0-255 range if
training masks are present otherwise None

Return type:

ndarray | None

property has_landmarks: bool: True if this object contains landmarks

height: The height (in pixels) of the face’s bounding box as discovered in plugins.extract.detect

property identity: dict[str, ndarray]: Identity mechanism as key, identity embedding as value

image: This is a generic image placeholder that should not be relied on to be holding a particular image. It may hold the source frame that holds the face, a cropped face or a scaled image depending on the method using this object.

property landmarks_xy: ndarray: The frame space 2D landmarks for this detected face.

left: The left most point (in pixels) of the face’s bounding box as discovered in plugins.extract.detect

load_aligned(image: np.ndarray | None, size: int = 256, dtype: str | None = None, centering: CenteringType = 'head', coverage_ratio: float = 1.0, y_offset: float = 0.0, force: bool = False, is_aligned: bool = False, is_legacy: bool = False) → None

Align a face from a given image.

Aligning a face is a relatively expensive task and is not required for all uses of the DetectedFace object, so call this function explicitly to load an aligned face.

This method plugs into lib.align.AlignedFace to perform face alignment based on this face’s landmarks_xy. If the face has already been aligned, then this function will return having performed no action.

Parameters:

image (np.ndarray | None) – The image that contains the face to be aligned. Default: None
size (int) – The size of the output face in pixels. Default: 256
dtype (str | None) – Optionally set a dtype for the final face to be formatted in. Default: None
centering (Literal["legacy", "face", "head"]) – The type of extracted face that should be loaded. “legacy” places the nose in the center of the image (the original method for aligning). “face” aligns for the nose to be in the center of the face (top to bottom) but the center of the skull for left to right. “head” aligns for the center of the skull (in 3D space) being the center of the extracted image, with the crop holding the full head. Default: “head”
coverage_ratio (float) – The amount of the aligned image to return. A ratio of 1.0 will return the full contents of the aligned image. A ratio of 0.5 will return an image of the given size, but will crop to the central 50%% of the image. Default: 1.0
y_offset (float) – The amount to adjust the aligned face along the y_axis in -1. to 1. range. Default: 0.0
force (bool) – Force an update of the aligned face, even if it is already loaded. Default: False
is_aligned (bool) – Indicates that the image is an aligned face rather than a frame. Default: False
is_legacy (bool) – Only used if is_aligned is True. True indicates that the aligned image being loaded is a legacy extracted face rather than a current head extracted face

Return type:

None

Notes

This method must be executed to get access to the following a lib.align.aligned_face.AlignedFace object

mask: The generated mask(s) for the face as generated in plugins.extract.mask

property right: int: Right point (in pixels) of face detection bounding box within the parent image

store_training_masks(masks: list[ndarray | None], delete_masks: bool = False) → None

Concatenate and compress the given training masks and store for retrieval.

Parameters:

masks (list[ | None]) – A list of training mask. Must be all be uint-8 3D arrays of the same size in 0-255 range
delete_masks (bool) – True to delete any of the Mask objects owned by this detected face. Use to free up non-required memory usage. Default: False

Return type:

None

to_alignment() → FileAlignments

Return the detected face formatted for an alignments file

Returns:

The alignment dict will be returned with the keys x, w, y, h,
landmarks_xy, mask. The additional key thumb will be provided if the
detected face object contains a thumbnail.

Return type:

FileAlignments

to_png_meta() → PNGAlignments

Return the detected face formatted for insertion into a png itxt header.

Returns:

The alignments dict will be returned with the keys x, w, y, h,
landmarks_xy and mask

Return type:

PNGAlignments

top: The top most point (in pixels) of the face’s bounding box as discovered in plugins.extract.detect

width: The width (in pixels) of the face’s bounding box as discovered in plugins.extract.detect