Alignments

class lib.align.alignments.Alignments(folder: str, filename: str = 'alignments')

Bases: object

The alignments file is a custom serialized .fsa file that holds information for each frame for a video or series of images.

Specifically, it holds a list of faces that appear in each frame. Each face contains information detailing their detected bounding box location within the frame, the 68 point facial landmarks and any masks that have been extracted.

Additionally it can also hold video meta information (timestamp and whether a frame is a key frame.)

Parameters:

folder (str) – The folder that contains the alignments .fsa file
filename (str) – The filename of the .fsa alignments file. If not provided then the given folder will be checked for a default alignments file filename. Default: “alignments”

Attributes Summary

`data`	The loaded alignments `file` in dictionary form.
`faces_count`	The total number of faces that appear in the alignments `data`
`file`	The full path to the currently loaded alignments file.
`frames_count`	The number of frames that appear in the alignments `data`.
`have_alignments_file`	`True` if an alignments file exists at location `file` otherwise `False`.
`mask_summary`	The mask type names stored in the alignments `data` as key with the number of faces which possess the mask type as value.
`thumbnails`	The low resolution thumbnail images that exist within the alignments file
`version`	The alignments file version number.
`video_meta_data`	The frame meta data stored in the alignments file.

Methods Summary

`add_face`(frame_name, face)	Add a new face for the given frame_name in `data` and return it's index.
`backup`()	Create a backup copy of the alignments `file`.
`count_faces_in_frame`(frame_name)	Return number of faces that appear within `data` for the given frame_name.
`delete_face_at_index`(frame_name, face_index)	Delete the face for the given frame_name at the given face index from `data`.
`filter_faces`(filter_dict[, filter_out])	Remove faces from `data` based on a given filter list.
`frame_exists`(frame_name)	Check whether a given frame_name exists within the alignments `data`.
`frame_has_faces`(frame_name)	Check whether a given frame_name exists within the alignments `data` and contains at least 1 face.
`frame_has_multiple_faces`(frame_name)	Check whether a given frame_name exists within the alignments `data` and contains more than 1 face.
`get_faces_in_frame`(frame_name)	Obtain the faces from `data` associated with a given frame_name.
`mask_is_valid`(mask_type)	Ensure the given `mask_type` is valid for the alignments `data`.
`save`()	Write the contents of `data` and `_meta` to a serialized `.fsa` file at the location `file`.
`save_video_meta_data`(pts_time, keyframes)	Save video meta data to the alignments file.
`update_face`(frame_name, face_index, face)	Update the face for the given frame_name at the given face index in `data`.
`update_from_dict`(data)	Replace all alignments with the contents of the given dictionary
`update_legacy_has_source`(filename)	Update legacy alignments files when we have the source filename available.
`yield_faces`()	Generator to obtain all faces with meta information from `data`.

Attributes Documentation

data: The loaded alignments file in dictionary form.

faces_count: The total number of faces that appear in the alignments data

file: The full path to the currently loaded alignments file.

frames_count: The number of frames that appear in the alignments data.

have_alignments_file: True if an alignments file exists at location file otherwise False.

mask_summary: The mask type names stored in the alignments data as key with the number of faces which possess the mask type as value.

thumbnails: The low resolution thumbnail images that exist within the alignments file

version

The alignments file version number.

Type:: float

video_meta_data: The frame meta data stored in the alignments file. If data does not exist in the alignments file then None is returned

Methods Documentation

add_face(frame_name: str, face: FileAlignments) → int

Add a new face for the given frame_name in data and return it’s index.

Parameters:

frame_name (str) – The frame name to add the face to. This should be the base name of the frame, not the full path
face (FileAlignments) – The face information to add to the given frame_name, correctly formatted for storing in data

Return type:

The index of the newly added face within data for the given frame_name

backup() → None

Create a backup copy of the alignments file.

Creates a copy of the serialized alignments file appending a timestamp onto the end of the file name and storing in the same folder as the original file.

Return type:: None

count_faces_in_frame(frame_name: str) → int

Return number of faces that appear within data for the given frame_name.

Parameters:: frame_name (str) – The frame name to return the count for. This should be the base name of the frame, not the full path
Return type:: The number of faces that appear in the given frame_name

delete_face_at_index(frame_name: str, face_index: int) → bool

Delete the face for the given frame_name at the given face index from data.

Parameters:

frame_name (str) – The frame name to remove the face from. This should be the base name of the frame, not the full path
face_index (int) – The index number of the face within the given frame_name to remove

Return type:

True if a face was successfully deleted otherwise False

filter_faces(filter_dict: dict[str, list[int]], filter_out: bool = False) → None

Remove faces from data based on a given filter list.

Parameters:

filter_dict (dict[str, list[int]]) – Dictionary of source filenames as key with a list of face indices to filter as value.
filter_out (bool) – True if faces should be removed from data when there is a corresponding match in the given filter_dict. False if faces should be kept in data when there is a corresponding match in the given filter_dict, but removed if there is no match. Default: False

Return type:

None

frame_exists(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data.

Parameters:

frame_name (str) – The frame name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data otherwise
False

Return type:

bool

frame_has_faces(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data and contains at least 1 face.

Parameters:

frame_name (str) – The frame name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data and has at least
1 face associated with it, otherwise False

Return type:

bool

frame_has_multiple_faces(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data and contains more than 1 face.

Parameters:

frame_name (str) – The frame_name name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data and has more
than 1 face associated with it, otherwise False

Return type:

bool

get_faces_in_frame(frame_name: str) → list[FileAlignments]

Obtain the faces from data associated with a given frame_name.

Parameters:: frame_name (str) – The frame name to return faces for. This should be the base name of the frame, not the full path
Return type:: The list of face dictionaries that appear within the requested frame_name

mask_is_valid(mask_type: str) → bool

Ensure the given mask_type is valid for the alignments data.

Every face in the alignments data must have the given mask type to successfully pass the test.

Parameters:

mask_type (str) – The mask type to check against the current alignments data

Returns:

True if all faces in the current alignments possess the given mask_type otherwise
False

Return type:

bool

save() → None

Write the contents of data and _meta to a serialized .fsa file at the location file.

Return type:: None

save_video_meta_data(pts_time: list[int], keyframes: list[int]) → None

Save video meta data to the alignments file.

If the alignments file does not have an entry for every frame (e.g. if Extract Every N was used) then the frame is added to the alignments file with no faces, so that they video meta data can be stored.

Parameters:

pts_time (list[int]) – A list of presentation timestamps (int) in frame index order for every frame in the input video
keyframes (list[int]) – A list of frame indices corresponding to the key frames in the input video

Return type:

None

update_face(frame_name: str, face_index: int, face: FileAlignments) → None

Update the face for the given frame_name at the given face index in data.

Parameters:

frame_name (str) – The frame name to update the face for. This should be the base name of the frame, not the full path
face_index (int) – The index number of the face within the given frame_name to update
face (FileAlignments) – The face information to update to the given frame_name at the given face_index, correctly formatted for storing in data

Return type:

None

update_from_dict(data: dict[str, AlignmentsEntry]) → None

Replace all alignments with the contents of the given dictionary

Parameters:: data (dict[str, AlignmentsEntry]) – The alignments, in correctly formatted dictionary form, to be populated into this Alignments
Return type:: None

update_legacy_has_source(filename: str) → None

Update legacy alignments files when we have the source filename available.

Updates here can only be performed when we have the source filename

Parameters:: filename (str) – The filename/folder of the original source images/video for the current alignments
Return type:: None

yield_faces() → Generator[tuple[str, list[FileAlignments], int, str], None, None]

Generator to obtain all faces with meta information from data. The results are yielded by frame.

Notes

The yielded order is non-deterministic.

Yields:

frame_name – The frame name that the face belongs to. This is the base name of the frame, as it appears in data, not the full path
faces – The list of face dict objects that exist for this frame
face_count – The number of faces that exist within data for this frame
frame_fullname – The full path (folder and filename) for the yielded frame

Return type:

Generator[tuple[str, list[FileAlignments], int, str], None, None]

add_face(frame_name: str, face: FileAlignments) → int

Add a new face for the given frame_name in data and return it’s index.

Parameters:

frame_name (str) – The frame name to add the face to. This should be the base name of the frame, not the full path
face (FileAlignments) – The face information to add to the given frame_name, correctly formatted for storing in data

Return type:

The index of the newly added face within data for the given frame_name

backup() → None

Create a backup copy of the alignments file.

Creates a copy of the serialized alignments file appending a timestamp onto the end of the file name and storing in the same folder as the original file.

Return type:: None

count_faces_in_frame(frame_name: str) → int

Return number of faces that appear within data for the given frame_name.

Parameters:: frame_name (str) – The frame name to return the count for. This should be the base name of the frame, not the full path
Return type:: The number of faces that appear in the given frame_name

property data: dict[str, AlignmentsEntry]: The loaded alignments file in dictionary form.

delete_face_at_index(frame_name: str, face_index: int) → bool

Delete the face for the given frame_name at the given face index from data.

Parameters:

frame_name (str) – The frame name to remove the face from. This should be the base name of the frame, not the full path
face_index (int) – The index number of the face within the given frame_name to remove

Return type:

True if a face was successfully deleted otherwise False

property faces_count: int: The total number of faces that appear in the alignments data

property file: str: The full path to the currently loaded alignments file.

filter_faces(filter_dict: dict[str, list[int]], filter_out: bool = False) → None

Remove faces from data based on a given filter list.

Parameters:

filter_dict (dict[str, list[int]]) – Dictionary of source filenames as key with a list of face indices to filter as value.
filter_out (bool) – True if faces should be removed from data when there is a corresponding match in the given filter_dict. False if faces should be kept in data when there is a corresponding match in the given filter_dict, but removed if there is no match. Default: False

Return type:

None

frame_exists(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data.

Parameters:

frame_name (str) – The frame name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data otherwise
False

Return type:

bool

frame_has_faces(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data and contains at least 1 face.

Parameters:

frame_name (str) – The frame name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data and has at least
1 face associated with it, otherwise False

Return type:

bool

frame_has_multiple_faces(frame_name: str) → bool

Check whether a given frame_name exists within the alignments data and contains more than 1 face.

Parameters:

frame_name (str) – The frame_name name to check. This should be the base name of the frame, not the full path

Returns:

True if the given frame_name exists within the alignments data and has more
than 1 face associated with it, otherwise False

Return type:

bool

property frames_count: int: The number of frames that appear in the alignments data.

get_faces_in_frame(frame_name: str) → list[FileAlignments]

Obtain the faces from data associated with a given frame_name.

Parameters:: frame_name (str) – The frame name to return faces for. This should be the base name of the frame, not the full path
Return type:: The list of face dictionaries that appear within the requested frame_name

property have_alignments_file: bool: True if an alignments file exists at location file otherwise False.

mask_is_valid(mask_type: str) → bool

Ensure the given mask_type is valid for the alignments data.

Every face in the alignments data must have the given mask type to successfully pass the test.

Parameters:

mask_type (str) – The mask type to check against the current alignments data

Returns:

True if all faces in the current alignments possess the given mask_type otherwise
False

Return type:

bool

property mask_summary: dict[str, int]: The mask type names stored in the alignments data as key with the number of faces which possess the mask type as value.

save() → None

Write the contents of data and _meta to a serialized .fsa file at the location file.

Return type:: None

save_video_meta_data(pts_time: list[int], keyframes: list[int]) → None

Save video meta data to the alignments file.

If the alignments file does not have an entry for every frame (e.g. if Extract Every N was used) then the frame is added to the alignments file with no faces, so that they video meta data can be stored.

Parameters:

pts_time (list[int]) – A list of presentation timestamps (int) in frame index order for every frame in the input video
keyframes (list[int]) – A list of frame indices corresponding to the key frames in the input video

Return type:

None

property thumbnails: Thumbnails: The low resolution thumbnail images that exist within the alignments file

update_face(frame_name: str, face_index: int, face: FileAlignments) → None

Update the face for the given frame_name at the given face index in data.

Parameters:

frame_name (str) – The frame name to update the face for. This should be the base name of the frame, not the full path
face_index (int) – The index number of the face within the given frame_name to update
face (FileAlignments) – The face information to update to the given frame_name at the given face_index, correctly formatted for storing in data

Return type:

None

update_from_dict(data: dict[str, AlignmentsEntry]) → None

Replace all alignments with the contents of the given dictionary

Parameters:: data (dict[str, AlignmentsEntry]) – The alignments, in correctly formatted dictionary form, to be populated into this Alignments
Return type:: None

update_legacy_has_source(filename: str) → None

Update legacy alignments files when we have the source filename available.

Updates here can only be performed when we have the source filename

Parameters:: filename (str) – The filename/folder of the original source images/video for the current alignments
Return type:: None

property version: float

The alignments file version number.

Type:: float

property video_meta_data: dict[Literal['pts_time', 'keyframes'], list[int]] | None: The frame meta data stored in the alignments file. If data does not exist in the alignments file then None is returned

yield_faces() → Generator[tuple[str, list[FileAlignments], int, str], None, None]

Generator to obtain all faces with meta information from data. The results are yielded by frame.

Notes

The yielded order is non-deterministic.

Yields:

frame_name – The frame name that the face belongs to. This is the base name of the frame, as it appears in data, not the full path
faces – The list of face dict objects that exist for this frame
face_count – The number of faces that exist within data for this frame
frame_fullname – The full path (folder and filename) for the yielded frame

Return type:

Generator[tuple[str, list[FileAlignments], int, str], None, None]