manual package

Subpackages 

The following subpackages handle the main two display areas of the Manual Tool’s GUI.

manual module 

The Manual Module is the main entry point into the Manual Editor Tool.

Module Summary

`Aligner`	The `Aligner` class sets up an extraction pipeline for each of the current Faceswap Aligners, along with the Landmarks based Maskers.
`FrameLoader`	Loads the frames, sets the frame count to `TkGlobals.frame_count` and handles the return of the correct frame for the GUI.
`Manual`	The main entry point for Faceswap's Manual Editor Tool.
`TkGlobals`	Holds Tkinter Variables and other frame information that need to be accessible from all areas of the GUI.

Module

The Manual Tool is a tkinter driven GUI app for editing alignments files with visual tools. This module is the main entry point into the Manual Tool.

class tools.manual.manual.Aligner(tk_globals: TkGlobals, exclude_gpus: list[int] | None)

Bases: object

The Aligner class sets up an extraction pipeline for each of the current Faceswap Aligners, along with the Landmarks based Maskers. When new landmarks are required, the bounding boxes from the GUI are passed to this class for pushing through the pipeline. The resulting Landmarks and Masks are then returned.

Parameters:

tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
exclude_gpus (list or None) – A list of indices correlating to connected GPUs that Tensorflow should not use. Pass None to not exclude any GPUs.

get_landmarks(frame_index: int, face_index: int, aligner: Literal['FAN', 'cv2-dnn', 'mask']) → ndarray

Feed the detected face into the alignment pipeline and retrieve the landmarks.

The face to feed into the aligner is generated from the given frame and face indices.

Parameters:

frame_index (int) – The frame index to extract the aligned face for
face_index (int) – The face index within the current frame to extract the face for
aligner (Literal["FAN", "cv2-dnn"]) – The aligner to use to extract the face

Returns:

The 68 point landmark alignments

Return type:

numpy.ndarray

get_masks(frame_index: int, face_index: int) → dict[str, Mask]

Feed the aligned face into the mask pipeline and retrieve the updated masks.

The face to feed into the aligner is generated from the given frame and face indices. This is to be called when a manual update is done on the landmarks, and new masks need generating.

Parameters:

frame_index (int) – The frame index to extract the aligned face for
face_index (int) – The face index within the current frame to extract the face for

Returns:

The updated masks

Return type:

dict[str, Mask]

property is_initialized: bool

The Aligners are initialized in a background thread so that other tasks can be performed whilst we wait for initialization. True is returned if the aligner has completed initialization otherwise False.

Type:: bool

link_faces(detected_faces: DetectedFaces) → None

As the Aligner has the potential to take the longest to initialize, it is kicked off as early as possible. At this time DetectedFaces is not yet available.

Once the Aligner has initialized, this function is called to add the DetectedFaces class as a property of the Aligner.

Parameters:: detected_faces (DetectedFaces) – The class that holds the DetectedFace objects for the current Manual session

set_normalization_method(method: Literal['none', 'clahe', 'hist', 'mean']) → None

Change the normalization method for faces fed into the aligner. The normalization method is user adjustable from the GUI. When this method is triggered the method is updated for all aligner pipelines.

Parameters:: method (Literal["none", "clahe", "hist", "mean"]) – The normalization method to use

class tools.manual.manual.FrameLoader(tk_globals, frames_location, video_meta_data)

Bases: object

Loads the frames, sets the frame count to TkGlobals.frame_count and handles the return of the correct frame for the GUI.

Parameters:

tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
frames_location (str) – The path to the input frames
video_meta_data (dict) – The meta data held within the alignments file, if it exists and the input is a video

property is_initialized

True if the Frame Loader has completed initialization otherwise False.

Type:: bool

property video_meta_data

The pts_time and key frames for the loader.

Type:: dict

class tools.manual.manual.Manual(arguments)

Bases: Tk

The main entry point for Faceswap’s Manual Editor Tool. This tool is part of the Faceswap Tools suite and should be called from python tools.py manual command.

Allows for visual interaction with frames, faces and alignments file to perform various adjustments to the alignments file.

Parameters:: arguments (argparse.Namespace) – The argparse arguments as passed in from tools.py

process()

The entry point for the Visual Alignments tool from lib.tools.manual.cli.

Launch the tkinter Visual Alignments Window and run main loop.

class tools.manual.manual.TkGlobals(input_location)

Bases: object

Holds Tkinter Variables and other frame information that need to be accessible from all areas of the GUI.

Parameters:: input_location (str) – The location of the input folder of frames or video file

property current_frame

The currently displayed frame in the frame viewer with it’s meta information. Key and Values are as follows:

image (numpy.ndarry): The currently displayed frame in original dimensions

scale (float): The scaling factor to use to resize the image to the display window

interpolation (int): The opencv interpolator ID to use for resizing the image to the display window

display_dims (tuple): The size of the currently displayed frame, sized for the display window

filename (str): The filename of the currently displayed frame

Type:: dict

property face_index

The currently displayed face index when in zoomed mode.

Type:: int

property filter_mode

The currently selected navigation mode.

Type:: str

property frame_count

The total number of frames for the input location

Type:: int

property frame_display_dims

The (width, height) of the video display frame in pixels.

Type:: tuple

property frame_index

The currently displayed frame index. NB This returns -1 if there are no frames that meet the currently selected filter criteria.

Type:: int

property is_video

True if the input is a video file, False if it is a folder of images.

Type:: bool

property is_zoomed

True if the frame viewer is zoomed into a face, False if the frame viewer is displaying a full frame.

Type:: bool

set_current_frame(image, filename)

Set the frame and meta information for the currently displayed frame. Populates the attribute current_frame

Parameters:

image (numpy.ndarray) – The image used to display in the Frame Viewer
filename (str) – The filename of the current frame

set_frame_count(count)

Set the count of total number of frames to frame_count when the FramesLoader has completed loading.

Parameters:: count (int) – The number of frames that exist for this session

set_frame_display_dims(width, height)

Set the size, in pixels, of the video frame display window and resize the displayed frame.

Used on a frame resize callback, sets the :attr:frame_display_dims`.

Parameters:

width (int) – The width of the frame holding the video canvas in pixels
height (int) – The height of the frame holding the video canvas in pixels

property tk_face_index

The variable that holds the face index of the selected face within the current frame when in zoomed mode.

Type:: tkinter.IntVar

property tk_faces_size

The variable holding the currently selected Faces Viewer thumbnail size.

Type:: tkinter.StringVar

property tk_filter_distance

The variable holding the currently selected threshold distance for misaligned filter mode.

Type:: tkinter.DoubleVar

property tk_filter_mode

The variable holding the currently selected navigation filter mode.

Type:: tkinter.StringVar

property tk_frame_index

The variable holding the current frame index.

Type:: tkinter.IntVar

property tk_is_zoomed

The variable holding the value indicating whether the frame viewer is zoomed into a face or zoomed out to the full frame.

Type:: tkinter.BooleanVar

property tk_transport_index

The current index of the display frame’s transport slider.

Type:: tkinter.IntVar

property tk_update

The variable holding the trigger that indicates that a full update needs to occur.

Type:: tkinter.BooleanVar

property tk_update_active_viewport

Boolean Variable that is traced by the viewport’s active frame to update..

Type:: tkinter.BooleanVar

detected_faces module 

Module Summary

`DetectedFaces`	Handles the manipulation of `DetectedFace` objects stored in the alignments file.
`FaceUpdate`	Perform updates on `DetectedFace` objects stored in `DetectedFaces` when changes are made within the GUI.
`Filter`	Returns stats and frames for filtered frames based on the user selected navigation mode filter.

Module

Alignments handling for Faceswap’s Manual Adjustments tool. Handles the conversion of alignments data to DetectedFace objects, and the update of these faces when edits are made in the GUI.

class tools.manual.detected_faces.DetectedFaces(tk_globals: manual.TkGlobals, alignments_path: str, input_location: str, extractor: manual.Aligner)

Bases: object

Handles the manipulation of DetectedFace objects stored in the alignments file. Acts as a parent class for the IO operations (saving and loading from an alignments file), the face update operations (when changes are made to alignments in the GUI) and the face filters (when a user changes the filter navigation mode.)

Parameters:

tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
alignments_path (str) – The full path to the alignments file
input_location (str) – The location of the input folder of frames or video file
extractor (Aligner) – The pipeline for passing faces through the aligner and retrieving results

property available_masks: dict[str, int]

The mask type names stored in the alignments; type as key with the number of faces which possess the mask type as value.

Type:: dict[str, int]

property current_faces: list[list[lib.align.detected_face.DetectedFace]]

The most up to date full list of detected face objects.

Type:: list[list[DetectedFace]]

extract() → None: Extract the faces in the current video to a user supplied folder.

property extractor: manual.Aligner

The pipeline for passing faces through the aligner and retrieving results.

Type:: Aligner

property face_count_per_index: list[int]

Count of faces for each frame. List is in frame index order.

The list needs to be calculated on the fly as the number of faces in a frame can change based on user actions.

Type:: list[int]

property filter: Filter

Handles returning of faces and stats based on the current user set navigation mode filter.

Type:: Filter

is_frame_updated(frame_index: int) → bool

Check whether the given frame index has been updated

Parameters:: frame_index (int) – The frame index to check
Returns:: True if the given frame index has updated faces within it otherwise False
Return type:: bool

load_faces() → None: Load the faces as DetectedFace objects from the alignments file.

revert_to_saved(frame_index)

Revert the frame’s alignments to their saved version for the given frame index.

Parameters:: frame_index (int) – The frame that should have their faces reverted to their saved version

save() → None: Save the alignments file with the latest edits.

save_video_meta_data(pts_time: list[float], keyframes: list[int]) → None

Save video meta data to the alignments file. This is executed if the video meta data does not already exist in the alignments file, so the video does not need to be scanned on every use of the Manual Tool.

Parameters:

pts_time (list[float]) – A list of presentation timestamps in frame index order for every frame in the input video
keyframes (list[int]) – A list of frame indices corresponding to the key frames in the input video.

property tk_edited: BooleanVar

The variable indicating whether an edit has occurred meaning a GUI redraw needs to be triggered.

Type:: tkinter.BooleanVar

property tk_face_count_changed: BooleanVar

The variable indicating whether a face has been added or removed meaning the FaceViewer grid redraw needs to be triggered.

Type:: tkinter.BooleanVar

property tk_unsaved: BooleanVar

The variable indicating whether the alignments have been updated since the last save.

Type:: tkinter.BooleanVar

property update: FaceUpdate

Handles the adding, removing and updating of DetectedFace stored within the alignments file.

Type:: FaceUpdate

property video_meta_data: dict[str, list[int] | list[float] | None]

The frame meta data stored in the alignments file. If data does not exist in the alignments file then None is returned for each Key

Type:: dict[str, list[int] | list[float] | None]

class tools.manual.detected_faces.FaceUpdate(detected_faces: DetectedFaces)

Bases: object

Perform updates on DetectedFace objects stored in DetectedFaces when changes are made within the GUI.

Parameters:: detected_faces (DetectedFaces) – The parent DetectedFaces object

add(frame_index: int, pnt_x: int, width: int, pnt_y: int, height: int) → None

Add a DetectedFace object to the current frame with the given dimensions.

Parameters:

frame_index (int) – The frame that the face is being set for
pnt_x (int) – The left point of the bounding box
width (int) – The width of the bounding box
pnt_y (int) – The top point of the bounding box
height (int) – The height of the bounding box

bounding_box(frame_index: int, face_index: int, pnt_x: int, width: int, pnt_y: int, height: int, aligner: manual.TypeManualExtractor = 'FAN') → None

Update the bounding box for the DetectedFace object at the given frame and face indices, with the given dimensions and update the 68 point landmarks from the Aligner for the updated bounding box.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
pnt_x (int) – The left point of the bounding box
width (int) – The width of the bounding box
pnt_y (int) – The top point of the bounding box
height (int) – The height of the bounding box
aligner (["cv2-dnn", "FAN"], optional) – The aligner to use to generate the landmarks. Default: “FAN”

copy(frame_index: int, direction: Literal['prev', 'next']) → None

Copy the alignments from the previous or next frame that has alignments to the current frame.

Parameters:

frame_index (int) – The frame that the needs to have alignments copied to it
direction (["prev", "next"]) – Whether to copy alignments from the previous frame with alignments, or the next frame with alignments

delete(frame_index: int, face_index: int) → None

Delete the DetectedFace object for the given frame and face indices.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame

landmark(frame_index: int, face_index: int, landmark_index: int, shift_x: int, shift_y: int, is_zoomed: bool) → None

Shift a single landmark point for the DetectedFace object at the given frame and face indices by the given x and y values.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
landmark_index (int or list) – The landmark index to shift. If a list is provided, this should be a list of landmark indices to be shifted
shift_x (int) – The amount to shift the landmark by along the x axis
shift_y (int) – The amount to shift the landmark by along the y axis
is_zoomed (bool) – True if landmarks are being adjusted on a zoomed image otherwise False

landmarks(frame_index: int, face_index: int, shift_x: int, shift_y: int) → None

Shift all of the landmarks and bounding box for the DetectedFace object at the given frame and face indices by the given x and y values and update the masks.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
shift_x (int) – The amount to shift the landmarks by along the x axis
shift_y (int) – The amount to shift the landmarks by along the y axis

Notes

Whilst the bounding box does not need to be shifted, it is anyway, to ensure that it is aligned with the newly adjusted landmarks.

landmarks_rotate(frame_index: int, face_index: int, angle: float, center: ndarray) → None

Rotate the landmarks on an Extract Box rotate for the DetectedFace object at the given frame and face indices for the given angle from the given center point.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
angle (float) – The angle, in radians to rotate the points by
center (numpy.ndarray) – The center point of the Landmark’s Extract Box

landmarks_scale(frame_index: int, face_index: int, scale: ndarray, center: ndarray) → None

Scale the landmarks on an Extract Box resize for the DetectedFace object at the given frame and face indices from the given center point.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
scale (float) – The amount to scale the landmarks by
center (numpy.ndarray) – The center point of the Landmark’s Extract Box

mask(frame_index: int, face_index: int, mask: ndarray, mask_type: str) → None

Update the mask on an edit for the DetectedFace object at the given frame and face indices, for the given mask and mask type.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame
mask (class:numpy.ndarray:) – The mask to replace
mask_type (str) – The name of the mask that is to be replaced

post_edit_trigger(frame_index: int, face_index: int) → None

Update the jpg thumbnail, the viewport thumbnail, the landmark masks and the aligned face on a face edit.

Parameters:

frame_index (int) – The frame that the face is being set for
face_index (int) – The face index within the frame

class tools.manual.detected_faces.Filter(detected_faces: DetectedFaces)

Bases: object

Returns stats and frames for filtered frames based on the user selected navigation mode filter.

Parameters:: detected_faces (DetectedFaces) – The parent DetectedFaces object

property count: int

The number of frames that meet the filter criteria returned by filter_mode.

Type:: int

property frame_meets_criteria: bool

True if the current frame meets the selected filter criteria otherwise False

Type:: bool

property frames_list: list[int]

The list of frame indices that meet the filter criteria returned by filter_mode.

Type:: list[int]

property raw_indices: dict[Literal['frame', 'face'], list[int]]

The frame and face indices that meet the current filter criteria for each displayed face.

Type:: dict[str, int]

thumbnails module 

Thumbnail generator for the manual tool

class tools.manual.thumbnails.ProgressBar(pbar: tqdm | None = None)

Bases: object

Thread-safe progress bar for tracking thumbnail generation progress

lock = <unlocked _thread.lock object>

pbar: tqdm | None = None

class tools.manual.thumbnails.ThumbsCreator(detected_faces: DetectedFaces, input_location: str, single_process: bool)

Bases: object

Background loader to generate thumbnails for the alignments file. Generates low resolution thumbnails in parallel threads for faster processing.

Parameters:

detected_faces (DetectedFaces) – The DetectedFace objects for this video
input_location (str) – The location of the input folder of frames or video file
single_process (bool) – True to generated thumbs in a single process otherwise False

generate_cache() → None: Extract the face thumbnails from a video or folder of images into the alignments file.

property has_thumbs: bool

True if the underlying alignments file holds thumbnail images otherwise False.

Type:: bool

class tools.manual.thumbnails.VideoMeta(key_frames: list[int] | None = None, pts_times: list[float] | None = None)

Bases: object

Holds meta information about a video file

Parameters:

key_frames (list[int]) – List of key frame indices for the video
pts_times (list[float]) – List of presentation timestams for the video

key_frames: list[int] | None = None

pts_times: list[float] | None = None

manual package

Subpackages

manual module

detected_faces module

thumbnails module

Subpackages 

manual module 

detected_faces module 

thumbnails module 