manual package

manual module

The Manual Module is the main entry point into the Manual Editor Tool.

Module Summary

Aligner The Aligner class sets up an extraction pipeline for each of the current Faceswap Aligners, along with the Landmarks based Maskers.
FrameLoader Loads the frames, sets the frame count to TkGlobals.frame_count and handles the return of the correct frame for the GUI.
Manual The main entry point for Faceswap’s Manual Editor Tool.
TkGlobals Holds Tkinter Variables and other frame information that need to be accessible from all areas of the GUI.

Module

The Manual Tool is a tkinter driven GUI app for editing alignments files with visual tools. This module is the main entry point into the Manual Tool.

class tools.manual.manual.Aligner(tk_globals, exclude_gpus)

Bases: object

The Aligner class sets up an extraction pipeline for each of the current Faceswap Aligners, along with the Landmarks based Maskers. When new landmarks are required, the bounding boxes from the GUI are passed to this class for pushing through the pipeline. The resulting Landmarks and Masks are then returned.

Parameters:
  • tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
  • exclude_gpus (list or None) – A list of indices correlating to connected GPUs that Tensorflow should not use. Pass None to not exclude any GPUs.
get_landmarks(frame_index, face_index, aligner)

Feed the detected face into the alignment pipeline and retrieve the landmarks.

The face to feed into the aligner is generated from the given frame and face indices.

Parameters:
  • frame_index (int) – The frame index to extract the aligned face for
  • face_index (int) – The face index within the current frame to extract the face for
  • aligner (["FAN", "cv2-dnn"]) – The aligner to use to extract the face
Returns:

The 68 point landmark alignments

Return type:

numpy.ndarray

get_masks(frame_index, face_index)

Feed the aligned face into the mask pipeline and retrieve the updated masks.

The face to feed into the aligner is generated from the given frame and face indices. This is to be called when a manual update is done on the landmarks, and new masks need generating

Parameters:
  • frame_index (int) – The frame index to extract the aligned face for
  • face_index (int) – The face index within the current frame to extract the face for
Returns:

The updated masks

Return type:

dict

is_initialized

The Aligners are initialized in a background thread so that other tasks can be performed whilst we wait for initialization. True is returned if the aligner has completed initialization otherwise False.

Type:bool

As the Aligner has the potential to take the longest to initialize, it is kicked off as early as possible. At this time DetectedFaces is not yet available.

Once the Aligner has initialized, this function is called to add the DetectedFaces class as a property of the Aligner.

Parameters:detected_faces (DetectedFaces) – The class that holds the DetectedFace objects for the current Manual session
set_normalization_method(method)

Change the normalization method for faces fed into the aligner. The normalization method is user adjustable from the GUI. When this method is triggered the method is updated for all aligner pipelines.

Parameters:method (str) – The normalization method to use
class tools.manual.manual.FrameLoader(tk_globals, frames_location, video_meta_data)

Bases: object

Loads the frames, sets the frame count to TkGlobals.frame_count and handles the return of the correct frame for the GUI.

Parameters:
  • tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
  • frames_location (str) – The path to the input frames
  • video_meta_data (dict) – The meta data held within the alignments file, if it exists and the input is a video
is_initialized

True if the Frame Loader has completed initialization otherwise False.

Type:bool
video_meta_data

The pts_time and key frames for the loader.

Type:dict
class tools.manual.manual.Manual(arguments)

Bases: tkinter.Tk

The main entry point for Faceswap’s Manual Editor Tool. This tool is part of the Faceswap Tools suite and should be called from python tools.py manual command.

Allows for visual interaction with frames, faces and alignments file to perform various adjustments to the alignments file.

Parameters:arguments (argparse.Namespace) – The argparse arguments as passed in from tools.py
process()

The entry point for the Visual Alignments tool from lib.tools.manual.cli.

Launch the tkinter Visual Alignments Window and run main loop.

class tools.manual.manual.TkGlobals(input_location)

Bases: object

Holds Tkinter Variables and other frame information that need to be accessible from all areas of the GUI.

Parameters:input_location (str) – The location of the input folder of frames or video file
current_frame

The currently displayed frame in the frame viewer with it’s meta information. Key and Values are as follows:

image (numpy.ndarry): The currently displayed frame in original dimensions

scale (float): The scaling factor to use to resize the image to the display window

interpolation (int): The opencv interpolator ID to use for resizing the image to the display window

display_dims (tuple): The size of the currently displayed frame, sized for the display window

filename (str): The filename of the currently displayed frame

Type:dict
face_index

The currently displayed face index when in zoomed mode.

Type:int
filter_mode

The currently selected navigation mode.

Type:str
frame_count

The total number of frames for the input location

Type:int
frame_display_dims

The (width, height) of the video display frame in pixels.

Type:tuple
frame_index

The currently displayed frame index. NB This returns -1 if there are no frames that meet the currently selected filter criteria.

Type:int
is_video

True if the input is a video file, False if it is a folder of images.

Type:bool
is_zoomed

True if the frame viewer is zoomed into a face, False if the frame viewer is displaying a full frame.

Type:bool
set_current_frame(image, filename)

Set the frame and meta information for the currently displayed frame. Populates the attribute current_frame

Parameters:
  • image (numpy.ndarray) – The image used to display in the Frame Viewer
  • filename (str) – The filename of the current frame
set_frame_count(count)

Set the count of total number of frames to frame_count when the FramesLoader has completed loading.

Parameters:count (int) – The number of frames that exist for this session
set_frame_display_dims(width, height)

Set the size, in pixels, of the video frame display window and resize the displayed frame.

Used on a frame resize callback, sets the :attr:frame_display_dims`.

Parameters:
  • width (int) – The width of the frame holding the video canvas in pixels
  • height (int) – The height of the frame holding the video canvas in pixels
tk_face_index

The variable that holds the face index of the selected face within the current frame when in zoomed mode.

Type:tkinter.IntVar
tk_faces_size

The variable holding the currently selected Faces Viewer thumbnail size.

Type:tkinter.StringVar
tk_filter_mode

The variable holding the currently selected navigation filter mode.

Type:tkinter.StringVar
tk_frame_index

The variable holding the current frame index.

Type:tkinter.IntVar
tk_is_zoomed

The variable holding the value indicating whether the frame viewer is zoomed into a face or zoomed out to the full frame.

Type:tkinter.BooleanVar
tk_transport_index

The current index of the display frame’s transport slider.

Type:tkinter.IntVar
tk_update

The variable holding the trigger that indicates that a full update needs to occur.

Type:tkinter.BooleanVar
tk_update_active_viewport

Boolean Variable that is traced by the viewport’s active frame to update..

Type:tkinter.BooleanVar

detected_faces module

Module Summary

DetectedFaces Handles the manipulation of DetectedFace objects stored in the alignments file.
FaceUpdate Perform updates on DetectedFace objects stored in DetectedFaces when changes are made within the GUI.
Filter Returns stats and frames for filtered frames based on the user selected navigation mode filter.
ThumbsCreator Background loader to generate thumbnails for the alignments file.

Module

Alignments handling for Faceswap’s Manual Adjustments tool. Handles the conversion of alignments data to DetectedFace objects, and the update of these faces when edits are made in the GUI.

class tools.manual.detected_faces.DetectedFaces(tk_globals, alignments_path, input_location, extractor)

Bases: object

Handles the manipulation of DetectedFace objects stored in the alignments file. Acts as a parent class for the IO operations (saving and loading from an alignments file), the face update operations (when changes are made to alignments in the GUI) and the face filters (when a user changes the filter navigation mode.)

Parameters:
  • tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI
  • alignments_path (str) – The full path to the alignments file
  • input_location (str) – The location of the input folder of frames or video file
  • extractor (Aligner) – The pipeline for passing faces through the aligner and retrieving results
available_masks

The mask type names stored in the alignments; type as key with the number of faces which possess the mask type as value.

Type:dict
current_faces

The most up to date full list of DetectedFace objects.

Type:list
extract()

Extract the faces in the current video to a user supplied folder.

extractor

The pipeline for passing faces through the aligner and retrieving results.

Type:Aligner
face_count_per_index

Count of faces for each frame. List is in frame index order.

The list needs to be calculated on the fly as the number of faces in a frame can change based on user actions.

Type:list
filter

Handles returning of faces and stats based on the current user set navigation mode filter.

Type:Filter
is_frame_updated(frame_index)

bool: True if the given frame index has updated faces within it otherwise False

load_faces()

Load the faces as DetectedFace objects from the alignments file.

revert_to_saved(frame_index)

Revert the frame’s alignments to their saved version for the given frame index.

Parameters:frame_index (int) – The frame that should have their faces reverted to their saved version
save()

Save the alignments file with the latest edits.

save_video_meta_data(pts_time, keyframes)

Save video meta data to the alignments file. This is executed if the video meta data does not already exist in the alignments file, so the video does not need to be scanned on every use of the Manual Tool.

Parameters:
  • pts_time (list) – A list of presentation timestamps (float) in frame index order for every frame in the input video
  • keyframes (list) – A list of frame indices corresponding to the key frames in the input video.
tk_edited

The variable indicating whether an edit has occurred meaning a GUI redraw needs to be triggered.

Type:tkinter.BooleanVar
tk_face_count_changed

The variable indicating whether a face has been added or removed meaning the FaceViewer grid redraw needs to be triggered.

Type:tkinter.BooleanVar
tk_unsaved

The variable indicating whether the alignments have been updated since the last save.

Type:tkinter.BooleanVar
update

Handles the adding, removing and updating of DetectedFace stored within the alignments file.

Type:FaceUpdate
video_meta_data

The frame meta data stored in the alignments file. If data does not exist in the alignments file then None is returned for each Key

Type:dict
class tools.manual.detected_faces.FaceUpdate(detected_faces)

Bases: object

Perform updates on DetectedFace objects stored in DetectedFaces when changes are made within the GUI.

Parameters:detected_faces (DetectedFaces) – The parent DetectedFaces object
add(frame_index, pnt_x, width, pnt_y, height)

Add a DetectedFace object to the current frame with the given dimensions.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • pnt_x (int) – The left point of the bounding box
  • width (int) – The width of the bounding box
  • pnt_y (int) – The top point of the bounding box
  • height (int) – The height of the bounding box
bounding_box(frame_index, face_index, pnt_x, width, pnt_y, height, aligner='FAN')

Update the bounding box for the DetectedFace object at the given frame and face indices, with the given dimensions and update the 68 point landmarks from the Aligner for the updated bounding box.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • pnt_x (int) – The left point of the bounding box
  • width (int) – The width of the bounding box
  • pnt_y (int) – The top point of the bounding box
  • height (int) – The height of the bounding box
  • aligner (["cv2-dnn", "FAN"], optional) – The aligner to use to generate the landmarks. Default: “FAN”
copy(frame_index, direction)

Copy the alignments from the previous or next frame that has alignments to the current frame.

Parameters:
  • frame_index (int) – The frame that the needs to have alignments copied to it
  • direction (["prev", "next"]) – Whether to copy alignments from the previous frame with alignments, or the next frame with alignments
delete(frame_index, face_index)

Delete the DetectedFace object for the given frame and face indices.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
landmark(frame_index, face_index, landmark_index, shift_x, shift_y, is_zoomed)

Shift a single landmark point for the DetectedFace object at the given frame and face indices by the given x and y values.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • landmark_index (int or list) – The landmark index to shift. If a list is provided, this should be a list of landmark indices to be shifted
  • shift_x (int) – The amount to shift the landmark by along the x axis
  • shift_y (int) – The amount to shift the landmark by along the y axis
  • is_zoomed (bool) – True if landmarks are being adjusted on a zoomed image otherwise False
landmarks(frame_index, face_index, shift_x, shift_y)

Shift all of the landmarks and bounding box for the DetectedFace object at the given frame and face indices by the given x and y values and update the masks.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • shift_x (int) – The amount to shift the landmarks by along the x axis
  • shift_y (int) – The amount to shift the landmarks by along the y axis

Notes

Whilst the bounding box does not need to be shifted, it is anyway, to ensure that it is aligned with the newly adjusted landmarks.

landmarks_rotate(frame_index, face_index, angle, center)

Rotate the landmarks on an Extract Box rotate for the DetectedFace object at the given frame and face indices for the given angle from the given center point.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • angle (numpy.ndarray) – The angle, in radians to rotate the points by
  • center (numpy.ndarray) – The center point of the Landmark’s Extract Box
landmarks_scale(frame_index, face_index, scale, center)

Scale the landmarks on an Extract Box resize for the DetectedFace object at the given frame and face indices from the given center point.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • scale (float) – The amount to scale the landmarks by
  • center (numpy.ndarray) – The center point of the Landmark’s Extract Box
mask(frame_index, face_index, mask, mask_type)

Update the mask on an edit for the DetectedFace object at the given frame and face indices, for the given mask and mask type.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
  • mask (class:numpy.ndarray:) – The mask to replace
  • mask_type (str) – The name of the mask that is to be replaced
post_edit_trigger(frame_index, face_index)

Update the jpg thumbnail and the viewport thumbnail on a face edit.

Parameters:
  • frame_index (int) – The frame that the face is being set for
  • face_index (int) – The face index within the frame
class tools.manual.detected_faces.Filter(detected_faces)

Bases: object

Returns stats and frames for filtered frames based on the user selected navigation mode filter.

Parameters:detected_faces (DetectedFaces) – The parent DetectedFaces object
count

The number of frames that meet the filter criteria returned by filter_mode.

Type:int
frames_list

The list of frame indices that meet the filter criteria returned by filter_mode.

Type:list
raw_indices

The frame and face indices that meet the current filter criteria for each displayed face.

Type:dict
class tools.manual.detected_faces.ThumbsCreator(detected_faces, input_location, single_process)

Bases: object

Background loader to generate thumbnails for the alignments file. Generates low resolution thumbnails in parallel threads for faster processing.

Parameters:
  • detected_faces (DetectedFaces) – The DetectedFace objects for this video
  • input_location (str) – The location of the input folder of frames or video file
generate_cache()

Extract the face thumbnails from a video or folder of images into the alignments file.

has_thumbs

True if the underlying alignments file holds thumbnail images otherwise False.

Type:bool