faceviewer package

Handles the display of faces in the Face Viewer section of Faceswap’s Manual Tool.

frame module

Module Summary

ContextMenu

Enables a right click context menu for the FacesViewer.

FacesActionsFrame

The left hand action frame holding the optional annotation buttons.

FacesFrame

The faces display frame (bottom section of GUI).

FacesViewer

The tkinter.Canvas that holds the faces viewer section of the Manual Tool.

Grid

Holds information on the current filtered grid layout.

Module

The Faces Viewer Frame and Canvas for Faceswap’s Manual Tool.

class tools.manual.faceviewer.frame.ContextMenu(canvas, detected_faces)

Bases: object

Enables a right click context menu for the FacesViewer.

Parameters:

  • canvas (tkinter.Canvas) – The FacesViewer canvas

  • detected_faces (detected_faces) – The manual tool’s detected faces class

class tools.manual.faceviewer.frame.FacesActionsFrame(parent: FacesFrame)

Bases: Frame

The left hand action frame holding the optional annotation buttons.

Parameters:

parent (FacesFrame) – The Faces frame that this actions frame reside in

property key_bindings: dict[str, Literal['mask', 'mesh']]

The mapping of key presses to optional annotations to display. Keyboard shortcuts utilize the function keys.

Type:

dict

on_click(display: Literal['mesh', 'mask']) → None

Click event for the optional annotation buttons. Loads and unloads the annotations from the faces viewer.

Parameters:

display (Literal["mesh", "mask"]) – The display name for the button that has called this event as exists in _buttons

class tools.manual.faceviewer.frame.FacesFrame(parent: ttk.PanedWindow, tk_globals: TkGlobals, detected_faces: DetectedFaces, display_frame: DisplayFrame)

Bases: Frame

The faces display frame (bottom section of GUI). This frame holds the faces viewport and the tkinter objects.

Parameters:

  • parent (ttk.PanedWindow) – The paned window that the faces frame resides in

  • tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI

  • detected_faces (DetectedFaces) – The DetectedFace objects for this video

  • display_frame (DisplayFrame) – The section of the Manual Tool that holds the frames viewer

canvas_scroll(direction: Literal['up', 'down', 'page-up', 'page-down']) → None

Scroll the canvas on an up/down or page-up/page-down key press.

Notes

To protect against a held down key press stacking tasks and locking up the GUI a background thread is launched and discards subsequent key presses whilst the previous update occurs.

Parameters:

direction (["up", "down", "page-up", "page-down"]) – The request page scroll direction and amount.

set_annotation_display(key: str) → None

Set the optional annotation overlay based on keyboard shortcut.

Parameters:

key (str) – The pressed key

class tools.manual.faceviewer.frame.FacesViewer(parent: ttk.Frame, tk_globals: TkGlobals, tk_action_vars: dict[T.Literal['mesh', 'mask'], tk.BooleanVar], detected_faces: DetectedFaces, display_frame: DisplayFrame, event: Event)

Bases: Canvas

The tkinter.Canvas that holds the faces viewer section of the Manual Tool.

Parameters:

  • parent (tkinter.ttk.Frame) – The parent frame for the canvas

  • tk_globals (TkGlobals) – The tkinter variables that apply to the whole of the GUI

  • tk_action_vars (dict) – The tkinter.BooleanVar objects for selectable optional annotations as set by the buttons in the FacesActionsFrame

  • detected_faces (DetectedFaces) – The DetectedFace objects for this video

  • display_frame (DisplayFrame) – The section of the Manual Tool that holds the frames viewer

  • event (threading.Event) – The threading event object for repeated key press protection

canvas_scroll(amount: int, units: Literal['pages', 'units'], event: Event) → None

Scroll the canvas on an up/down or page-up/page-down key press.

Parameters:

  • amount (int) – The number of units to scroll the canvas

  • units (Literal["pages", "units"]) – The unit type to scroll by

  • event (threading.Event) – event to indicate to the calling process whether the scroll is still updating

property control_colors: dict[str, str]

The frame Editor name as key with the current user selected hex code as value.

Type:

dict[str, str]

property face_size: int

The currently selected thumbnail size in pixels

Type:

int

get_muted_color(color_key: str) → str

Creates a muted version of the given annotation color for non-active faces.

Parameters:

color_key (str) – The annotation key to obtain the color for from control_colors

Returns:

The hex color code of the muted color

Return type:

str

property layout: Grid

The grid for the current FacesViewer.

Type:

Grid

property optional_annotations: dict[Literal['mesh', 'mask'], bool]

The values currently set for the selectable optional annotations.

Type:

dict[Literal[“mesh”, “mask”], bool]

refresh_grid(trigger_var: BooleanVar, retain_position: bool = False) → None

Recalculate the full grid and redraw. Used when the active filter pull down is used, a face has been added or removed, or the face thumbnail size has changed.

Parameters:

  • trigger_var (tkinter.BooleanVar) – The tkinter variable that has triggered the grid update. Will either be the variable indicating that the face size have been changed, or the variable indicating that the selected filter mode has been changed.

  • retain_position (bool, optional) – True if the grid should be set back to the position it was at after the update has been processed, otherwise False. Default: False.

property selected_mask: str

The currently selected mask from the display frame control panel.

Type:

str

property viewport: Viewport

The viewport area of the faces viewer.

Type:

Viewport

class tools.manual.faceviewer.frame.Grid(canvas: FacesViewer, detected_faces: DetectedFaces)

Bases: object

Holds information on the current filtered grid layout.

The grid keeps information on frame indices, face indices, x and y positions and detected face objects laid out in a numpy array to reflect the current full layout of faces within the face viewer based on the currently selected filter and face thumbnail size.

Parameters:
property columns_rows: tuple[int, int]

the (columns, rows) required to hold all display images.

Type:

tuple

property dimensions: tuple[int, int]

The (width, height) required to hold all display images.

Type:

tuple

property face_size: int

The pixel size of each thumbnail within the face viewer.

Type:

int

frame_has_faces(frame_index: int) → bool | bool_

Check whether the given frame index contains any faces.

Parameters:

frame_index (int) – The frame index to locate in the grid

Returns:

True if there are faces in the given frame otherwise False

Return type:

bool

property is_valid: bool

True if the current filter means that the grid holds faces. False if there are no faces displayed in the grid.

Type:

bool

transport_index_from_frame(frame_index: int) → int | None

Return the main frame’s transport index for the given frame index based on the current filter criteria.

Parameters:

frame_index (int) – The absolute index for the frame within the full frames list

Returns:

The index of the requested frame within the filtered frames view. None if no valid frames

Return type:

int | None

update() → None

Update the underlying grid.

Called on initialization, on a filter change or on add/remove faces. Recalculates the underlying grid for the current filter view and updates the attributes _grid, _display_faces, _raw_indices, _frames_list and is_valid

property visible_area: tuple[numpy.ndarray, numpy.ndarray]

Tuple containing 2 arrays.

1st array contains an array of shape (4, rows, columns) corresponding to the viewable area of the display grid. 1st dimension contains frame indices, 2nd dimension face indices. The 3rd and 4th dimension contain the x and y position of the top left corner of the face respectively.

2nd array contains DetectedFace objects laid out in (rows, columns)

Any locations that are not populated by a face will have a frame and face index of -1

Type:

tuple[numpy.ndarray, numpy.ndarray]

y_coord_from_frame(frame_index: int) → int

Return the y coordinate for the first face that appears in the given frame.

Parameters:

frame_index (int) – The frame index to locate in the grid

Returns:

The y coordinate of the first face for the given frame

Return type:

int

interact module

Module Summary

ActiveFrame

Handles the display of faces and annotations for the currently active frame.

Asset

Holds all of the display assets identifiers for the active frame's face viewer objects

HoverBox

Handle the current mouse location when over the Viewport.

Module

Handles the viewport area for mouse hover actions and the active frame

class tools.manual.faceviewer.interact.ActiveFrame(viewport: Viewport, tk_edited_variable: tk.BooleanVar)

Bases: object

Handles the display of faces and annotations for the currently active frame.

Parameters:

  • canvas (tkinter.Canvas) – The FacesViewer canvas

  • tk_edited_variable (tkinter.BooleanVar) – The tkinter callback variable indicating that a face has been edited

property current_frame: ndarray

A BGR version of the frame currently being displayed.

Type:

numpy.ndarray

property frame_index: int

The frame index of the currently displayed frame.

Type:

int

move_to_top() → None

Move the currently selected frame’s faces to the top of the viewport if they are moving off the bottom of the viewer.

reload_annotations() → None

Handles the reloading of annotations for the currently active faces.

Highlights the faces within the viewport of those faces that exist in the currently displaying frame. Applies annotations based on the optional annotations and current editor selections.

class tools.manual.faceviewer.interact.Asset(images: list[int], meshes: list[dict[T.Literal['polygon', 'line'], list[int]]], faces: list[DetectedFace], boxes: list[int])

Bases: object

Holds all of the display assets identifiers for the active frame’s face viewer objects

Parameters:

  • images (list[int]) – Indices for a frame’s tk image ids displayed in the active frame

  • meshes (list[dict[Literal["polygon", "line"], list[int]]]) – Indices for a frame’s tk line/polygon object ids displayed in the active frame

  • faces (list[DetectedFace]) – DetectedFace objects that exist in the current frame

  • boxes (list[int]) – Indices for a frame’s bounding box object ids displayed in the active frame

boxes: list[int]

Indices for a frame’s bounding box object ids displayed in the active frame

Type:

list[int]

faces: list[DetectedFace]

DetectedFace objects that exist in the current frame

Type:

list[DetectedFace]

images: list[int]

Indices for a frame’s tk image ids displayed in the active frame

Type:

list[int]

meshes: list[dict[T.Literal['polygon', 'line'], list[int]]]

Indices for a frame’s tk line/polygon object ids displayed in the active frame

Type:

list[dict[Literal[“polygon”, “line”], list[int]]]

class tools.manual.faceviewer.interact.HoverBox(viewport: Viewport)

Bases: object

Handle the current mouse location when over the Viewport.

Highlights the face currently underneath the cursor and handles actions when clicking on a face.

Parameters:

viewport (Viewport) – The viewport object for the FacesViewer canvas

on_hover(event: Event | None) → None

Highlight the face and set the mouse cursor for the mouse’s current location.

Parameters:

event (tkinter.Event or None) – The tkinter mouse event. Provides the current location of the mouse cursor. If None is passed as the event (for example when this function is being called outside of a mouse event) then the location of the cursor will be calculated

viewport module

Module Summary

TKFace

An object that holds a single tkinter.PhotoImage face, ready for placement in the Viewport, Handles the placement of and removal of masks for the face as well as updates on any edits.

Viewport

Handles the display of faces and annotations in the currently viewable area of the canvas.

VisibleObjects

Holds the objects from the Grid that appear in the viewable area of the Viewport.

Module

Handles the visible area of the FacesViewer canvas.

class tools.manual.faceviewer.viewport.Recycler(canvas: FacesViewer)

Bases: object

Tkinter can slow down when constantly creating new objects.

This class delivers recycled objects, if stale objects are available, otherwise creates a new object

:param FacesViewer: The canvas that holds the faces display

get_image(coordinates: tuple[float | int, float | int]) → int

Obtain a recycled or new image object ID

Parameters:

coordinates (tuple[float | int, float | int]) – The co-ordinates that the image should be displayed at

Returns:

The canvas object id for the created image

Return type:

int

get_mesh(face: DetectedFace) → dict[T.Literal['polygon', 'line'], list[int]]

Get the mesh annotation for the landmarks. This is made up of a series of polygons or lines, depending on which part of the face is being annotated. Creates a new series of objects, or pulls existing objects from the recycled objects pool if they are available.

Parameters:

face (DetectedFace) – The detected face object to obrain the mesh for

Returns:

The dictionary of line and polygon tkinter canvas object ids for the mesh annotation

Return type:

dict[Literal[“polygon”, “line”], list[int]]

recycle_assets(asset_ids: list[int]) → None

Recycle assets that are no longer required

Parameters:

asset_ids (list[int]) – The IDs of the assets to be recycled

class tools.manual.faceviewer.viewport.TKFace(face: ndarray, size: int = 128, mask: ndarray | None = None)

Bases: object

An object that holds a single tkinter.PhotoImage face, ready for placement in the Viewport, Handles the placement of and removal of masks for the face as well as updates on any edits.

Parameters:

  • face (numpy.ndarray) – The face, sized correctly as a 3 channel BGR image or an encoded jpg to create a tkinter.PhotoImage from

  • size (int, optional) – The pixel size of the face image. Default: 128

  • mask (numpy.ndarray or None, optional) – The mask to be applied to the face image. Pass None if no mask is to be used. Default None

property photo: PhotoImage

The face in a format that can be placed on the FacesViewer canvas.

Type:

tkinter.PhotoImage

update(face: ndarray, mask: ndarray) → None

Update the photo with the given face and mask.

Parameters:

  • face (numpy.ndarray) – The face, sized correctly as a 3 channel BGR image

  • mask (numpy.ndarray or None) – The mask to be applied to the face image. Pass None if no mask is to be used

update_mask(mask: ndarray | None) → None

Update the mask in the 4th channel of photo to the given mask.

Parameters:

mask (numpy.ndarray or None) – The mask to be applied to the face image. Pass None if no mask is to be used

class tools.manual.faceviewer.viewport.Viewport(canvas: FacesViewer, tk_edited_variable: tk.BooleanVar)

Bases: object

Handles the display of faces and annotations in the currently viewable area of the canvas.

Parameters:

  • canvas (tkinter.Canvas) – The FacesViewer canvas

  • tk_edited_variable (tkinter.BooleanVar) – The variable that indicates that a face has been edited

face_from_point(point_x: int, point_y: int) → ndarray

Given an (x, y) point on the Viewport, obtain the face information at that location.

Parameters:

  • point_x (int) – The x position on the canvas of the point to retrieve the face for

  • point_y (int) – The y position on the canvas of the point to retrieve the face for

Returns:

Array of shape (4, ) containing the (frame index, face index, x_point of top left corner, y point of top left corner) of the face at the given coordinates.

If the given coordinates are not over a face, then the frame and face indices will be -1

Return type:

numpy.ndarray

property face_size: int

The pixel size of each thumbnail

Type:

int

get_landmarks(frame_index: int, face_index: int, face: DetectedFace, top_left: list[float], refresh: bool = False) → dict[T.Literal['polygon', 'line'], list[np.ndarray]]

Obtain the landmark points for each mesh annotation.

First tries to obtain the aligned landmarks from the cache. If the landmarks do not exist in the cache, or a refresh has been requested, then the landmarks are calculated from the detected face object.

Parameters:

  • frame_index (int) – The frame index to obtain the face for

  • face_index (int) – The face index of the face within the requested frame

  • face (lib.align.DetectedFace) – The detected face object to obtain landmarks for

  • top_left (list[float]) – The top left (x, y) points of the face’s bounding box within the viewport

  • refresh (bool, optional) – Whether to force a reload of the face’s aligned landmarks, even if they already exist within the cache. Default: False

Returns:

The key is the tkinter canvas object type for each part of the mesh annotation (polygon, line). The value is a list containing the (x, y) coordinates of each part of the mesh annotation, from the top left corner location.

Return type:

dict

get_tk_face(frame_index: int, face_index: int, face: DetectedFace) → TKFace

Obtain the TKFace object for the given face from the cache. If the face does not exist in the cache, then it is generated and added prior to returning.

Parameters:

  • frame_index (int) – The frame index to obtain the face for

  • face_index (int) – The face index of the face within the requested frame

  • face (DetectedFace) – The detected face object, containing the thumbnail jpg

Returns:

An object for displaying in the faces viewer canvas populated with the aligned mesh landmarks and face thumbnail

Return type:

TKFace

property hover_box: HoverBox

The hover box for the viewport.

Type:

HoverBox

property mesh_kwargs: dict[Literal['polygon', 'line'], dict[str, Any]]

Dynamic keyword arguments defining the color and state for the objects that make up a single face’s mesh annotation based on the current user selected options. Values are the keyword arguments for that given type.

Type:

dict[Literal[“polygon”, “line”], str | int]

move_active_to_top() → None

Check whether the active frame is going off the bottom of the viewport, if so: move it to the top of the viewport.

reset() → None

Reset all the cached objects on a face size change.

property selected_editor: str

The currently selected editor.

Type:

str

toggle_mask(state: Literal['hidden', 'normal'], mask_type: str) → None

Toggles the mask optional annotation on and off.

Parameters:

  • state (Literal["hidden", "normal"]) – Whether the mask should be displayed or hidden

  • mask_type (str) – The type of mask to overlay onto the face

toggle_mesh(state: Literal['hidden', 'normal']) → None

Toggles the mesh optional annotations on and off.

Parameters:

state (Literal["hidden", "normal"]) – The state to set the mesh annotations to

update(refresh_annotations: bool = False) → None

Update the viewport.

Parameters:

  • refresh_annotations (bool, optional) – True if mesh annotations should be re-calculated otherwise False. Default: False

  • canvas (Obtains the objects that are currently visible. Updates the visible area of the) –

  • annotations. (and reloads the active frame's) –

class tools.manual.faceviewer.viewport.VisibleObjects(viewport: Viewport)

Bases: object

Holds the objects from the Grid that appear in the viewable area of the Viewport.

Parameters:

viewport (Viewport) – The viewport object for the FacesViewer canvas

property images: ndarray

The viewport’s tkinter canvas image objects.

A numpy array of shape (rows, columns) corresponding to the viewable area of the display grid and containing the tkinter canvas image object for the face at the corresponding location.

Type:

numpy.ndarray

property meshes: ndarray

The viewport’s tkinter canvas mesh annotation objects.

A numpy array of shape (rows, columns) corresponding to the viewable area of the display grid and containing a dictionary of the corresponding tkinter polygon and line objects required to build a face’s mesh annotation for the face at the corresponding location.

Type:

numpy.ndarray

update() → None

Load and unload thumbnails in the visible area of the faces viewer.

property visible_faces: ndarray

The currently visible DetectedFace objects.

A numpy array of shape (rows, columns) corresponding to the viewable area of the display grid and containing the detected faces at their currently viewable position.

Any locations that are not populated by a face will have None in it’s place.

Type:

numpy.ndarray

property visible_grid: ndarray

The currently visible section of the Grid

A numpy array of shape (4, rows, columns) corresponding to the viewable area of the display grid. 1st dimension contains frame indices, 2nd dimension face indices. The 3rd and 4th dimension contain the x and y position of the top left corner of the face respectively.

Any locations that are not populated by a face will have a frame and face index of -1.

Type:

numpy.ndarray