convert package

The Convert Package handles the various plugins available for performing conversion in Faceswap

colour package

plugins.convert.color.avg_color Module

Average colour adjustment color matching adjustment plugin for faceswap.py converter

class plugins.convert.color.avg_color.Color(config_file=None, config=None)

Adjust the mean of the color channels to be the same for the swap and old frame

process(old_face: ndarray, new_face: ndarray, raw_mask: ndarray) → ndarray

Adjust the mean of the original face and the new face to be the same

Parameters:

• old_face (numpy.ndarray) – The original face

• new_face (numpy.ndarray) – The Faceswap generated face

• raw_mask (numpy.ndarray) – A raw mask for including the face area only

Returns:

The adjusted face patch

Return type:

numpy.ndarray

Classes

Color([config_file, config])

Adjust the mean of the color channels to be the same for the swap and old frame

Class Inheritance Diagram

plugins.convert.color.color_transfer Module

Color Transfer adjustment color matching adjustment plugin for faceswap.py converter source: https://github.com/jrosebr1/color_transfer The MIT License (MIT)

Copyright (c) 2014 Adrian Rosebrock, http://www.pyimagesearch.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

class plugins.convert.color.color_transfer.Color(config_file=None, config=None)

Transfers the color distribution from the source to the target image using the mean and standard deviations of the L*a*b* color space.

This implementation is (loosely) based on to the “Color Transfer between Images” paper by Reinhard et al., 2001.

static image_stats(image)

Parameters:

image (NumPy array) – OpenCV image in L*a*b* color space

Returns:

• Tuple of mean and standard deviations for the L, a*, and b**

• channels, respectively

process(old_face, new_face, raw_mask)

Parameters:

• source (NumPy array) – OpenCV image in BGR color space (the source image)

• target (NumPy array) – OpenCV image in BGR color space (the target image)

• clip (Should components of L*a*b* image be scaled by np.clip before) – converting back to BGR color space? If False then components will be min-max scaled appropriately. Clipping will keep target image brightness truer to the input. Scaling will adjust image brightness to avoid washed out portions in the resulting color transfer that can be caused by clipping.

• preserve_paper (Should color transfer strictly follow methodology) – layed out in original paper? The method does not always produce aesthetically pleasing results. If False then L*a*b* components will scaled using the reciprocal of the scaling factor proposed in the paper. This method seems to produce more consistently aesthetically pleasing results

Returns:

transfer – OpenCV image (w, h, 3) NumPy array (uint8)

Return type:

NumPy array

Classes

Color([config_file, config])

Transfers the color distribution from the source to the target image using the mean and standard deviations of the L*a*b* color space.

Class Inheritance Diagram

plugins.convert.color.manual_balance Module

Manual Balance colour adjustment plugin for faceswap.py converter

class plugins.convert.color.manual_balance.Color(config_file=None, config=None)

Adjust the mean of the color channels to be the same for the swap and old frame

adjust_contrast(image)

Adjust image contrast and brightness.

convert_colorspace(new_face, to_bgr=False)

Convert colorspace based on mode or back to bgr

process(old_face, new_face, raw_mask)

Override for specific color adjustment process

Classes

Color([config_file, config])

Adjust the mean of the color channels to be the same for the swap and old frame

Class Inheritance Diagram

plugins.convert.color.match_hist Module

Match histogram colour adjustment color matching adjustment plugin for faceswap.py converter

class plugins.convert.color.match_hist.Color(config_file=None, config=None)

Match the histogram of the color intensity of each channel

static hist_match(old_channel, new_channel, mask_indices, threshold)

Construct the histogram of the color intensity of a channel for the swap and the original. Match the histogram of the original by interpolation

process(old_face, new_face, raw_mask)

Override for specific color adjustment process

Classes

Color([config_file, config])

Match the histogram of the color intensity of each channel

Class Inheritance Diagram

plugins.convert.color.seamless_clone Module

Seamless clone adjustment plugin for faceswap.py converter NB: This probably isn’t the best place for this, but it is independent of color adjustments and does not have a natural home, so here for now and called as an extra plugin from lib/convert.py

class plugins.convert.color.seamless_clone.Color(config_file=None, config=None)

Seamless clone the swapped face into the old face with cv2 NB: This probably isn’t the best place for this, but it doesn’t work well and does not have a natural home, so here for now.

process(old_face, new_face, raw_mask)

Override for specific color adjustment process

Classes

Color([config_file, config])

Seamless clone the swapped face into the old face with cv2 NB: This probably isn't the best place for this, but it doesn't work well and does not have a natural home, so here for now.

Class Inheritance Diagram

mask package

plugins.convert.mask.mask_blend Module

Plugin to blend the edges of the face between the swap and the original face.

class plugins.convert.mask.mask_blend.Mask(mask_type: str, output_size: int, coverage_ratio: float, config_file: str | None = None)

Manipulations to perform to the mask that is to be applied to the output of the Faceswap model.

Parameters:

• mask_type (str) – The mask type to use for this plugin

• output_size (int) – The size of the output from the Faceswap model.

• coverage_ratio (float) – The coverage ratio that the Faceswap model was trained at.

• config_file (str, Optional) – Optional location of custom configuration ini file. If None then use the default config location. Default: None

run(detected_face: DetectedFace, source_offset: ndarray, target_offset: ndarray, centering: Literal['legacy', 'face', 'head'], landmarks_mask: LandmarksMask | None = None, predicted_mask: ndarray | None = None) → tuple[ndarray, ndarray]

Obtain the requested mask type and perform any defined mask manipulations.

Parameters:

• detected_face (lib.align.detected_face.DetectedFace) – The DetectedFace object as returned from scripts.convert.Predictor.

• source_offset (numpy.ndarray) – The (x, y) offset for the mask at its stored centering

• target_offset (numpy.ndarray) – The (x, y) offset for the mask at the requested target centering

• centering ([“legacy”, “face”, “head”]) – The centering to obtain the mask for

• landmarks_mask (lib.align.aligned_mask.LandmarksMask | None, optional) – The landmarks mask object, if requested or None. Default: None

• predicted_mask (numpy.ndarray | None, optional) – The predicted mask as output from the Faceswap Model, if the model was trained with a mask, otherwise None. Default: None.

Returns:

• mask (numpy.ndarray) – The mask with all requested manipulations applied

• raw_mask (numpy.ndarray) – The mask with no erosion/dilation applied

Return type:

tuple[ndarray, ndarray]

Classes

Mask(mask_type, output_size, coverage_ratio)

Manipulations to perform to the mask that is to be applied to the output of the Faceswap model.

scaling package

plugins.convert.scaling.sharpen Module

Sharpening for enlarged face for faceswap.py converter

class plugins.convert.scaling.sharpen.Scaling(config_file=None)

Sharpening Adjustments for the face applied after warp to final frame

classmethod box(new_face: ndarray, kernel_size: tuple[int, int], radius: int, amount: float) → ndarray

Sharpen using box filter

Parameters:

• new_face (numpy.ndarray) – The batch of swapped image patches that is to have sharpening applied

• kernel_size (tuple[int, int]) – The sharpening kernel size

• radius (int) – The pixel radius the kernel

• amount (float) – The amount of sharpening to apply

Returns:

The batch of swapped faces with box sharpening applied

Return type:

numpy.ndarray

classmethod gaussian(new_face: ndarray, kernel_size: tuple[int, int], radius: float, amount: float) → ndarray

Sharpen using gaussian filter

Parameters:

• new_face (numpy.ndarray) – The batch of swapped image patches that is to have sharpening applied

• kernel_size (tuple[int, int]) – The sharpening kernel size

• radius (int) – The pixel radius the kernel. Unused

• amount (float) – The amount of sharpening to apply

Returns:

The batch of swapped faces with gaussian sharpening applied

Return type:

numpy.ndarray

classmethod get_kernel_size(new_face: ndarray, radius_percent: float) → tuple[tuple[int, int], int]

Return the kernel size and central point for the given radius

relative to frame width.

Parameters:

• new_face (numpy.ndarray) – The swapped image patch that is to have sharpening applied

• radius_percent (float) – The percentage of the image size to use as the sharpening kernel

Returns:

• kernel_size (tuple[int, int]) – The sharpening kernel

• radius (int) – The pixel radius the kernel

Return type:

tuple[tuple[int, int], int]

process(new_face: ndarray) → ndarray

Sharpen using the requested technique

Parameters:

new_face (numpy.ndarray) – A batch of swapped image patch that is to have sharpening applied

Returns:

The batch of swapped faces with sharpening applied

Return type:

numpy.ndarray

classmethod unsharp_mask(new_face: ndarray, kernel_size: tuple[int, int], center: float, amount: float) → ndarray

Sharpen using unsharp mask

Parameters:

• new_face (numpy.ndarray) – The batch of swapped image patches that is to have sharpening applied

• kernel_size (tuple[int, int]) – The sharpening kernel size

• radius (int) – The pixel radius the kernel. Unused

• amount (float) – The amount of sharpening to apply

• center (float)

Returns:

The batch of swapped faces with unsharp-mask sharpening applied

Return type:

numpy.ndarray

Classes

Scaling([config_file])

Sharpening Adjustments for the face applied after warp to final frame

Class Inheritance Diagram

writer package

plugins.convert.writer.ffmpeg Module

Video output writer for faceswap.py converter

class plugins.convert.writer.ffmpeg.Writer(output_folder: str, total_count: int, frame_ranges: list[tuple[int, int]] | None, source_video: str, **kwargs)

Video output writer using pyAV.

Parameters:

• output_folder (str) – The folder to save the output video to

• total_count (int) – The total number of frames to be converted

• frame_ranges (list or None) – List of tuples for starting and end values of each frame range to be converted or None if all frames are to be converted

• source_video (str) – The full path to the source video for obtaining fps and audio

• kwargs (dict) – Any additional standard plugins.convert.writer._base.Output key word arguments.

close() → None

Close the ffmpeg writer

Return type:

None

write(filename: str, image: npt.NDArray[np.uint8]) → None

Frames come from the pool in arbitrary order, so frames are cached for writing out in the correct order.

Parameters:

• filename (str) – The incoming frame filename.

• image (numpy.ndarray) – The converted image to be written

Return type:

None

Classes

Writer(output_folder, total_count, ...)

Video output writer using pyAV.

Class Inheritance Diagram

plugins.convert.writer.gif Module

Animated GIF writer for faceswap.py converter

class plugins.convert.writer.gif.Writer(output_folder: str, total_count: int, frame_ranges: list[tuple[int, int]] | None, **kwargs)

GIF output writer using PIL.

Parameters:

• output_folder (str) – The folder to save the output gif to

• total_count (int) – The total number of frames to be converted

• frame_ranges (list[tuple[int, int]] | None) – List of tuples for starting and end values of each frame range to be converted or None if all frames are to be converted

• kwargs – Any additional standard plugins.convert.writer._base.Output key word arguments.

close() → None

Close the GIF writer on completion.

Return type:

None

write(filename: str, image: npt.NDArray[np.uint8]) → None

Frames come from the pool in arbitrary order, so frames are cached for writing out in the correct order.

Parameters:

• filename (str) – The incoming frame filename.

• image (npt.NDArray[np.uint8]) – The converted image to be written

Return type:

None

Classes

Writer(output_folder, total_count, ...)

GIF output writer using PIL.

Class Inheritance Diagram

plugins.convert.writer.opencv Module

Image output writer for faceswap.py converter Uses cv2 for writing as in testing this was a lot faster than both Pillow and ImageIO

class plugins.convert.writer.opencv.Writer(output_folder: str, **kwargs)

Images output writer using cv2

Parameters:

• output_folder (str) – The full path to the output folder where the converted media should be saved

• config_file (str, optional) – The full path to a custom configuration ini file. If None is passed then the file is loaded from the default location. Default: None.

close() → None

Does nothing as OpenCV writer does not need a close method

Return type:

None

property output_alpha: bool

OpenCV can output alpha channel.

Type:

bool

pre_encode(image: ndarray, **kwargs) → list[bytes]

Pre_encode the image in lib/convert.py threads as it is a LOT quicker.

Parameters:

image (numpy.ndarray) – A 3 or 4 channel BGR swapped frame

Returns:

List of bytes objects ready for writing. The list will be of length 1 with image bytes object as the only member unless separate mask has been requested, in which case it will be length 2 with the image in position 0 and mask in position 1

Return type:

list

write(filename: str, image: list[bytes]) → None

Write out the pre-encoded image to disk. If separate mask has been selected, write out the encoded mask to a sub-folder in the output directory.

Parameters:

• filename (str) – The full path to write out the image to.

• image (list) – List of bytes objects of length 1 (containing just the image to write out) or length 2 (containing the image and mask to write out)

Return type:

None

Classes

Writer(output_folder, **kwargs)

Images output writer using cv2

Class Inheritance Diagram

plugins.convert.writer.patch Module

Face patch output writer for faceswap.py converter Extracts the swapped Face Patch from faceswap rather than the final composited frame along with the transformation matrix for re-inserting the face into the original frame

class plugins.convert.writer.patch.Writer(output_folder: str, patch_size: int, **kwargs)

Face patch writer for outputting swapped face patches and transformation matrices

Parameters:

• output_folder (str) – The full path to the output folder where the face patches should be saved

• patch_size (int) – The size of the face patch output from the model

• config_file (str, optional) – The full path to a custom configuration ini file. If None is passed then the file is loaded from the default location. Default: None.

close() → None

Outputs json file if requested

Return type:

None

pre_encode(image: ndarray, **kwargs) → list[list[bytes]]

Pre_encode the image in lib/convert.py threads as it is a LOT quicker.

Parameters:

• image (numpy.ndarray) – A 3 or 4 channel BGR swapped face batch as float32

• canvas_size (tuple[int, int]) – The size of the canvas (x, y) that the transformation matrix applies to.

• matrices (numpy.ndarray, optional) – The transformation matrices for extracting the face patches from the original frame. Must be provided if an image is provided, otherwise None to insert a dummy matrix

Returns:

List of bytes objects ready for writing. The list will be of length 1 with image bytes object as the only member unless separate mask has been requested, in which case it will be length 2 with the image in position 0 and mask in position 1

Return type:

list

write(filename: str, image: list[list[bytes]]) → None

Write out the pre-encoded image to disk. If separate mask has been selected, write out the encoded mask to a sub-folder in the output directory.

Parameters:

• filename (str) – The full path to write out the image to.

• image (list[list[bytes]]) – List of list of bytes objects of containing all swapped faces from a frame to write out. The inner list will be of length 1 (mask included in the alpha channel) or length 2 (mask to write out separately)

Return type:

None

Classes

Writer(output_folder, patch_size, **kwargs)

Face patch writer for outputting swapped face patches and transformation matrices

Class Inheritance Diagram

plugins.convert.writer.pillow Module

Image output writer for faceswap.py converter

class plugins.convert.writer.pillow.Writer(output_folder: str, **kwargs)

Images output writer using Pillow

Parameters:

• output_folder (str) – The full path to the output folder where the converted media should be saved

• config_file (str, optional) – The full path to a custom configuration ini file. If None is passed then the file is loaded from the default location. Default: None.

close() → None

Does nothing as Pillow writer does not need a close method

Return type:

None

property output_alpha: bool

Pillow can output alpha channel. Returns True

Type:

bool

pre_encode(image: ndarray, **kwargs) → list[BytesIO]

Pre_encode the image in lib/convert.py threads as it is a LOT quicker

Parameters:

image (numpy.ndarray) – A 3 or 4 channel BGR swapped frame

Returns:

List of BytesIO objects ready for writing. The list will be of length 1 with image bytes object as the only member unless separate mask has been requested, in which case it will be length 2 with the image in position 0 and mask in position 1

Return type:

list

write(filename: str, image: list[BytesIO]) → None

Write out the pre-encoded image to disk. If separate mask has been selected, write out the encoded mask to a sub-folder in the output directory.

Parameters:

• filename (str) – The full path to write out the image to.

• image (list) – List of BytesIO objects of length 1 (containing just the image to write out) or length 2 (containing the image and mask to write out)

Return type:

None

Classes

Writer(output_folder, **kwargs)

Images output writer using Pillow

Class Inheritance Diagram