model package
The Model Package handles interfacing with the neural network backend and holds custom objects.
model.backup_restore module
Functions for backing up, restoring and creating model snapshots.
- class lib.model.backup_restore.Backup(model_dir, model_name)
Bases:
object
Performs the back up of models at each save iteration, and the restoring of models from their back up location.
- Parameters:
model_dir (str) – The folder that contains the model to be backed up
model_name (str) – The name of the model that is to be backed up
- static backup_model(full_path)
Backup a model file.
The backed up file is saved with the original filename in the original location with .bk appended to the end of the name.
- Parameters:
full_path (str) – The full path to a .h5 model file or a .json state file
- restore()
Restores a model from backup.
The original model files are migrated into a folder within the original model folder named <model_name>_archived_<timestamp>. The .bk backup files are then moved to the location of the previously existing model files. Logs that were generated after the the last backup was taken are removed.
- snapshot_models(iterations)
Take a snapshot of the model at the current state and back it up.
The snapshot is a copy of the model folder located in the same root location as the original model file, with the number of iterations appended to the end of the folder name.
- Parameters:
iterations (int) – The number of iterations that the model has trained when performing the snapshot.
model.initializers module
Module Summary
Initializer that generates orthogonal convolution filters in the Fourier space. |
|
ICNR initializer for checkerboard artifact free sub pixel convolution |
|
Computes the number of input and output units for a weight shape. |
Custom Initializers for faceswap.py
- class lib.model.initializers.ConvolutionAware(eps_std=0.05, seed=None, initialized=False)
Bases:
Initializer
Initializer that generates orthogonal convolution filters in the Fourier space. If this initializer is passed a shape that is not 3D or 4D, orthogonal initialization will be used.
Adapted, fixed and optimized from: https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/initializers/convaware.py
- Parameters:
eps_std (float, optional) – The Standard deviation for the random normal noise used to break symmetry in the inverse Fourier transform. Default: 0.05
seed (int, optional) – Used to seed the random generator. Default:
None
initialized (bool, optional) – This should always be set to
False
. To avoid Keras re-calculating the values every time the model is loaded, this parameter is internally set on first time initialization. Default:False
- Returns:
The modified kernel weights
- Return type:
tensor
References
Armen Aghajanyan, https://arxiv.org/abs/1702.06295
- get_config()
Return the Convolutional Aware Initializer configuration.
- Returns:
The configuration for ICNR Initialization
- Return type:
dict
- class lib.model.initializers.ICNR(initializer, scale=2)
Bases:
Initializer
ICNR initializer for checkerboard artifact free sub pixel convolution
- Parameters:
initializer (
keras.initializers.Initializer
) – The initializer used for sub kernels (orthogonal, glorot uniform, etc.)scale (int, optional) – scaling factor of sub pixel convolution (up sampling from 8x8 to 16x16 is scale 2). Default: 2
- Returns:
The modified kernel weights
- Return type:
tensor
Example
>>> x = conv2d(... weights_initializer=ICNR(initializer=he_uniform(), scale=2))
References
Andrew Aitken et al. Checkerboard artifact free sub-pixel convolution https://arxiv.org/pdf/1707.02937.pdf, https://distill.pub/2016/deconv-checkerboard/
- get_config()
Return the ICNR Initializer configuration.
- Returns:
The configuration for ICNR Initialization
- Return type:
dict
- lib.model.initializers.compute_fans(shape, data_format='channels_last')
Computes the number of input and output units for a weight shape.
Ported directly from Keras as the location moves between keras and tensorflow-keras
- Parameters:
shape (tuple) – shape tuple of integers
data_format (str) – Image data format to use for convolution kernels. Note that all kernels in Keras are standardized on the “channels_last” ordering (even when inputs are set to “channels_first”).
- Returns:
A tuple of scalars, (fan_in, fan_out).
- Return type:
tuple
- Raises:
ValueError – In case of invalid data_format argument.
model.layers module
Module Summary
Global minimum pooling operation for spatial data. |
|
Global standard deviation pooling operation for spatial data. |
|
A custom upscale function that uses |
|
Normalizes a tensor w.r.t. |
|
PixelShuffler layer for Keras. |
|
Applies GELU approximation that is fast but somewhat inaccurate. |
|
Reflection-padding layer for 2D input (e.g. |
|
Sub-pixel convolutional up-scaling layer. |
|
Swish Activation Layer implementation for Keras. |
Custom Layers for faceswap.py.
- class lib.model.layers.GlobalMinPooling2D(*args, **kwargs)
Bases:
_GlobalPooling2D
Global minimum pooling operation for spatial data.
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- class lib.model.layers.GlobalStdDevPooling2D(*args, **kwargs)
Bases:
_GlobalPooling2D
Global standard deviation pooling operation for spatial data.
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- class lib.model.layers.KResizeImages(*args, **kwargs)
Bases:
Layer
A custom upscale function that uses
keras.backend.resize_images
to upsample.- Parameters:
size (int or float, optional) – The scale to upsample to. Default: 2
interpolation (["nearest", "bilinear"], optional) – The interpolation to use. Default: “nearest”
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
- call(inputs: Tensor, *args, **kwargs) Tensor
Call the upsample layer
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- compute_output_shape(input_shape: tuple[int, ...]) tuple[int, ...]
Computes the output shape of the layer.
This is the input shape with size dimensions multiplied by
size
- Parameters:
input_shape (tuple or list of tuples) – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns:
An input shape tuple
- Return type:
tuple
- get_config() dict[str, Any]
Returns the config of the layer.
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.layers.L2_normalize(*args, **kwargs)
Bases:
Layer
Normalizes a tensor w.r.t. the L2 norm alongside the specified axis.
- Parameters:
axis (int) – The axis to perform normalization across
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- get_config() dict[str, Any]
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.layers.PixelShuffler(*args, **kwargs)
Bases:
Layer
PixelShuffler layer for Keras.
This layer requires a Convolution2D prior to it, having output filters computed according to the formula \(filters = k * (scale_factor * scale_factor)\) where k is a user defined number of filters (generally larger than 32) and scale_factor is the up-scaling factor (generally 2).
This layer performs the depth to space operation on the convolution filters, and returns a tensor with the size as defined below.
Notes
In practice, it is useful to have a second convolution layer after the
PixelShuffler
layer to speed up the learning process. However, if you are stacking multiplePixelShuffler
blocks, it may increase the number of parameters greatly, so the Convolution layer afterPixelShuffler
layer can be removed.Example
>>> # A standard sub-pixel up-scaling block >>> x = Convolution2D(256, 3, 3, padding="same", activation="relu")(...) >>> u = PixelShuffler(size=(2, 2))(x) [Optional] >>> x = Convolution2D(256, 3, 3, padding="same", activation="relu")(u)
- Parameters:
size (tuple, optional) – The (h, w) scaling factor for up-scaling. Default: (2, 2)
data_format ([“channels_first”, “channels_last”,
None
], optional) – The data format for the input. Default:None
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
References
https://gist.github.com/t-ae/6e1016cc188104d123676ccef3264981
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- compute_output_shape(input_shape: tuple[int, ...]) tuple[int, ...]
Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- Parameters:
input_shape (tuple or list of tuples) – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns:
An input shape tuple
- Return type:
tuple
- get_config() dict[str, Any]
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.layers.QuickGELU(*args, **kwargs)
Bases:
Layer
Applies GELU approximation that is fast but somewhat inaccurate.
- Parameters:
name (str, optional) – The name for the layer. Default: “QuickGELU”
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
- call(inputs: Tensor, *args, **kwargs) Tensor
Call the QuickGELU layerr
- Parameters:
inputs (
tf.Tensor
) – The input Tensor- Returns:
The output Tensor
- Return type:
tf.Tensor
- class lib.model.layers.ReflectionPadding2D(*args, **kwargs)
Bases:
Layer
Reflection-padding layer for 2D input (e.g. picture).
This layer can add rows and columns at the top, bottom, left and right side of an image tensor.
- Parameters:
stride (int, optional) – The stride of the following convolution. Default: 2
kernel_size (int, optional) – The kernel size of the following convolution. Default: 5
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
- build(input_shape: Tensor) None
Creates the layer weights.
Must be implemented on all layers that have weights.
- Parameters:
input_shape (
tf.Tensor
) – Keras tensor (future input to layer) orlist
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- compute_output_shape(input_shape: tuple[int, ...]) tuple[int, ...]
Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- Parameters:
input_shape (tuple or list of tuples) – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns:
An input shape tuple
- Return type:
tuple
- get_config() dict[str, Any]
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.layers.SubPixelUpscaling(*args, **kwargs)
Bases:
Layer
Sub-pixel convolutional up-scaling layer.
This layer requires a Convolution2D prior to it, having output filters computed according to the formula \(filters = k * (scale_factor * scale_factor)\) where k is a user defined number of filters (generally larger than 32) and scale_factor is the up-scaling factor (generally 2).
This layer performs the depth to space operation on the convolution filters, and returns a tensor with the size as defined below.
Notes
This method is deprecated as it just performs the same as
PixelShuffler
using explicit Tensorflow ops. The method is kept in the repository to support legacy models that have been created with this layer.In practice, it is useful to have a second convolution layer after the
SubPixelUpscaling
layer to speed up the learning process. However, if you are stacking multipleSubPixelUpscaling
blocks, it may increase the number of parameters greatly, so the Convolution layer afterSubPixelUpscaling
layer can be removed.Example
>>> # A standard sub-pixel up-scaling block >>> x = Convolution2D(256, 3, 3, padding="same", activation="relu")(...) >>> u = SubPixelUpscaling(scale_factor=2)(x) [Optional] >>> x = Convolution2D(256, 3, 3, padding="same", activation="relu")(u)
- Parameters:
size (int, optional) – The up-scaling factor. Default: 2
data_format ([“channels_first”, “channels_last”,
None
], optional) – The data format for the input. Default:None
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
References
based on the paper “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network” (https://arxiv.org/abs/1609.05158).
- build(input_shape: tuple[int, ...]) None
Creates the layer weights.
Must be implemented on all layers that have weights.
- Parameters:
input_shape (tensor) – Keras tensor (future input to layer) or
list
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs: Tensor, *args, **kwargs) Tensor
This is where the layer’s logic lives.
- Parameters:
inputs (
tf.Tensor
) – Input tensor, or list/tuple of input tensors- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- compute_output_shape(input_shape: tuple[int, ...]) tuple[int, ...]
Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- Parameters:
input_shape (tuple or list of tuples) – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- Returns:
An input shape tuple
- Return type:
tuple
- get_config() dict[str, Any]
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.layers.Swish(*args, **kwargs)
Bases:
Layer
Swish Activation Layer implementation for Keras.
- Parameters:
beta (float, optional) – The beta value to apply to the activation function. Default: 1.0
kwargs (dict) – The standard Keras Layer keyword arguments (if any)
References
Swish: a Self-Gated Activation Function: https://arxiv.org/abs/1710.05941v1
- call(inputs, *args, **kwargs)
Call the Swish Activation function.
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tf.Tensor
- get_config()
Returns the config of the layer.
Adds the
beta
to config.- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
model.losses module
Module Summary
model.nets module
Module Summary
model.nn_blocks module
Module Summary
A standard Keras Convolution 2D layer with parameters updated to be more appropriate for Faceswap architecture. |
|
A standard Convolution 2D layer which applies user specified configuration to the layer. |
|
A Convolution 2D layer that separates out the activation layer to explicitly set the data type on the activation to float 32 to fully support mixed precision training. |
|
Residual block from dfaker. |
|
Seperable Convolution Block. |
|
Custom hybrid upscale layer for sub-pixel up-scaling. |
|
An upscale layer for sub-pixel up-scaling. |
|
Set the global configuration parameters from the user's config file. |
Neural Network Blocks for faceswap.py.
- class lib.model.nn_blocks.Conv2D(*args, **kwargs)
Bases:
Conv2D
A standard Keras Convolution 2D layer with parameters updated to be more appropriate for Faceswap architecture.
Parameters are the same, with the same defaults, as a standard
keras.layers.Conv2D
except where listed below. The default initializer is updated to he_uniform or convolutional aware based on user configuration settings.- Parameters:
padding (str, optional) – One of “valid” or “same” (case-insensitive). Default: “same”. Note that “same” is slightly inconsistent across backends with strides != 1, as described here.
is_upscale (bool, optional) –
True
if the convolution is being called from an upscale layer. This causes the instance to check the user configuration options to see if ICNR initialization has been selected and should be applied. This should only be passed in asTrue
fromUpscaleBlock
layers. Default:False
- class lib.model.nn_blocks.Conv2DBlock(filters: int, kernel_size: int | tuple[int, int] = 5, strides: int | tuple[int, int] = 2, padding: str = 'same', normalization: str | None = None, activation: str | None = 'leakyrelu', use_depthwise: bool = False, relu_alpha: float = 0.1, **kwargs)
Bases:
object
A standard Convolution 2D layer which applies user specified configuration to the layer.
Adds reflection padding if it has been selected by the user, and other post-processing if requested by the plugin.
Adds instance normalization if requested. Adds a LeakyReLU if a residual block follows.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. NB: If use_depthwise is
True
then a value must still be provided here, but it will be ignored. Default: 5strides (tuple or int, optional) – An integer or tuple/list of 2 integers, specifying the strides of the convolution along the height and width. Can be a single integer to specify the same value for all spatial dimensions. Default: 2
padding (["valid", "same"], optional) – The padding to use. NB: If reflect padding has been selected in the user configuration options, then this argument will be ignored in favor of reflect padding. Default: “same”
normalization (str or
None
, optional) – Normalization to apply after the Convolution Layer. Select one of “batch” or “instance”. Set toNone
to not apply normalization. Default:None
activation (str or
None
, optional) – The activation function to use. This is applied at the end of the convolution block. Select one of “leakyrelu”, “prelu” or “swish”. Set toNone
to not apply an activation function. Default: “leakyrelu”use_depthwise (bool, optional) – Set to
True
to use a Depthwise Convolution 2D layer rather than a standard Convolution 2D layer. Default:False
relu_alpha (float) – The alpha to use for LeakyRelu Activation. Default=`0.1`
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- class lib.model.nn_blocks.Conv2DOutput(filters: int, kernel_size: int | tuple[int], activation: str = 'sigmoid', padding: str = 'same', **kwargs)
Bases:
object
A Convolution 2D layer that separates out the activation layer to explicitly set the data type on the activation to float 32 to fully support mixed precision training.
The Convolution 2D layer uses default parameters to be more appropriate for Faceswap architecture.
Parameters are the same, with the same defaults, as a standard
keras.layers.Conv2D
except where listed below. The default initializer is updated to he_uniform or convolutional aware based on user config settings.- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int or tuple/list of 2 ints) – The height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.
activation (str, optional) – The activation function to apply to the output. Default: “sigmoid”
padding (str, optional) –
One of “valid” or “same” (case-insensitive). Default: “same”. Note that “same” is slightly inconsistent across backends with strides != 1, as described here.
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- class lib.model.nn_blocks.DepthwiseConv2D(*args, **kwargs)
Bases:
DepthwiseConv2D
A standard Keras Depthwise Convolution 2D layer with parameters updated to be more appropriate for Faceswap architecture.
Parameters are the same, with the same defaults, as a standard
keras.layers.DepthwiseConv2D
except where listed below. The default initializer is updated to he_uniform or convolutional aware based on user configuration settings.- Parameters:
padding (str, optional) –
One of “valid” or “same” (case-insensitive). Default: “same”. Note that “same” is slightly inconsistent across backends with strides != 1, as described here.
is_upscale (bool, optional) –
True
if the convolution is being called from an upscale layer. This causes the instance to check the user configuration options to see if ICNR initialization has been selected and should be applied. This should only be passed in asTrue
fromUpscaleBlock
layers. Default:False
- class lib.model.nn_blocks.ResidualBlock(filters: int, kernel_size: int | tuple[int, int] = 3, padding: str = 'same', **kwargs)
Bases:
object
Residual block from dfaker.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 3
padding (["valid", "same"], optional) – The padding to use. Default: “same”
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- Returns:
The output tensor from the Upscale layer
- Return type:
tensor
- class lib.model.nn_blocks.SeparableConv2DBlock(filters: int, kernel_size: int | tuple[int, int] = 5, strides: int | tuple[int, int] = 2, **kwargs)
Bases:
object
Seperable Convolution Block.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 5
strides (tuple or int, optional) – An integer or tuple/list of 2 integers, specifying the strides of the convolution along the height and width. Can be a single integer to specify the same value for all spatial dimensions. Default: 2
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Separable Convolutional 2D layer
- class lib.model.nn_blocks.Upscale2xBlock(filters: int, kernel_size: int | tuple[int, int] = 3, padding: str = 'same', activation: str | None = 'leakyrelu', interpolation: str = 'bilinear', sr_ratio: float = 0.5, scale_factor: int = 2, fast: bool = False, **kwargs)
Bases:
object
Custom hybrid upscale layer for sub-pixel up-scaling.
Most of up-scaling is approximating lighting gradients which can be accurately achieved using linear fitting. This layer attempts to improve memory consumption by splitting with bilinear and convolutional layers so that the sub-pixel update will get details whilst the bilinear filter will get lighting.
Adds reflection padding if it has been selected by the user, and other post-processing if requested by the plugin.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 3
padding (["valid", "same"], optional) – The padding to use. Default: “same”
activation (str or
None
, optional) – The activation function to use. This is applied at the end of the convolution block. Select one of “leakyrelu”, “prelu” or “swish”. Set toNone
to not apply an activation function. Default: “leakyrelu”interpolation (["nearest", "bilinear"], optional) – Interpolation to use for up-sampling. Default: “bilinear”
scale_factor (int, optional) – The amount to upscale the image. Default: 2
sr_ratio (float, optional) – The proportion of super resolution (pixel shuffler) filters to use. Non-fast mode only. Default: 0.5
fast (bool, optional) – Use a faster up-scaling method that may appear more rugged. Default:
False
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- class lib.model.nn_blocks.UpscaleBlock(filters: int, kernel_size: int | tuple[int, int] = 3, padding: str = 'same', scale_factor: int = 2, normalization: str | None = None, activation: str | None = 'leakyrelu', **kwargs)
Bases:
object
An upscale layer for sub-pixel up-scaling.
Adds reflection padding if it has been selected by the user, and other post-processing if requested by the plugin.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 3
padding (["valid", "same"], optional) – The padding to use. NB: If reflect padding has been selected in the user configuration options, then this argument will be ignored in favor of reflect padding. Default: “same”
scale_factor (int, optional) – The amount to upscale the image. Default: 2
normalization (str or
None
, optional) – Normalization to apply after the Convolution Layer. Select one of “batch” or “instance”. Set toNone
to not apply normalization. Default:None
activation (str or
None
, optional) – The activation function to use. This is applied at the end of the convolution block. Select one of “leakyrelu”, “prelu” or “swish”. Set toNone
to not apply an activation function. Default: “leakyrelu”kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- class lib.model.nn_blocks.UpscaleDNYBlock(filters: int, kernel_size: int | tuple[int, int] = 3, padding: str = 'same', activation: str | None = 'leakyrelu', size: int = 2, interpolation: str = 'bilinear', **kwargs)
Bases:
object
Upscale block that implements methodology similar to the Disney Research Paper using an upsampling2D block and 2 x convolutions
Adds reflection padding if it has been selected by the user, and other post-processing if requested by the plugin.
References
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 3
activation (str or
None
, optional) – The activation function to use. This is applied at the end of the convolution block. Select one of “leakyrelu”, “prelu” or “swish”. Set toNone
to not apply an activation function. Default: “leakyrelu”size (int, optional) – The amount to upscale the image. Default: 2
interpolation (["nearest", "bilinear"], optional) – Interpolation to use for up-sampling. Default: “bilinear”
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layers
- class lib.model.nn_blocks.UpscaleResizeImagesBlock(filters: int, kernel_size: int | tuple[int, int] = 3, padding: str = 'same', activation: str | None = 'leakyrelu', scale_factor: int = 2, interpolation: Literal['nearest', 'bilinear'] = 'bilinear')
Bases:
object
Upscale block that uses the Keras Backend function resize_images to perform the up scaling Similar in methodology to the
Upscale2xBlock
Adds reflection padding if it has been selected by the user, and other post-processing if requested by the plugin.
- Parameters:
filters (int) – The dimensionality of the output space (i.e. the number of output filters in the convolution)
kernel_size (int, optional) – An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. Default: 3
padding (["valid", "same"], optional) – The padding to use. Default: “same”
activation (str or
None
, optional) – The activation function to use. This is applied at the end of the convolution block. Select one of “leakyrelu”, “prelu” or “swish”. Set toNone
to not apply an activation function. Default: “leakyrelu”scale_factor (int, optional) – The amount to upscale the image. Default: 2
interpolation (["nearest", "bilinear"], optional) – Interpolation to use for up-sampling. Default: “bilinear”
kwargs (dict) – Any additional Keras standard layer keyword arguments to pass to the Convolutional 2D layer
- lib.model.nn_blocks.set_config(configuration: dict) None
Set the global configuration parameters from the user’s config file.
These options are used when creating layers for new models.
- Parameters:
configuration (dict) – The configuration options that exist in the training configuration files that pertain specifically to Custom Faceswap Layers. The keys should be: icnr_init, conv_aware_init and ‘reflect_padding’
model.normalization module
Module Summary
Instance normalization layer (Lei Ba et al, 2016, Ulyanov et al., 2016). |
Normalization methods for faceswap.py specific to Tensorflow backend
- class lib.model.normalization.AdaInstanceNormalization(*args, **kwargs)
Bases:
Layer
Adaptive Instance Normalization Layer for Keras.
- Parameters:
axis (int, optional) – The axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format=”channels_first”, set axis=1 in
InstanceNormalization
. Setting axis=None will normalize all values in each instance of the batch. Axis 0 is the batch dimension. axis cannot be set to 0 to avoid errors. Default:None
momentum (float, optional) – Momentum for the moving mean and the moving variance. Default: 0.99
epsilon (float, optional) – Small float added to variance to avoid dividing by zero. Default: 1e-3
center (bool, optional) – If
True
, add offset of beta to normalized tensor. IfFalse
, beta is ignored. Default:True
scale (bool, optional) – If
True
, multiply by gamma. IfFalse
, gamma is not used. When the next layer is linear (also e.g. relu), this can be disabled since the scaling will be done by the next layer. Default:True
References
Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization - https://arxiv.org/abs/1703.06868
- build(input_shape)
Creates the layer weights.
- Parameters:
input_shape (tensor) – Keras tensor (future input to layer) or
list
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs, training=None)
This is where the layer’s logic lives.
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- compute_output_shape(input_shape)
Calculate the output shape from this layer.
- Parameters:
input_shape (tuple) – The input shape to the layer
- Returns:
The output shape to the layer
- Return type:
int
- get_config()
Returns the config of the layer.
The Keras configuration for the layer.
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.normalization.GroupNormalization(*args, **kwargs)
Bases:
Layer
Group Normalization
- Parameters:
axis (int, optional) – The axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format=”channels_first”, set axis=1 in
InstanceNormalization
. Setting axis=None will normalize all values in each instance of the batch. Axis 0 is the batch dimension. axis cannot be set to 0 to avoid errors. Default:None
gamma_init (str, optional) – Initializer for the gamma weight. Default: “one”
beta_init (str, optional) – Initializer for the beta weight. Default “zero”
gamma_regularizer (varies, optional) – Optional regularizer for the gamma weight. Default:
None
beta_regularizer (varies, optional) – Optional regularizer for the beta weight. Default
None
epsilon (float, optional) – Small float added to variance to avoid dividing by zero. Default: 1e-3
group (int, optional) – The group size. Default: 32
data_format (["channels_first", "channels_last"], optional) – The required data format. Optional. Default:
None
kwargs (dict) – Any additional standard Keras Layer key word arguments
References
Shaoanlu GAN: https://github.com/shaoanlu/faceswap-GAN
- build(input_shape)
Creates the layer weights.
- Parameters:
input_shape (tensor) – Keras tensor (future input to layer) or
list
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs, *args, **kwargs)
This is where the layer’s logic lives.
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- get_config()
Returns the config of the layer.
The Keras configuration for the layer.
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.normalization.InstanceNormalization(*args, **kwargs)
Bases:
Layer
Instance normalization layer (Lei Ba et al, 2016, Ulyanov et al., 2016).
Normalize the activations of the previous layer at each step, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.
- Parameters:
axis (int, optional) – The axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format=”channels_first”, set axis=1 in
InstanceNormalization
. Setting axis=None will normalize all values in each instance of the batch. Axis 0 is the batch dimension. axis cannot be set to 0 to avoid errors. Default:None
epsilon (float, optional) – Small float added to variance to avoid dividing by zero. Default: 1e-3
center (bool, optional) – If
True
, add offset of beta to normalized tensor. IfFalse
, beta is ignored. Default:True
scale (bool, optional) – If
True
, multiply by gamma. IfFalse
, gamma is not used. When the next layer is linear (also e.g. relu), this can be disabled since the scaling will be done by the next layer. Default:True
beta_initializer (str, optional) – Initializer for the beta weight. Default: “zeros”
gamma_initializer (str, optional) – Initializer for the gamma weight. Default: “ones”
beta_regularizer (str, optional) – Optional regularizer for the beta weight. Default:
None
gamma_regularizer (str, optional) – Optional regularizer for the gamma weight. Default:
None
beta_constraint (float, optional) – Optional constraint for the beta weight. Default:
None
gamma_constraint (float, optional) – Optional constraint for the gamma weight. Default:
None
References
Layer Normalization - https://arxiv.org/abs/1607.06450
Instance Normalization: The Missing Ingredient for Fast Stylization - https://arxiv.org/abs/1607.08022
- build(input_shape)
Creates the layer weights.
- Parameters:
input_shape (tensor) – Keras tensor (future input to layer) or
list
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs, training=None)
This is where the layer’s logic lives.
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- get_config()
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
- class lib.model.normalization.RMSNormalization(*args, **kwargs)
Bases:
Layer
Root Mean Square Layer Normalization (Biao Zhang, Rico Sennrich, 2019)
RMSNorm is a simplification of the original layer normalization (LayerNorm). LayerNorm is a regularization technique that might handle the internal covariate shift issue so as to stabilize the layer activations and improve model convergence. It has been proved quite successful in NLP-based model. In some cases, LayerNorm has become an essential component to enable model optimization, such as in the SOTA NMT model Transformer.
RMSNorm simplifies LayerNorm by removing the mean-centering operation, or normalizing layer activations with RMS statistic.
- Parameters:
axis (int) – The axis to normalize across. Typically this is the features axis. The left-out axes are typically the batch axis/axes. This argument defaults to -1, the last dimension in the input.
epsilon (float, optional) – Small float added to variance to avoid dividing by zero. Default: 1e-8
partial (float, optional) – Partial multiplier for calculating pRMSNorm. Valid values are between 0.0 and 1.0. Setting to 0.0 or 1.0 disables. Default: 0.0
bias (bool, optional) – Whether to use a bias term for RMSNorm. Disabled by default because RMSNorm does not enforce re-centering invariance. Default
False
kwargs (dict) – Standard keras layer kwargs
References
RMS Normalization - https://arxiv.org/abs/1910.07467
Official implementation - https://github.com/bzhangGo/rmsnorm
- build(input_shape)
Validate and populate
axis
- Parameters:
input_shape (tensor) – Keras tensor (future input to layer) or
list
/tuple
of Keras tensors to reference for weight shape computations.
- call(inputs, *args, **kwargs)
Call Root Mean Square Layer Normalization
- Parameters:
inputs (tensor) – Input tensor, or list/tuple of input tensors
- Returns:
A tensor or list/tuple of tensors
- Return type:
tensor
- compute_output_shape(input_shape)
The output shape of the layer is the same as the input shape.
- Parameters:
input_shape (tuple) – The input shape to the layer
- Returns:
The output shape to the layer
- Return type:
tuple
- get_config()
Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstated later (without its trained weights) from this configuration.
The configuration of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- Returns:
A python dictionary containing the layer configuration
- Return type:
dict
model.optimizers module
Module Summary
model.session module
Settings manager for Keras Backend
- class lib.model.session.KSession(name: str, model_path: str, model_kwargs: dict | None = None, allow_growth: bool = False, exclude_gpus: list[int] | None = None, cpu_mode: bool = False)
Bases:
object
Handles the settings of backend sessions for inference models.
This class acts as a wrapper for various
keras.Model()
functions, ensuring that actions performed on a model are handled consistently and can be performed in parallel in separate threads.This is an early implementation of this class, and should be expanded out over time.
Notes
The documentation refers to
keras
. This is a pseudonym for eitherkeras
ortensorflow.keras
depending on the backend in use.- Parameters:
name (str) – The name of the model that is to be loaded
model_path (str) – The path to the keras model file
model_kwargs (dict, optional) – Any kwargs that need to be passed to
keras.models.load_models()
. Default:None
allow_growth (bool, optional) – Enable the Tensorflow GPU allow_growth configuration option. This option prevents Tensorflow from allocating all of the GPU VRAM, but can lead to higher fragmentation and slower performance. Default:
False
exclude_gpus (list, optional) – A list of indices correlating to connected GPUs that Tensorflow should not use. Pass
None
to not exclude any GPUs. Default:None
cpu_mode (bool, optional) –
True
run the model on CPU. Default:False
- append_softmax_activation(layer_index: int = -1) None
Append a softmax activation layer to a model
Occasionally a softmax activation layer needs to be added to a model’s output. This is a convenience function to append this layer to the loaded model.
- Parameters:
layer_index (int, optional) – The layer index of the model to select the output from to use as an input to the softmax activation layer. Default: -1 (The final layer of the model)
- define_model(function: Callable) None
Defines a model from the given function.
This method acts as a wrapper for
keras.models.Model()
.- Parameters:
function (function) – A function that defines a
keras.Model
and returns it’sinputs
andoutputs
. The function that generates these results should be passed in, NOT the results themselves, as the function needs to be executed within the correct context.
- load_model() None
Loads a model.
This method is a wrapper for
keras.models.load_model()
. Loads a model and its weights frommodel_path
defined during initialization of this class. Any additionalkwargs
to be passed tokeras.models.load_model()
should also be defined during initialization of the class.For Tensorflow backends, the make_predict_function method is called on the model to make it thread safe.
- load_model_weights() None
Load model weights for a defined model inside the correct session.
This method is a wrapper for
keras.load_weights()
. Once a model has been defined indefine_model()
this method can be called to load its weights from themodel_path
defined during initialization of this class.For Tensorflow backends, the make_predict_function method is called on the model to make it thread safe.
- predict(feed: list[numpy.ndarray] | ndarray, batch_size: int | None = None) list[numpy.ndarray] | ndarray
Get predictions from the model.
This method is a wrapper for
keras.predict()
function. For Tensorflow backends this is a straight call to the predict function.- Parameters:
feed (numpy.ndarray or list) – The feed to be provided to the model as input. This should be a
numpy.ndarray
for single inputs or a list ofnumpy.ndarray
objects for multiple inputs.batchsize (int, optional) – The batch size to run prediction at. Default
None
- Returns:
The predictions from the model
- Return type:
numpy.ndarray