Model

class plugins.train.model.original.Model(*args, **kwargs)

Bases: ModelBase

Original Faceswap Model.

This is the original faceswap model and acts as a template for plugin development.

All plugins must define the following attribute override after calling the parent’s __init__() method:

input_shape (tuple or list): a tuple of ints defining the shape of the faces that the model takes as input. If the input size is the same for both sides, this can be a single 3 dimensional tuple. If the inputs have different sizes for “A” and “B” this should be a list of 2 3 dimensional shape tuples, 1 for each side.

Any additional attributes used exclusively by this model should be defined here, but make sure that you are not accidentally overriding any existing ModelBase attributes.

Parameters:

args (varies) – The default command line arguments passed in from Train or Convert
kwargs (varies) – The default keyword arguments passed in from Train or Convert

Attributes Summary

`coverage_ratio`	The ratio of the training image to crop out and train on as defined in user configuration options.
`freeze_layers`	Override to set plugin specific layers that can be frozen.
`input_shapes`	A flattened list corresponding to all of the inputs to the model.
`io`	Input/Output operations for the model
`iterations`	The total number of iterations that the model has trained.
`load_layers`	Override to set plugin specific layers that can be loaded.
`model`	The compiled model for this plugin.
`model_name`	The name of the keras model.
`name`	The name of this model based on the plugin name.
`output_shapes`	A flattened list corresponding to all of the outputs of the model.
`state`	The state settings for the current plugin.

Methods Summary

`add_history`(loss)	Add the current iteration's loss history to `_io.history`.
`build`()	Build the model and assign to `model`.
`build_model`(inputs)	Create the model's structure.
`decoder`(side)	The original Faceswap Decoder Network.
`encoder`()	The original Faceswap Encoder Network.

Attributes Documentation

coverage_ratio

The ratio of the training image to crop out and train on as defined in user configuration options.

NB: The coverage ratio is a raw float, but will be applied to integer pixel images.

To ensure consistent rounding and guaranteed even image size, the calculation for coverage should always be: \((original_size * coverage_ratio // 2) * 2\)

freeze_layers: Override to set plugin specific layers that can be frozen. Defaults to [“encoder”]

input_shapes: A flattened list corresponding to all of the inputs to the model.

io: Input/Output operations for the model

iterations: The total number of iterations that the model has trained.

load_layers: Override to set plugin specific layers that can be loaded. Defaults to [“encoder”]

model: The compiled model for this plugin.

model_name: The name of the keras model. Generally this will be the same as name but some plugins will override this when they contain multiple architectures

name: The name of this model based on the plugin name.

output_shapes: A flattened list corresponding to all of the outputs of the model.

state: The state settings for the current plugin.

Methods Documentation

add_history(loss: np.ndarray) → None

Add the current iteration’s loss history to _io.history.

Called from the trainer after each iteration, for tracking loss drop over time between save iterations.

Parameters:: loss (np.ndarray) – The loss values for the A and B side for the current iteration. This should be the collated loss values for each side.
Return type:: None

build() → None

Build the model and assign to model.

Within the defined strategy scope, either builds the model from scratch or loads an existing model if one exists.

If running inference, then the model is built only for the required side to perform the swap function, otherwise the model is then compiled with the optimizer and chosen loss function(s).

Finally, a model summary is outputted to the logger at verbose level.

Return type:: None

build_model(inputs)

Create the model’s structure.

This function is automatically called immediately after __init__() has been called if a new model is being created. It is ignored if an existing model is being loaded from disk as the model structure will be defined in the saved model file.

The model’s final structure is defined here.

For the original model, An encoder instance is defined, then the same instance is referenced twice, one for each input “A” and “B” so that the same model is used for both inputs.

2 Decoders are then defined (one for each side) with the encoder instances passed in as input to the corresponding decoders.

The final output of the model should always call lib.model.nn_blocks.Conv2DOutput so that the correct data type is set for the final activation, to support Mixed Precision Training. Failure to do so is likely to lead to issues when Mixed Precision is enabled.

Parameters:: inputs (list) – A list of input tensors for the model. This will be a list of 2 tensors of shape input_shape, the first for side “a”, the second for side “b”.
Returns:: See Keras documentation for the correct structure, but note that parameter name is a required rather than an optional argument in Faceswap. You should assign this to the attribute self.name that is automatically generated from the plugin’s filename.
Return type:: keras.models.Model

decoder(side)

The original Faceswap Decoder Network.

The decoders for the original model have separate weights for each side “A” and “B”, so two instances are created in build_model(), one for each side.

Parameters:: side (str) – Either “a or “b”. This is used for naming the decoder model.
Returns:: The Keras decoder model. This will be called twice, once for each side.
Return type:: keras.models.Model

encoder()

The original Faceswap Encoder Network.

The encoder for the original model has it’s weights shared between both the “A” and “B” side of the model, so only one instance is created build_model(). However this same instance is then used twice (once for A and once for B) meaning that the weights get shared.

Returns:: The Keras encoder model, for sharing between inputs from both sides.
Return type:: keras.models.Model