Model Parameter Glossary

The model configuration is a JSON file where the user specifies the model parameters for the session. There are 5 classes that can be configured: strategy, model, ml-task, optimizer and differential_privacy_params

strategy

Required: "name", name of the federated learning strategy, "FedAvg"

Required: "params", 3 parameters can be specified

1. "fraction_fit"

   • type: number, minimum: 0, maximum: 1

   • description: Proportion of clients to use for training. If fraction * total_num_users is smaller than min_num_clients set in the session config, then min_num_clients will be used.

2. "fraction_eval":

   • type: number, minimum: 0, maximum: 1

   • description: Proportion of clients to use for evaluation. If fraction * total_num_users is smaller than min_num_clients set in the session config, then min_num_clients will be used.

3. "accept_failures":

   • type: boolean

   • description: Whether to accept failures during training and evaluation. If False, the training process will be stopped when a client fails, otherwise, the failed client will be ignored.

model

Required: "params", options below, along with which standard model type they relate to.

1. "input_size" (glm & ffnet)

   • type: integer, minimum: 0

   • description: Number of input features/predictors.

2. "output_activation" (glm & ffnet)

   • enum: softmax, sigmoid, tanh, exp

   • description: Activation function for the output layer. For classification tasks, softmax has been integrated into the loss fn, thus do not specify it here. For GLM, this is equivalent to the inverse link function. For example, using exp as the output activation means using log link.

3. "hidden_activation" (ffnet)

   • type: enum, [ "relu", "sigmoid", "tanh" ]

   • description: Activation function for all hidden layers (excluding the output layer).

4. "hidden_layer_sizes" (ffnet)

   • type: array, items type: integer, minimum: 0

   • description: Number of hidden nodes in hidden layers, e.g., [3, 3] corresponds to 2 hidden layers, each with 3 nodes.

5. "output_size" (ffnet)

   • type: integer, minimum: 0

   • description": Number of nodes in the output layer. For regression tasks, output_size=1, while for classification tasks, output_size equals to the number of classes/categories.

ml_task

Required: "type", specifies the federated learning strategy, of which there are 7 available in PowerFlow.

• 2 for FFNet models, which are "classification" and "regression",

• 5 for GLM models, which are "logistic", "normal", "poisson", "gamma", "inverseGaussian".

Required: "params", what parameters are required for ml-task depends on which ml-task is chosen.

• If a FFNet type is chosen ("classification" or "regression"), only type is required and params can be left empty or optionally, user can specify the alpha and l1_ratio to impose the elastic net penalty (see more details below).

• If FFNet type is "classification", you can optionally also specify loss_weights as a param (see more details below)

  • "loss_weights" (type: object, default: None)

    • Use if you want to assign specific weights to each label loss (None is the default, 'auto': we calculate the weights based on the ratio of labels, a list of numbers like [1., 2., 1.] is when you want to assign weights manually based on some business logic)

    • Possible values: None, 'auto', array of numbers representing the columns in dataset

    • Mutually exclusive with the balance_train_datasets param

• If a GLM type is chosen (""logistic", "normal", "poisson", "gamma", "inverseGaussian"), user must specify a value within "params" for

  • "alpha" (type: number, above 0)

    • alpha is the multiplicative factor of the elastic net penalty. alpha=0 means no penalty.

  • "l1_ratio" (type: number, between 0 and 1)

    • l1_ratio is the weight of l1 norm in the elastic net penalty. l1_ratio=0 means l2 regularization, and l1_ratio=1 means l1 regularization.

balance_train_datasets

• type: boolean

• default: False

• description: Use if you want to use under-sampling approach to balance the dataset labels. This is only applicable to classification models. It is mutually exclusive to the loss_weights param in ml_task

optimizer

Required: "name", name of the optimizer to use. Choose from the list of optimizers supported by PyTorch here. Please use the exact class name, e.g., SGD or Adam

Required: "params", the configurable parameters for the corresponding optimizer. For example, for Adam, the configurable parameters include lr, betas, weight_decay etc. For the full list of configurable parameters of different optimizers, please follow the link above to the PyTorch page of individual optimizers. NOTE: the parameter learning_rate can still be used in place of lr, but it will be deprecated in future versions in favour of the PyTorch naming lr

differential_privacy_params

Required: "epsilon",

• type: number, minimum: 0

• description: Privacy budget. Larger values correspond to less privacy protection, and potentially better model performance.

Required: "max_grad_norm",

• type: number, minimum: 0

• description: The upper bound for clipping gradients. A hyper-parameter to tune.

Read more about our differential privacy feature here.