Federated Learning with Weights & Biases

In this workspace we can see an example of a federated learning project that utilised Weights & Biases (W&B). This project used the Python federated learning framework Flower to train 3 clients on subsets of the CIFAR-10 image classification dataset.

Federated Learning enables learning from large distributed datasets in a privacy preserving manner where it is not possible due to technical or legal reasons to centralise the data for non-federated machine/deep learning.

Runs as Federated Groups

Along the left hand side we can see that each individual experiment (training session) is grouped by a key called 'federated_group'. This group contains the four individual runs that comprise a single federated learning experiment. One run corresponds to the federated learning server (which is used as the value for this key) alongside a run for each client. Using the grouping functionality in W&B in this way, we can analyse and monitor the experiments in a natural way.

Jobs

If you click the Jobs tab to the left, you will see a distinct 'job' (basically a run template) for the server code, and another for the client code. By logging these as jobs within W&B we can then use them as part of an automated workflow.

Automations

If you click the Automat. tab on the left hand side, you will see a series of automations that have been created. We have setup automations such that once a federated learning experiment is started (i.e., the server.py is run), then we automatically spin up three clients that will connect to the server. These clients can be running on different nodes in different networks, but the automations functionality of W&B removes the manual processes and steps required to conduct realistic federated learning experiments.

Reports

If you click the Reports tab on the left hand side you will see reports that have been created that explains the project and presents results and analysis of experiments.

Artifacts

If you click the Artifacts tab on the left hand side you will see a series of artifacts that are being logged, versioned and tracked in this project. These artifacts include the versions of datasets used for each experiment, the models that were produced from training, as well as capturing which datasets and models were involved in which federation.

The Workspace

If you scroll down you will see the rest of a W&B project workspace. This workspace provides a series of visualisations and analysis to compare and understand the experiments carried out so far.

The workspace is divided into a natural way for federated learning, with a panel visualising the global performance, with a series of panels containing the performance of individual clients in the federated learning experiments, providing both a high level overview and granular insights into performance.