Federated averaging

Averaging client models weighted by their data size.

Federated averaging (FedAvg) is the standard aggregation rule. After local training, the server averages the client models, weighting each by its share of the data:

w_{t + 1} = k = 1 \sum K \frac{n _{k}}{n} w_{t + 1}^{(k)}, n = k \sum n_{k}

Doing several local epochs before averaging dramatically reduces how many communication rounds are needed compared with sending a gradient every step.

The catch: with very different client data, local models drift in different directions and their average can be poor. Variants like FedProx add a term keeping local models close to the global one.

Formula to remember

Federated averaging

w_{t + 1} = k = 1 \sum K \frac{n _{k}}{n} w_{t + 1}^{(k)}

Global model = data-weighted average of client models.

Quick check · +5 XP

Which technique lets the server see only the sum of client updates?

Your notes

Saved in this browser as you type.