Federated Learning
Overview and Motivation
Federated Learning (FL) is a decentralized machine learning paradigm that enables multiple edge devices referred to as clients to collaboratively train a shared model without transferring their private data to a central location. Each client performs local training using its own dataset and communicates only model updates (such as gradients or weights) to an orchestrating server or aggregator. These updates are then aggregated to produce a new global model that is redistributed to the clients for further training. This process continues iteratively, allowing the model to learn from distributed data sources while preserving the privacy and autonomy of each client. By design, FL shifts the focus from centralized data collection to collaborative model development, introducing a new direction in scalable, privacy-preserving machine learning [1].
The motivation for Federated Learning arises from growing concerns around data privacy, security, and communication efficiency particularly in edge computing environments where data is generated in massive volumes across geographically distributed and often resource-constrained devices. Centralized learning architectures struggle in such contexts due to limited bandwidth, high transmission costs, and strict regulatory frameworks such as the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA). FL inherently mitigates these issues by allowing data to remain on-device, thereby minimizing the risk of data exposure and reducing reliance on constant connectivity to cloud services. Furthermore, by exchanging only lightweight model updates instead of full datasets, FL significantly decreases communication overhead, making it well-suited for real-time learning in mobile and edge networks [2].
Within the broader ecosystem of edge computing, FL represents a paradigm shift that enables distributed intelligence under conditions of partial availability, device heterogeneity, and non-identically distributed (non-IID) data. Clients in FL systems can participate asynchronously, tolerate network interruptions, and adapt their computational loads based on local capabilities. This flexibility is particularly important in edge scenarios where devices may differ in processor power, battery life, and storage. Moreover, FL supports the development of personalized and locally adapted models through techniques such as federated personalization and clustered aggregation. These properties make FL not only an effective solution for collaborative learning at the edge but also a foundational approach for building scalable, secure, and trustworthy AI systems that are aligned with emerging demands in distributed computing and privacy-preserving technologies [1][2][3].
Federated Learning Architectures
Federated Learning (FL) can be implemented through various architectural configurations, each defining how clients interact, how updates are aggregated, and how trust and responsibility are distributed. These architectures play a central role in determining the scalability, fault tolerance, communication overhead, and privacy guarantees of a federated system. In edge computing environments, where client devices are heterogeneous and network reliability varies, the choice of architecture significantly affects the efficiency and robustness of learning. The three dominant paradigms are centralized, decentralized, and hierarchical architectures. Each of these approaches balances different trade-offs in terms of coordination complexity, system resilience, and resource allocation.
Centralized Architecture
In the centralized FL architecture, a central server or cloud orchestrator is responsible for all coordination, aggregation, and distribution activities. The server begins each round by broadcasting a global model to a selected subset of client devices, which then perform local training using their private data. After completing local updates, clients send their modified model parameters usually in the form of weight vectors or gradients back to the server. The server performs aggregation, typically using algorithms such as Federated Averaging (FedAvg), and sends the updated global model to the clients for the next round of training.
The centralized model is appealing for its simplicity and compatibility with existing cloud to client infrastructures. It is relatively easy to deploy, manage, and scale in environments with stable connectivity and limited client churn. However, its reliance on a single server introduces critical vulnerabilities. The server becomes a bottleneck under high communication loads and a single point of failure if it experiences downtime or compromise. Furthermore, this architecture requires clients to trust the central aggregator with metadata, model parameters, and access scheduling. In privacy-sensitive or high availability contexts, these limitations can restrict centralized FL’s applicability [1].
Decentralized Architecture
Decentralized FL removes the need for a central server altogether. Instead, client devices interact directly with each other to share and aggregate model updates. These peer-to-peer (P2P) networks may operate using structured overlays, such as ring topologies or blockchain systems, or employ gossip-based protocols for stochastic update dissemination. In some implementations, clients collaboratively compute weighted averages or perform federated consensus to update the global model in a distributed fashion.
This architecture significantly enhances system robustness, resilience, and trust decentralization. There is no single point of failure, and the absence of a central coordinator eliminates risks of aggregator bias or compromise. Moreover, decentralized FL supports federated learning in contexts where participants belong to different organizations or jurisdictions and cannot rely on a neutral third party. However, these benefits come at the cost of increased communication overhead, complex synchronization requirements, and difficulties in managing convergence, especially under non-identical data distributions and asynchronous updates. Protocols for secure communication, update verification, and identity authentication are necessary to prevent malicious behavior and ensure model integrity. Due to these complexities, decentralized FL is an active area of research and is best suited for scenarios requiring strong autonomy and fault tolerance [2].
Hierarchical Architecture
Hierarchical FL is a hybrid architecture that introduces one or more intermediary layers—often called edge servers or aggregators between clients and the global coordinator. In this model, clients are organized into logical or geographical groups, with each group connected to an edge server. Clients send their local model updates to their respective edge aggregator, which performs preliminary aggregation. The edge servers then send their aggregated results to the cloud server, where final aggregation occurs to produce the updated global model.
This multi-tiered architecture is designed to address the scalability and efficiency challenges inherent in centralized systems while avoiding the coordination overhead of full decentralization. Hierarchical FL is especially well-suited for edge computing environments where data, clients, and compute resources are distributed across structured clusters, such as hospitals within a healthcare network or base stations in a telecommunications infrastructure.
One of the key advantages of hierarchical FL is communication optimization. By aggregating locally at edge nodes, the amount of data transmitted over wide-area networks is significantly reduced. Additionally, this model supports region-specific model personalization by allowing edge servers to maintain specialized sub-models adapted to local client behavior. Hierarchical FL also enables asynchronous and fault-tolerant training by isolating disruptions within specific clusters. However, this architecture still depends on reliable edge aggregators and introduces new challenges in cross-layer consistency, scheduling, and privacy preservation across multiple tiers [1][3].
Aggregation Algorithms and Communication Efficiency
Aggregation is a fundamental operation in Federated Learning (FL), where updates from multiple edge clients are merged to form a new global model. The quality, stability, and efficiency of the federated learning process depend heavily on the aggregation strategy employed. In edge environments characterized by device heterogeneity and non-identical data distributions choosing the right aggregation algorithm is essential to ensure reliable convergence and effective collaboration.
Key Aggregation Algorithms
| Algorithm | Description | Handles Non-IID Data | Server-Side Optimization | Typical Use Case |
|---|---|---|---|---|
| FedAvg | Performs weighted averaging of client models based on dataset size. Simple and communication-efficient. | Limited | No | Basic federated setups with IID or mildly non-IID data. |
| FedProx | Adds a proximal term to the local loss function to prevent client drift. Stabilizes training with diverse data. | Yes | No | Suitable for edge deployments with high data heterogeneity or resource-limited clients. |
| FedOpt | Applies adaptive optimizers (e.g., FedAdam, FedYogi) on aggregated updates. Enhances convergence in dynamic systems. | Yes | Yes | Used in large-scale systems or settings with unstable participation and gradient variability. |
Aggregation is the cornerstone of Federated Learning (FL), where locally computed model updates from edge devices are combined into a global model. The most widely adopted aggregation method is Federated Averaging (FedAvg), introduced in the foundational work by McMahan et al. FedAvg operates by averaging model parameters received from participating clients, typically weighted by the size of each client’s local dataset. This simple yet powerful method reduces the frequency of communication by allowing each device to perform multiple local updates before sending gradients to the server. However, FedAvg performs optimally only when data across clients is balanced and independent and identically distributed (IID)—conditions rarely satisfied in edge computing environments, where client datasets are often highly non-IID, sparse, or skewed [1][2].
To address these limitations, several advanced aggregation algorithms have been proposed. One notable extension is FedProx, which modifies the local optimization objective by adding a proximal term that penalizes large deviations from the global model. This constrains local training and improves stability in heterogeneous data scenarios. FedProx also allows flexible participation by clients with limited resources or intermittent connectivity, making it more robust in practical edge deployments. Another family of aggregation algorithms is FedOpt, which includes adaptive server-side optimization techniques such as FedAdam and FedYogi. These algorithms build on optimization methods used in centralized training and apply them at the aggregation level, enabling faster convergence and improved generalization under complex, real-world data distributions. Collectively, these variants of aggregation address critical FL challenges such as slow convergence, client drift, and update divergence due to heterogeneity in both data and device capabilities [1][3].
Communication Efficiency in Edge-Based FL
Communication remains one of the most critical bottlenecks in deploying FL at the edge, where devices often suffer from limited bandwidth, intermittent connectivity, and energy constraints. To address this, several strategies have been developed. **Gradient quantization** reduces the size of transmitted updates by lowering numerical precision (e.g., from 32-bit to 8-bit values). **Gradient sparsification** limits communication to only the most significant changes in the model, transmitting top-k updates while discarding negligible ones. **Local update batching** allows devices to perform multiple rounds of local training before sending updates, reducing the frequency of synchronization.
Further, **client selection strategies** dynamically choose a subset of devices to participate in each round, based on criteria like availability, data quality, hardware capacity, or trust level. These communication optimizations are crucial for ensuring that FL remains scalable, efficient, and deployable across millions of edge nodes without overloading the network or draining device batteries [1][2][3].
Privacy Mechanisms
Privacy and data confidentiality are central design goals of Federated Learning (FL), particularly in edge computing scenarios where numerous IoT devices (e.g., hospital servers, autonomous vehicles) gather sensitive data. Although FL does not require the raw data to leave each client’s device, model updates can still leak private information or be correlated to individual data points. To address these challenges, various privacy-preserving mechanisms have been proposed in the literature [1][2][3].
Differential Privacy (DP)
Differential Privacy is a formal framework ensuring that the model’s outputs (e.g., parameter updates) do not reveal individual records. In FL, DP often involves injecting calibrated noise into gradients or model weights on each client. This noise is designed so that the global model’s performance remains acceptable, yet attackers cannot reliably infer any single data sample’s presence in the training set. A step-by-step timeline of DP in an FL context can be summarized as follows: 1. Clients fetch the global model and compute local gradients. 2. Before transmitting gradients, clients add randomized noise to mask specific data patterns. 3. The central server aggregates the noisy gradients to produce a new global model. 4. Clients download the updated global model for further local training. By carefully tuning the “privacy budget” (ε and δ), DP can balance privacy against model utility [1][4].
Secure Aggregation (SecAgg)
Secure Aggregation, or SecAgg, is a protocol that encrypts local updates before they are sent to the server, ensuring that only the aggregated result is revealed. A typical SecAgg workflow includes: 1. Each client randomly splits its model updates into multiple shares. 2. These shares are exchanged among clients and the server in a way that no single party sees the entirety of any update. 3. The server only obtains the sum of all client updates, rather than individual parameters. This approach can thwart internal adversaries who might try to reconstruct local data from raw updates [2]. SecAgg is crucial for preserving confidentiality, especially in IoT-based FL systems where data privacy regulations (GDPR, HIPAA) prohibit raw data exposure.
Homomorphic Encryption and SMPC
Homomorphic Encryption (HE) supports computations on encrypted data without the need for decryption. In FL, a homomorphically encrypted gradient can be aggregated securely by the server, preventing it from seeing cleartext updates. This approach, however, introduces higher computational overhead, which can be burdensome for resource-limited IoT edge devices [3]. Secure Multi-Party Computation (SMPC) is a related set of techniques that enables multiple parties to perform joint computations on secret inputs. In the context of FL, SMPC allows clients to compute sums of model updates without revealing individual updates. Although performance optimizations exist, SMPC remains challenging for large-scale models with millions of parameters [1][5].
IoT-Specific Considerations
In edge computing, IoT devices often capture highly sensitive data (patient records, vehicle sensor logs, etc.). Privacy measures must therefore operate seamlessly on low-power hardware while accommodating intermittent connectivity. For instance, a smart healthcare device storing patient records may use DP-based local training and SecAgg to encrypt updates before uploading. Meanwhile, an autonomous vehicle might adopt HE to guard sensor patterns relevant to real-time traffic analysis. Together, these techniques form a multi-layered privacy defense tailored for distributed, resource-constrained IoT ecosystems [4][5].
Security Threats
While Federated Learning (FL) enhances data privacy by ensuring that raw data remains on edge devices, it introduces significant security vulnerabilities due to its decentralized design and reliance on untrusted participants. In edge computing environments, where clients often operate with limited computational power and over unreliable networks, these threats are particularly pronounced.
Model Poisoning Attacks
Model poisoning occurs when malicious clients deliberately manipulate their local model updates to corrupt the global model. These clients may inject backdoors or skew the model's behavior toward specific target outcomes. Since aggregation algorithms such as FedAvg average local updates without deep validation, even a small number of compromised clients can have a disproportionate effect on the global model—especially under non-IID data conditions common in edge environments [1][4].
Defensive strategies include robust aggregation techniques like Krum, Trimmed Mean, and Bulyan, which aim to detect and exclude anomalous updates. However, their computational requirements may be unsuitable for resource-constrained edge devices [1].
Data Poisoning Attacks
In data poisoning attacks, adversaries modify the training data itself on edge devices to distort model learning. Poisoned data can be mislabeled or adversarially crafted inputs designed to bias the training outcome. Because FL systems do not inspect clients' private data, poisoned datasets remain undetected and influence global training through seemingly benign updates [3][2].
In edge computing, data heterogeneity and lack of transparency make it difficult to distinguish between legitimate variation and malicious manipulation [4].
Inference and Membership Attacks
FL is also vulnerable to inference attacks, where adversaries analyze shared model updates to extract information about clients’ private datasets. One prominent form is the membership inference attack, where an attacker determines whether a specific data point was used during training. These attacks exploit overfitting and gradient leakage, particularly in scenarios involving small or biased local datasets—common characteristics of edge FL deployments [2][3].
Defenses like Differential Privacy (DP) and Secure Aggregation aim to obscure individual contributions but often introduce trade-offs in model performance or computational overhead [4].
Sybil and Free-Rider Attacks
In Sybil attacks, an adversary spawns multiple fake clients to gain undue influence over the global model. These fake identities may submit poisoned updates or manipulate consensus protocols in decentralized FL settings [1]. The lack of strong identity verification in cross-device FL makes these attacks feasible, especially when client selection is randomized.
Free-rider attacks involve clients who participate in training without contributing meaningful updates. These clients may reuse stale models or fabricate updates while benefiting from the evolving global model. Such behavior is particularly attractive in energy-constrained IoT settings and undermines collaborative learning [3].
Proposed countermeasures include reputation-based systems and update auditing, though they add complexity and may conflict with privacy goals.
Malicious Server Attacks
In centralized FL, the server is responsible for aggregating updates and broadcasting the global model. If compromised, the server can perform gradient inversion, drop or alter updates, or inject adversarial parameters. This creates a single point of failure [1][3]. Even with secure aggregation protocols, a malicious server could infer sensitive information if not properly constrained.
Decentralized architectures reduce this risk by removing the central aggregator and distributing coordination across edge clients. However, this introduces new issues like synchronization and communication overhead [2][4].
References
- Gabrielli, E., Pica, G., Tolomei, G. A Survey on Decentralized Federated Learning, 2023.
- Kairouz, P., et al. Advances and Open Problems in Federated Learning, 2021.
- Nguyen, D. C., et al. Federated Learning for Internet of Things: A Comprehensive Survey, IEEE Communications Surveys & Tutorials, vol. 23, no. 3, 2021.
- Abreha, H. G., Hayajneh, M., Serhani, M. A. Federated Learning in Edge Computing: A Systematic Survey, Sensors, vol. 22, 2022.
- Pinyoanuntapong, P., et al. EdgeML: Towards Network-Accelerated Federated Learning over Wireless Edge, 2022.
References
[1] Abreha, H.G., Hayajneh, M., & Serhani, M.A. (2022). Federated Learning in Edge Computing: A Systematic Survey. Sensors, 22(2), 450.
[2] Kairouz, P., et al. (2019). Advances and Open Problems in Federated Learning. arXiv:1912.04977.
[3] Gabrielli, E., Pica, G., & Tolomei, G. (2023). A Survey on Decentralized Federated Learning. ACM Transactions on Intelligent Systems and Technology.
[4] Li, T., Sahu, A.K., Talwalkar, A., & Smith, V. (2020). Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine, 37(3), 50–60.
[5] Nguyen, D.C., et al. (2021). Federated Learning for Internet of Things: A Comprehensive Survey. IEEE Communications Surveys & Tutorials, 23(3), 1622–1677.