Federated Learning

5.1 Overview and Motivation

Federated Learning (FL) is a decentralized approach to machine learning where many edge devices—called clients—work together to train a shared model without sending their private data to a central server. Each client trains the model locally using its own data and then sends only the model updates (like gradients or weights) to a central server, often called an aggregator. The server combines these updates to create a new global model, which is then sent back to the clients for the next round of training. This cycle repeats, allowing the model to improve while keeping the raw data on the devices. FL shifts the focus from central data collection to collaborative training, supporting privacy and scalability in machine learning systems.^[1]

The main reason for using FL is to address concerns about data privacy, security, and communication efficiency—especially in edge computing, where huge amounts of data are created across many different, often limited, devices. Centralized learning struggles here due to limited bandwidth, high data transfer costs, and strict privacy regulations like the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA). FL helps solve these problems by keeping data on the device, reducing the risk of leaks and the need for constant cloud connectivity. It also lowers communication costs by sharing small model updates instead of full datasets, making it ideal for real-time learning in mobile and edge networks.^[2]

In edge computing, FL is a major shift in how we do machine learning. It supports distributed intelligence even when devices have limited resources, unreliable connections, or very different types of data (non-IID). Clients can join training sessions at different times, work around network delays, and adjust based on their hardware limitations. This makes FL a flexible option for edge environments with varying battery levels, processing power, and storage. FL can also support personalized models using techniques like federated personalization or clustered aggregation. Overall, it provides a strong foundation for building AI systems that are scalable, private, and better suited for the challenges of modern distributed computing.^[1][2][3]

5.2 Federated Learning Architectures

Federated Learning (FL) can be implemented through various architectural configurations, each defining how clients interact, how updates are aggregated, and how trust and responsibility are distributed. These architectures play a central role in determining the scalability, fault tolerance, communication overhead, and privacy guarantees of a federated system. In edge computing environments, where client devices are heterogeneous and network reliability varies, the choice of architecture significantly affects the efficiency and robustness of learning. The three dominant paradigms are centralized, decentralized, and hierarchical architectures. Each of these approaches balances different trade-offs in terms of coordination complexity, system resilience, and resource allocation.

Visual comparison of Cloud-Based, Edge-Based, and Hierarchical Federated Learning architectures. Source: ^[1]

5.2.1 Centralized Architecture

In the centralized FL architecture, a central server or cloud orchestrator is responsible for all coordination, aggregation, and distribution activities. The server begins each round by broadcasting a global model to a selected subset of client devices, which then perform local training using their private data. After completing local updates, clients send their modified model parameters—usually in the form of weight vectors or gradients—back to the server. The server performs aggregation, typically using algorithms such as Federated Averaging (FedAvg), and sends the updated global model to the clients for the next round of training.

The centralized model is appealing for its simplicity and compatibility with existing cloud-to-client infrastructures. It is relatively easy to deploy, manage, and scale in environments with stable connectivity and limited client churn. However, its reliance on a single server introduces critical vulnerabilities. The server becomes a bottleneck under high communication loads and a single point of failure if it experiences downtime or compromise. Furthermore, this architecture requires clients to trust the central aggregator with metadata, model parameters, and access scheduling. In privacy-sensitive or high-availability contexts, these limitations can restrict centralized FL’s applicability. ^[1]

5.2.2 Decentralized Architecture

Decentralized FL removes the need for a central server altogether. Instead, client devices interact directly with each other to share and aggregate model updates. These peer-to-peer (P2P) networks may operate using structured overlays, such as ring topologies or blockchain systems, or employ gossip-based protocols for stochastic update dissemination. In some implementations, clients collaboratively compute weighted averages or perform federated consensus to update the global model in a distributed fashion.

This architecture significantly enhances system robustness, resilience, and trust decentralization. There is no single point of failure, and the absence of a central coordinator eliminates risks of aggregator bias or compromise. Moreover, decentralized FL supports federated learning in contexts where participants belong to different organizations or jurisdictions and cannot rely on a neutral third party. However, these benefits come at the cost of increased communication overhead, complex synchronization requirements, and difficulties in managing convergence—especially under non-identical data distributions and asynchronous updates. Protocols for secure communication, update verification, and identity authentication are necessary to prevent malicious behavior and ensure model integrity. Due to these complexities, decentralized FL is an active area of research and is best suited for scenarios requiring strong autonomy and fault tolerance. ^[2]

5.2.3 Hierarchical Architecture

Hierarchical FL is a hybrid architecture that introduces one or more intermediary layers—often called edge servers or aggregators—between clients and the global coordinator. In this model, clients are organized into logical or geographical groups, with each group connected to an edge server. Clients send their local model updates to their respective edge aggregator, which performs preliminary aggregation. The edge servers then send their aggregated results to the cloud server, where final aggregation occurs to produce the updated global model.

This multi-tiered architecture is designed to address the scalability and efficiency challenges inherent in centralized systems while avoiding the coordination overhead of full decentralization. Hierarchical FL is especially well-suited for edge computing environments where data, clients, and compute resources are distributed across structured clusters, such as hospitals within a healthcare network or base stations in a telecommunications infrastructure.

One of the key advantages of hierarchical FL is communication optimization. By aggregating locally at edge nodes, the amount of data transmitted over wide-area networks is significantly reduced. Additionally, this model supports region-specific model personalization by allowing edge servers to maintain specialized sub-models adapted to local client behavior. Hierarchical FL also enables asynchronous and fault-tolerant training by isolating disruptions within specific clusters. However, this architecture still depends on reliable edge aggregators and introduces new challenges in cross-layer consistency, scheduling, and privacy preservation across multiple tiers. ^[1][3]

5.3 Aggregation Algorithms and Communication Efficiency

Aggregation is a fundamental operation in Federated Learning (FL), where updates from multiple edge clients are merged to form a new global model. The quality, stability, and efficiency of the FL process depend heavily on the aggregation strategy employed. In edge environments characterized by device heterogeneity and non-identical data distributions, choosing the right aggregation algorithm is essential to ensure reliable convergence and effective collaboration.

Federated Learning protocol showing client selection, local training, model update, and aggregation. Source: Adapted from Federated Learning in Edge Computing: A Systematic Survey ^[1]

5.3.1 Key Aggregation Algorithms

Comparison of Aggregation Algorithms in Federated Learning

Algorithm	Description	Handles Non-IID Data	Server-Side Optimization	Typical Use Case
FedAvg	Performs weighted averaging of client models based on dataset size. Simple and communication-efficient.	Limited	No	Basic federated setups with IID or mildly non-IID data.
FedProx	Adds a proximal term to the local loss function to prevent client drift. Stabilizes training with diverse data.	Yes	No	Suitable for edge deployments with high data heterogeneity or resource-limited clients.
FedOpt	Applies adaptive optimizers (e.g., FedAdam, FedYogi) on aggregated updates. Enhances convergence in dynamic systems.	Yes	Yes	Used in large-scale systems or settings with unstable participation and gradient variability.

Aggregation is the cornerstone of FL, where locally computed model updates from edge devices are combined into a global model. The most widely adopted aggregation method is Federated Averaging (FedAvg), introduced in the foundational work by McMahan et al.^[1] FedAvg operates by averaging model parameters received from participating clients, typically weighted by the size of each client’s local dataset. This simple yet powerful method reduces the frequency of communication by allowing each device to perform multiple local updates before sending gradients to the server. However, FedAvg performs optimally only when data across clients is balanced and independent and identically distributed (IID)—conditions rarely satisfied in edge computing environments, where client datasets are often highly non-IID, sparse, or skewed.^[2]

To address these limitations, several advanced aggregation algorithms have been proposed. One notable extension is FedProx, which modifies the local optimization objective by adding a proximal term that penalizes large deviations from the global model. This constrains local training and improves stability in heterogeneous data scenarios. FedProx also allows flexible participation by clients with limited resources or intermittent connectivity, making it more robust in practical edge deployments. Another family of aggregation algorithms is FedOpt, which includes adaptive server-side optimization techniques such as FedAdam and FedYogi. These algorithms build on optimization methods used in centralized training and apply them at the aggregation level, enabling faster convergence and improved generalization under complex, real-world data distributions. Collectively, these variants of aggregation address critical FL challenges such as slow convergence, client drift, and update divergence due to heterogeneity in both data and device capabilities.^[1][3]

5.3.2 Communication Efficiency in Edge-Based FL

Communication remains one of the most critical bottlenecks in deploying FL at the edge, where devices often suffer from limited bandwidth, intermittent connectivity, and energy constraints. To address this, several strategies have been developed. Gradient quantization reduces the size of transmitted updates by lowering numerical precision (e.g., from 32-bit to 8-bit values). Gradient sparsification limits communication to only the most significant changes in the model, transmitting top-k updates while discarding negligible ones. Local update batching allows devices to perform multiple rounds of local training before sending updates, reducing the frequency of synchronization.

Further, client selection strategies dynamically choose a subset of devices to participate in each round, based on criteria like availability, data quality, hardware capacity, or trust level. These communication optimizations are crucial for ensuring that FL remains scalable, efficient, and deployable across millions of edge nodes without overloading the network or draining device batteries.^[1][2][3]

5.4 Privacy Mechanisms

Privacy and data confidentiality are central design goals of Federated Learning (FL), particularly in edge computing scenarios where numerous IoT devices (e.g., hospital servers, autonomous vehicles) gather sensitive data. Although FL does not require the raw data to leave each client’s device, model updates can still leak private information or be correlated to individual data points. To address these challenges, various privacy-preserving mechanisms have been proposed in the literature.^[1][2][3]

5.4.1 Differential Privacy (DP)

Differential Privacy (DP) is a formal framework ensuring that the model’s outputs (e.g., parameter updates) do not reveal individual records. In FL, DP often involves injecting calibrated noise into gradients or model weights on each client. This noise is designed so that the global model’s performance remains acceptable, yet attackers cannot reliably infer any single data sample’s presence in the training set.

A step-by-step timeline of DP in an FL context can be summarized as follows:

1. Clients fetch the global model and compute local gradients.
2. Before transmitting gradients, clients add randomized noise to mask specific data patterns.
3. The central server aggregates the noisy gradients to produce a new global model.
4. Clients download the updated global model for further local training.

By carefully tuning the “privacy budget” (ε and δ), DP can balance privacy against model utility.^[1][4]

5.4.2 Secure Aggregation

Secure Aggregation, or SecAgg, is a protocol that encrypts local updates before they are sent to the server, ensuring that only the aggregated result is revealed. A typical SecAgg workflow includes:

1. Each client randomly splits its model updates into multiple shares.
2. These shares are exchanged among clients and the server in a way that no single party sees the entirety of any update.
3. The server only obtains the sum of all client updates, rather than individual parameters.

This approach can thwart internal adversaries who might try to reconstruct local data from raw updates.^[2] SecAgg is crucial for preserving confidentiality, especially in IoT-based FL systems where data privacy regulations (such as GDPR and HIPAA) prohibit raw data exposure.

5.4.3 Homomorphic Encryption and SMPC

Homomorphic Encryption (HE) supports computations on encrypted data without the need for decryption. In FL, a homomorphically encrypted gradient can be aggregated securely by the server, preventing it from seeing cleartext updates. This approach, however, introduces higher computational overhead, which can be burdensome for resource-limited IoT edge devices.^[3]

Secure Multi-Party Computation (SMPC) is a related set of techniques that enables multiple parties to perform joint computations on secret inputs. In the context of FL, SMPC allows clients to compute sums of model updates without revealing individual updates. Although performance optimizations exist, SMPC remains challenging for large-scale models with millions of parameters.^[1][5]

5.4.4 IoT-Specific Considerations

In edge computing, IoT devices often capture highly sensitive data (patient records, vehicle sensor logs, etc.). Privacy measures must therefore operate seamlessly on low-power hardware while accommodating intermittent connectivity. For instance, a smart healthcare device storing patient records may use DP-based local training and SecAgg to encrypt updates before uploading. Meanwhile, an autonomous vehicle might adopt HE to guard sensor patterns relevant to real-time traffic analysis.

Together, these techniques form a multi-layered privacy defense tailored for distributed, resource-constrained IoT ecosystems.^[4][5]

System model illustrating privacy-preserving federated learning using homomorphic encryption.
Adapted from Privacy-Preserving Federated Learning Using Homomorphic Encryption.

5.5 Security Threats

While Federated Learning (FL) enhances data privacy by ensuring that raw data remains on edge devices, it introduces significant security vulnerabilities due to its decentralized design and reliance on untrusted participants. In edge computing environments, where clients often operate with limited computational power and over unreliable networks, these threats are particularly pronounced.

5.5.1 Model Poisoning Attacks

Model poisoning attacks are a critical threat in FL, especially in edge computing environments where the infrastructure is distributed and clients may be untrusted or loosely regulated. In this type of attack, malicious clients intentionally craft and submit harmful model updates during the training process to compromise the performance or integrity of the global model. These attacks are typically categorized as either untargeted aimed at degrading general model accuracy—or targeted (backdoor attacks), where the global model is manipulated to behave incorrectly in specific scenarios while appearing normal in others. For instance, an attacker might train its local model with a backdoor trigger, such as a specific pixel pattern in an image, so that the global model misclassifies inputs containing that pattern, even though it performs well on standard test cases.^[1][4]

FL's reliance on aggregation algorithms like Federated Averaging (FedAvg), which simply compute the average of local updates, makes it susceptible to these attacks. Since raw data is never shared, poisoned updates can be hard to detect—especially in non-IID settings where variability is expected. Robust aggregation techniques like Krum, Trimmed Mean, and Bulyan have been proposed to resist such manipulation by filtering or down-weighting outlier contributions. However, these algorithms often introduce computational and communication overheads that are impractical for edge devices with limited power and processing capabilities.^[2][4] Furthermore, adversaries can design subtle attacks that mimic benign statistical patterns, making them indistinguishable from legitimate updates. Emerging research explores anomaly detection based on update similarity and trust scoring, yet these solutions face limitations when applied to large-scale or asynchronous FL deployments. Developing lightweight, real-time, and scalable defenses that remain effective under device heterogeneity and unreliable network conditions remains an unresolved challenge.^[3][4]

Federated Learning-based architecture for adversarial sample detection and defense in IoT environments. Adapted from Federated Learning for Internet of Things: A Comprehensive Survey.

5.5.2 Data Poisoning Attacks

Data poisoning attacks target the integrity of FL by manipulating the training data on individual clients before model updates are generated. Unlike model poisoning, which corrupts the gradients or weights directly, data poisoning occurs at the dataset level—allowing adversaries to stealthily influence model behavior through biased or malicious data. This includes techniques such as label flipping (e.g., changing labels of one class to another), outlier injection, or clean-label attacks that subtly alter data without visible artifacts. Since FL assumes client data remains private and uninspected, such poisoned data can easily propagate harmful patterns into the global model, especially in non-IID edge scenarios.^[2][3]

Edge devices are especially vulnerable to this form of attack due to their limited compute and energy resources, which often preclude comprehensive input validation. The highly diverse and fragmented nature of edge data—like medical readings or driving behavior—makes it difficult to establish a clear baseline for anomaly detection. Defense strategies include robust aggregation (e.g., Median, Trimmed Mean), anomaly detection, and Differential Privacy (DP). However, these methods can reduce model accuracy or increase complexity.^[1][4] There is currently no foolproof solution to detect data poisoning without violating privacy principles. As FL continues to be deployed in critical domains, mitigating these attacks while preserving user data locality and system scalability remains an open and urgent research challenge.^[3][4]

5.5.3 Inference and Membership Attacks

Inference attacks represent a subtle yet powerful class of threats in FL, where adversaries seek to extract sensitive information from shared model updates rather than raw data. These attacks exploit the iterative nature of FL training. By analyzing updates—especially in over-parameterized models—attackers can infer properties of the data or even reconstruct inputs. A key example is the membership inference attack, where an adversary determines if a specific data point was used in training. This becomes more effective in edge scenarios, where updates often correlate strongly with individual devices.^[2][3]

As model complexity increases, so does the risk of gradient-based information leakage. Small datasets on edge devices amplify this vulnerability. Attackers with access to multiple rounds of updates may perform gradient inversion to reconstruct training inputs. These risks are especially serious in sensitive fields like healthcare. Mitigations include Differential Privacy and Secure Aggregation, but both reduce accuracy or add system overhead.^[4] Designing FL systems that preserve utility while protecting against inference remains a major ongoing challenge.^[1][4]

5.5.4 Sybil and Free-Rider Attacks

Sybil attacks occur when a single adversary creates multiple fake clients (Sybil nodes) to manipulate the FL process. These clients may collude to skew aggregation results or overwhelm honest participants. This is especially dangerous in decentralized or large-scale FL environments, where authentication is weak or absent.^[1] Without strong identity verification, an attacker can inject numerous poisoned updates, degrading model accuracy or blocking convergence entirely.

Traditional defenses like IP throttling or user registration may violate privacy principles or be infeasible at the edge. Cryptographic registration, proof-of-work, or client reputation scoring have been explored, but each has trade-offs. Clustering and anomaly detection can identify Sybil patterns, though adversaries may adapt their behavior to avoid detection.^[4]

Free-rider attacks involve clients that participate in training but contribute little or nothing—e.g., sending stale models or dummy gradients—while still downloading and benefiting from the global model. This undermines fairness and wastes resources, especially on networks where honest clients use real bandwidth and energy.^[3] Mitigation strategies include contribution-aware aggregation and client auditing.

5.5.5 Malicious Server Attacks

In classical centralized FL, the server acts as coordinator—receiving updates and distributing models. However, if compromised, the server becomes a major threat. It can perform inference attacks, drop honest client updates, tamper with model weights, or inject adversarial logic. This poses significant risk in domains like healthcare and autonomous systems.^[1][3]

Secure Aggregation protects client updates by encrypting them, ensuring only aggregate values are visible. Homomorphic Encryption allows encrypted computation, while SMPC enables privacy-preserving joint computation. However, all three approaches involve high computational or communication costs.^[4] Decentralized or hierarchical architectures reduce single-point-of-failure risk, but introduce new challenges around trust coordination and efficiency.^[2][4]

5.6 Resource-Efficient Model Training

In Federated Learning (FL), especially within edge computing environments, resource-efficient model training is crucial due to the inherent limitations of edge devices, such as constrained computational power, limited memory, and restricted communication bandwidth. Addressing these challenges involves implementing strategies that optimize the use of local resources while maintaining the integrity and performance of the global model. Key approaches include model compression techniques, efficient communication protocols, and adaptive client selection methods.

5.6.1 Model Compression Techniques

Model compression techniques are essential for reducing the computational and storage demands of FL models on edge devices. By decreasing the model size, these techniques facilitate more efficient local training and minimize the communication overhead during model updates. Common methods include:

• Pruning: This technique involves removing less significant weights or neurons from the neural network, resulting in a sparser model that requires less computation and storage. For instance, a study proposed a framework where the global model is pruned on a powerful server before being distributed to clients, effectively reducing the computational load on resource-constrained edge devices.^[1]

• Quantization: This method reduces the precision of the model's weights, such as converting 32-bit floating-point numbers to 8-bit integers, thereby decreasing the model size and accelerating computations. However, careful implementation is necessary to balance the trade-off between model size reduction and potential accuracy loss.

• Knowledge Distillation: In this approach, a large, complex model (teacher) is used to train a smaller, simpler model (student) by transferring knowledge, allowing the student model to achieve comparable performance with reduced resource requirements. This technique has been effectively applied in FL to accommodate the constraints of edge devices.^[2]

5.6.2 Efficient Communication Protocols

Efficient communication protocols are vital for mitigating the communication bottleneck in FL, as frequent transmission of model updates between clients and the central server can overwhelm limited network resources. Strategies to enhance communication efficiency include:

• Update Sparsification: This technique involves transmitting only the most significant updates or gradients, reducing the amount of data sent during each communication round. By focusing on the most impactful changes, update sparsification decreases communication overhead without substantially affecting model performance.

• Compression Algorithms: Applying data compression methods to model updates before transmission can significantly reduce the data size. For example, using techniques like Huffman coding or run-length encoding can compress the updates, leading to more efficient communication.^[3]

• Adaptive Communication Frequency: Adjusting the frequency of communications based on the training progress or model convergence can help in conserving bandwidth. For instance, clients may perform multiple local training iterations before sending updates to the server, thereby reducing the number of communication rounds required.

5.6.3 Adaptive Client Selection Methods

Adaptive client selection methods focus on optimizing the selection of participating clients during each training round to enhance resource utilization and overall model performance. Approaches include:

• Resource-Aware Selection: Prioritizing clients with higher computational capabilities and better network connectivity can lead to more efficient training processes. By assessing the resource availability of clients, the FL system can make informed decisions on which clients to involve in each round.^[4]

• Clustered Federated Learning: Grouping clients based on similarities in data distribution or system characteristics allows for more efficient training. Clients within the same cluster can collaboratively train a sub-model, which is then aggregated to form the global model, reducing the overall communication and computation burden.^[5]

• Early Stopping Strategies: Implementing mechanisms to terminate training early when certain criteria are met can conserve resources. For example, if a client's local model reaches a predefined accuracy threshold, it can stop training and send the update to the server, thereby saving computational resources.^[6]

Incorporating these strategies into the FL framework enables more efficient utilization of the limited resources available on edge devices. By tailoring model training processes to the specific constraints of these devices, it is possible to achieve effective and scalable FL deployments in edge computing environments.

5.8 Challenges in Federated Learning

Federated Learning (FL) helps protect user data by training models directly on devices. While useful, FL faces many challenges that make it hard to use in real-world systems [1][2][3][4][5][6]:

• System Heterogeneity

Devices used in FL—like smartphones, sensors, or edge servers—have different speeds, memory, and battery. Some may be too slow or lose power during training, causing delays. Solutions include using smaller models, pruning, partial updates, and early stopping [1][6].

• Communication Bottlenecks

FL requires frequent communication between devices and a central server. Sending full model updates can overload weak or slow networks. To fix this, researchers use update compression techniques like quantization, sparsification, and knowledge distillation [2][3].

• Statistical Heterogeneity (Non-IID Data)

Each device collects different types of data depending on users and context, making the data non-IID. This harms the global model’s accuracy. Solutions include clustered FL (grouping similar clients), personalized FL, and meta-learning to adapt locally [2][5].

• Privacy and Security

Even if raw data stays on devices, updates can leak private info or be attacked. Malicious devices may poison the model. Solutions include secure aggregation, differential privacy, and encryption—though they may increase cost [2][3].

• Scalability

FL needs to support thousands of devices, many of which may go offline. Picking reliable clients and using hierarchical FL where some devices act as local aggregators—can help scale better [4].

• Incentive Mechanisms

Users may not want to spend energy or bandwidth on FL. Without rewards, participation drops. Research explores systems like blockchain tokens or credit points, but real-world use is still rare [2].

• Lack of Standardization and Benchmarks

FL lacks shared benchmarks and datasets, making it hard to compare approaches. Simulations often ignore real-world issues like device failures or concept drift. Frameworks like LEAF, FedML, and Flower help but more real-world data is needed [2][3].

• Concept Drift and Continual Learning

User data changes over time (concept drift), so the model can become outdated. Continual learning helps the model adapt over time. Techniques include memory replay, adaptive learning rates, and early stopping [1][6].

• Deployment Complexity

Devices use different hardware, software, and networks. It’s hard to manage updates or fix bugs across so many different systems.

• Reliability and Fault Tolerance

Devices can crash, go offline, or send incorrect updates. FL must be designed to handle such failures without harming the global model.

• Monitoring and Debugging

In FL, it's difficult to track and debug model behavior across many devices. New tools are needed to observe and fix problems in distributed training.

5.9 References

1. Efficient Pruning Strategies for Federated Edge Devices. Springer Journal, 2023. https://link.springer.com/article/10.1007/s40747-023-01120-5
2. Federated Learning with Knowledge Distillation. arXiv preprint, 2020. https://arxiv.org/abs/2007.14513
3. Communication-Efficient FL using Data Compression. MDPI Applied Sciences, 2022. https://www.mdpi.com/2076-3417/12/18/9124
4. Resource-Aware Federated Client Selection. arXiv preprint, 2021. https://arxiv.org/pdf/2111.01108
5. Clustered Federated Learning for Heterogeneous Clients. MDPI Telecom, 2023. https://www.mdpi.com/2673-2688/6/2/30
6. Early Stopping in FL for Energy-Constrained Devices. arXiv preprint, 2023. https://arxiv.org/abs/2310.09789