Machine Learning at the Edge - Revision history

Ciarang: /* Dynamic Models */

2025-04-25T05:18:23Z

Dynamic Models

← Older revision		Revision as of 05:18, 25 April 2025
Line 166:		Line 166:
	Given the dynamic nature of the environments that edge devices must function, as well as the heterogeneity of the devices themselves, a dynamic model of machine learning is often employed. Such models must keep track of the current available resources including computation usage and power, as well as network traffic. These may change very often depending on the workloads and devices in the system. As such, training models to continuously monitor and dynamically distribute the workloads is a very important part of optimization. Simply offloading larger tasks to more powerful devices may be obsolete if the devices has all of its computing resources or network capabilities being used up by another workload.		Given the dynamic nature of the environments that edge devices must function, as well as the heterogeneity of the devices themselves, a dynamic model of machine learning is often employed. Such models must keep track of the current available resources including computation usage and power, as well as network traffic. These may change very often depending on the workloads and devices in the system. As such, training models to continuously monitor and dynamically distribute the workloads is a very important part of optimization. Simply offloading larger tasks to more powerful devices may be obsolete if the devices has all of its computing resources or network capabilities being used up by another workload.

	The way this is commonly done is by using the profiling step described above as a baseline. Then, a machine learning model utilizes the data to estimate the performance of devices and/or layers. During runtime, a similar process is employed which may update the data used and help the model refine its predictions. Network traffic is also taken into account at this stage in order to preserve the edge computing benefit of providing lower latency. Using all of this data and updates at runtime, the partitioning model is able to dynamically distribute workloads at runtime in order to optimize the workflow and ensure each device is utilizing its resources in the most efficient manner. 2 very good examples of how such a system is specifically deployed are the Neurosurgeon and EdgeShard systems, shown above.		The way this is commonly done is by using the profiling step described above as a baseline. Then, a machine learning model utilizes the data to estimate the performance of devices and/or layers. During runtime, a similar process is employed which may update the data used and help the model refine its predictions. Network traffic is also taken into account at this stage in order to preserve the edge computing benefit of providing lower latency. Using all of this data and updates at runtime, the partitioning model is able to dynamically distribute workloads at runtime in order to optimize the workflow and ensure each device is utilizing its resources in the most efficient manner. Two very good examples of how such a system is specifically deployed are the Neurosurgeon and EdgeShard systems, shown above.

	==='''Horizontal and Vertical Partitioning'''===		==='''Horizontal and Vertical Partitioning'''===

Ciarang: /* Usage and Applications of AI Agents */

2025-04-25T05:16:48Z

Usage and Applications of AI Agents

← Older revision		Revision as of 05:16, 25 April 2025
Line 143:		Line 143:
	'''Tools:''' These tasks, however, are impossible to accomplish without the correct tools. Even if an AI agent understands how to go about carrying a task for optimal results, it must have the actual means to do it. This can include the ability to use and interact with APIs, interpret code, and access certain databases.		'''Tools:''' These tasks, however, are impossible to accomplish without the correct tools. Even if an AI agent understands how to go about carrying a task for optimal results, it must have the actual means to do it. This can include the ability to use and interact with APIs, interpret code, and access certain databases.

	Utilizing AI agents on edge devices can be tricky due to the computational and reasoning power needed. However, there are methods to accomplish this such as SLMs which query LLMs as ~~need~~ (discussed later), or utilizing more powerful edge devices to carry out tasks. However, utilizing edge devices can be paramount if latency is a major issue, or if the agent is exposed to sensitive user data. Additionally, by using edge devices specific to a user, it may be able to better learn a user's patterns and preferences and react accordingly to provide the best possible outcome for that user.		Utilizing AI agents on edge devices can be tricky due to the computational and reasoning power needed. However, there are methods to accomplish this such as SLMs which query LLMs as needed (discussed later), or utilizing more powerful edge devices to carry out tasks. However, utilizing edge devices can be paramount if latency is a major issue, or if the agent is exposed to sensitive user data. Additionally, by using edge devices specific to a user, it may be able to better learn a user's patterns and preferences and react accordingly to provide the best possible outcome for that user.

	=='''4.3 ML Model Optimization at the Edge'''==		=='''4.3 ML Model Optimization at the Edge'''==

Ciarang: /* Challenges of Machine Learning Using Edge Computing */

2025-04-25T05:15:44Z

Challenges of Machine Learning Using Edge Computing

← Older revision		Revision as of 05:15, 25 April 2025
Line 16:		Line 16:
	==='''Challenges of Machine Learning Using Edge Computing'''===		==='''Challenges of Machine Learning Using Edge Computing'''===

	'''Potentially Low Computational Power:''' Many edge devices lack the computational power ~~needed for~~ needed for deep learning applications, especially given the large amount of data and complex operations that may have to take place [1].		'''Potentially Low Computational Power:''' Many edge devices lack the computational power needed for deep learning applications, especially given the large amount of data and complex operations that may have to take place [1].

	'''Energy Management:''' Given that edge devices may consist of sensors, or other devices that are meant to have low energy consumption, the tasks associated with machine learning could quickly drain their power, even if they have sufficient power to run the workloads [6].		'''Energy Management:''' Given that edge devices may consist of sensors, or other devices that are meant to have low energy consumption, the tasks associated with machine learning could quickly drain their power, even if they have sufficient power to run the workloads [6].

Ciarang: /* The Need for Model Optimization at the Edge */

2025-04-25T05:14:49Z

The Need for Model Optimization at the Edge

← Older revision		Revision as of 05:14, 25 April 2025
Line 148:		Line 148:

	==='''The Need for Model Optimization at the Edge'''===		==='''The Need for Model Optimization at the Edge'''===
	Given the constrained resources~~, along with~~ the inherently dynamic ~~environment that~~ edge devices must ~~function in~~, model optimization is a crucial part of machine learning in edge computing. The current most widely used methodology consists of simply specifying an exceptionally large set of parameters, and giving it to the model to train on. This can be feasible when hardware is very advanced and powerful, and is necessary for systems such as Large Language Models (LLMs). However, this is no longer viable when dealing with the devices and environments at the edge. It is crucial to identify the best parameters and training methodology so as to minimize the amount of work done by these devices, while compromising as little as possible on the accuracy of the models. There are multiple ways to this, and they include either optimization or augmentation of the dataset itself, or optimization of the partition of work among the edge devices.		Given the constrained resources and the inherently dynamic environments in which edge devices must operate, model optimization is a crucial part of machine learning in edge computing. The current most widely used methodology consists of simply specifying an exceptionally large set of parameters, and giving it to the model to train on. This can be feasible when hardware is very advanced and powerful, and is necessary for systems such as Large Language Models (LLMs). However, this is no longer viable when dealing with the devices and environments at the edge. It is crucial to identify the best parameters and training methodology so as to minimize the amount of work done by these devices, while compromising as little as possible on the accuracy of the models. There are multiple ways to this, and they include either optimization or augmentation of the dataset itself, or optimization of the partition of work among the edge devices.

	==='''Edge and Cloud Collaboration'''===		==='''Edge and Cloud Collaboration'''===

Ciarang: /* 4.2 ML Training at the Edge */

2025-04-25T05:10:52Z

4.2 ML Training at the Edge

← Older revision		Revision as of 05:10, 25 April 2025
Line 130:		Line 130:
	\| Effective when deploying accurate, lightweight models under data or resource constraints.		\| Effective when deploying accurate, lightweight models under data or resource constraints.
	\|}		\|}


	==='''Usage and Applications of AI Agents'''===		==='''Usage and Applications of AI Agents'''===

Ciarang: /* Dealing With Challenges */

2025-04-25T05:10:12Z

Dealing With Challenges

@@ Line 72: / Line 72: @@
 A well known example is Apple’s use of on-device training for personalized voice recognition with Siri. Instead of uploading user voice data to the cloud, Apple uses local training to improve accuracy over time while maintaining user privacy.
-==='''Dealing With Challenges'''===
+==='''Model Compression Techniques'''===
 Despite the challenges, of ML at the edge, there are a variety of methods that can be used to provide a more efficient means of training, and making the heavy workloads compatible with even the limited computing power of certain edge devices.
@@ Line 101: / Line 101: @@
 [[File:Knowledge_Distillation.png|700px|thumb|center|This shows how a complex teacher model transfers learned knowledge to a smaller student model using soft predictions to enable efficient edge deployment]]
 ==='''Usage and Applications of AI Agents'''===

Ciarang: /* Usage and Applications of AI Agents */

2025-04-25T05:01:21Z

Usage and Applications of AI Agents

← Older revision		Revision as of 05:01, 25 April 2025
Line 103:		Line 103:

	==='''Usage and Applications of AI Agents'''===		==='''Usage and Applications of AI Agents'''===
	As ~~AI develops~~ and ~~is able~~ to ~~identify different images~~, ~~generate chats and utilize human language~~, ~~and optimize different systems~~, the ~~next step is for it to be able to~~ efficiently ~~perform~~ tasks specified by users. ~~This is known as an~~ AI ~~agent~~. ~~The~~ agent must first perceive its environment and ~~task, often specified~~ by a user, ~~then~~ it must ~~think~~ about ~~what is~~ the ~~best means~~ to accomplish ~~the~~ task, and finally ~~be able to~~ act on it. ~~3 aspects of AI agents are crucial in their development~~ to ~~efficiently~~ and ~~correctly carry out task:~~		As artificial intelligence and machine learning technologies continue to mature, they pave the way for the development of intelligent AI agents capable of autonomous, context-aware behavior, with the goal of efficiently performing tasks specified by users. These agents combine perception, reasoning, and decision-making to execute tasks with minimal human intervention. When deployed on edge devices, AI agents can operate with low latency, preserve user privacy, and adapt to local data—making them ideal for real-time, personalized applications in homes, vehicles, factories, and beyond.

			To function effectively, an agent must first perceive its environment and understand the task—often defined by the user. Then, it must reason about the optimal steps to accomplish that task, and finally, it must act on those decisions. These three components—perception, reasoning, and action—are essential to the agent’s ability to operate accurately and autonomously in dynamic environments.

	'''Reasoning:''' The agent must be able to think sequentially, and decompose its specified tasks into a sequence of specific steps in order to accomplish its goal. It must also have some memory storage in order to remember what it has done, as well as the results of its sequence of actions in order to learn for future steps.		'''Reasoning:''' The agent must be able to think sequentially, and decompose its specified tasks into a sequence of specific steps in order to accomplish its goal. It must also have some memory storage in order to remember what it has done, as well as the results of its sequence of actions in order to learn for future steps.

Ciarang: /* 4.3 ML Model Optimization at the Edge */

2025-04-25T04:45:13Z

4.3 ML Model Optimization at the Edge

← Older revision		Revision as of 04:45, 25 April 2025
Line 142:		Line 142:

	[[File:Edge_computing_layers.png\|400px\|thumb\|center\|An example of different layers with multiple devices]]		[[File:Edge_computing_layers.png\|400px\|thumb\|center\|An example of different layers with multiple devices]]


	==='''Distributed Learning'''===		==='''Distributed Learning'''===
Line 171:		Line 170:

	One way that a similar system can be achieved on edge devices is by using SLMs. These are not as accurate and do not have the vast knowledge of LLMs or the amount of data they are trained on, but for the purposes of basic applications and edge devices, they can be sufficient to accomplish many tasks. They are also often fine-tuned and trained to accomplish the specific tasks which they are deployed for and are much faster and resource-efficient than LLMs. They can also provide much more privacy because they are able to be run on local devices without sharing user data to the cloud. This can be useful for a wide variety of edge applications. If needed and privacy constraints permit, they can query and LLM as needed for more complicated tasks. This means that not every prompt leads to a query, and thus network traffic, privacy, and latency constraints are still preserved.		One way that a similar system can be achieved on edge devices is by using SLMs. These are not as accurate and do not have the vast knowledge of LLMs or the amount of data they are trained on, but for the purposes of basic applications and edge devices, they can be sufficient to accomplish many tasks. They are also often fine-tuned and trained to accomplish the specific tasks which they are deployed for and are much faster and resource-efficient than LLMs. They can also provide much more privacy because they are able to be run on local devices without sharing user data to the cloud. This can be useful for a wide variety of edge applications. If needed and privacy constraints permit, they can query and LLM as needed for more complicated tasks. This means that not every prompt leads to a query, and thus network traffic, privacy, and latency constraints are still preserved.

			==='''Synthesis'''===

			In summary, edge-oriented machine learning optimization requires an integrated approach that combines model-level compression with system-level orchestration. Techniques such as quantization, structured and unstructured pruning, and knowledge distillation reduce the computational footprint and memory requirements of deep learning models, enabling deployment on resource-constrained devices without substantial loss in inference accuracy. Concurrently, dynamic workload partitioning, heterogeneity-aware scheduling, and adaptive runtime profiling allow the system to allocate tasks across edge and cloud tiers based on real-time availability of compute, bandwidth, and energy resources. This joint optimization across model architecture and execution environment is essential to meet the latency, privacy, and resilience demands of edge AI deployments.

	=='''References'''==		=='''References'''==

Ciarang: /* References */

2025-04-25T04:31:28Z

References

← Older revision		Revision as of 04:31, 25 April 2025
Line 181:		Line 181:
	[https://dl.acm.org/doi/abs/10.1145/3093337.3037698 4. Kang, Yiping and Hauswald, Johann and Gao, Cao and Rovinski, Austin and Mudge, Trevor and Mars, Jason and Tang, Lingjia, "Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge" 2017 Association for Computing Machinery, New York, NY, USA, 2017, doi: 10.1145/3093337.3037698.]		[https://dl.acm.org/doi/abs/10.1145/3093337.3037698 4. Kang, Yiping and Hauswald, Johann and Gao, Cao and Rovinski, Austin and Mudge, Trevor and Mars, Jason and Tang, Lingjia, "Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge" 2017 Association for Computing Machinery, New York, NY, USA, 2017, doi: 10.1145/3093337.3037698.]

	[https://dl.acm.org/doi/pdf/10.1145/3555802 5. Hua, ~~Haochen~~, ~~et al~~. "Edge computing with artificial intelligence: A machine learning perspective." ACM Computing Surveys 55.9 (2023): ~~1-35~~.]		[https://dl.acm.org/doi/pdf/10.1145/3555802 5. H. Hua, Y. Li, T. Wang, N. Dong, W. Li, and J. Cao, "Edge computing with artificial intelligence: A machine learning perspective," ACM Computing Surveys, vol. 55, no. 9, Art. no. 184, pp. 1–35, Jan. 2023, doi: 10.1145/3555802..]

	[https://www.mdpi.com/2079-9292/13/3/640 6. Grzesik, Piotr, and Dariusz Mrozek. "Combining machine learning and edge computing: Opportunities, challenges, platforms, frameworks, and use cases." Electronics 13.3 (2024): 640.]		[https://www.mdpi.com/2079-9292/13/3/640 6. Grzesik, Piotr, and Dariusz Mrozek. "Combining machine learning and edge computing: Opportunities, challenges, platforms, frameworks, and use cases." Electronics 13.3 (2024): 640.]

Ciarang: /* Benefits of Machine Learning using Edge Computing */

2025-04-25T04:29:00Z

Benefits of Machine Learning using Edge Computing

← Older revision		Revision as of 04:29, 25 April 2025
Line 8:		Line 8:

	==='''Benefits of Machine Learning using Edge Computing'''===		==='''Benefits of Machine Learning using Edge Computing'''===
	'''Data Volumes:''' Traditional cloud computing architecture may not be able to keep up with the massive volume of data that is often generated by IoT devices and sensors, thus increasing costs as well as pressure and congestion on the networks transmitting data [5]. With edge computing ~~architecture~~, machine learning can be deployed in a distributed nature, allowing ML models to be deployed and trained across many edge devices. The nature of edge devices sharing computational tasks makes it perfect for taking large datasets and splitting the computational load so that many devices can compute what a single device might not have been able to handle [5]. More than just splitting input data across edge devices, machine learning models can be split up across edge devices as well. This allows for deployment of models that would be otherwise too large to be deployed on a single edge device with limited memory. In such cases, edge nodes are able to collaboratively pass data between them, and the global ML model is later updated based on the workings of each smaller portion [5]. Additionally, the sheer amount of data that processed from edge devices means more training data for machine learning models, which generally increases accuracy, and thus edge computing could be an effective solution to providing lots of good and relevant data for a variety of applications.		'''Data Volumes:''' Traditional cloud computing architecture may not be able to keep up with the massive volume of data that is often generated by IoT devices and sensors, thus increasing costs as well as pressure and congestion on the networks transmitting data [5]. With edge computing architectures, machine learning can be deployed in a distributed nature, allowing ML models to be deployed and trained across many edge devices. The nature of edge devices sharing computational tasks makes it perfect for taking large datasets and splitting the computational load so that many devices can compute what a single device might not have been able to handle [5]. More than just splitting input data across edge devices, machine learning models can be split up across edge devices as well. This allows for deployment of models that would be otherwise too large to be deployed on a single edge device with limited memory. In such cases, edge nodes are able to collaboratively pass data between them, and the global ML model is later updated based on the workings of each smaller portion [5]. Additionally, the sheer amount of data that processed from edge devices means more training data for machine learning models, which generally increases accuracy, and thus edge computing could be an effective solution to providing lots of good and relevant data for a variety of applications.

	'''Lower Latency:''' For certain real-time applications, such as Virtual and Augmented Reality(VR/AR) or smart cars, the latency required to transmit data and process it to the cloud may be too high to make these applications efficient and safe. A key benefit of edge computing lies in the reduced latency provided by putting devices closer to the users, and this is applicable to machine learning as well [5].		'''Lower Latency:''' For certain real-time applications, such as Virtual and Augmented Reality(VR/AR) or smart cars, the latency required to transmit data and process it to the cloud may be too high to make these applications efficient and safe. A key benefit of edge computing lies in the reduced latency provided by putting devices closer to the users, and this is applicable to machine learning as well [5].