{"id":94473,"date":"2023-05-31T08:00:00","date_gmt":"2023-05-31T15:00:00","guid":{"rendered":""},"modified":"2023-08-01T17:37:27","modified_gmt":"2023-08-02T00:37:27","slug":"on-device-training-efficient-training-on-the-edge-with-onnx-runtime","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2023\/05\/31\/on-device-training-efficient-training-on-the-edge-with-onnx-runtime\/","title":{"rendered":"On-Device Training: Efficient training on the edge with ONNX Runtime"},"content":{"rendered":"\n

We are introducing On-Device Training<\/a>, a new capability in ONNX Runtime (ORT) which enables training models on edge devices without the data ever leaving the device. The edge devices can be any compute-enabled devices like laptops, smartphones, gaming consoles, or other embedded devices. This capability opens new opportunities for application developers, as they can now personalize experiences for users without compromising privacy. This blog post provides a quick overview of On-Device Training with ORT and resources to help you get started.<\/p>\n\n\n\n

ONNX Runtime at a glance<\/h2>\n\n\n\n

ORT<\/a> is a high-performance cross-platform inference and training engine that can run a variety of machine learning models. ORT provides an easy-to-use experience for the AI developers to run models on multiple hardware and software platforms. Beyond accelerating server-side inference and training, ORT is also available for inferencing on mobile devices and on web browsers.<\/p>\n\n\n\n

The new On-Device Training capability extends the ORT-Mobile inference offering to enable training on the edge devices. The goal is to make it easy for developers to take an inference model and train it locally on-device\u2014with data present on-device\u2014to provide an improved user experience for end customers.<\/p>\n\n\n\n

On-Device Training with ONNX Runtime<\/h2>\n\n\n\n

As opposed to traditional deep learning (DL) model training, On-Device Training requires efficient use of compute and memory resources. Additionally, edge devices vary greatly in compute and memory configurations. To support these unique needs of edge device training, we created On-Device Training capability that is framework agnostic and builds on top of the existing C++ ORT core functionality.<\/p>\n\n\n\n

With On-Device Training, application developers can now infer and train using the same binaries. At the end of a training session, the runtime produces optimized inference ready models which can then be used for a more personalized experience on the device. For scenarios like federated learning, the runtime provides model differences since the aggregation happens on the server side.<\/p>\n\n\n

\"High-Level
Figure 1: High-Level workflow for personalization with ONNX Runtime<\/figcaption><\/figure>\n\n\n\n

Key benefits<\/h2>\n\n\n\n