{"id":86937,"date":"2021-07-13T09:00:14","date_gmt":"2021-07-13T16:00:14","guid":{"rendered":"https:\/\/cloudblogs.microsoft.com\/opensource\/?p=86937"},"modified":"2025-05-30T15:18:18","modified_gmt":"2025-05-30T22:18:18","slug":"onnx-runtime-release-1-8-1-previews-support-for-accelerated-training-on-amd-gpus-with-the-amd-rocm-open-software-platform","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2021\/07\/13\/onnx-runtime-release-1-8-1-previews-support-for-accelerated-training-on-amd-gpus-with-the-amd-rocm-open-software-platform\/","title":{"rendered":"ONNX Runtime release 1.8.1 previews support for accelerated training on AMD GPUs with the AMD ROCm\u2122 Open Software Platform"},"content":{"rendered":"\n

This post was co-authored by Jeff Daily, a Principal Member of Technical Staff, Deep Learning Software for AMD.<\/em><\/p>\n\n\n\n

\"ONNX<\/figure>\n\n\n\n

ONNX Runtime<\/a> is an open-source project that is designed to accelerate machine learning across a wide range of frameworks, operating systems, and hardware platforms. Today, we are excited to announce a preview version of ONNX Runtime in release 1.8.1 featuring support for AMD Instinct\u2122 GPUs facilitated by the AMD ROCm\u2122 open software platform. Users can now use AMD Instinct\u2122 GPUs with ONNX Runtime to accelerate distributed training for large-scale DNN models. AMD ROCm\u2122 becomes the latest ONNX Runtime execution provider, continuing the Microsoft mission to endorse choice and versatility in targeting different compute devices and server platforms.<\/p>\n\n\n\n

\"Selection<\/figure>\n\n\n\n

Figure 1: Selection interface showing AMD GPU support<\/a>.<\/em><\/p>\n\n\n\n

The ROCm Open Software Platform<\/h2>\n\n\n\n

ROCm is AMD’s open software platform for GPU-accelerated high-performance computing and machine learning workloads.  Since the first ROCm release in 2016, the ROCm platform has evolved to support additional math, AI and machine learning, and communication libraries and tools, a wider set of Linux\u00ae distributions, and a range of new GPUs. This includes the AMD Instinct\u2122 MI100 GPU, the first AMD data center accelerator based on the compute-optimized AMD CDNA\u2122 architecture.<\/p>\n\n\n\n

The primary focus of ROCm has been high-performance computing at scale. The combined capabilities of ROCm and the AMD Instinct family of data center accelerators are well suited to accelerate AI\/ML training using ONNX Runtime.<\/p>\n\n\n\n

Accelerated training with ONNX Runtime on AMD GPUs<\/h2>\n\n\n\n

Large transformer models like GPT2 have proven themselves state of the art in natural language processing (NLP) tasks like NLP understanding, generation, and translation. They are also proving useful in applications like time-series prediction and computer vision. Due to their size, these models need to be trained in a large\u2011scale, distributed GPU environment. ONNX Runtime, with support from AMD (rocBLAS, MIOpen, hipRAND, and RCCL) libraries, enables users to train large transformer models in mixed\u2011precision in a distributed AMD GPU environment. Thus, ONNX Runtime on ROCm supports training state-of-art models like BERT, GPT-2, T5, BART, and more using AMD Instinct\u2122 GPUs. Data scientists, researchers, students, and others in the community have an option to accelerate workloads using ONNX Runtime on AMD GPUs. This includes AMD Instinct\u2122 MI100<\/a>, AMD Radeon Instinct\u2122 MI50, and AMD Radeon\u2122 Pro VII GPUs.<\/p>\n\n\n\n

Today, we are happy to announce the preview of Python\u2122 packages supporting ONNX Runtime on ROCm, making it easy to get started with ROCm and ONNX Runtime.<\/p>\n\n\n\n

Training performance acceleration<\/h3>\n\n\n\n

In this preview, we have demonstrated clear performance gains with ONNX runtime using AMD GPUs for fine-tuning GPT2 using HuggingFace<\/a> on eight AMD Instinct\u2122 MI100 GPUs. We see an 18 percent performance gain in these experiments relative to standalone PyTorch along and validated well-matched loss curves.<\/p>\n\n\n\n

\"Using<\/figure>\n\n\n\n

Figure 2: Using ONNX runtime gets 18 percent perf gains over standalone PyTorch. Configuration details are listed below.<\/em><\/p>\n\n\n\n

\"chart,<\/figure>\n\n\n\n

Figure 3: Training loss comparing the PyTorch and PyTorch and ONNX Runtime experiments.<\/em><\/p>\n\n\n\n

In general, the preview ONNX Runtime-ROCm library can be used in multi-node MI100 AMD GPU configurations with high-speed interconnects for inter-GPU communications. As we proceed to our official release, we expect users to see excellent performance across a wide range of Transformer models and ML\/AI workloads, offering users a highly performant choice for their datacenter applications.<\/p>\n\n\n\n

Getting started with ONNX runtime on AMD GPUs<\/h3>\n\n\n\n

With Python packages<\/h4>\n\n\n\n

In a ROCm enabled environment, users can get off to a quick start with a pip install:<\/p>\n\n\n

\npip install onnxruntime-training -f\n<\/pre><\/div>\n\n\n
https:\/\/onnxruntimepackages.z14.web.core.windows.net\/onnxruntime_stable_torch190.rocm42.html<\/a><\/p>\n\n\n\n
Plugging their Pytorch script to ONNX runtime only* requires wrapping the model with<\/p>\n\n\n
\nmodel = ORTModule(model)\n<\/pre><\/div>\n\n\n
More details are available at pytorch\/ort: Accelerate PyTorch models with ONNX Runtime<\/a>.<\/p>\n\n\n\n
*PyTorch model should use standard PyTorch to support export to ONNX.<\/em><\/p>\n\n\n\n
With Dockerfiles<\/h4>\n\n\n\n
Users can also take advantage of a simple Dockerfile to get pre-configured packages of the ROCm libraries, Pytorch, and ONNX Runtime.<\/p>\n\n\n\n
The stable ONNX runtime 1.8.1 release is now available at ort\/Dockerfile.ort-torch181-onnxruntime-stable-rocm4.2-ubuntu18.04 at main \u00b7 pytorch\/ort<\/a><\/p>\n\n\n\n
More details are available at pytorch\/ort<\/a>.<\/p>\n\n\n\n
More information about ONNX Runtime<\/h2>\n\n\n\n