{"id":94855,"date":"2023-10-04T08:00:00","date_gmt":"2023-10-04T15:00:00","guid":{"rendered":""},"modified":"2024-08-23T09:13:54","modified_gmt":"2024-08-23T16:13:54","slug":"accelerating-over-130000-hugging-face-models-with-onnx-runtime","status":"publish","type":"post","link":"https:\/\/opensource.microsoft.com\/blog\/2023\/10\/04\/accelerating-over-130000-hugging-face-models-with-onnx-runtime\/","title":{"rendered":"Accelerating over 130,000 Hugging Face models with ONNX Runtime"},"content":{"rendered":"\n

There are currently over 320,000 models on Hugging Face (HF), and this number continues to grow every day. Only about 6,000 of these models have an indication of ONNX support in the HF Model Hub, but over 130,000 support the ONNX format.<\/p>\n\n\n\n

ONNX models can be accelerated with ONNX Runtime (ORT), which works cross-platform and provides coverage for many cloud models and language models. Updating the HF Model Hub with more accurate information about ONNX coverage will ensure that users can leverage all the benefits of ORT when deploying HF models. This blog post will provide an overview of HF model architectures with ORT support, discuss ORT coverage for cloud models and language models, and provide the next steps for increasing the number of ONNX models listed in the HF Model Hub. Ultimately, readers will have a better understanding of why they should use ONNX Runtime<\/a> to accelerate open source machine learning models from Hugging Face.<\/p>\n\n\n\n

HF ORT support overview<\/h2>\n\n\n\n

Hugging Face provides a list of supported model architectures in its Transformers documentation<\/a>. Model architectures are groups of models with similar operators, meaning that if one model within a model architecture is supported by ONNX, the other models in the architecture are supported by ONNX as well (with rare exceptions). Models in the HF Model Hub can be filtered by model architecture using search queries (e.g., the number of models from the BERT model architecture can be found using this Hugging Face tool<\/a>).<\/p>\n\n\n\n

ORT supports model architectures where:<\/p>\n\n\n\n