In the rapidly evolving world of artificial intelligence, having access to pre-trained models can significantly accelerate the development process. Azure AI Studio, a leading platform in this domain, offers a diverse range of models through its Model Catalog. This article aims to guide you through the exploration of these models, shedding light on their variety, functionality, and how they can be effectively utilized in your AI projects.
Understanding the Models Catalog in Azure AI Studio
The Models Catalog in Azure AI Studio is a central hub for discovering foundation models. It includes a wide variety of models curated by Microsoft, Hugging Face, and Meta. These models are pre-trained and optimized for use in Azure AI Studio, making them ready for out-of-the-box usage. Each model comes with a model card that provides a brief description of the model, along with links to samples for code-based inferencing, fine-tuning, and evaluation of the model.
The Role of the Models Catalog in Azure AI Studio
The Models Catalog plays a crucial role in Azure AI Studio. It allows developers and data scientists to quickly find and implement the right model for their specific task. Whether you’re working on natural language processing, computer vision, or any other AI task, the Models Catalog has a model that can help.
The Models Catalog also includes a Sample Inference feature, which allows you to test the model with your own sample input. This feature is incredibly useful for understanding how a model will perform with your specific data.
Moreover, the Models Catalog provides model benchmarks, allowing you to compare the performance of different models on various datasets. This feature is particularly useful when you’re trying to decide between several models for a specific task.
Model Card in Azure AI Studio
A model card in Azure AI Studio is a detailed document that provides essential information about a pre-trained model. It serves as a guide for developers and users to understand the model’s purpose, performance, and how to use it.
Here are some key features of a model card in Azure AI Studio:
Sample Inference: You can quickly try out any pre-trained model using the Sample Inference widget on the model card, providing your sample input to test the result.
Model Description: The model card includes a brief description of the model.
Links to Samples: The model card provides links to samples for code-based inferencing, fine-tuning, and evaluation of the model.
You can filter the model catalog by collection, model name, or task to find the model that best suits your needs.
Filtering Models
In Azure AI Studio’s Models Catalog, you can filter models by collection, model name, or task. This is an essential feature that allows you to navigate through the vast array of models available.
Filtering is important for finding the right model for your specific needs. With the vast number of models available in the Models Catalog, it can be overwhelming to find the one that best suits your task. By filtering models based on your requirements, you can narrow down the options and make the selection process more manageable.
Moreover, different models are designed for different tasks and perform optimally under different conditions. Therefore, understanding your task requirements and using the filtering options effectively can help you select a model that will deliver the best results.
To filter models, you simply use the filtering options provided in the Models Catalog. You can choose to filter by:
1. Collection
The Models Catalog includes models from various collections, including those curated by Microsoft, Hugging Face, and Meta. You can filter by collection to view models from a specific source. This is particularly useful if you’re looking for models that have been optimized or vetted by a particular organization.
Below is the collection of models provided by Azure AI Studio:
Curated by Azure AI models: These models are used for a wide range of tasks such as chat, question answering, and text generation. They are optimized for use in Azure AI Studio and can be used for tasks like image recognition, speech recognition, and most recently on language understanding tasks.
Azure OpenAI models: These models are used for a variety of tasks such as translation, question answering, and summarization. They can also be used for writing assistance, code generation, and reasoning over data.
Meta AI models: Meta’s models are used for a wide range of tasks including content understanding, feeds, generative AI, and ads ranking. They can also be used for tasks like image and object recognition, speech recognition, and most recently on language understanding tasks.
Hugging Face models: Hugging Face models are used for a wide range of tasks such as natural language processing (translation, summarization, and text generation), audio-related functions (automatic speech recognition, voice activity detection, and text-to-speech), and computer vision tasks (depth estimation, image classification, and image-to-image processing).
NVIDIA AI models: NVIDIA AI models are used for a wide range of tasks including computer vision, speech, language understanding, molecule generation, and more. They can also be used for tasks like image recognition, speech recognition, and most recently on language understanding tasks.
Microsoft Research AI models: Microsoft Research AI models are used for a wide range of tasks including content generation, image creation, drug discovery, and IT service automation.
Deci AI models: Deci AI models are used for a wide range of tasks including computer vision, Generative AI, and NLP applications.
Mistral AI models: Mistral AI models are used for a wide range of tasks including text generation, code completion, and text-to-image generation.
2. Inference Task
Different models are designed for different inference tasks, such as text classification, image recognition, or language translation. By filtering by inference task, you can view models that are specifically designed for the task you’re working on. This can help you find a model that’s likely to perform well on your task.
Azure AI Studio provides a list of inference tasks that you can use for various purposes. Here are some of them:
Conversational: This task involves generating responses in a conversation, similar to a chatbot.
Fill Mask: This task involves predicting missing words or phrases in a sentence.
Question Answering: This task involves answering questions based on a given context.
Summarization: This task involves generating a concise summary of a larger text.
Table Question Answering: This task involves answering questions based on a given table of data.
Text Classification: This task involves categorizing a piece of text into one or more predefined categories.
Text Generation: This task involves generating text based on a given prompt.
Text-to-Text Generation: This task involves generating a piece of text based on another piece of text.
Token Classification: This task involves classifying individual tokens (words, subwords) in a text.
Translation: This task involves translating text from one language to another.
Zero-shot Classification: This task involves classifying text without any prior examples of those categories.
Speech Recognition: This task involves transcribing spoken language into written text.
Chat Completion: This task involves generating responses in a conversation, similar to a chatbot.
Embeddings: This task involves generating vector representations of text that capture their meanings.
Image Classification: This task involves categorizing an image into one of several predefined categories.
Image Segmentation: This task involves dividing an image into multiple segments, each of which corresponds to an object or a part of an object.
Image to Image: This task involves transforming one image into another, such as converting a sketch into a colored image.
Image to Text: This task involves generating a textual description of an image.
Multi-Object Tracking: This task involves tracking multiple objects as they move through a series of images or video frames.
Object Detection: This task involves identifying objects within an image and typically includes providing bounding boxes around the objects.
Text Summarization: This task involves generating a concise summary of a larger text.
Text to Image: This task involves generating an image based on a textual description.
Text Translation: This task involves translating text from one language to another.
Visual Question Answering: This task involves answering questions based on visual content.
Zero-Shot Image Classification: This task involves classifying images without any prior examples of those categories.
3. Fine-tuning Task
Some models in the catalog are designed to be fine-tuned on a specific task. By filtering by fine-tuning task, you can find models that have been pre-trained on a large dataset and then fine-tuned on a specific task. These models can often provide better performance on the fine-tuning task than models that haven’t been fine-tuned.
Here are some of them:
Image Classification: This task involves categorizing an image into one of several predefined categories.
Image Segmentation: This task involves dividing an image into multiple segments, each of which corresponds to an object or a part of an object.
Object Detection: This task involves identifying objects within an image and typically includes providing bounding boxes around the objects.
Question Answering: This task involves answering questions based on a given context.
Summarization: This task involves generating a concise summary of a larger text.
Text Classification: This task involves categorizing a piece of text into one or more predefined categories.
Text Generation: This task involves generating text based on a given prompt.
Token Classification: This task involves classifying individual tokens (words, subwords) in a text.
Translation: This task involves translating text from one language to another.
Multi-Object Tracking: This task involves tracking multiple objects as they move through a series of images or video frames.
4. License
Models in the catalog are subject to different license terms. Some models are open source and can be used freely, while others may have restrictions on their use. By filtering by license, you can ensure that the models you’re considering are suitable for your intended use.
Here is the list:
Apache-2.0: The Apache License 2.0 is a permissive free software license written by the Apache Software Foundation (ASF). It allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties.
MIT: The MIT License is a permissive free software license originating at the Massachusetts Institute of Technology (MIT). It puts only very limited restrictions on reuse and has, therefore, high license compatibility.
CreativeML-OpenRail-M
OpenRail
AFL-3.0: The Academic Free License (AFL) is a permissive free software license. The terms of the license make it suitable for academics and other developers who want to create proprietary derivatives.
CC-BY-4.0: Creative Commons Attribution 4.0 International (CC BY 4.0) is a public copyright license that enables the free distribution of an otherwise copyrighted work. A CC BY license allows for others to distribute, remix, adapt, and build upon the work, even commercially, as long as they credit the original creation.
CC-BY-NC-4.0: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) is similar to CC-BY-4.0 but with the additional stipulation that you can’t use the material for commercial purposes.
CC-BY-NC-SA-4.0: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) is similar to CC-BY-NC-4.0 but with the additional stipulation that if you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
CC-BY-SA-4.0: Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) is similar to CC-BY-4.0 but with the additional stipulation that if you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
GPL-3.0: The GNU General Public License (GPL) is a widely-used free software license, which guarantees end users the freedom to run, study, share and modify the software.
BigScience-Bloom-Rail~1.0: This could be a specific license for models related to the BigScience Bloom Rail project. I would need to search for more specific information about this license.
BSD~3-Clause: The BSD 3-Clause License is a permissive free software license, imposing minimal restrictions on the use and distribution of covered software. This is in contrast to copyleft licenses, which have reciprocity share-alike requirements.
BigScience-OpenRail-M
AGPL~3.0: The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software.
BSD: The BSD License is a permissive free software license, imposing minimal restrictions on the use and redistribution of covered software.
Conclusion
The Model Catalog in Azure AI Studio serves as a comprehensive hub for discovering and utilizing pre-trained models. With its user-friendly interface and extensive collection of models from Microsoft, Hugging Face, and Meta, it empowers users to quickly find and deploy models that best suit their needs. The ability to test models with the Sample Inference widget and compare benchmarks across models and datasets further enhances its utility.
However, users must be mindful of the third-party license terms when using models from Hugging Face and Meta. As Azure AI Studio continues to evolve, the Model Catalog is set to play a pivotal role in simplifying and accelerating the AI development process.