• Article

July 2019

Volume 34 Number 7

[Azure ML]

Understanding Azure AI Services

By Ashish Sahu | July 2019

Cloud computing changed the way products and services are imagined and created. It changed how software solutions were architected and enabled developers to build apps on a planetary scale. And it paved the way to democratize artificial intelligence (AI) for developers across the globe.

Microsoft Azure leads the pack in empowering developers with a spectrum of AI services. But as so often happens when we’re spoiled with choices, it becomes difficult to discern what services are the best ones to use and when. In this article, we walk through the AI services available on Microsoft Azure and provide the information and context you need to analyze your requirements and make informed decisions about which services to use based on your requirements. In some architectures, you might end up with a pipeline of multiple AI services, working in concert to achieve your goals.

Among the range of services available, Microsoft Azure provides more than 35 services targeted at AI and machine learning (ML). These services range from ready-to-use RESTful APIs to the services and SDKs that you can use to create an AI model from scratch using your own data. As we go forward, we’ll discuss these services and share code that you can try yourself. We will also explore the scenarios, requirements and target audience for these services, so you can understand when and what services make a case for themselves.

As a practicing software or ML developer/architect, you may already be aware that the terms AI and ML refer to a vast domain of software tools, frameworks, algorithms and data processing techniques. The features that are powered by AI can range from the simple—say, a personalized recommendation system—to extremely complex—such as identifying actions in videos.

The AI and ML services on Azure cater to everything under the sun and include pre-built AI services that you can integrate into your solution with a few RESTful API calls. If your specific requirement mandates a custom ML model, there are tools and services to build and use those, as well. Openness has been a core tenet of the Microsoft Azure platform, and the Microsoft AI platform allows you to use any toolset or framework that you use today, and still leverage Azure AI Services to streamline the ML workflow end-to-end.

An Overview of AI Services

For simplicity, we can arrange AI Services into three high-level categories: AI Apps and Agents, Knowledge Mining and Machine Learning. Let’s take a moment to explore these categories.

AI Apps and Agents: As a category, AI Apps and Agents are a subset of services offered by Microsoft Cognitive Services and the Azure Bot Service. Microsoft Cognitive Services are pre-built AI services that allow developers to quickly add intelligent features in their apps and services with just a few RESTful API calls. Some of these services provide customization options to suit your specification requirements, but they’re largely powered by models developed, trained and hosted by Microsoft, so you don’t really need to go through the complete process of developing a model from scratch.

Azure Bot Service is powered by the Microsoft Bot Framework platform and provides you with ways to quickly get started with functioning bots that you can extend using Microsoft Cognitive Services or other AI services from Azure. You can also integrate any AI services that you’re developing on your own. The Speech and Language Cognitive Services are also referred to as Conversational AI services in conjunction with Azure Bot Service.

Knowledge Mining: Knowledge Mining refers to a branch of Azure AI platform where intelligent information extraction is the key channel to surface insights from a corpus of structured and unstructured data. Microsoft Azure Search (with its Cognitive Skills) and Azure Form Recognizer service are two shining examples of services that are part of this category.

Machine Learning: Both the AI Apps and Agents and Knowledge Mining categories include pre-built AI services. However, Azure AI platform includes bespoke services that will make any data scientist feel at home. The Machine Learning category includes Microsoft AI Platform services to bring your ML models to life. These Azure services help you manage your ML experiments from data prep stages, to testing and managing the training, to the evaluation of runtimes. They also provide a flexible compute targets option so you can focus solely on your experiment code, rather than worrying about infrastructure, platforms and scaling. Azure Machine Learning Studio, Automated Machine Learning service (also with SDK), Azure Notebooks and Azure Machine Learning Services provide you with platform tools and services to make your experiment development more productive.

Getting Practical

It’s one thing to review the list of available services. It’s quite another to understand how they work and interact in the context of a realistic scenario that applies to a broad range of organizations. With that in mind, we’ll describe a scenario and then use that to show how specific Azure components can address associated tasks.

For our journey today let’s start with a bit of background. Suppose you’re a lead developer, architect or AI engineer at everyone’s favorite company, Contoso. Contoso recently announced the modernization of its business processes, with one of the projects aimed to transform an expense management solution built in-house a decade ago. While the core business logic and approval workflows might not necessarily change, AI can be used to improve user interaction, data input and information processing.

To make this clearer, let’s understand the possible tasks involved in the project and then assign a specific AI service to each task for implementation. Figure 1 illustrates the tasks that are relevant to this project.

Task Map for Contoso Expense Modernization
Figure 1 Task Map for Contoso Expense Modernization

Now let’s walk through the tasks in Figure 1 and see which Microsoft AI service can be used for each task.

Information Extraction from Receipts: Simple & Complex

In this scenario, the requirement is to automate information retrieval from scanned or digital receipts uploaded by users. This stage extracts fields, amounts, and vendor information from the receipts and pushes them to the data store or to a UI for review in the expense application.

Relevant Services: Azure Cognitive Services, Azure Search, Azure Form Recognizer Service

Scenarios that require scanning an image to recognize text (optical character recognition [OCR]) or a set of images and documents call for Azure Search with Cognitive Skills, Computer Vision, and Text Analytics Cognitive Services. The last service is used to extract entities and key phrases (for example, “Seattle” and “dinner”) to complete this task.

Target audience: AI engineers and application developers.

Hands-on Samples:

  • Quickstart: Create an AI indexing pipeline using cognitive skills in Azure Search (bit.ly/2WNDooN)
  • Computer Vision API sample in JavaScript (bit.ly/2YCcFvB)
  • Quickstart: Use the .NET SDK and C# to explore the Text Analytics service (bit.ly/2VKkyST)

The data science community thrives on collaboration and knowledge sharing. But as the scale and complexity of both data and requirements increase, you may find reasons to build your own ML models, which are more effective in understanding the patterns in the data at large scale. These models are referred to as custom ML models.

The lifecycle of shipping intelligent features or building custom ML models is called Team Data Science Process (TDSP) or Data Science Lifecyle. The stages of TDSP bring structure to the development process just like an application development lifecycle. The process starts with an understanding of the business requirements and ends with customer acceptance. In between are intermediate stages—like data acquisition and understanding, and modeling and deployment--that can be iterated as needed. You can learn more about the data science lifecycle at bit.ly/2XbmKj3.

In our example, the in-house data science team is creating a statistical modeling of past spends and forecasting future expense patterns to help the finance department with planning next fiscal year. The data science experts sample data from an expense data store and perform Exploratory Data Analysis (EDA) using famil­iar tools. The resulting custom ML model is tuned to specific requirements and then learns from the gathered data.

Relevant Services: Azure Data Science Virtual Machine (DSVM), Azure Notebooks, Azure ML Services, Azure ML Studio

DSVM is a templated image of OS images built to speed up development of custom ML experiments, while Azure Notebooks and Visual Studio can provide a great environment for running data science experiment projects. Azure ML Services helps teams of data scientists to collaborate and maintain their models using Notebook VMs, code and compute environments, and manage the application lifecycle. Automated ML Service is also an option to generate high-performing models. You can also use Azure ML Studio if you’d rather experiment using a visual drag-and-drop interface. Azure Notebooks are another way to host Jupyter notebooks publicly or privately that can still connect to your ML compute for training and deployment.

Target audience: Data scientists and data scientist teams, developers, analysts, architects

Hands-on Samples:

Train the Custom Model at Scale with Actual Past Data and Various Data Sources

In this scenario, the data scientists in your organization want to run their models on actual data to understand the accuracy of their forecasts. One key factor here is to minimize the training time as much as possible.

Relevant Services: Azure Databricks, Azure ML Services, Azure Data Lake

Usually experimentation happens on sampled data to create custom ML models. After the model is developed with sample data, it’s trained with actual data, which might be gigabytes or terabytes in size. Distributed computing and Big Data storage are essential for optimal performance when learning with data at this scale. Azure ML Services helps you to choose compute targets for training at scale, ranging from a GPU virtual machine (VM) to an Azure Kubernetes cluster. If there are Big Data experts on the team, you might choose to run the training-at-scale scenarios using Azure Databricks for the compute layer (for Spark/MMLSpark deployments) or using Azure Data Lake Analytics (if the data resides in Azure Data Lake).

Azure Databricks also provides a data-engineering platform to help with data ingestion, ETL and orchestration tasks.

Target audience: Data scientists, data engineers, data developers

Hands-on Samples:

  • Quickstart: Run a Spark job on Azure Databricks using the Azure Portal (bit.ly/2EevltO)
  • Tutorial: Extract, transform, and load data by using Azure Databricks - bit.ly/30pj3bs

Package the Model and Deploy It for Use by Apps

The custom machine model is ready! While there are various ways to evaluate, most applications will interact with models through an API. Moreover, the hosting platform (OS) of your customer environment can vary based on a lot of factors. Hence, it’s important for your DevOps teams and ML teams to work closely to ensure that models are accessible, available and can run cross-platform.

Relevant Services: Azure IOT Edge, Azure Container Instances, Azure ML Services

Container platforms have gained a lot of momentum in application hosting scenarios as they package apps and dependencies in one portable unit. Containers provide flexibility of model packaging in cloud, hybrid, edge or private hosting scenarios. Here you can use Azure ML Services to deploy the model files to a multitude of environments, such as Local Web Service, Azure IOT Edge, Azure Kubernetes Service, Azure Container Instances or Azure ML Compute. AI models exported from Computer Vision can be run as containers on various OS and orchestration platforms.

Target audience: Application developers, DevOps engineers, data scientists

Hands-on Samples:

Consume Models on and Develop Model Apps with ML.NET

Most of your Contoso line-of-business apps have been written in .NET Core, while some solutions run on the edge or as mobile apps. This is where the application teams and architects need to load the ML models developed by data science teams into the mobile and edge apps.

Relevant Services: ML.NET, Azure Cognitive Services

ML.NET is a .NET Core-based, cross-platform framework for developers familiar with .NET to start developing ML applications in their language of choice. You can load ONNX and TensorFlow models in your .NET Core apps through ML.NET. You can also create a model from data, save it and then load it for evaluation in other ML.NET apps. For full-stack application developers, there’s an option to infuse AI into apps through a wide range of Azure Cognitive Services.

Models exported from some Cognitive Services can also be loaded into iOS and Android apps.

Target audience: Application developers, .NET developers, mobile applications developers, AI developers

Hands-on Samples:

  • Use ML.NET to classify handwritten digits from zero to nine using the MNIST dataset (bit.ly/2EfpzZ4)
  • Comprehensive Cognitive Services SDK sample (bit.ly/30pPTcf)

Provide Interactive Experience and Insight Through an Enterprise Chatbot

Filing expenses is a mundane task that only gets worse with repetition. Bots powered with AI capabilities can lessen the sting, reducing the time that employees spend inputting data. These bots are created with Azure Bot Service and are Web apps that you can extend using Computer Vision API to accept, analyze and perform OCR on the receipts. You can then store this information to build insights in the longer run.

Relevant Services: Azure Bot Service, Language Understanding Intelligence Service (LUIS), Virtual Assistant, Speech Services, QnA Maker

Bots powered with Azure Bot Service along with LUIS and the Computer Vision cognitive service can provide a conversational experience to users filing expenses. The addition of the Speech cognitive service allows for building a natural speech-enabled bot that people can use to provide input using voice. You can also explore the open source Virtual Assistant solution to quickly get a fully functional bot running, which you can then customize to your requirements. Finally, the QnA Maker service can be used to provide assistance with first line of questioning coming from the users.

Target audience: Application developers, AI developers

Hands-on Samples:

  • Comprehensive Cognitive Services SDK sample (bit.ly/30pPTcf)
  • Quickstart: Recognize speech with the Speech SDK for .NET Core (bit.ly/2VvN7P1)
  • Virtual Assistance solution overview (bit.ly/2YyfKgf)
  • Quickstart: Create your first Virtual Assistant (C#) (bit.ly/2Hy69zc)
  • Quickstart: Create a knowledge base using the QnA Maker API service portal (bit.ly/2EhddzD)

Wrapping Up

This completes the coverage of the scenarios detailed in Figure 1 and provides an overview of the spread of Azure AI Services that apply in a typical AI feature integration and development scenario.

If you’re looking to add intelligent features in your apps and evaluating AI services available on Azure, you’ll need first to decide if you can use any of the Microsoft Cognitive Services, or if you need to build a custom ML model. In many cases, you can use a combination of Cognitive Services and custom ML models to achieve the desired end result.

The pre-built AI services from Microsoft Cognitive Services often meet the requirement for simpler scenarios, such as performing OCR on digital documents, categorizing images based on content, and adding conversational AI in your apps. Some of these services, such as Custom Vision Service and LUIS, also provide extensive customization to meet less-than-simple requirements.

Big Data analytics services such as Azure Databricks and Azure Data Warehouse, along with Azure ML service, can be used for complex scenarios at any scale. Azure ML with automated ML and ML pipelines can increase the productivity of the data science team in an organization by streamlining DevOps processes and reducing time for experimentation.

Now that you’ve explored Azure AI Services and walked through the expense modernization requirements at Contoso, take a moment to test your understanding by taking the online interactive quiz at ashisa.github.io/AzureAIQuiz.

Ashish Sahu is a senior technology evangelist, working with Microsoft India and helping ISVs and startups overcome technical challenges, adopt the latest technologies, and evolve their solutions to the next level.

Sagar Bhanudas Joshi has been working with developers to help embrace the Universal Windows Platform and the Microsoft Azure platform for more than six years. His current role includes helping SaaS companies architect, design and on-board solutions to Microsoft Cloud Platform, with special focus on ML and AI.

THANKS to the following Microsoft technical expert for reviewing this article: Sandeep Alur

Discuss this article in the MSDN Magazine forum