Recently, everyone seems to have been preoccupied with generative AI, of which OpenAI’s ChatGPT, with its ability write articles and code and even solve math problems, has definitely had the strongest impact. As Microsoft has had a long standing partnership with OpenAI, in this article I will discuss how OpenAI services and models will be integrated into Microsoft’s products and services and show their location in the Microsoft Azure and Power Platform service architecture.
Introduction—types of artificial intelligence
At its core, AI is about creating intelligent machines that can perform tasks that would normally require human intelligence, such as understanding natural language, recognizing images, and making decisions. There are several subsets of AI, including machine learning, deep learning, and generative AI—see Figure 1.
- Machine learning is a subset of AI that involves training algorithms to learn from data and make predictions or decisions. This approach allows AI models to improve their performance over time as they are exposed to more data.
- Deep learning is a type of machine learning that uses neural networks to model complex patterns in data. These networks are composed of multiple layers of interconnected nodes, allowing them to learn and represent increasingly abstract features of the data.
- Generative AI is a type of deep learning that can produce new content or data based on existing data. This approach allows machines to generate new, original content that is similar to the input data, but not identical. For example, a generative natural language model could be trained on a dataset of texts and then generate new texts that are similar in style and content to the input texts.
Fig. 1 The subsets of AI
Beyond GPT 3—available Open AI models
Many Azure OpenAI services are built on deep learning models, while Open AI offers a set of generative AI models that can produce language, code, and images. These are the families of models provided by Open AI:
- GPT 4—All GPT models can understand and generate natural language or code. GPT 4 is an improvement over the publicly released GPT 3.5 model and is one of the most advanced AI models available.
- GPT-3.5—A set of models that improve on the less powerful GPT-3. It is possibly the most commonly used of all AI models, as it has been made available to the public through OpenAI’s chat service as part of the organization’s AI research.
- GPT-3—A precursor to the GPT-3.5 and GPT-4 models.
- DALL-E—A model that can generate and edit images based on natural language prompts.
- Whisper—A model that can convert sound to text.
- Embeddings—A set of models that can convert text into a numerical form.
- Moderation—A refined model that can detect whether text may be sensitive or dangerous.
- Codex—A set of models that can understand and generate code, including translating natural language into code.
Benefits and risks associated with AI
Generative AI technology offers many potential benefits for humanity, such as increasing creativity and innovation, improving quality and efficiency, and supporting education and culture. For example, a teacher can use generative models to generate questions based on some text or exercises in mere minutes, while a writer can get a rough draft of an article or blog post. Generative AI can help create new content or solve problems in an original, creative, and unconventional way. It can also help automate or improve tasks or processes that require large amounts of data or time.
On the other hand, we must also look at generative AI technology in the context of potential threats and challenges, such as violating copyright and ethical standards, losing control and trust, and reducing value and meaning. For example, generative AI may violate the copyrights or ethics of the creators of original content or data if it does not properly indicate its sources or obtain consent. Chat GPT 3.5 in its publicly available version does not cite sources, while image generation models, such as DALL-E and Stable Diffusion, have been accused of violating the copyright of artists whose style they imitate.
AI may also create false information—even inadvertently, as generative models are prone to hallucinate, especially if information in their training data is lacking. Bad actors may also use generative AI models to create content that is harmful to other people or the environment. Additionally, public AI models may collect a lot of sensitive information, such as personally identifying information, user account information, user content, and social media information.
Confidentiality and OpenAI
When using and interacting with AI services, users automatically send information such as log data, usage data, location data and mapping, device information, cookies, and analytics. Open AI has a privacy policy that outlines how they collect, use, and share user data and mentions several data use cases, including administration, maintenance, and analysis of the service; service improvement and research; user communication; development of new programs and services; fraud prevention and security; and protection of IT infrastructure, architecture, and networks. Data may also be used to ensure compliance with legal obligations and court processes, as well as for the protection of rights, privacy, security, or property of their companies.
It is important for users to be aware of the potential risks associated with sharing their data with Open AI and to take steps to protect their privacy. For example, users can opt-out of sharing certain data with Open AI or take other measures to limit the amount of data they share. It is also important for users to be aware of the potential risks associated with using AI models that have been trained on large amounts of data, as these models may inadvertently reveal sensitive information about individuals or groups.
The EU AI Act
In this context it is worth commenting on the proposed European law on artificial intelligence, commonly known as the EU AI Act, which assigns applications of AI to three risk categories:
- Applications and systems that create an unacceptable risk, such as those involved with government-run social scoring of the type used in China, which are banned.
- High-risk applications, such as a CV-scanning tool that ranks job applicants, which are subject to specific legal requirements.
- Applications that are not explicitly banned or listed as high-risk are largely left
This law could become a global standard, determining the extent to which AI has a positive rather than negative effect on our lives.
What is the Azure OpenAI Service?
Acknowledging the many risks and issues associated with AI development, Microsoft has stated its commitment to responsible AI and data handling protections within its new Azure OpenAI Service. What distinguishes the use of AI models through Azure OpenAI is that the service does not use customer data to train public Microsoft AI models. Data is stored only in the context of a given trained model, known as the fine-tuned model.
What, then, is this new Azure OpenAI Service? Azure OpenAI provides REST API access to powerful Open AI language models, including the GPT-3, Codex, and Embeddings series. The ChatGPT model, gpt-35-turbo, and the GPT-4 models, gpt-4 and gpt-4-32k, are now available in Azure OpenAI Service in preview. These large language models can be easily customized for specific tasks, including generating content, summarizing it, performing semantic search, and translating natural language into code. Users can access the service via REST API, Python SDK, or the Azure OpenAI Studio web interface.
Access to OpenAI’s models within the Azure OpenAI Service is available to users with an Azure account and subscription. It consists of four components:
- pre-trained generative AI models,
- customization capabilities for fine-tuning AI models using your own data,
- built-in tools for detecting and limiting harmful use cases, so that users can responsibly deploy AI,
- corporate-class security through role-based access control (RBAC) and private networks.
Using Azure OpenAI allows users to work with AI models securely and at scale while taking advantage of Azure’s private networks, regional availability, and responsible AI content filtering. The models made available through Microsoft Azure can be divided into three broad categories:
- Generating natural language, which involves summarizing complex text for different reading levels, suggesting alternative sentence formulations, and more.
- Generating code, which includes translating code from one programming language to another, identifying and troubleshooting code errors, and more.
- Generating images, which involves generating images for publication based on textual descriptions and much more.
Data security in the Azure OpenAI Service
After deploying a model (consisting of a fine-tuned client model or base model endpoint) in Azure OpenAI, it is possible to send text prompts to the model via the REST API, client library, or Azure OpenAI Studio. The model generates textual outputs in response, which are returned through the API. Content filters are applied to both input prompts and generated output, ensuring that they are not stored or used for training or improving the models. Additionally, Azure OpenAI incorporates a content management system that filters out potentially harmful content by processing input and generated output through a team of classification models—see Figure 2.
Fig. 2 Azure OpenAI Service boundary
Here is how the data is stored and what control clients have over it:
- Training, validation, and training results are stored in Azure storage, encrypted with Microsoft Managed keys, and can be deleted by the user.
- Fine-tuned OpenAI models are stored in Azure Storage, encrypted, and can be deleted by the user.
- Prompts and outputs are only temporarily stored by Azure OpenAI (for up to 30 days), are encrypted, and are accessible only to authorized Microsoft employees for debugging and abuse pattern analysis. However, if a client is approved for modified abuse monitoring, prompt and completion data is not stored and Microsoft employees do not have access to that data.
Azure Cognitive Services
The Microsoft Azure OpenAI Service is a part of Azure Cognitive Services, which are a collection of AI-based services that help developers add intelligent features to their applications without direct AI or data science skills or knowledge. These services are available through REST APIs and client SDK libraries in popular programming languages. Azure Cognitive Services includes a range of AI services for tasks such as vision, speech, language, and decision-making.
- Vision services allow for the processing of images and the return of information using advanced cognitive algorithms.
- Speech services allow for the addition of speech-based features to applications.
- Language services allow for the understanding and analysis of text using various natural language processing (NLP) functions.
- Decision-making services allow for the monitoring and detection of anomalies in time series data, content moderation, and selecting the best experience to show to users, learning from their behavior in real-time.
- The Azure OpenAI Service provides powerful language models, including the GPT series, Codex, and Embeddings.
Azure Cognitive Services is a powerful tool for professionals looking to leverage AI capabilities in their applications, providing secure support for private networks and endpoints and offering a layered security model. Access to resources can be restricted using request filtering based on IP addresses, IP ranges, or Azure Virtual Networks subnets. Authentication is managed through Azure Active Directory (Azure AD) service credentials or a valid API key.
It is worth noting that the Azure Open AI service, being part of Azure Cognitive Services, also supports VNets and private endpoints, enabling secure network-based authentication. A secure login architecture could be structured as follows—see Figure 3:
- Client applications access the Azure OpenAI Service endpoints for text generation or model training.
- The Azure Application Gateway provides a single access point to Azure OpenAI Service models and offers load balancing for API interfaces.
- Azure API Management enables security control, inspection, and monitoring of Azure OpenAI Service models.
- API Management connects to all Azure platform resources through Azure Private Link, ensuring enhanced security for all traffic through private endpoints and encapsulating it within a private network.
Fig. 3 A secure login architecture for Azure OpenAI Service
Azure OpenAI Service in MS Power Platform
Azure OpenAI Service is also available in Microsoft Power Platform, a suite of low-code tools to build automation and business apps.
In fact, Microsoft has been consistently infusing AI capabilities into its low-code platform for several years. Back in May 2021, they added a feature based on GPT-3 to Power Apps, making it possible to translate language prompt into Power Fx formulas. In 2022, the preview of express design in Power Apps was launched, making it possible to instantly build low-code apps with design files (e.g., from Figma) and images (e.g., PDFs or sketches on a whiteboard) as an input. In the same year, Power Automate gained the ability to understand natural language and created flows using AI models like GPT-3.
In 2023, the process of building AI capabilities in MS Power Platform has accelerated. Power Virtual Agents got a conversation booster, allowing users to build chat bots based on their company-specific resources, like a website, FAQs, or internal knowledge base. AI Builder, a tool giving access to AI capabilities in low-code apps, can now use the GPT-3 model to create text based on different input data in the apps created via Power Apps. This feature is still in the preview mode but it has huge potential. It can be used in tools like Power Apps, Power Automate, or Power Virtual Agents to fill in forms, generate reports, write emails, summarize text or meetings, or build intelligent chatbots.
Another important new feature is Copilot, which brings AI-powered assistance into Power Apps, Power Virtual Agents, and Power Automate. Copilot provides makers with a live in-studio assistant that helps them build solutions and provides suggestions for improvement. To build an app, flow, or bot, you can describe it using natural language and Copilot can build it in seconds. Copilot is designed to accelerate development for both citizen and professional developers alike. It is built on the Azure OpenAI Service with GPT and is aligned with Microsoft’s responsible AI principles.
Of course, as we already know, Azure OpenAI is part of the Azure Cognitive Services that are also available in MS Power Platform.
Azure OpenAI Service—potential use cases
Some of the potential use cases for the Azure OpenAI Service include:
#1 Customer support chatbots: Use large language models in the Azure OpenAI Service to create a company-specific chatbot that can answer customers’ questions about products and pricing, solutions, and support services.
#2 Summarizing documents: Use the large language models in the Azure OpenAI Service to create a tool that can summarize large amounts of documentation in concise, natural language.
#3 Analyzing CVs: Use the Azure OpenAI Service to analyze and summarize résumés and cover letters for HR departments, speeding up the recruitment process.
#4 Internal company search engine: Use the Azure OpenAI Service to create an advanced search engine for internal company resources, allowing employees to easily find the information they need.
#5 Coding and language models: Use AI large language models in the Azure OpenAI Service to convert natural language into code, allowing users to create cutting edge applications without knowing how to write computer code.
#6 Email management: Using the Azure OpenAI Service to categorize, prioritize, and generate responses to incoming emails, saving employees time.
#7 Content generation: Use the Azure OpenAI Service to produce text in natural language, such as reports, emails, newsletters, or messages to customers.
These are just a few examples of the many potential use cases for the Azure OpenAI Service. With access to advanced AI models and enterprise-grade security and compliance features, the possibilities are virtually endless.
Azure OpenAI Service—conclusion
In conclusion, the Azure OpenAI Service is a powerful tool that brings the power of generative AI to users of the Microsoft Azure environment. With access to some of the most advanced AI models, including GPT-3 and DALL-E, developers and business executives alike can create innovative solutions for various problems. From the HR and marketing team, through software developers, partners, and employees, to the corporate vice president, access to OpenAI’s most advanced AI models through the Azure OpenAI Service allows for various tools to be used to improve and modernize processes in any business or organization.
