-2. [Use Azure Storage Explorer to manage Azure managed disks](../virtual-machines/disks-use-storage-explorer-managed-disks.md) and access permissions.

-If you have Software Assurance, you can enable the [Azure Hybrid Benefit](../virtual-machines/windows/hybrid-use-benefit-licensing.md). You can see potential savings using [Azure Hybrid Benefit Calculator](https://azure.microsoft.com/pricing/hybrid-benefit/#calculator).

-[Azure Hybrid Benefit for Linux](../virtual-machines/linux/azure-hybrid-benefit-linux.md) is a licensing benefit that helps you to significantly reduce the costs of running your Red Hat Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SLES) virtual machines (VMs) in the cloud.

-This query identifies Virtual Machines that aren't properly deallocated. If a virtual machineΓÇÖs status is Stopped rather than Stopped (Deallocated), you're still billed for the resource as the hardware remains allocated for you. Learn more about [States and billing status of Azure Virtual Machines](../virtual-machines/states-billing.md).

-We evaluate if workloads are eligible to run on specialized SKUs called **Burstable SKUs** that support variable workload performance requirements and are less expensive than general purpose SKUs. Learn more about burstable SKUs here: [B-series burstable - Azure Virtual Machines](../virtual-machines/sizes-b-series-burstable.md).

-Learn more about [Virtual machine - AzureStorageVmUltraDisk (Take advantage of Ultra Disk low latency for your log disks and improve your database workload performance.)](../virtual-machines/disks-enable-ultra-ssd.md?tabs=azure-portal).

-Learn how to deploy the Azure AI services [Anomaly Detector](../anomaly-detector-container-howto.md) container to Azure [Container Instances](../../../container-instances/index.yml). This procedure demonstrates the creation of an Anomaly Detector resource. Then we discuss pulling the associated container image. Finally, we highlight the ability to exercise the orchestration of the two from a browser. Using containers can shift the developers' attention away from managing infrastructure to instead focusing on application development.

-* Export the tested LUIS app version into a [LUIS docker container](./luis-container-howto.md?tabs=v3) and deploy the LUIS container to Azure [Container instances](../../container-instances/index.yml).

-**Docker Engine**: You must have Docker Engine installed locally. Docker provides packages that configure the Docker environment on [macOS](https://docs.docker.com/docker-for-mac/), [Linux](https://docs.docker.com/engine/installation/#supported-platforms), and [Windows](https://docs.docker.com/docker-for-windows/). On Windows, Docker must be configured to support Linux containers. Docker containers can also be deployed directly to [Azure Kubernetes Service](/azure/aks/) or [Azure Container Instances](../container-instances/index.yml).

-In our example, we utilize an [NCv3 series VM](../../virtual-machines/ncv3-series.md) that has one v100 GPU.

-An Azure Virtual Machine with a GPU can also be used to run Spatial Analysis. The example below will use a [NCv3 series VM](../../virtual-machines/ncv3-series.md) that has one v100 GPU.

-Once the extension is successfully applied, navigate to the VM main page in the Azure portal and select `Connect`. The VM can be accessed either through SSH or RDP. RDP is helpful as it enables viewing of the visualizer window (explained later). Configure the RDP access by following [these steps](../../virtual-machines/linux/use-remote-desktop.md) and opening a remote desktop connection to the VM.

-With the following steps, scale Azure AI services applications in the cloud easily with Azure [Container Instances](../../container-instances/index.yml). Containerization helps you focus on building your applications instead of managing the infrastructure. For more information on using containers, see [features and benefits](../cognitive-services-container-support.md#features-and-benefits).

-* [Azure Container Instances](../../../container-instances/index.yml).

-> You may need to enable TLS for your web app in order to view it at its `https` address. Follow the instructions in [Enable a TLS endpoint](../../container-instances/container-instances-container-group-ssl.md) to set up a sidecar container than enables TLS/SSL for your web app.

-1. On the **Role** tab, search for and select the **Storage Blob Data Reader** permission and select **Next**.

- **Resolution**: Make sure you grant your Document Intelligence managed identity the role of **Storage Blob Data Reader** and enabled **Trusted services** access or **Resource instance** rules on the networking tab.

-* Get started with Azure AI Document Intelligence latest preview version (2024-02-29-preview).

-description: Learn how to set up and use Document Intelligence Studio to test features of Azure AI Document Intelligence on the web.

- - ignite-2023

-<!-- markdownlint-disable MD033 -->

-# What is Document Intelligence Studio?

-> [!IMPORTANT]

->

-> * There are separate URLs for Document Intelligence Studio sovereign cloud regions.

-> * Azure for US Government: [Document Intelligence Studio (Azure Fairfax cloud)](https://formrecognizer.appliedai.azure.us/studio)

-> * Microsoft Azure operated by 21Vianet: [Document Intelligence Studio (Azure in China)](https://formrecognizer.appliedai.azure.cn/studio)

-[Document Intelligence Studio](https://documentintelligence.ai.azure.com/studio/) is an online tool to visually explore, understand, train, and integrate features from the Document Intelligence service into your applications. The studio provides a platform for you to experiment with the different Document Intelligence models and sample returned data in an interactive manner without the need to write code. Use the Document Intelligence Studio to:

-## Authorization policies

-## Document Intelligence model support

-## Try a Document Intelligence model

-### View resource details

-## Analyze options

-### Auto label documents with prebuilt models or one of your own models

-### Auto label tables

-### Add test files directly to your training dataset

-### Make use of the document list options and filters in custom projects

-### Project sharing

-## Troubleshooting

-### Manage third-party settings for Studio access

-* Go to **Settings** for Edge

-* Add exception for **https://documentintelligence.ai.azure.com** or the Document Intelligence Studio URL of your environment

- "projectKind": "CustomMultiLabelClassification",

- "storageInputContainerName": "{CONTAINER-NAME}",

- "projectName": "{PROJECT-NAME}",

- "multilingual": false,

- "description": "Project-description",

- "language": "en-us"

- "projectKind": "CustomMultiLabelClassification",

- "classes": [

- {

- "category": "Class1"

- },

- {

- "category": "Class2"

- }

- ],

- "documents": [

- {

- "location": "{DOCUMENT-NAME}",

- "language": "{LANGUAGE-CODE}",

- "dataset": "{DATASET}",

- "classes": [

- {

- "category": "Class1"

- },

- {

- "category": "Class2"

- }

- ]

- }

- ]

- }

- "projectKind": "CustomSingleLabelClassification",

- "storageInputContainerName": "{CONTAINER-NAME}",

- "settings": {},

- "projectName": "{PROJECT-NAME}",

- "multilingual": false,

- "description": "Project-description",

- "language": "en-us"

- "projectKind": "CustomSingleLabelClassification",

- "classes": [

- {

- "category": "Class1"

- },

- {

- "category": "Class2"

- }

- ],

- "documents": [

- {

- "location": "{DOCUMENT-NAME}",

- "language": "{LANGUAGE-CODE}",

- "dataset": "{DATASET}",

- "class": {

- "category": "Class2"

- }

- },

- {

- "location": "{DOCUMENT-NAME}",

- "language": "{LANGUAGE-CODE}",

- "dataset": "{DATASET}",

- "class": {

- "category": "Class1"

- }

- }

- ]

- }

-See the [ACI regional support](../../../../container-instances/container-instances-region-availability.md) article for availability information.

-* Store your embeddings and perform vector (similarity) search using your choice of Azure service:

-## Text streaming

-To use the text streaming feature, connect to the websocket V2 endpoint: `wss://{region}.tts.speech.microsoft.com/cognitiveservices/websocket/v2`

-* [Azure Container Instances](~/articles/container-instances/index.yml).

-| `value` | The requested duration of the output audio in either seconds, such as `2s`, or milliseconds, such as `2000ms`.<br/><br/>This value should be within `0.5` to `2` times the original audio without any other rate settings. For example, if the requested duration of your audio is `30s`, then the original audio must otherwise be between 15 and 60 seconds. If you set a value outside of these boundaries, the duration is set according to the respective minimum or maximum multiple.<br/><br/>Given your requested output audio duration, the Speech service adjusts the speaking rate accordingly. Use the [voice list](rest-text-to-speech.md#get-a-list-of-voices) API and check the `WordsPerMinute` attribute to find out the speaking rate of the neural voice that you're using. You can divide the number of words in your input text by the value of the `WordsPerMinute` attribute to get the approximate original output audio duration. The output audio sounds most natural when you set the audio duration closest to the estimated duration.| Required |

-The default content filtering configuration for the GPT model series is set to filter at the medium severity threshold for all four content harm categories (hate, violence, sexual, and self-harm) and applies to both prompts (text, multi-modal text/image) and completions (text). This means that content that is detected at severity level medium or high is filtered, while content detected at severity level low isn't filtered by the content filters. For DALL-E, the default severity threshold is set to low for both prompts (text) and completions (images), so content detected at severity levels low, medium, or high is filtered. The configurability feature is available in preview and allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels as described in the table below:

-description: Learn how to use Jamba-Instruct chat models with Azure AI Studio.

-zone_pivot_groups: azure-ai-model-catalog-samples-chat

-# How to use Jamba-Instruct chat models

-In this article, you learn about Jamba-Instruct chat models and how to use them.

-The Jamba-Instruct model is AI21's production-grade Mamba-based large language model (LLM) which uses AI21's hybrid Mamba-Transformer architecture. It's an instruction-tuned version of AI21's hybrid structured state space model (SSM) transformer Jamba model. The Jamba-Instruct model is built for reliable commercial use with respect to quality and performance.

-> [!TIP]

-> See our announcements of AI21's Jamba-Instruct model available now on Azure AI Model Catalog through [AI21's blog](https://aka.ms/ai21-jamba-instruct-blog) and [Microsoft Tech Community Blog](https://aka.ms/ai21-jamba-instruct-announcement).

-## Jamba-Instruct chat models

-You can learn more about the models in their respective model card:

-* [AI21-Jamba-Instruct](https://aka.ms/azureai/landing/AI21-Jamba-Instruct)

-## Prerequisites

-To use Jamba-Instruct chat models with Azure AI Studio, you need the following prerequisites:

-### A model deployment

-**Deployment to serverless APIs**

-Jamba-Instruct chat models can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need.

-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).

-> [!div class="nextstepaction"]

-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)

-### The inference package installed

-You can consume predictions from this model by using the `azure-ai-inference` package with Python. To install this package, you need the following prerequisites:

-* Python 3.8 or later installed, including pip.

-* The endpoint URL. To construct the client library, you need to pass in the endpoint URL. The endpoint URL has the form `https://your-host-name.your-azure-region.inference.ai.azure.com`, where `your-host-name` is your unique model deployment host name and `your-azure-region` is the Azure region where the model is deployed (for example, eastus2).

-* Depending on your model deployment and authentication preference, you need either a key to authenticate against the service, or Microsoft Entra ID credentials. The key is a 32-character string.

-

-Once you have these prerequisites, install the Azure AI inference package with the following command:

-```bash

-pip install azure-ai-inference

-```

-Read more about the [Azure AI inference package and reference](https://aka.ms/azsdk/azure-ai-inference/python/reference).

-## Work with chat completions

-In this section, you use the [Azure AI model inference API](https://aka.ms/azureai/modelinference) with a chat completions model for chat.

-> [!TIP]

-> The [Azure AI model inference API](https://aka.ms/azureai/modelinference) allows you to talk with most models deployed in Azure AI Studio with the same code and structure, including Jamba-Instruct chat models.

-### Create a client to consume the model

-First, create the client to consume the model. The following code uses an endpoint URL and key that are stored in environment variables.

-```python

-import os

-from azure.ai.inference import ChatCompletionsClient

-from azure.core.credentials import AzureKeyCredential

-client = ChatCompletionsClient(

- endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],

- credential=AzureKeyCredential(os.environ["AZURE_INFERENCE_CREDENTIAL"]),

-)

-```

-### Get the model's capabilities

-The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:

-```python

-model_info = client.get_model_info()

-```

-The response is as follows:

-```python

-print("Model name:", model_info.model_name)

-print("Model type:", model_info.model_type)

-print("Model provider name:", model_info.model_provider_name)

-```

-```console

-Model name: AI21-Jamba-Instruct

-Model type: chat-completions

-Model provider name: AI21

-```

-### Create a chat completion request

-The following example shows how you can create a basic chat completions request to the model.

-```python

-from azure.ai.inference.models import SystemMessage, UserMessage

-response = client.complete(

- messages=[

- SystemMessage(content="You are a helpful assistant."),

- UserMessage(content="How many languages are in the world?"),

- ],

-)

-```

-The response is as follows, where you can see the model's usage statistics:

-```python

-print("Response:", response.choices[0].message.content)

-print("Model:", response.model)

-print("Usage:")

-print("\tPrompt tokens:", response.usage.prompt_tokens)

-print("\tTotal tokens:", response.usage.total_tokens)

-print("\tCompletion tokens:", response.usage.completion_tokens)

-```

-```console

-Response: As of now, it's estimated that there are about 7,000 languages spoken around the world. However, this number can vary as some languages become extinct and new ones develop. It's also important to note that the number of speakers can greatly vary between languages, with some having millions of speakers and others only a few hundred.

-Model: AI21-Jamba-Instruct

-Usage:

- Prompt tokens: 19

- Total tokens: 91

- Completion tokens: 72

-```

-Inspect the `usage` section in the response to see the number of tokens used for the prompt, the total number of tokens generated, and the number of tokens used for the completion.

-#### Stream content

-By default, the completions API returns the entire generated content in a single response. If you're generating long completions, waiting for the response can take many seconds.

-You can _stream_ the content to get it as it's being generated. Streaming content allows you to start processing the completion as content becomes available. This mode returns an object that streams back the response as [data-only server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events). Extract chunks from the delta field, rather than the message field.

-```python

-result = client.complete(

- messages=[

- SystemMessage(content="You are a helpful assistant."),

- UserMessage(content="How many languages are in the world?"),

- ],

- temperature=0,

- top_p=1,

- max_tokens=2048,

- stream=True,

-)

-```

-To stream completions, set `stream=True` when you call the model.

-To visualize the output, define a helper function to print the stream.

-```python

-def print_stream(result):

- """

- Prints the chat completion with streaming.

- """

- import time

- for update in result:

- if update.choices:

- print(update.choices[0].delta.content, end="")

-```

-You can visualize how streaming generates content:

-```python

-print_stream(result)

-```

-#### Explore more parameters supported by the inference client

-Explore other parameters that you can specify in the inference client. For a full list of all the supported parameters and their corresponding documentation, see [Azure AI Model Inference API reference](https://aka.ms/azureai/modelinference).

-```python

-from azure.ai.inference.models import ChatCompletionsResponseFormat

-response = client.complete(

- messages=[

- SystemMessage(content="You are a helpful assistant."),

- UserMessage(content="How many languages are in the world?"),

- ],

- presence_penalty=0.1,

- frequency_penalty=0.8,

- max_tokens=2048,

- stop=["<|endoftext|>"],

- temperature=0,

- top_p=1,

- response_format={ "type": ChatCompletionsResponseFormat.TEXT },

-)

-```

-> [!WARNING]

-> Jamba doesn't support JSON output formatting (`response_format = { "type": "json_object" }`). You can always prompt the model to generate JSON outputs. However, such outputs are not guaranteed to be valid JSON.

-If you want to pass a parameter that isn't in the list of supported parameters, you can pass it to the underlying model using *extra parameters*. See [Pass extra parameters to the model](#pass-extra-parameters-to-the-model).

-### Pass extra parameters to the model

-The Azure AI Model Inference API allows you to pass extra parameters to the model. The following code example shows how to pass the extra parameter `logprobs` to the model.

-Before you pass extra parameters to the Azure AI model inference API, make sure your model supports those extra parameters. When the request is made to the underlying model, the header `extra-parameters` is passed to the model with the value `pass-through`. This value tells the endpoint to pass the extra parameters to the model. Use of extra parameters with the model doesn't guarantee that the model can actually handle them. Read the model's documentation to understand which extra parameters are supported.

-```python

-response = client.complete(

- messages=[

- SystemMessage(content="You are a helpful assistant."),

- UserMessage(content="How many languages are in the world?"),

- ],

- model_extras={

- "logprobs": True

- }

-)

-```

-### Apply content safety

-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.

-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.

-```python

-from azure.ai.inference.models import AssistantMessage, UserMessage, SystemMessage

-try:

- response = client.complete(

- messages=[

- SystemMessage(content="You are an AI assistant that helps people find information."),

- UserMessage(content="Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."),

- ]

- )

- print(response.choices[0].message.content)

-except HttpResponseError as ex:

- if ex.status_code == 400:

- response = ex.response.json()

- if isinstance(response, dict) and "error" in response:

- print(f"Your request triggered an {response['error']['code']} error:\n\t {response['error']['message']}")

- else:

- raise

- raise

-```

-> [!TIP]

-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).

-## Jamba-Instruct chat models

-You can learn more about the models in their respective model card:

-* [AI21-Jamba-Instruct](https://aka.ms/azureai/landing/AI21-Jamba-Instruct)

-## Prerequisites

-To use Jamba-Instruct chat models with Azure AI Studio, you need the following prerequisites:

-### A model deployment

-**Deployment to serverless APIs**

-Jamba-Instruct chat models can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need.

-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).

-> [!div class="nextstepaction"]

-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)

-### The inference package installed

-You can consume predictions from this model by using the `@azure-rest/ai-inference` package from `npm`. To install this package, you need the following prerequisites:

-* LTS versions of `Node.js` with `npm`.

-* The endpoint URL. To construct the client library, you need to pass in the endpoint URL. The endpoint URL has the form `https://your-host-name.your-azure-region.inference.ai.azure.com`, where `your-host-name` is your unique model deployment host name and `your-azure-region` is the Azure region where the model is deployed (for example, eastus2).

-* Depending on your model deployment and authentication preference, you need either a key to authenticate against the service, or Microsoft Entra ID credentials. The key is a 32-character string.

-Once you have these prerequisites, install the Azure Inference library for JavaScript with the following command:

-```bash

-npm install @azure-rest/ai-inference

-```

-## Work with chat completions

-In this section, you use the [Azure AI model inference API](https://aka.ms/azureai/modelinference) with a chat completions model for chat.

-> [!TIP]

-> The [Azure AI model inference API](https://aka.ms/azureai/modelinference) allows you to talk with most models deployed in Azure AI Studio with the same code and structure, including Jamba-Instruct chat models.

-### Create a client to consume the model

-First, create the client to consume the model. The following code uses an endpoint URL and key that are stored in environment variables.

-```javascript

-import ModelClient from "@azure-rest/ai-inference";

-import { isUnexpected } from "@azure-rest/ai-inference";

-import { AzureKeyCredential } from "@azure/core-auth";

-const client = new ModelClient(

- process.env.AZURE_INFERENCE_ENDPOINT,

- new AzureKeyCredential(process.env.AZURE_INFERENCE_CREDENTIAL)

-);

-```

-### Get the model's capabilities

-The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:

-```javascript

-var model_info = await client.path("/info").get()

-```

-The response is as follows:

-```javascript

-console.log("Model name: ", model_info.body.model_name)

-console.log("Model type: ", model_info.body.model_type)

-console.log("Model provider name: ", model_info.body.model_provider_name)

-```

-```console

-Model name: AI21-Jamba-Instruct

-Model type: chat-completions

-Model provider name: AI21

-```

-### Create a chat completion request

-The following example shows how you can create a basic chat completions request to the model.

-```javascript

-var messages = [

- { role: "system", content: "You are a helpful assistant" },

- { role: "user", content: "How many languages are in the world?" },

-];

-var response = await client.path("/chat/completions").post({

- body: {

- messages: messages,

- }

-});

-```

-The response is as follows, where you can see the model's usage statistics:

-```javascript

-if (isUnexpected(response)) {

- throw response.body.error;

-}

-console.log("Response: ", response.body.choices[0].message.content);

-console.log("Model: ", response.body.model);

-console.log("Usage:");

-console.log("\tPrompt tokens:", response.body.usage.prompt_tokens);

-console.log("\tTotal tokens:", response.body.usage.total_tokens);

-console.log("\tCompletion tokens:", response.body.usage.completion_tokens);

-```

-```console

-Response: As of now, it's estimated that there are about 7,000 languages spoken around the world. However, this number can vary as some languages become extinct and new ones develop. It's also important to note that the number of speakers can greatly vary between languages, with some having millions of speakers and others only a few hundred.

-Model: AI21-Jamba-Instruct

-Usage:

- Prompt tokens: 19

- Total tokens: 91

- Completion tokens: 72

-```

-Inspect the `usage` section in the response to see the number of tokens used for the prompt, the total number of tokens generated, and the number of tokens used for the completion.

-#### Stream content

-By default, the completions API returns the entire generated content in a single response. If you're generating long completions, waiting for the response can take many seconds.

-You can _stream_ the content to get it as it's being generated. Streaming content allows you to start processing the completion as content becomes available. This mode returns an object that streams back the response as [data-only server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events). Extract chunks from the delta field, rather than the message field.

-```javascript

-var messages = [

- { role: "system", content: "You are a helpful assistant" },

- { role: "user", content: "How many languages are in the world?" },

-];

-var response = await client.path("/chat/completions").post({

- body: {

- messages: messages,

- }

-}).asNodeStream();

-```

-To stream completions, use `.asNodeStream()` when you call the model.

-You can visualize how streaming generates content:

-```javascript

-var stream = response.body;

-if (!stream) {

- stream.destroy();

- throw new Error(`Failed to get chat completions with status: ${response.status}`);

-}

-if (response.status !== "200") {

- throw new Error(`Failed to get chat completions: ${response.body.error}`);

-}

-var sses = createSseStream(stream);

-for await (const event of sses) {

- if (event.data === "[DONE]") {

- return;

- }

- for (const choice of (JSON.parse(event.data)).choices) {

- console.log(choice.delta?.content ?? "");

- }

-}

-```

-#### Explore more parameters supported by the inference client

-Explore other parameters that you can specify in the inference client. For a full list of all the supported parameters and their corresponding documentation, see [Azure AI Model Inference API reference](https://aka.ms/azureai/modelinference).

-```javascript

-var messages = [

- { role: "system", content: "You are a helpful assistant" },

- { role: "user", content: "How many languages are in the world?" },

-];

-var response = await client.path("/chat/completions").post({

- body: {

- messages: messages,

- presence_penalty: "0.1",

- frequency_penalty: "0.8",

- max_tokens: 2048,

- stop: ["<|endoftext|>"],

- temperature: 0,

- top_p: 1,

- response_format: { type: "text" },

- }

-});

-```

-> [!WARNING]

-> Jamba doesn't support JSON output formatting (`response_format = { "type": "json_object" }`). You can always prompt the model to generate JSON outputs. However, such outputs are not guaranteed to be valid JSON.

-If you want to pass a parameter that isn't in the list of supported parameters, you can pass it to the underlying model using *extra parameters*. See [Pass extra parameters to the model](#pass-extra-parameters-to-the-model).

-### Pass extra parameters to the model

-The Azure AI Model Inference API allows you to pass extra parameters to the model. The following code example shows how to pass the extra parameter `logprobs` to the model.

-Before you pass extra parameters to the Azure AI model inference API, make sure your model supports those extra parameters. When the request is made to the underlying model, the header `extra-parameters` is passed to the model with the value `pass-through`. This value tells the endpoint to pass the extra parameters to the model. Use of extra parameters with the model doesn't guarantee that the model can actually handle them. Read the model's documentation to understand which extra parameters are supported.

-```javascript

-var messages = [

- { role: "system", content: "You are a helpful assistant" },

- { role: "user", content: "How many languages are in the world?" },

-];

-var response = await client.path("/chat/completions").post({

- headers: {

- "extra-params": "pass-through"

- },

- body: {

- messages: messages,

- logprobs: true

- }

-});

-```

-### Apply content safety

-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.

-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.

-```javascript

-try {

- var messages = [

- { role: "system", content: "You are an AI assistant that helps people find information." },

- { role: "user", content: "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills." },

- ];

- var response = await client.path("/chat/completions").post({

- body: {

- messages: messages,

- }

- });

- console.log(response.body.choices[0].message.content);

-}

-catch (error) {

- if (error.status_code == 400) {

- var response = JSON.parse(error.response._content);

- if (response.error) {

- console.log(`Your request triggered an ${response.error.code} error:\n\t ${response.error.message}`);

- }

- else

- {

- throw error;

- }

- }

-}

-```

-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).

-## Jamba-Instruct chat models

-You can learn more about the models in their respective model card:

-* [AI21-Jamba-Instruct](https://aka.ms/azureai/landing/AI21-Jamba-Instruct)

-## Prerequisites

-To use Jamba-Instruct chat models with Azure AI Studio, you need the following prerequisites:

-### A model deployment

-**Deployment to serverless APIs**

-Jamba-Instruct chat models can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need.

-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).

-> [!div class="nextstepaction"]

-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)

-### The inference package installed

-You can consume predictions from this model by using the `Azure.AI.Inference` package from [Nuget](https://www.nuget.org/). To install this package, you need the following prerequisites:

-* The endpoint URL. To construct the client library, you need to pass in the endpoint URL. The endpoint URL has the form `https://your-host-name.your-azure-region.inference.ai.azure.com`, where `your-host-name` is your unique model deployment host name and `your-azure-region` is the Azure region where the model is deployed (for example, eastus2).

-* Depending on your model deployment and authentication preference, you need either a key to authenticate against the service, or Microsoft Entra ID credentials. The key is a 32-character string.

-Once you have these prerequisites, install the Azure AI inference library with the following command:

-```dotnetcli

-dotnet add package Azure.AI.Inference --prerelease

-```

-You can also authenticate with Microsoft Entra ID (formerly Azure Active Directory). To use credential providers provided with the Azure SDK, install the `Azure.Identity` package:

-```dotnetcli

-dotnet add package Azure.Identity

-```

-Import the following namespaces:

-```csharp

-using Azure;

-using Azure.Identity;

-using Azure.AI.Inference;

-```

-This example also use the following namespaces but you may not always need them:

-```csharp

-using System.Text.Json;

-using System.Text.Json.Serialization;

-using System.Reflection;

-```

-## Work with chat completions

-In this section, you use the [Azure AI model inference API](https://aka.ms/azureai/modelinference) with a chat completions model for chat.

-> [!TIP]

-> The [Azure AI model inference API](https://aka.ms/azureai/modelinference) allows you to talk with most models deployed in Azure AI Studio with the same code and structure, including Jamba-Instruct chat models.

-### Create a client to consume the model

-First, create the client to consume the model. The following code uses an endpoint URL and key that are stored in environment variables.

-```csharp

-ChatCompletionsClient client = new ChatCompletionsClient(

- new Uri(Environment.GetEnvironmentVariable("AZURE_INFERENCE_ENDPOINT")),

- new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_INFERENCE_CREDENTIAL"))

-);

-```

-### Get the model's capabilities

-The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:

-```csharp

-Response<ModelInfo> modelInfo = client.GetModelInfo();

-```

-The response is as follows:

-```csharp

-Console.WriteLine($"Model name: {modelInfo.Value.ModelName}");

-Console.WriteLine($"Model type: {modelInfo.Value.ModelType}");

-Console.WriteLine($"Model provider name: {modelInfo.Value.ModelProviderName}");

-```

-```console

-Model name: AI21-Jamba-Instruct

-Model type: chat-completions

-Model provider name: AI21

-```

-### Create a chat completion request

-The following example shows how you can create a basic chat completions request to the model.

-```csharp

-ChatCompletionsOptions requestOptions = new ChatCompletionsOptions()

-{

- Messages = {

- new ChatRequestSystemMessage("You are a helpful assistant."),

- new ChatRequestUserMessage("How many languages are in the world?")

- },

-};

-Response<ChatCompletions> response = client.Complete(requestOptions);

-```

-The response is as follows, where you can see the model's usage statistics:

-```csharp

-Console.WriteLine($"Response: {response.Value.Choices[0].Message.Content}");

-Console.WriteLine($"Model: {response.Value.Model}");

-Console.WriteLine("Usage:");

-Console.WriteLine($"\tPrompt tokens: {response.Value.Usage.PromptTokens}");

-Console.WriteLine($"\tTotal tokens: {response.Value.Usage.TotalTokens}");

-Console.WriteLine($"\tCompletion tokens: {response.Value.Usage.CompletionTokens}");

-```

-```console

-Response: As of now, it's estimated that there are about 7,000 languages spoken around the world. However, this number can vary as some languages become extinct and new ones develop. It's also important to note that the number of speakers can greatly vary between languages, with some having millions of speakers and others only a few hundred.

-Model: AI21-Jamba-Instruct

-Usage:

- Prompt tokens: 19

- Total tokens: 91

- Completion tokens: 72

-```

-Inspect the `usage` section in the response to see the number of tokens used for the prompt, the total number of tokens generated, and the number of tokens used for the completion.

-#### Stream content

-By default, the completions API returns the entire generated content in a single response. If you're generating long completions, waiting for the response can take many seconds.

-You can _stream_ the content to get it as it's being generated. Streaming content allows you to start processing the completion as content becomes available. This mode returns an object that streams back the response as [data-only server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events). Extract chunks from the delta field, rather than the message field.

-```csharp

-static async Task StreamMessageAsync(ChatCompletionsClient client)

-{

- ChatCompletionsOptions requestOptions = new ChatCompletionsOptions()

- {

- Messages = {

- new ChatRequestSystemMessage("You are a helpful assistant."),

- new ChatRequestUserMessage("How many languages are in the world? Write an essay about it.")

- },

- MaxTokens=4096

- };

- StreamingResponse<StreamingChatCompletionsUpdate> streamResponse = await client.CompleteStreamingAsync(requestOptions);

- await PrintStream(streamResponse);

-}

-```

-To stream completions, use `CompleteStreamingAsync` method when you call the model. Notice that in this example we the call is wrapped in an asynchronous method.

-To visualize the output, define an asynchronous method to print the stream in the console.

-```csharp

-static async Task PrintStream(StreamingResponse<StreamingChatCompletionsUpdate> response)

-{

- await foreach (StreamingChatCompletionsUpdate chatUpdate in response)

- {

- if (chatUpdate.Role.HasValue)

- {

- Console.Write($"{chatUpdate.Role.Value.ToString().ToUpperInvariant()}: ");

- }

- if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate))

- {

- Console.Write(chatUpdate.ContentUpdate);

- }

- }

-}

-```

-You can visualize how streaming generates content:

-```csharp

-StreamMessageAsync(client).GetAwaiter().GetResult();

-```

-#### Explore more parameters supported by the inference client

-Explore other parameters that you can specify in the inference client. For a full list of all the supported parameters and their corresponding documentation, see [Azure AI Model Inference API reference](https://aka.ms/azureai/modelinference).

-```csharp

-requestOptions = new ChatCompletionsOptions()

-{

- Messages = {

- new ChatRequestSystemMessage("You are a helpful assistant."),

- new ChatRequestUserMessage("How many languages are in the world?")

- },

- PresencePenalty = 0.1f,

- FrequencyPenalty = 0.8f,

- MaxTokens = 2048,

- StopSequences = { "<|endoftext|>" },

- Temperature = 0,

- NucleusSamplingFactor = 1,

- ResponseFormat = new ChatCompletionsResponseFormatText()

-};

-response = client.Complete(requestOptions);

-Console.WriteLine($"Response: {response.Value.Choices[0].Message.Content}");

-```

-> [!WARNING]

-> Jamba doesn't support JSON output formatting (`response_format = { "type": "json_object" }`). You can always prompt the model to generate JSON outputs. However, such outputs are not guaranteed to be valid JSON.

-If you want to pass a parameter that isn't in the list of supported parameters, you can pass it to the underlying model using *extra parameters*. See [Pass extra parameters to the model](#pass-extra-parameters-to-the-model).

-### Pass extra parameters to the model

-The Azure AI Model Inference API allows you to pass extra parameters to the model. The following code example shows how to pass the extra parameter `logprobs` to the model.

-Before you pass extra parameters to the Azure AI model inference API, make sure your model supports those extra parameters. When the request is made to the underlying model, the header `extra-parameters` is passed to the model with the value `pass-through`. This value tells the endpoint to pass the extra parameters to the model. Use of extra parameters with the model doesn't guarantee that the model can actually handle them. Read the model's documentation to understand which extra parameters are supported.

-```csharp

-requestOptions = new ChatCompletionsOptions()

-{

- Messages = {

- new ChatRequestSystemMessage("You are a helpful assistant."),

- new ChatRequestUserMessage("How many languages are in the world?")

- },

- AdditionalProperties = { { "logprobs", BinaryData.FromString("true") } },

-};

-response = client.Complete(requestOptions, extraParams: ExtraParameters.PassThrough);

-Console.WriteLine($"Response: {response.Value.Choices[0].Message.Content}");

-```

-### Apply content safety

-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.

-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.

-```csharp

-try

-{

- requestOptions = new ChatCompletionsOptions()

- {

- Messages = {

- new ChatRequestSystemMessage("You are an AI assistant that helps people find information."),

- new ChatRequestUserMessage(

- "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."

- ),

- },

- };

- response = client.Complete(requestOptions);

- Console.WriteLine(response.Value.Choices[0].Message.Content);

-}

-catch (RequestFailedException ex)

-{

- if (ex.ErrorCode == "content_filter")

- {

- Console.WriteLine($"Your query has trigger Azure Content Safety: {ex.Message}");

- }

- else

- {

- throw;

- }

-}

-```

-> [!TIP]

-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).

-## Jamba-Instruct chat models

-You can learn more about the models in their respective model card:

-* [AI21-Jamba-Instruct](https://aka.ms/azureai/landing/AI21-Jamba-Instruct)

-## Prerequisites

-To use Jamba-Instruct chat models with Azure AI Studio, you need the following prerequisites:

-### A model deployment

-**Deployment to serverless APIs**

-Jamba-Instruct chat models can be deployed to serverless API endpoints with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need.

-Deployment to a serverless API endpoint doesn't require quota from your subscription. If your model isn't deployed already, use the Azure AI Studio, Azure Machine Learning SDK for Python, the Azure CLI, or ARM templates to [deploy the model as a serverless API](deploy-models-serverless.md).

-> [!div class="nextstepaction"]

-> [Deploy the model to serverless API endpoints](deploy-models-serverless.md)

-### A REST client

-Models deployed with the [Azure AI model inference API](https://aka.ms/azureai/modelinference) can be consumed using any REST client. To use the REST client, you need the following prerequisites:

-* To construct the requests, you need to pass in the endpoint URL. The endpoint URL has the form `https://your-host-name.your-azure-region.inference.ai.azure.com`, where `your-host-name`` is your unique model deployment host name and `your-azure-region`` is the Azure region where the model is deployed (for example, eastus2).

-* Depending on your model deployment and authentication preference, you need either a key to authenticate against the service, or Microsoft Entra ID credentials. The key is a 32-character string.

-## Work with chat completions

-In this section, you use the [Azure AI model inference API](https://aka.ms/azureai/modelinference) with a chat completions model for chat.

-> [!TIP]

-> The [Azure AI model inference API](https://aka.ms/azureai/modelinference) allows you to talk with most models deployed in Azure AI Studio with the same code and structure, including Jamba-Instruct chat models.

-### Create a client to consume the model

-First, create the client to consume the model. The following code uses an endpoint URL and key that are stored in environment variables.

-### Get the model's capabilities

-The `/info` route returns information about the model that is deployed to the endpoint. Return the model's information by calling the following method:

-```http

-GET /info HTTP/1.1

-Host: <ENDPOINT_URI>

-Authorization: Bearer <TOKEN>

-Content-Type: application/json

-```

-The response is as follows:

-{

- "model_name": "AI21-Jamba-Instruct",

- "model_type": "chat-completions",

- "model_provider_name": "AI21"

-}

-### Create a chat completion request

-The following example shows how you can create a basic chat completions request to the model.

-```json

-{

- "messages": [

- {

- "role": "system",

- "content": "You are a helpful assistant."

- },

- {

- "role": "user",

- "content": "How many languages are in the world?"

- }

- ]

-}

-```

-The response is as follows, where you can see the model's usage statistics:

-```json

- "id": "0a1234b5de6789f01gh2i345j6789klm",

- "object": "chat.completion",

- "created": 1718726686,

- "model": "AI21-Jamba-Instruct",

- "choices": [

- {

- "index": 0,

- "message": {

- "role": "assistant",

- "content": "As of now, it's estimated that there are about 7,000 languages spoken around the world. However, this number can vary as some languages become extinct and new ones develop. It's also important to note that the number of speakers can greatly vary between languages, with some having millions of speakers and others only a few hundred.",

- "tool_calls": null

- },

- "finish_reason": "stop",

- "logprobs": null

- }

- ],

- "usage": {

- "prompt_tokens": 19,

- "total_tokens": 91,

- "completion_tokens": 72

- }

-Inspect the `usage` section in the response to see the number of tokens used for the prompt, the total number of tokens generated, and the number of tokens used for the completion.

-#### Stream content

-By default, the completions API returns the entire generated content in a single response. If you're generating long completions, waiting for the response can take many seconds.

-You can _stream_ the content to get it as it's being generated. Streaming content allows you to start processing the completion as content becomes available. This mode returns an object that streams back the response as [data-only server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events). Extract chunks from the delta field, rather than the message field.

-```json

- "messages": [

- {

- "role": "system",

- "content": "You are a helpful assistant."

- },

- {

- "role": "user",

- "content": "How many languages are in the world?"

- }

- ],

- "stream": true,

- "temperature": 0,

- "top_p": 1,

- "max_tokens": 2048

-You can visualize how streaming generates content:

-```json

- "id": "23b54589eba14564ad8a2e6978775a39",

- "object": "chat.completion.chunk",

- "created": 1718726371,

- "model": "AI21-Jamba-Instruct",

- "choices": [

- {

- "index": 0,

- "delta": {

- "role": "assistant",

- "content": ""

- },

- "finish_reason": null,

- "logprobs": null

- }

- ]

-The last message in the stream has `finish_reason` set, indicating the reason for the generation process to stop.

-```json

-{

- "id": "23b54589eba14564ad8a2e6978775a39",

- "object": "chat.completion.chunk",

- "created": 1718726371,

- "model": "AI21-Jamba-Instruct",

- "choices": [

- {

- "index": 0,

- "delta": {

- "content": ""

- },

- "finish_reason": "stop",

- "logprobs": null

- }

- ],

- "usage": {

- "prompt_tokens": 19,

- "total_tokens": 91,

- "completion_tokens": 72

- }

-}

-```

-#### Explore more parameters supported by the inference client

-Explore other parameters that you can specify in the inference client. For a full list of all the supported parameters and their corresponding documentation, see [Azure AI Model Inference API reference](https://aka.ms/azureai/modelinference).

-```json

-{

- "messages": [

- {

- "role": "system",

- "content": "You are a helpful assistant."

- },

- {

- "role": "user",

- "content": "How many languages are in the world?"

- }

- ],

- "presence_penalty": 0.1,

- "frequency_penalty": 0.8,

- "max_tokens": 2048,

- "stop": ["<|endoftext|>"],

- "temperature" :0,

- "top_p": 1,

- "response_format": { "type": "text" }

-}

-```

-```json

-{

- "id": "0a1234b5de6789f01gh2i345j6789klm",

- "object": "chat.completion",

- "created": 1718726686,

- "model": "AI21-Jamba-Instruct",

- "choices": [

- {

- "index": 0,

- "message": {

- "role": "assistant",

- "content": "As of now, it's estimated that there are about 7,000 languages spoken around the world. However, this number can vary as some languages become extinct and new ones develop. It's also important to note that the number of speakers can greatly vary between languages, with some having millions of speakers and others only a few hundred.",

- "tool_calls": null

- },

- "finish_reason": "stop",

- "logprobs": null

- }

- ],

- "usage": {

- "prompt_tokens": 19,

- "total_tokens": 91,

- "completion_tokens": 72

- }

-}

-```

-> [!WARNING]

-> Jamba doesn't support JSON output formatting (`response_format = { "type": "json_object" }`). You can always prompt the model to generate JSON outputs. However, such outputs are not guaranteed to be valid JSON.

-If you want to pass a parameter that isn't in the list of supported parameters, you can pass it to the underlying model using *extra parameters*. See [Pass extra parameters to the model](#pass-extra-parameters-to-the-model).

-### Pass extra parameters to the model

-The Azure AI Model Inference API allows you to pass extra parameters to the model. The following code example shows how to pass the extra parameter `logprobs` to the model.

-Before you pass extra parameters to the Azure AI model inference API, make sure your model supports those extra parameters. When the request is made to the underlying model, the header `extra-parameters` is passed to the model with the value `pass-through`. This value tells the endpoint to pass the extra parameters to the model. Use of extra parameters with the model doesn't guarantee that the model can actually handle them. Read the model's documentation to understand which extra parameters are supported.

-```http

-POST /chat/completions HTTP/1.1

-Host: <ENDPOINT_URI>

-Authorization: Bearer <TOKEN>

-Content-Type: application/json

-extra-parameters: pass-through

-```

-```json

-{

- "messages": [

- {

- "role": "system",

- "content": "You are a helpful assistant."

- },

- {

- "role": "user",

- "content": "How many languages are in the world?"

- }

- ],

- "logprobs": true

-}

-```

-### Apply content safety

-The Azure AI model inference API supports [Azure AI content safety](https://aka.ms/azureaicontentsafety). When you use deployments with Azure AI content safety turned on, inputs and outputs pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions.

-The following example shows how to handle events when the model detects harmful content in the input prompt and content safety is enabled.

-```json

- "messages": [

- {

- "role": "system",

- "content": "You are an AI assistant that helps people find information."

- },

- {

- "role": "user",

- "content": "Chopping tomatoes and cutting them into cubes or wedges are great ways to practice your knife skills."

- }

- ]

-```json

-{

- "error": {

- "message": "The response was filtered due to the prompt triggering Microsoft's content management policy. Please modify your prompt and retry.",

- "type": null,

- "param": "prompt",

- "code": "content_filter",

- "status": 400

- }

-}

-> [!TIP]

-> To learn more about how you can configure and control Azure AI content safety settings, check the [Azure AI content safety documentation](https://aka.ms/azureaicontentsafety).

-## More inference examples

-For more examples of how to use Jamba, see the following examples and tutorials:

-| Description | Language | Sample |

-|-|-|--|

-| Azure AI Inference package for JavaScript | JavaScript | [Link](https://aka.ms/azsdk/azure-ai-inference/javascript/samples) |

-| Azure AI Inference package for Python | Python | [Link](https://aka.ms/azsdk/azure-ai-inference/python/samples) |

-## Cost and quota considerations for Jamba family of models deployed as serverless API endpoints

-Jamba models deployed as a serverless API are offered by AI21 through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying the model.

-Each time a project subscribes to a given offer from the Azure Marketplace, a new resource is created to track the costs associated with its consumption. The same resource is used to track costs associated with inference; however, multiple meters are available to track each scenario independently.

-For more information on how to track costs, see [Monitor costs for models offered through the Azure Marketplace](costs-plan-manage.md#monitor-costs-for-models-offered-through-the-azure-marketplace).

-* [Azure AI Model Inference API](../reference/reference-model-inference-api.md)

-* [Deploy models as serverless APIs](deploy-models-serverless.md)

-* [Consume serverless API endpoints from a different Azure AI Studio project or hub](deploy-models-serverless-connect.md)

-* [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)

-* [Plan and manage costs (marketplace)](costs-plan-manage.md#monitor-costs-for-models-offered-through-the-azure-marketplace)

-Azure AI Studio is currently available in preview in the following Azure regions. You can create [Azure AI Studio hubs](../how-to/create-azure-ai-resource.md) and Azure AI Studio projects in these regions.

-Azure AI Studio preview is currently not available in Azure Government regions or air-gap regions.

-| [Service Fabric](../service-fabric/service-fabric-api-management-overview.md) | Developer, Premium | ❌ |❌ | ❌ | ❌ |

-|Hardware|[Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md)|[Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md)|

-Both App Service Environment v3 and the public multitenant offering run on [Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md), which means that both offerings benefit from the capabilities that scale sets provide. However, App Service Environment v3 is a dedicated environment, which means that even though it can scale out to more instances than the public multitenant offering, scaling out to multiple instances can be slower than the public multitenant offering.

-|Hardware |[Cloud Services (classic)](../../cloud-services/cloud-services-choose-me.md) |[Cloud Services (classic)](../../cloud-services/cloud-services-choose-me.md) |[Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md) |

-App Service Environment v3 runs on the latest [Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md) infrastructure while App Service Environment v1 and v2 run on [Cloud Services (classic)](../../cloud-services/cloud-services-choose-me.md). Because of this, App Service Environment v3 has the best performing and fastest scaling times across all versions.

-The **PremiumV3** pricing tier guarantees machines with faster processors (minimum 195 [ACU](../virtual-machines/acu.md) per virtual CPU), SSD storage, memory-optimized options and quadruple memory-to-core ratio compared to **Standard** tier. **PremiumV3** also supports higher scale via increased instance count while still providing all the advanced capabilities found in **Standard** tier. All features available in the existing **PremiumV2** tier are included in **PremiumV3**.

-App Service is a Platform-as-a-Service, which means that the OS and application stack are managed for you by Azure; you only manage your application and its data. More control over the OS and application stack is available for you in [Azure Virtual Machines](../virtual-machines/index.yml). With that in mind, it is nevertheless helpful for you as an App Service user to know more information, such as:

-Besides App Service, Azure offers other services that can be used for hosting websites and web applications. For most scenarios, App Service is the best choice. For microservice architecture, consider [Azure Spring Apps](../spring-apps/index.yml) or [Service Fabric](../service-fabric/index.yml). If you need more control over the VMs on which your code runs, consider [Azure Virtual Machines](../virtual-machines/index.yml). For more information about how to choose between these Azure services, see [Azure App Service, Virtual Machines, Service Fabric, and Cloud Services comparison](/azure/architecture/guide/technology-choices/compute-decision-tree).

-[WordPress](https://www.wordpress.org) is an open source Content Management System (CMS) used by over 40% of the web to create websites, blogs, and other applications. WordPress can be run on a few different Azure

-> As described, the HTTP protocol is used only to perform a handshake when establishing a WebSocket connection. Once the handshake is completed, a WebSocket connection gets opened for transmitting the data, and the Web Application Firewall (WAF) cannot parse any contents. Therefore, WAF doesn't perform any inspections on such data.

-As part of VNet migration process as documented at [IaaS resources migration page](../virtual-machines/migration-classic-resource-manager-ps.md), if you have an Application Gateway resource present on the VNet that you're trying to migrate to Resource Manager deployment model, the automatic migration wouldn't be successful.

-Please refer to [Frequently asked questions about classic to Azure Resource Manager migration](../virtual-machines/migration-classic-resource-manager-faq.yml)

-To get started see: [platform-supported migration of IaaS resources from classic to Resource Manager](../virtual-machines/migration-classic-resource-manager-ps.md)

-Azure PowerShell is used to create an IPv6 Azure Application Gateway and perform testing to ensure it works correctly. Application gateway can manage and secure web traffic to servers that you maintain. A [virtual machine scale set](../virtual-machine-scale-sets/overview.md) is for backend servers to manage web traffic. The scale set contains two virtual machine instances that are added to the default backend pool of the application gateway. For more information about the components of an application gateway, see [Application gateway components](application-gateway-components.md).

-[Cloud-init support for virtual machines in Azure](../virtual-machines/linux/using-cloud-init.md).

-You can use the Azure CLI to create an [application gateway](overview.md) with a certificate for TLS/SSL termination. A routing rule is used to redirect HTTP traffic to the HTTPS port in your application gateway. In this example, you also create a [Virtual Machine Scale Set](../virtual-machine-scale-sets/overview.md) for the backend pool of the application gateway that contains two virtual machine instances.

-You can use the Azure portal to create an [application gateway](overview.md) with a certificate for TLS termination. A routing rule is used to redirect HTTP traffic to the HTTPS port in your application gateway. In this example, you also create a [virtual machine scale set](../virtual-machine-scale-sets/overview.md) for the backend pool of the application gateway that contains two virtual machine instances.

-You can use the Azure PowerShell to create an [application gateway](overview.md) with a certificate for TLS/SSL termination. A routing rule is used to redirect HTTP traffic to the HTTPS port in your application gateway. In this example, you also create a [virtual machine scale set](../virtual-machine-scale-sets/overview.md) for the backend pool of the application gateway that contains two virtual machine instances.

-Application gateway is used to manage and secure web traffic to servers that you maintain. You can use the Azure CLI to create an [application gateway](overview.md) that uses a [Virtual Machine Scale Set](../virtual-machine-scale-sets/overview.md) for backend servers. In this example, the scale set contains two virtual machine instances. The scale set is added to the default backend pool of the application gateway.

-Application gateway is used to manage and secure web traffic to servers that you maintain. You can use Azure PowerShell to create an [application gateway](overview.md) that uses a [virtual machine scale set](../virtual-machine-scale-sets/overview.md) for backend servers to manage web traffic. In this example, the scale set contains two virtual machine instances that are added to the default backend pool of the application gateway.

-You can use the Azure CLI to create an [application gateway](overview.md) with a certificate for [TLS termination](ssl-overview.md). For backend servers, you can use a [Virtual Machine Scale Set](../virtual-machine-scale-sets/overview.md). In this example, the scale set contains two virtual machine instances that are added to the default backend pool of the application gateway.

-You can use Azure PowerShell to create an [application gateway](overview.md) with a certificate for [TLS/SSL termination](ssl-overview.md) that uses a [virtual machine scale set](../virtual-machine-scale-sets/overview.md) for backend servers. In this example, the scale set contains two virtual machine instances that are added to the default backend pool of the application gateway.

-You can use the Azure CLI to configure [URL path-based routing rules](tutorial-url-route-cli.md) when you create an [application gateway](./overview.md). In this tutorial, you create backend pools using [virtual machine scale sets](../virtual-machine-scale-sets/overview.md). You then create URL routing rules that make sure web traffic is redirected to the appropriate backend pool.

-You can use Azure PowerShell to configure [URL-based routing rules](./url-route-overview.md) when you create an [application gateway](./overview.md). In this article, you create backend pools using [virtual machine scale sets](../virtual-machine-scale-sets/overview.md). You then create URL routing rules that make sure web traffic is redirected to the appropriate backend pool.

-You can use Azure PowerShell to configure web traffic routing to specific scalable server pools based on the URL that is used to access your application. In this article, you create an [Azure Application Gateway](./overview.md) with three backend pools using [Virtual Machine Scale Sets](../virtual-machine-scale-sets/overview.md). Each of the backend pools serves a specific purpose such as, common data, images, and video. Routing traffic to separate pools ensures that your customers get the information that they need when they need it.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-| [Boot Diagnostics](../virtual-machines/boot-diagnostics.md) | Boot diagnostics is a debugging feature for Azure virtual machines (VM) that allows diagnosis of VM boot failures. Boot diagnostics enables a user to observe the state of their VM as it is booting up by collecting serial log information and screenshots. This will only be enabled for machines that are using managed disks. | Production, Dev/Test |

-| [Boot Diagnostics](../virtual-machines/boot-diagnostics.md) | Boot diagnostics is a debugging feature for Azure virtual machines (VM) that allows diagnosis of VM boot failures. Boot diagnostics enables a user to observe the state of their VM as it is booting up by collecting serial log information and screenshots. This will only be enabled for machines that are using managed disks. | Production, Dev/Test |

-| "OS Information: Name='(null)', ver='(null)', agent status='Not Ready'." | Ensure that you're running a [minimum supported agent version](/troubleshoot/azure/virtual-machines/support-extensions-agent-version), the agent is running ([Linux](/troubleshoot/azure/virtual-machines/linux-azure-guest-agent) and [Windows](/troubleshoot/azure/virtual-machines/windows-azure-guest-agent)), and that the agent is up to date ([Linux](../virtual-machines/extensions/update-linux-agent.md) and [Windows](../virtual-machines/extensions/agent-windows.md)). |

-| "Unable to determine the OS for the VM. Check that the VM Agent is running, the current status is Ready." | Ensure that you're running a [minimum supported agent version](/troubleshoot/azure/virtual-machines/support-extensions-agent-version), the agent is running ([Linux](/troubleshoot/azure/virtual-machines/linux-azure-guest-agent) and [Windows](/troubleshoot/azure/virtual-machines/windows-azure-guest-agent)), and that the agent is up to date ([Linux](../virtual-machines/extensions/update-linux-agent.md) and [Windows](../virtual-machines/extensions/agent-windows.md)). |

-* An [Azure virtual machine](../virtual-machines/windows/quick-create-powershell.md).

-You can also use Azure Resource Manager templates with Azure Desired State Configuration (DSC) extension to push configurations to your Azure VMs. The Azure DSC extension uses the Azure VM Agent framework to deliver, enact, and report on DSC configurations running on Azure VMs. For compilation details using Azure Resource Manager templates, see [Desired State Configuration extension with Azure Resource Manager templates](../virtual-machines/extensions/dsc-template.md#details).

-The Azure Desired State Configuration (DSC) VM [extension](../virtual-machines/extensions/dsc-overview.md) is updated as-needed to support enhancements and new capabilities delivered by Azure, Windows Server, and the Windows Management Framework (WMF) that includes Windows PowerShell.

- - Bug fixes & improvements that simplify using Azure Automation State Configuration in the portal and with a Resource Manager template. For more information, see [Default Configuration Script](../virtual-machines/extensions/dsc-overview.md) in the DSC extension documentation.

-The best way to register VMs from other Azure subscriptions is to use the DSC extension in an Azure Resource Manager deployment template. Examples are provided in [Desired State Configuration extension with Azure Resource Manager templates](../virtual-machines/extensions/dsc-template.md).

-You can correct drift of Windows virtual machines using the `Run` command feature. See [Run PowerShell scripts in your Windows VM with Run command](../virtual-machines/windows/run-command.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

- - For Azure VMs, install the Log Analytics agent for Linux using the [virtual machine extension for Linux](../virtual-machines/extensions/oms-linux.md). The extension installs the Log Analytics agent on Azure virtual machines, and enrolls virtual machines into an existing Log Analytics workspace. You can use an Azure Resource Manager template, the Azure CLI, or Azure Policy to assign the [Deploy Log Analytics agent for *Linux* or *Windows* VMs](../governance/policy/samples/built-in-policies.md#monitoring) built-in policy definition. Once the agent is installed, the machine can be added to a Hybrid Runbook Worker group in your Automation account.

- > If you are using the Log Analytics virtual machine extension for Linux for an Azure VM, we recommend setting `autoUpgradeMinorVersion` to `false` as auto-upgrading versions can cause issues with the Hybrid Runbook Worker. To learn how to upgrade the extension manually, see [Azure CLI deployment](../virtual-machines/extensions/oms-linux.md#azure-cli-deployment).

-With the hybrid sandbox, you can use `C:\temp` based on the availability of storage on a Hybrid Runbook Worker. However, per Azure VM recommendations, you shouldn't use the [temporary disk](../virtual-machines/managed-disks-overview.md#temporary-disk) on Windows or Linux for data that needs to be persisted.

-* A [virtual machine](../virtual-machines/windows/quick-create-portal.md) to enable for the feature.

- - For Azure VMs, install the Log Analytics agent for Windows using the [virtual machine extension for Windows](../virtual-machines/extensions/oms-windows.md). The extension installs the Log Analytics agent on Azure virtual machines, and enrolls virtual machines into an existing Log Analytics workspace. You can use an Azure Resource Manager template, PowerShell, or Azure Policy to assign the [Deploy Log Analytics agent for *Linux* or *Windows* VMs](../governance/policy/samples/built-in-policies.md#monitoring) built-in policy definition. Once the agent is installed, the machine can be added to a Hybrid Runbook Worker group in your Automation account.

-Log Analytics agent versions prior to [10.20.18053 (bundle) and 1.0.18053.0 (extension)](../virtual-machines/extensions/oms-windows.md#agent-and-vm-extension-version) use an older method of certificate handling, and hence it is **not recommended**. Hybrid Workers on the outdated agents will not be able to connect to Azure, and Azure Automation jobs executed by these Hybrid Workers will stop.

-2. If your Log Analytics agent version is prior to [10.20.18053 (bundle) and 1.0.18053.0 (extension)](../virtual-machines/extensions/oms-windows.md#agent-and-vm-extension-version), upgrade to the latest version of the Windows Log Analytics agent, following these [guidelines](../azure-monitor/agents/agent-manage.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-This article assumes that you have a VM to enable with Change Tracking and Inventory. If you don't have an Azure VM, you can [create a VM](../../virtual-machines/windows/quick-create-portal.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-[Agents that are on version - 10.20.18053 (bundle) and 1.0.18053.0 (extension)](../../virtual-machines/extensions/oms-windows.md#agent-and-vm-extension-version) or newer aren't affected in response to this change. If youΓÇÖre on an agent prior to that, your agent will be unable to connect, and the Change Tracking & Inventory pipeline & downstream activities can stop. You can check the current LA agent version in HeartBeat table within your LA Workspace.

-Hybrid Worker extension supports [Automatic upgrade](../virtual-machines/automatic-extension-upgrade.md) of minor versions by default. We recommend that you enable Automatic upgrades to take advantage of any security or feature updates without manual overhead. However, to prevent the extension from automatically upgrading (for example, if there is a strict change windows and can only be updated at specific time), you can opt out of this feature by setting the `enableAutomaticUpgrade`property in ARM, Bicep template, PowerShell cmdlets to *false*. Set the same property to *true* whenever you want to re-enable the Automatic upgrade.

-* Two or more [Azure virtual machines](../../virtual-machines/windows/quick-create-powershell.md). Since you stop and start these machines, they shouldn't be production VMs.

-* An [Azure virtual machine](../../virtual-machines/windows/quick-create-powershell.md). Since you stop and start this machine, it shouldn't be a production VM.

-1. If you're using Classic Run As accounts, ensure that you have [migrated](../virtual-machines/classic-vm-deprecation.md) resources deployed through classic deployment model to Azure Resource Manager.

-To install the `xDscDiagnostics` module on your Azure machine, use [Invoke-AzVMRunCommand](/powershell/module/az.compute/invoke-azvmruncommand). You can also use the **Run command** option in the Azure portal by following the steps in [Run PowerShell scripts in your Windows VM with Run Command](../../virtual-machines/windows/run-command.md).

-On the Troubleshoot Update Agent page, select **Run Checks** to start the troubleshooter. The troubleshooter uses [Run command](../../virtual-machines/linux/run-command.md) to run a script on the machine to verify the dependencies. When the troubleshooter is finished, it returns the result of the checks.

-To fix this issue, allow access to IP **169.254.169.254**. For more information, see [Access Azure Instance Metadata Service](../../virtual-machines/windows/instance-metadata-service.md#azure-instance-metadata-service-windows)

-Fix this issue by allowing the prerequisite Repo URL. For RHEL, see [here](../../virtual-machines/workloads/redhat/redhat-rhui.md#troubleshoot-connection-problems-to-azure-rhui).

-On the Troubleshoot Update Agent page, select **Run checks** to start the troubleshooter. The troubleshooter uses [Run Command](../../virtual-machines/windows/run-command.md) to run a script on the machine, to verify dependencies. When the troubleshooter is finished, it returns the result of the checks.

-To fix the issue, allow access to IP **169.254.169.254** </br> For more information see, [access Azure instance metadata service](../../virtual-machines/windows/instance-metadata-service.md#access-azure-instance-metadata-service)

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

- [Create your first Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-* An [Azure virtual machine](../../virtual-machines/windows/quick-create-portal.md), or VM or server registered with Azure Arc-enabled servers. Non-Azure VMs or servers need to have the [Log Analytics agent](../../azure-monitor/agents/log-analytics-agent.md) for Windows or Linux installed and reporting to the workspace linked to the Automation account where Update Management is enabled. We recommend installing the Log Analytics agent for Windows or Linux by first connecting your machine to [Azure Arc-enabled servers](../../azure-arc/servers/overview.md), and then use Azure Policy to assign the [Deploy Log Analytics agent to *Linux* or *Windows* Azure Arc machines](../../governance/policy/samples/built-in-policies.md#monitoring) built-in policy. Alternatively, if you plan to monitor the machines with Azure Monitor for VMs, instead use the [Enable Azure Monitor for VMs](../../governance/policy/samples/built-in-initiatives.md#monitoring) initiative.

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-* A [virtual machine](../../virtual-machines/windows/quick-create-portal.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> Update Management does not support automating update management across all instances in an Azure virtual machine scale set. [Automatic OS image upgrades](../../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md) is the recommended method for managing OS image upgrades on your scale set.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-Update Management requiresΓÇ»[Log Analytics agent](../../azure-monitor/agents/log-analytics-agent.md)ΓÇ» for its functioning. We recommend you to update Windows Log Analytics agent (also known as Windows Microsoft Monitoring Agent (MMA)) to the latest version to reduce security vulnerabilities and benefit from bug fixes. Log Analytics agent versions prior to [10.20.18053 (bundle) and 1.0.18053.0 (extension)](../../virtual-machines/extensions/oms-windows.md#agent-and-vm-extension-version) use an older method of certificate handling and hence it is not recommended. Older Windows Log Analytics agents would not be able to connect to Azure and Update Management would stop working on them.

-1. If your Log Analytics agent version is prior to [10.20.18053 (bundle) and 1.0.18053.0 (extension)](../../virtual-machines/extensions/oms-windows.md#agent-and-vm-extension-version), upgrade to the latest version of the Windows Log Analytics agent, following these [guidelines](../../azure-monitor/agents/agent-manage.md).ΓÇ»

-On Azure VMs, if the Log Analytics agent isn't already installed, when you enable Update Management for the VM it is automatically installed using the Log Analytics VM extension for [Windows](../../virtual-machines/extensions/oms-windows.md) or [Linux](../../virtual-machines/extensions/oms-linux.md). The agent is configured to report to the Log Analytics workspace linked to the Automation account Update Management is enabled in.

-For Red Hat Linux machines, see [IPs for the RHUI content delivery servers](../../virtual-machines/workloads/redhat/redhat-rhui.md#the-ips-for-the-rhui-content-delivery-servers) for required endpoints. For other Linux distributions, see your provider documentation.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-Before connecting, make sure that the SSH public/private key pair that you used when creating the cluster controller is installed on your local machine. Read the SSH keys documentation for [Windows](../virtual-machines/linux/ssh-from-windows.md) or for [Linux](../virtual-machines/linux/mac-create-ssh-keys.md) if you need help. If you used a password instead of a public key, you will be prompted to enter it when you connect.

-For additional information about these VMs, read the Microsoft Azure documentation: [Memory optimized virtual machine sizes](../virtual-machines/sizes-memory.md)

-* **Authentication type** - Choose either password or SSH public key authentication for connecting to the controller. The SSH public key method is recommended; read [How to create and use SSH keys](../virtual-machines/linux/ssh-from-windows.md) if you need help.

-> Managed identity can't be used to authenticate locally running applications. Your application must be deployed to an Azure service that supports Managed Identity. This article uses Azure App Service as an example. However, the same concept applies to any other Azure service that supports managed identity. For example, [Azure Kubernetes Service](/azure/aks/use-azure-ad-pod-identity), [Azure Virtual Machine](../active-directory/managed-identities-azure-resources/qs-configure-portal-windows-vm.md), and [Azure Container Instances](../container-instances/container-instances-managed-identity.md). If your workload is hosted in one of those services, you can also leverage the service's managed identity support.

- console.log("config.app.greeting:", config.greeting); // config.app.greeting: Hello World

- console.log("config.app.json:", config.json); // config.app.json: { myKey: 'myValue' }

-## Development

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-For a group of Arc-enabled servers undergoing an update, the Azure platform will orchestrate updates following the model described in the [Automation Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md#availability-first-updates). However, there are some notable differences between Arc-enabled servers and Azure VMs:

-> Azure Arc-enabled servers does not support deploying and managing VM extensions to Azure virtual machines. For Azure VMs, see the following [VM extension overview](../../virtual-machines/extensions/overview.md) article.

-> Azure Arc-enabled servers does not support deploying and managing VM extensions to Azure virtual machines. For Azure VMs, see the following [VM extension overview](../../virtual-machines/extensions/overview.md) article.

-> Azure Arc-enabled servers does not support deploying and managing VM extensions to Azure virtual machines. For Azure VMs, see the following [VM extension overview](../../virtual-machines/extensions/overview.md) article.

-> Azure Arc-enabled servers does not support deploying and managing VM extensions to Azure virtual machines. For Azure VMs, see the following [VM extension overview](../../virtual-machines/extensions/overview.md) article.

-To use the Custom Script extension, the following sample is provided to run on Windows and Linux. If you are unfamiliar with the Custom Script extension, see [Custom Script extension for Windows](../../virtual-machines/extensions/custom-script-windows.md) or [Custom Script extension for Linux](../../virtual-machines/extensions/custom-script-linux.md). There are a couple of differing characteristics that you should understand when using this extension with hybrid machines:

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> Azure Arc-enabled servers does not support deploying and managing VM extensions to Azure virtual machines. For Azure VMs, see the following [VM extension overview](../../virtual-machines/extensions/overview.md) article.

-|Microsoft Antimalware extension |Microsoft.Azure.Security |IaaSAntimalware |[Microsoft Antimalware extension for Windows](../../virtual-machines/extensions/iaas-antimalware-windows.md) |

-|Custom Script extension |Microsoft.Compute | CustomScriptExtension |[Windows Custom Script Extension](../../virtual-machines/extensions/custom-script-windows.md)|

-|Log Analytics agent |Microsoft.EnterpriseCloud.Monitoring |MicrosoftMonitoringAgent |[Log Analytics VM extension for Windows](../../virtual-machines/extensions/oms-windows.md)|

-|Azure Monitor for VMs (insights) |Microsoft.Azure.Monitoring.DependencyAgent |DependencyAgentWindows | [Dependency agent virtual machine extension for Windows](../../virtual-machines/extensions/agent-dependency-windows.md)|

-|Azure Key Vault Certificate Sync | Microsoft.Azure.Key.Vault |KeyVaultForWindows | [Key Vault virtual machine extension for Windows](../../virtual-machines/extensions/key-vault-windows.md) |

-|Windows Patch Extension |Microsoft.CPlat.Core |WindowsPatchExtension |[Automatic Guest Patching for Azure Virtual Machines and Scale Sets](../../virtual-machines/automatic-vm-guest-patching.md) |

-|Custom Script extension |Microsoft.Azure.Extensions |CustomScript |[Linux Custom Script Extension Version 2](../../virtual-machines/extensions/custom-script-linux.md) |

-|Log Analytics agent |Microsoft.EnterpriseCloud.Monitoring |OmsAgentForLinux |[Log Analytics VM extension for Linux](../../virtual-machines/extensions/oms-linux.md) |

-|Azure Monitor for VMs (insights) |Microsoft.Azure.Monitoring.DependencyAgent |DependencyAgentLinux |[Dependency agent virtual machine extension for Linux](../../virtual-machines/extensions/agent-dependency-linux.md) |

-|Azure Key Vault Certificate Sync | Microsoft.Azure.Key.Vault |KeyVaultForLinux | [Key Vault virtual machine extension for Linux](../../virtual-machines/extensions/key-vault-linux.md) |

-|Linux Patch Extension |Microsoft.CPlat.Core |LinuxPatchExtension |[Automatic Guest Patching for Azure Virtual Machines and Scale Sets](../../virtual-machines/automatic-vm-guest-patching.md)|

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-| CentOS 7 and 8 | 1.42 | 05/31/2025 | See the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md). |

-The VM that is migrated to Azure from on-premises doesn't have the Linux or Windows Azure Guest Agent installed. In these scenarios, you have to manually install the VM agent. For more information about how to install the VM Agent, see [Azure Virtual Machine Windows Agent Overview](../../virtual-machines/extensions/agent-windows.md) or [Azure Virtual Machine Linux Agent Overview](../../virtual-machines/extensions/agent-linux.md).

-After migration and completion of all post-migration configuration steps, you can now deploy the Azure VM extensions based on the VM extensions originally installed on your Azure Arc-enabled server. Review [Azure virtual machine extensions and features](../../virtual-machines/extensions/overview.md) to help plan your extension deployment.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-The E1 (preview) SKU is intended for dev/test scenarios, primarily. E1 runs on smaller [burstable VMs](../virtual-machines/b-series-cpu-credit-model/b-series-cpu-credit-model.md). Burstable VMs offer variable performance based on how much CPU is consumed. Unlike other Enterprise SKU offerings, you can't _scale out_ the E1 SKU, although it's still possible to _scale up_ to a larger SKU. The E1 SKU also doesn't support [active geo-replication](cache-how-to-active-geo-replication.md).

-Azure Cache for Redis Standard (Preview), Premium (Premium), and Enterprise tiers provide built-in redundancy by hosting each cache on two dedicated virtual machines (VMs). Even though these VMs are located in separate [Azure fault and update domains](../virtual-machines/availability.md) and highly available, they're susceptible to data center-level failures. Azure Cache for Redis also supports zone redundancy in its Standard (preview), Premium (preview) and Enterprise tiers. A zone-redundant cache runs on VMs spread across multiple [Availability Zones](../reliability/availability-zones-overview.md). It provides higher resilience and availability.

-The E1 SKU is intended primarily for dev/test scenarios. It runs on smaller [burstable virtual machines](../virtual-machines/b-series-cpu-credit-model/b-series-cpu-credit-model.md). As a result, E1 offers variable performance depending on how much CPU is consumed. Unlike other Enterprise offerings, it isn't possible to scale E1 out. However, it is still possible to scale up to a larger SKU. The E1 SKU also does not support [active geo-replication](cache-how-to-active-geo-replication.md).

-There's no extra charge for using Compute Fleet. You're only charged for the VMs your Compute Fleet launches per hour. For more information on virtual machine billing, see [states and billing status of Azure Virtual Machines](../virtual-machines/states-billing.md).

- - A **Password** must be at least 12 characters long and meet three out of the four following complexity requirements: one lower case character, one upper case character, one number, and one special character. For more information, see [username and password requirements](../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

- - If you select a Linux OS disk image, you can instead choose **SSH public key**. You can use an existing key or create a new one. In this example, Azure generates a new key pair for us. For more information on generating key pairs, see [create and use SSH keys](../virtual-machines/linux/mac-create-ssh-keys.md).

-"Instance" refers to a single instance of an orchestrator function running on a single small ([A1](../../virtual-machines/sizes-previous-gen.md)) VM in Azure App Service. In all cases, it is assumed that [extended sessions](#orchestrator-function-replay) are enabled. Actual results may vary depending on the CPU or I/O work performed by the function code.

-The Start/Stop VMs v2 feature starts or stops Azure Virtual Machines instances across multiple subscriptions. It starts or stops virtual machines on user-defined schedules, provides insights through [Azure Application Insights](../../azure-monitor/app/app-insights-overview.md), and send optional notifications by using [action groups](../../azure-monitor/alerts/action-groups.md). For most scenarios, Start/Stop VMs can manage virtual machines deployed and managed both by Azure Resource Manager and by Azure Service Manager (classic), which is [deprecated](../../virtual-machines/classic-vm-deprecation.md).

-See [Manage the availability of Windows virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/windows/toc.json) and [Manage the availability of Linux virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/linux/toc.json)

-See [Manage the availability of Windows virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/windows/toc.json) or [Manage the availability of Linux virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/linux/toc.json)

-See [Navigate and select Windows virtual machine images in Azure with PowerShell or the CLI](virtual-machines/windows/cli-ps-findimage.md?toc=/azure/virtual-machines/windows/toc.json)

-See [Manage the availability of Windows virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/windows/toc.json) and [Manage the availability of Linux virtual machines](./virtual-machines/availability.md?toc=/azure/virtual-machines/linux/toc.json)

-The software implementation of a physical computer that runs an operating system. Multiple virtual machines can run simultaneously on the same hardware. In Azure, virtual machines are available in a variety of sizes. For more information, see [Virtual Machines documentation](./virtual-machines/index.yml)

-See [About virtual machine extensions and features (Windows)](./virtual-machines/extensions/features-windows.md?toc=/azure/virtual-machines/windows/toc.json) or [About virtual machine extensions and features (Linux)](./virtual-machines/extensions/features-linux.md?toc=/azure/virtual-machines/linux/toc.json)

-[Azure Dedicated Host](../virtual-machines/dedicated-hosts.md) provides physical servers that can host one or more Azure VMs and are dedicated to one Azure subscription. You can provision dedicated hosts within a region, availability zone, and fault domain. You can then place [Windows](../virtual-machines/windows/overview.md), [Linux](../virtual-machines/linux/overview.md), and [SQL Server on Azure](/azure/azure-sql/virtual-machines/) VMs directly into provisioned hosts using whatever configuration best meets your needs. Dedicated Host provides hardware isolation at the physical server level, enabling you to place your Azure VMs on an isolated and dedicated physical server that runs only your organization's workloads to meet corporate compliance requirements.

-> You can deploy a dedicated host using the **[Azure portal, Azure PowerShell, and Azure CLI](../virtual-machines/dedicated-hosts-how-to.md)**.

-Microsoft provides detailed customer guidance on **[Windows](../virtual-machines/windows/quick-create-portal.md)** and **[Linux](../virtual-machines/linux/quick-create-portal.md)** Azure Virtual Machine provisioning using the Azure portal, Azure PowerShell, and Azure CLI. Table 5 summarizes the available security guidance for your virtual machines provisioned in Azure.

-|**Windows**|[Secure policies](../virtual-machines/security-policy.md) <br/>[Azure Disk Encryption](../virtual-machines/windows/disk-encryption-overview.md) <br/> [Built-in security controls](../virtual-machines/windows/security-baseline.md) <br/> [Security recommendations](../virtual-machines/security-recommendations.md)|

-|**Linux**|[Secure policies](../virtual-machines/security-policy.md) <br/> [Azure Disk Encryption](../virtual-machines/linux/disk-encryption-overview.md) <br/> [Built-in security controls](../virtual-machines/linux/security-baseline.md) <br/> [Security recommendations](../virtual-machines/security-recommendations.md)|

-Azure Compute offers virtual machine sizes that are [isolated to a specific hardware type](../virtual-machines/isolation.md) and dedicated to a single customer. These VM instances allow your workloads to be deployed on dedicated physical servers. Using Isolated VMs essentially guarantees that your VM will be the only one running on that specific server node. You can also choose to further subdivide the resources on these Isolated VMs by using [Azure support for nested Virtual Machines](/virtualization/hyper-v-on-windows/user-guide/nested-virtualization).

-**In-transit encryption for VMs** ΓÇô Remote sessions to Windows and Linux VMs deployed in Azure can be conducted over protocols that ensure data encryption in transit. For example, the [Remote Desktop Protocol (RDP)](/windows/win32/termserv/remote-desktop-protocol) initiated from your client computer to Windows and Linux VMs enables TLS protection for data in transit. You can also use [Secure Shell](../virtual-machines/linux/ssh-from-windows.md) (SSH) to connect to Linux VMs running in Azure. SSH is an encrypted connection protocol available by default for remote management of Linux VMs hosted in Azure.

-Because data encryption is performed by the Storage service, server-side encryption with CMK enables you to use any operating system types and images for your VMs. For your Windows and Linux IaaS VMs, Azure also provides Azure Disk encryption that enables you to encrypt managed disks with CMK within the Guest VM or EncryptionAtHost that encrypts disk data right at the host, as described in the next sections. Azure Storage service encryption also offers [double encryption of disk data at rest](../virtual-machines/disks-enable-double-encryption-at-rest-portal.md).

-Azure Storage service encryption encrypts the page blobs that store Azure Virtual Machine disks. Moreover, you may optionally use [Azure Disk encryption](../virtual-machines/disk-encryption-overview.md) to encrypt Azure [Windows](../virtual-machines/windows/disk-encryption-overview.md) and [Linux](../virtual-machines/linux/disk-encryption-overview.md) IaaS Virtual Machine disks to increase storage isolation and assure cryptographic certainty of your data stored in Azure. This encryption includes [managed disks](../virtual-machines/managed-disks-overview.md), as described later in this section. Azure disk encryption uses the industry standard [BitLocker](/windows/security/information-protection/bitlocker/bitlocker-overview) feature of Windows and the [DM-Crypt](https://en.wikipedia.org/wiki/Dm-crypt) feature of Linux to provide OS-based volume encryption that is integrated with Azure Key Vault.

-> Detailed instructions are available for creating and configuring a key vault for Azure Disk encryption with both **[Windows](../virtual-machines/windows/disk-encryption-key-vault.yml)** and **[Linux](../virtual-machines/linux/disk-encryption-key-vault.md)** VMs.

-For [Windows VMs](../virtual-machines/windows/disk-encryption-faq.yml), Azure Disk encryption selects the encryption method in BitLocker based on the version of Windows, for example, XTS-AES 256 bit for Windows Server 2012 or greater. These crypto modules are FIPS 140 validated as part of the Microsoft [Windows FIPS validation program](/windows/security/threat-protection/fips-140-validation#modules-used-by-windows-server). For [Linux VMs](../virtual-machines/linux/disk-encryption-faq.yml), Azure Disk encryption uses the decrypt default of aes-xts-plain64 with a 256-bit volume master key that is FIPS 140 validated as part of DM-Crypt validation obtained by suppliers of Linux IaaS VM images in Microsoft Azure Marketplace.

-[Azure managed disks](../virtual-machines/managed-disks-overview.md) are block-level storage volumes that are managed by Azure and used with Azure Windows and Linux virtual machines. They simplify disk management for Azure IaaS VMs by handling storage account management transparently for you. Azure managed disks automatically encrypt your data by default using [256-bit AES encryption](../virtual-machines/disk-encryption.md) that is FIPS 140 validated. For encryption key management, you have the following choices:

-Customer-managed keys (CMK) enable you to have [full control](../virtual-machines/disk-encryption.md#full-control-of-your-keys) over your encryption keys. You can grant access to managed disks in your Azure Key Vault so that your keys can be used for encrypting and decrypting the DEK. You can also disable your keys or revoke access to managed disks at any time. Finally, you have full audit control over key usage with Azure Key Vault monitoring to ensure that only managed disks or other authorized resources are accessing your encryption keys.

-Encryption at host works with server-side encryption to ensure data stored on the VM host is encrypted at rest and flows encrypted to the Storage service. The server hosting your VM encrypts your data with no performance impact or requirement for code running in your guest VM, and that encrypted data flows into Azure Storage using the keys configured for server-side encryption. For more information, see [Encryption at host - End-to-end encryption for your VM data](../virtual-machines/disk-encryption.md#encryption-at-hostend-to-end-encryption-for-your-vm-data). Encryption at host with CMK can use keys stored in Managed HSM or Key Vault, just like server-side encryption for managed disks.

-| [Azure Stack HCI](/azure-stack/hci/) | &#x2705; | &#x2705; |

-| [Container Instances](../../container-instances/index.yml) | &#x2705; | &#x2705; |

-| [Data Box](../../databox/index.yml) **&ast;** | &#x2705; | &#x2705; |

-| [Planned Maintenance for VMs](../../virtual-machines/maintenance-and-updates.md) | &#x2705; | &#x2705; |

-| [Power Pages](https://powerapps.microsoft.com/portals/) | &#x2705; | &#x2705; |

-| [Service Fabric](../../service-fabric/index.yml) | &#x2705; | &#x2705; |

-| [Storage: Disks (incl. managed disks)](../../virtual-machines/managed-disks-overview.md) | &#x2705; | &#x2705; |

-| [Virtual Machine Scale Sets](../../virtual-machine-scale-sets/index.yml) | &#x2705; | &#x2705; |

-| [Virtual Machines](../../virtual-machines/index.yml) | &#x2705; | &#x2705; |

-| [VM Image Builder](../../virtual-machines/image-builder-overview.md) | &#x2705; | &#x2705; |

-**&ast;&ast;&ast;** FedRAMP High authorization for edge devices (such as Azure Data Box and Azure Stack Edge) applies only to Azure services that support on-premises, customer-managed devices. For example, FedRAMP High authorization for Azure Data Box covers datacenter infrastructure services and Data Box pod and disk service, which are the online software components supporting your Data Box hardware appliance. You are wholly responsible for the authorization package that covers the physical devices. For assistance with accelerating your onboarding and authorization of devices, contact your Microsoft account representative.

-| [Azure Stack HCI](/azure-stack/hci/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

-| [Container Instances](../../container-instances/index.yml)| &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

-| [Planned Maintenance for VMs](../../virtual-machines/maintenance-and-updates.md) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

-| [Power Pages](https://powerapps.microsoft.com/portals/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

-| [Service Fabric](../../service-fabric/index.yml) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

-| [Storage: Disks (incl. managed disks)](../../virtual-machines/managed-disks-overview.md) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

-| [Virtual Machine Scale Sets](../../virtual-machine-scale-sets/index.yml) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

-| [Virtual Machines](../../virtual-machines/index.yml) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

-| [VM Image Builder](../../virtual-machines/image-builder-overview.md) | &#x2705; | &#x2705; | &#x2705; | | |

-**&ast;** Authorizations for edge devices (such as Azure Data Box and Azure Stack Edge) apply only to Azure services that support on-premises, customer-managed devices. You are wholly responsible for the authorization package that covers the physical devices. For assistance with accelerating your onboarding and authorization of devices, contact your Microsoft account representative.

-Azure [virtual machine (VM) extensions](../virtual-machines/extensions/features-windows.md) are small applications that provide post-deployment configuration and automation tasks on Azure VMs.

-* [Deploy a Windows virtual machine extension](../virtual-machines/extensions/features-windows.md#run-vm-extensions)

-* [Deploy a Linux virtual machine extension](../virtual-machines/extensions/features-linux.md#run-vm-extensions)

-IL5 separation requirements are stated in Section 5.2.2.3 (Page 51) of the [Cloud Computing SRG](https://public.cyber.mil/dccs/dccs-documents/). The SRG focuses on compute separation during "processing" of IL5 data. This separation ensures that a virtual machine that could potentially compromise the physical host can't affect a DoD workload. To remove the risk of runtime attacks and ensure long running workloads aren't compromised from other workloads on the same host, **all IL5 virtual machines and virtual machine scale sets** should be isolated by DoD mission owners via [Azure Dedicated Host](https://azure.microsoft.com/services/virtual-machines/dedicated-host/) or [isolated virtual machines](../virtual-machines/isolation.md). Doing so provides a dedicated physical server to host your Azure Virtual Machines (VMs) for Windows and Linux.

-### [Virtual machines](../virtual-machines/index.yml) and [virtual machine scale sets](../virtual-machine-scale-sets/index.yml)

-#### [Azure Dedicated Host](../virtual-machines/dedicated-hosts.md)

-#### [Isolated virtual machines](../virtual-machines/isolation.md)

-> As new hardware generations become available, some VM types might require reconfiguration (scale up or migration to a new VM SKU) to ensure they remain on properly dedicated hardware. For more information, see **[Virtual machine isolation in Azure](../virtual-machines/isolation.md).**

- - [Enable Azure Disk Encryption for Linux](../virtual-machines/linux/disk-encryption-overview.md)

- - [Enable Azure Disk Encryption for Windows](../virtual-machines/windows/disk-encryption-overview.md)

-### [Container Instances](../container-instances/index.yml)

-This tutorial shows how to connect to Azure Government, create a virtual network and a virtual machine on this network in Azure Government cloud. The Azure Government marketplace provides a VM library, in this tutorial we use "Data Science Virtual Machine - Windows 2016 CSP". To learn more about Azure Virtual Machines and see end-to-end scenarios, see [Virtual Machines Documentation](../virtual-machines/index.yml).

- b. Select the *Availability options*. To learn about availability sets, see [Availability sets overview](../virtual-machines/availability-set-overview.md).

- > For instructions on creating an SSH RSA public-private key pair for SSH client connections, see **[Create and manage SSH keys for authentication to a Linux VM in Azure](../virtual-machines/linux/create-ssh-keys-detailed.md).**

-To learn more about backup and restore options for virtual machines in Azure, continue to [Overview of backup options for VMs](../virtual-machines/backup-recovery.md).

- b. Select the *Availability options*. To learn about availability sets, see [Availability sets overview](../virtual-machines/availability-set-overview.md).

-To learn more about backup and restore options for virtual machines in Azure, continue to [Overview of backup options for VMs](../virtual-machines/backup-recovery.md).

-* [Azure Container Instances](../../articles/container-instances/index.yml).

-Also on the right is the [Azure proximity placement group's name](../virtual-machines/co-location.md).

-For time synchronization, Azure Linux reads the time from the Azure VM host using [chronyd](../../articles/virtual-machines/linux/time-sync.md#chrony) and the /dev/ptp device.

-[nvidia-v100]: ../virtual-machines/ncv3-series.md

-[nvidia-t4]: ../virtual-machines/nct4-v3-series.md

-For more information about creating SSH keys, see [Create and manage SSH keys for authentication in Azure](../../articles/virtual-machines/linux/create-ssh-keys-detailed.md).

-For more information about creating SSH keys, see [Create and manage SSH keys for authentication in Azure](../../articles/virtual-machines/linux/create-ssh-keys-detailed.md).

-| [Route] | Yes | One request = 1 transaction<br><ul><li>If using the Route Matrix, each cell in the Route Matrix request generates a billable Route transaction.</li><li>If using Batch Directions, each route query (route origin/destination coordinate pair and waypoints) in the Batch request call generates a billable Route transaction. Note, the billable Route transaction usage results generated by the batch request has **-Batch** appended to the API name of your Azure portal metrics report.</li></ul> | <ul><li>Location Insights Routing (Gen2 pricing)</li><li>Standard S1 Routing Transactions (Gen1 S1 pricing)</li><li>Standard Services API Transactions (Gen1 S0 pricing)</li></ul> |

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-* Focus support on any of the [Azure Linux Endorsed distros](../../virtual-machines/linux/endorsed-distros.md). There might be some delay between a new distro/version being Azure Linux Endorsed and it being supported for the Log Analytics Linux agent.

-> For Azure VMs, we recommend that you install the agent on them by using the [Azure Log Analytics VM extension](../../virtual-machines/extensions/oms-linux.md) for Linux.

-| Azure VM | Log Analytics agent VM extension for Windows/Linux | The agent is automatically upgraded [after the VM model changes](../../virtual-machines/extensions/features-linux.md#how-agents-and-extensions-are-updated), unless you configured your Azure Resource Manager template to opt out by setting the property `autoUpgradeMinorVersion` to **false**. Once deployed, however, the extension won't upgrade minor versions unless redeployed, even with this property set to **true**. Only the Linux agent supports automatic update post deployment with `enableAutomaticUpgrade` property (see [Enable Auto-update for the Linux agent](#enable-auto-update-for-the-linux-agent)). Major version upgrade is always manual (see [VirtualMachineExtensionInner.AutoUpgradeMinorVersion Property](/dotnet/api/microsoft.azure.management.compute.fluent.models.virtualmachineextensioninner.autoupgrademinorversion)). |

-We recommend that you enable [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) by using these commands to update the agent automatically.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-We strongly recommended to always update to the latest version, or opt in to the [Automatic Extension Update](../../virtual-machines/automatic-extension-upgrade.md) feature.

-| August 2024 | **Windows**<ul><li>Added columns to the SecurityEvent table: Keywords, Opcode, Correlation, ProcessId, ThreadId, EventRecordId.</li><li>AMA: Support AMA Client Installer for selected partners.</li></ul>**Linux Features**<ul><li>Enable Dynamic Linking of OpenSSL 1.1 in all regions</li><li>Add Computer field to Custom Logs</li><li>Add EventHub upload support for Custom Logs </li><li>Reliability improvement for upload task scheduling</li><li>Added support for SUSE15 SP5, and AWS 3 distributions</li></ul>**Linux Fixes**<ul><li>Fix Direct upload to storage for perf counters when no other destination is configured. You don't see perf counters If storage was the only configured destination for perf counters, they wouldn't see perf counters in their blob or table.</li><li>Fluent-Bit updated to version 3.0.7. This fixes the issue with Fluent-Bit creating junk files in the root directory on process shutdown.</li><li>Fix proxy for system-wide proxy using http(s)_proxy env var </li><li>Support for syslog hostnames that are up to 255characters</li><li>Stop sending rows longer than 1MB. This exceeds ingestion limits and destabilizes the agent. Now the row is gracefully dropped and a diagnostic message is written.</li><li>Set max disk space used for rsyslog spooling to 1GB. There was no limit before which could lead to high memory usage.</li><li>Use random available TCP port when there is a port conflict with AMA port 28230 and 28330 . This resolved issues where port 28230 and 28330 were already in uses by the customer which prevented data upload to Azure.</li></ul>| 1.29 | 1.32.5 |

-> The recommendation is to enable [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) to update installed extensions to the released (latest) version across all regions. Upgrades are issued in batches, so you may see some of your virtual machines, scale-sets or Arc-enabled servers get upgraded before others. If you need to upgrade an extension immediately, you may use the manual instructions below.

-We recommend that you enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature. Go to your virtual machine or scale set, select the **Extensions** tab and select **AzureMonitorWindowsAgent** or **AzureMonitorLinuxAgent**. In the dialog that opens, select **Enable automatic upgrade**.

-We recommend that you enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature by using the following PowerShell commands.

-We recommend that you enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature by using the following CLI commands.

-Azure Monitor Agent is implemented as an [Azure VM extension](../../virtual-machines/extensions/overview.md) with the details in the following table. You can install it by using any of the methods to install virtual machine extensions. For version information, see [Azure Monitor agent extension versions](./azure-monitor-agent-extension-versions.md).

-> CentOS is a Linux distribution that is nearing End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

- - IMDS service isn't running/accessible from the virtual machine. [Check if you can access IMDS from the machine](../../virtual-machines/windows/instance-metadata-service.md?tabs=windows).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-For information on how to install and configure the diagnostics extension, see [Install and configure Azure Diagnostics extension for Windows (WAD)](../agents/diagnostics-extension-windows-install.md) or [Use Azure Diagnostics extension for Linux to monitor metrics and logs](../../virtual-machines/extensions/diagnostics-linux.md). You can specify the storage account and configure collection of the data that you want to forward to Azure Monitor Logs.

-The Azure Diagnostics extension for both Windows and Linux always collects data into an Azure Storage account. For a list of specific tables and blobs where this data is collected, see [Install and configure Azure Diagnostics extension for Windows](diagnostics-extension-windows-install.md) and [Use Azure Diagnostics extension for Linux to monitor metrics and logs](../../virtual-machines/extensions/diagnostics-linux.md).

-The diagnostics extension is implemented as a [virtual machine extension](../../virtual-machines/extensions/overview.md) in Azure, so it supports the same installation options using Azure Resource Manager templates, PowerShell, and the Azure CLI. For information on installing and maintaining virtual machine extensions, see [Virtual machine extensions and features for Windows](../../virtual-machines/extensions/features-windows.md) and [Virtual machine extensions and features for Linux](../../virtual-machines/extensions/features-linux.md).

-* [Use Linux diagnostics extension to monitor metrics and logs](../../virtual-machines/extensions/diagnostics-linux.md)

-[Monitor and diagnose services in a local machine development setup](../../service-fabric/service-fabric-diagnostics-how-to-monitor-and-diagnose-services-locally.md)

-The diagnostics extension is implemented as a [virtual machine extension](../../virtual-machines/extensions/overview.md) in Azure. It supports the same installation options by using Azure Resource Manager templates, PowerShell, and the Azure CLI. For information on how to install and maintain virtual machine extensions, see [Virtual machine extensions and features for Windows](../../virtual-machines/extensions/features-windows.md).

-For information on how to deploy the diagnostics extension with Azure Resource Manager templates, see [Use monitoring and diagnostics with a Windows VM and Azure Resource Manager templates](../../virtual-machines/extensions/diagnostics-template.md).

-See also [Use PowerShell to enable Azure Diagnostics in a virtual machine running Windows](../../virtual-machines/extensions/diagnostics-windows.md).

-> Configuring the Azure Load Balancer using the **Basic SKU**, requires that Azure virtual machines belong to an Availability Set. To learn more about availability sets, see [Manage the availability of Windows virtual machines in Azure](../../virtual-machines/availability.md). To add existing virtual machines to an availability set, refer to [Set Azure Resource Manager VM Availability Set](/troubleshoot/azure/virtual-machines/allocation-failure#resize-a-vm-or-add-vms-to-an-existing-availability-set).

-The following sample installs the Log Analytics agent on an Azure virtual machine. This is done by enabling the [Log Analytics virtual machine extension for Windows](../../virtual-machines/extensions/oms-windows.md).

-The following sample installs the Log Analytics agent on a Linux Azure virtual machine. This is done by enabling the [Log Analytics virtual machine extension for Linux](../../virtual-machines/extensions/oms-linux.md).

-The following sample enables and configures the diagnostic extension on a Linux Azure virtual machine. For details on the configuration, see [Windows diagnostics extension schema](../../virtual-machines/extensions/diagnostics-linux.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

- * [Install the Azure VM Agent](../../virtual-machines/extensions/agent-windows.md#install-the-azure-windows-vm-agent).

-For more information, see [Troubleshooting Windows extensions](../../virtual-machines/extensions/oms-windows.md).

- * [Install the Azure VM Agent on Linux VMs](../../virtual-machines/extensions/agent-linux.md#installation).

-You can manually create alert rules for any of your Azure resources using the appropriate metric values or log queries as a signal. You must create and maintain each alert rule for each resource individually, so you will probably want to use one of the other options when they're applicable and only manually create alert rules for special cases. Multiple services in Azure have documentation articles that describe recommended telemetry to collect and alert rules that are recommended for that service. These articles are typically found in the **Monitor** section of the service's documentation. For example, [Monitor Azure virtual machines](../../virtual-machines/monitor-vm.md) and [Monitor Azure Kubernetes Service (AKS)](/azure/aks/monitor-aks).

-You can create a resource group by using the [az group create](/cli/azure/group#az-group-create) command. For information about creating virtual machines, see [Create a Windows virtual machine with the Azure CLI](../../virtual-machines/windows/quick-create-cli.md), [Create a Linux virtual machine with the Azure CLI](../../virtual-machines/linux/quick-create-cli.md), and the [az vm create](/cli/azure/vm#az-vm-create) command.

-Enabling monitoring for your ASP.NET and ASP.NET Core IIS-hosted applications running on [Azure Virtual Machines](https://azure.microsoft.com/services/virtual-machines/) or [Azure Virtual Machine Scale Sets](../../virtual-machine-scale-sets/index.yml) is now easier than ever. Get all the benefits of using Application Insights without modifying your code.

-* Learn how to [deploy an application to an Azure virtual machine scale set](../../virtual-machine-scale-sets/virtual-machine-scale-sets-deploy-app.md).

-If either preceding case is true, see [Use PowerShell to enable Azure Diagnostics in a virtual machine running Windows](../../virtual-machines/extensions/diagnostics-windows.md) to configure and update your Azure VM Diagnostics extension to enable the metric. The article also includes a sample diagnostics configuration file.

-* [Automatically scale a virtual machine scale](../../virtual-machine-scale-sets/tutorial-autoscale-powershell.md)

-> [Availability sets](/archive/blogs/kaevans/autoscaling-azurevirtual-machines) are an older scaling feature for virtual machines with limited support. We recommend migrating to [Azure Virtual Machine Scale Sets](../../virtual-machine-scale-sets/overview.md) for faster and more reliable autoscale support.

-* [Tutorial: Automatically scale a virtual machine scale set with the Azure CLI](../../virtual-machine-scale-sets/tutorial-autoscale-cli.md)

-* [Tutorial: Automatically scale a virtual machine scale set with an Azure template](../../virtual-machine-scale-sets/tutorial-autoscale-powershell.md)

-| Azure Virtual Machines Scale Sets | [Overview of autoscale with Azure Virtual Machine Scale Sets](../../virtual-machine-scale-sets/virtual-machine-scale-sets-autoscale-overview.md) |

-* [Tutorial: Automatically scale a virtual machine scale set with the Azure CLI](../../virtual-machine-scale-sets/tutorial-autoscale-cli.md)

-* [Tutorial: Automatically scale a virtual machine scale set with Azure PowerShell](../../virtual-machine-scale-sets/tutorial-autoscale-powershell.md)

-| Azure Container Instances | [Logging for Azure Container Instances](../../container-instances/container-instances-log-analytics.md#log-schema) |

-| [Azure Monitor for SAP solutions](../../virtual-machines/workloads/sap/monitor-sap-on-azure.md) | Preview | No | An Azure-native monitoring product for anyone running their SAP landscapes on Azure. It works with both SAP on Azure Virtual Machines and SAP on Azure Large Instances. Collects telemetry data from Azure infrastructure and databases in one central location and visually correlates the data for faster troubleshooting. You can monitor different components of an SAP landscape, such as Azure virtual machines (VMs), high-availability clusters, SAP HANA database, and SAP NetWeaver, by adding the corresponding provider for that component. |

-| Data Manager for Energy | [OEPDataplaneLogs](/azure/azure-monitor/reference/tables/OEPDataplaneLogs) |

-| Dedicated SQL Pool | [SynapseSqlPoolSqlRequests](/azure/azure-monitor/reference/tables/synapsesqlpoolsqlrequests)<br>[SynapseSqlPoolRequestSteps](/azure/azure-monitor/reference/tables/synapsesqlpoolrequeststeps)<br>[SynapseSqlPoolExecRequests](/azure/azure-monitor/reference/tables/synapsesqlpoolexecrequests)<br>[SynapseSqlPoolDmsWorkers](/azure/azure-monitor/reference/tables/synapsesqlpooldmsworkers)<br>[SynapseSqlPoolWaits](/azure/azure-monitor/reference/tables/synapsesqlpoolwaits) |

- For information about how to add the Diagnostics extension to your deployment template, see [Use monitoring and diagnostics with a Windows VM and Azure Resource Manager templates](../../virtual-machines/extensions/diagnostics-template.md).

-> This scenario describes how to implement complete monitoring of your Azure and hybrid virtual machine environment. To get started monitoring your first Azure virtual machine, see [Monitor Azure virtual machines](../../virtual-machines/monitor-vm.md).

-The Azure Monitor agent is implemented as a [virtual machine extension](../../virtual-machines/extensions/overview.md), so you can install it using a variety of standard methods including PowerShell, CLI, and Resource Manager templates. See [Manage Azure Monitor Agent](../agents/azure-monitor-agent-manage.md) for details on each. Other notable methods for installation are described below.

-> This scenario describes how to implement complete monitoring of your Azure and hybrid virtual machine environment. To get started monitoring your first Azure virtual machine, see [Monitor Azure virtual machines](../../virtual-machines/monitor-vm.md) or [Tutorial: Collect guest logs and metrics from Azure virtual machine](tutorial-monitor-vm-guest.md).

-> This scenario describes how to implement complete monitoring of your Azure and hybrid virtual machine environment. To get started monitoring your first Azure virtual machine, see [Monitor Azure virtual machines](../../virtual-machines/monitor-vm.md).

-> This guide describes how to implement complete monitoring of your enterprise Azure and hybrid virtual machine environment. To get started monitoring your first Azure virtual machine, see [Monitor Azure virtual machines](../../virtual-machines/monitor-vm.md).

-One of the most common monitoring requirements for a virtual machine is to create an alert if it stops running. The best method for this is to create a metric alert rule in Azure Monitor using the [VM availability](../../virtual-machines/monitor-vm-reference.md#vm-availability-metric-preview) metric which is currently in public preview.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-|Azure VM | Dependency agent VM extension for [Windows](../../virtual-machines/extensions/agent-dependency-windows.md) and [Linux](../../virtual-machines/extensions/agent-dependency-linux.md) | Agent is automatically upgraded by default unless you configured your Azure Resource Manager template to opt out by setting the property *autoUpgradeMinorVersion* to **false**. The upgrade for minor version where auto upgrade is disabled, and a major version upgrade follow the same method - uninstall and reinstall the extension. |

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-Virtual-Machines|[Monitor Azure virtual machines](../../articles/virtual-machines/monitor-vm.md)|VM scenario updates for Azure Monitor Agent.|

-Virtual-Machines|[Monitor Azure virtual machines](../../articles/virtual-machines/monitor-vm.md)|Updated for Azure Monitor Agent and availability metric.|

-* You must create a proximity placement group (PPG) and anchor it to your SAP HANA compute resources. Application volume group for SAP HANA needs this setup to search for an Azure NetApp Files resource that is close to the SAP HANA servers. For more information, see [Best practices about Proximity Placement Groups](#best-practices-about-proximity-placement) and [Create a Proximity Placement Group using the Azure portal](../virtual-machines/windows/proximity-placement-groups-portal.md).

- 1. For SUSE on Azure Large Instances, set up SUSE Subscription Management Tool (SMT). For more information, see [Install and configure SAP HANA (Large Instances) on Azure](../virtual-machines/workloads/sap/hana-installation.md#operating-system).

- - Oracle Database release 12 or later (see [Oracle VM images and their deployment on Microsoft Azure](../virtual-machines/workloads/oracle/oracle-vm-solutions.md) for details)

- - IBM Db2 for LUW on Linux-only version 10.5 or later (see [IBM Db2 Azure Virtual Machines DBMS deployment for SAP workload](../virtual-machines/workloads/sap/dbms_guide_ibm.md) for details)

-The snapshot tools can be used in the following [Supported scenarios for HANA Large Instances](../virtual-machines/workloads/sap/hana-supported-scenario.md) and [SAP HANA with Azure NetApp Files](../virtual-machines/workloads/sap/hana-vm-operations-netapp.md).

-Microsoft provides many storage options for deploying databases such as SAP HANA. For details about some of these options, see [Azure Storage types for SAP workload](../virtual-machines/workloads/sap/planning-guide-storage.md). There's also a [cost-conscious solution with Azure premium storage](../virtual-machines/workloads/sap/hana-vm-premium-ssd-v1.md#cost-conscious-solution-with-azure-premium-storage).

-For best results, ensure that you are using a virtual machine (VM) instance that is appropriately sized to perform the tests. The following examples use a Standard_D32s_v3 instance. For more information about VM instance sizes, see [Sizes for Windows virtual machines in Azure](../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-network%2ftoc.json) for Windows-based VMs, and [Sizes for Linux virtual machines in Azure](../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json) for Linux-based VMs.

-* [Oracle VM images and their deployment on Microsoft Azure: Shared storage configuration options](../virtual-machines/workloads/oracle/oracle-vm-solutions.md#shared-storage-configuration-options)

-* [High availability for SAP NetWeaver on Azure VMs on SUSE Linux Enterprise Server with Azure NetApp Files for SAP applications](../virtual-machines/workloads/sap/high-availability-guide-suse-netapp-files.md)

-* [High availability for SAP NetWeaver on Azure VMs on Red Hat Enterprise Linux with Azure NetApp Files for SAP applications](../virtual-machines/workloads/sap/high-availability-guide-rhel-netapp-files.md)

-* [High availability for SAP NetWeaver on Azure VMs on Windows with Azure NetApp Files (SMB) for SAP applications](../virtual-machines/workloads/sap/high-availability-guide-windows-netapp-files-smb.md)

-* [Using Windows DFS-N to support flexible SAPMNT share creation for SMB-based file share](../virtual-machines/workloads/sap/high-availability-guide-windows-dfs.md)

-* [High availability for SAP NetWeaver on Azure VMs on Red Hat Enterprise Linux for SAP applications multi-SID guide](../virtual-machines/workloads/sap/high-availability-guide-rhel-multi-sid.md)

-* [SAP HANA Azure virtual machine storage configurations](../virtual-machines/workloads/sap/hana-vm-operations-storage.md)

-* [NFS v4.1 volumes on Azure NetApp Files for SAP HANA](../virtual-machines/workloads/sap/hana-vm-operations-netapp.md)

-* [High availability of SAP HANA Scale-up with Azure NetApp Files on Red Hat Enterprise Linux](../virtual-machines/workloads/sap/sap-hana-high-availability-netapp-files-red-hat.md)

-* [SAP HANA scale-out with standby node on Azure VMs with Azure NetApp Files on SUSE Linux Enterprise Server](../virtual-machines/workloads/sap/sap-hana-scale-out-standby-netapp-files-suse.md)

-* [SAP HANA scale-out with standby node on Azure VMs with Azure NetApp Files on Red Hat Enterprise Linux](../virtual-machines/workloads/sap/sap-hana-scale-out-standby-netapp-files-rhel.md)

-* [SAP HANA scale-out with HSR and Pacemaker on RHEL - Azure Virtual Machines](../virtual-machines/workloads/sap/sap-hana-high-availability-scale-out-hsr-rhel.md)

-* [Oracle Azure Virtual Machines DBMS deployment for SAP workload - Azure Virtual Machines](../virtual-machines/workloads/sap/dbms_guide_oracle.md)

-* [IBM Db2 Azure Virtual Machines DBMS deployment for SAP workload using Azure NetApp Files](../virtual-machines/workloads/sap/dbms_guide_ibm.md#using-azure-netapp-files)

-* [Striping Disks in Azure](../virtual-machines/premium-storage-performance.md#disk-striping)

-For example, [Deploy a SAP HANA scale-out system with standby node on Azure VMs by using Azure NetApp Files on SUSE Linux Enterprise Server](../virtual-machines/workloads/sap/sap-hana-scale-out-standby-netapp-files-suse.md#mount-the-azure-netapp-files-volumes) shows the 256-KiB `rsize` and `wsize` maximum as follows:

-This section details the criteria to be considered in selecting [VMs](../virtual-machines/sizes.md) for best performance and the rationale behind selections made for testing. Azure NetApp Files is a Network Attached Storage (NAS) service, therefore appropriate network bandwidth sizing is critical for optimal performance.

-* [Design and implement an Oracle database in Azure](../virtual-machines/workloads/oracle/oracle-design.md)

-* [Reference architectures for Oracle Database Enterprise Edition on Azure](../virtual-machines/workloads/oracle/oracle-reference-architecture.md)

-Using a 1-TiB working set size and an 80% read, 20% update SQL Server workload, performance capabilities of most the instances in the D instance class were measured; most, not all, as the D2 and D64 instances themselves were excluded from testing. The former was left out as it doesn't support accelerated networking, and the latter because it's the comparison point. See the following graph to understand the limits of D4s_v4, D8s_v4, D16s_v4, and D32s_v4, respectively. Managed disk storage tests are not shown in the graph. Comparison values are drawn directly from the [Azure Virtual Machine limits table](../virtual-machines/dv3-dsv3-series.md) for the D class instance type.

-Many applications are built for HA across multiple availability zones using application-based replication and failover technologies, like [SQL Server Always-On Availability Groups (AOAG)](/sql/database-engine/availability-groups/windows/always-on-availability-groups-sql-server), [SAP HANA with HANA System Replication (HSR)](../virtual-machines/workloads/sap/sap-hana-high-availability-netapp-files-suse.md), and [Oracle with Data Guard](../virtual-machines/workloads/oracle/oracle-reference-architecture.md#high-availability-for-oracle-databases).

-The virtual machine name is hard-coded as **SimpleWinVM** in the ARM template, to match what's used in the `main.bicep` file that creates the dashboard. You'll need to create your own administration username and password for this VM. This is a new username and password (not the account you use to sign in to Azure). The password must be complex. For more information, see [username requirements](../virtual-machines/windows/faq.yml#what-are-the-username-requirements-when-creating-a-vm-)

-and [password requirements](../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

- 1. Enter a username and password for the VM. This is a new username and password (not the account you use to sign in to Azure). The password must be complex. For more information, see [username requirements](../virtual-machines/windows/faq.yml#what-are-the-username-requirements-when-creating-a-vm-) and [password requirements](../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

-> This is a new username and password (not the account you use to sign in to Azure). The password must be complex. For more information, see [username requirements](../virtual-machines/windows/faq.yml#what-are-the-username-requirements-when-creating-a-vm-)

-and [password requirements](../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

-Store login credentials for the VM in a variable. The password must be complex. This is a new user name and password; it's not, for example, the account you use to sign in to Azure. For more information, see [username requirements](../virtual-machines/windows/faq.yml#what-are-the-username-requirements-when-creating-a-vm-)

-and [password requirements](../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

-1. [Create an Azure VM](../virtual-machines/windows/quick-create-portal.md). For this sample, use a Windows 10 image.

-1. Copy the executable under \<your local path\>\RoleBasedAccessControl\bin\Debug folder to the VM. You can use RDP to connect to your Azure VM. For more information, see [How to connect and sign on to an Azure virtual machine running Windows](../virtual-machines/windows/connect-logon.md).

-For this test, create a virtual machine by following the steps in the [Create a Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md)

-This article describes the syntax you use to export and import shared functionality, as well as namespaces for Bicep extensions.

-## Exporting types, variables and functions

-The `@export()` decorator indicates that a given statement can be imported by another file. This decorator is only valid on type, variable, and function statements. Variable statements marked with `@export()` must be compile-time constants.

-## Import types, variables, and functions

-Only statements that have been [exported](#exporting-types-variables-and-functions) in the file being referenced are available to be imported.

-In Bicep, a union type allows the creation of a combined type consisting of a set of sub-types. An assignment is valid if any of the individual sub-type assignments are permitted. The `|` character separates individual sub-types using an _or_ condition. For example, the syntax _'a' | 'b'_ means that a valid assignment could be either _'a'_ or _'b'_. Union types are translated into the [allowed-value](../templates/definitions.md#allowed-values) constraint in Bicep, so only literals are permitted as members. Unions can include any number of literal-typed expressions.

-Any type expression can be used as a sub-type in a union type declaration (between `|` characters). For example, the following examples are all valid:

-* All literals must be of the same primitive data type (e.g., all strings or all integers).

-Secure string uses the same format as string, and secure object uses the same format as object. With Bicep, you add the `@secure()` [decorator](./parameters.md#decorators) to a string or object.

-Getting a parameter value from a file is helpful when you need to provide configuration values. For example, you can provide [cloud-init values for a Linux virtual machine](../../virtual-machines/linux/using-cloud-init.md).

-Getting a parameter value from a file is helpful when you need to provide configuration values. For example, you can provide [cloud-init values for a Linux virtual machine](../../virtual-machines/linux/using-cloud-init.md).

-Learn how to create a development environment for developing and testing deployment scripts with a deployment script image. You can either create an [Azure container instance](../../container-instances/container-instances-overview.md) or use [Docker](https://docs.docker.com/get-docker/). Both options are covered in this article.

-The deployment script resource is available only in the regions where Azure Container Instances is available. For more information, see [Resource availability for Azure Container Instances in Azure regions](../../container-instances/container-instances-region-availability.md).

-| `DeploymentScriptContainerGroupNameInvalid` | The specified container instance name doesn't meet the Azure Container Instances requirements. See [Troubleshoot common issues in Azure Container Instances](../../container-instances/container-instances-troubleshooting.md#issues-during-container-group-deployment).|

-You can set environment variables (`EnvironmentVariable`) in your container instances to provide dynamic configuration of the application or script that the container runs. A deployment script handles nonsecured and secured environment variables in the same way as Azure Container Instances. For more information, see [Set environment variables in container instances](../../container-instances/container-instances-environment-variables.md#secure-values).

-## Parameters

-Use parameters for values that need to vary for different deployments. You can define a default value for the parameter that is used if no value is provided during deployment.

-For example, you can add a SKU parameter to specify different sizes for a resource. You might pass in different values depending on whether you're deploying to test or production.

-```bicep

-param storageSKU string = 'Standard_LRS'

-```

-The parameter is available for use in your Bicep file.

-```bicep

-sku: {

- name: storageSKU

-}

-```

-For more information, see [Parameters in Bicep](./parameters.md).

-## Parameter decorators

-You can add one or more decorators for each parameter. These decorators describe the parameter and define constraints for the values that are passed in. The following example shows one decorator but many others are available.

-```bicep

-@allowed([

- 'Standard_LRS'

- 'Standard_GRS'

- 'Standard_ZRS'

- 'Premium_LRS'

-])

-param storageSKU string = 'Standard_LRS'

-```

-For more information, including descriptions of all available decorators, see [Decorators](parameters.md#decorators).

-## Variables

-You can make your Bicep file more readable by encapsulating complex expressions in a variable. For example, you might add a variable for a resource name that is constructed by concatenating several values together.

-```bicep

-var uniqueStorageName = '${storagePrefix}${uniqueString(resourceGroup().id)}'

-```

-Apply this variable wherever you need the complex expression.

-```bicep

-resource stg 'Microsoft.Storage/storageAccounts@2023-04-01' = {

- name: uniqueStorageName

-```

-For more information, see [Variables in Bicep](./variables.md).

-For more information, see [Use Bicep modules](./modules.md).

-## Resource and module decorators

-You can add a decorator to a resource or module definition. The supported decorators are `batchSize(int)` and `description`. You can only apply it to a resource or module definition that uses a `for` expression.

-By default, resources are deployed in parallel. When you add the `batchSize(int)` decorator, you deploy instances serially.

-```bicep

-@batchSize(3)

-resource storageAccountResources 'Microsoft.Storage/storageAccounts@2023-04-01' = [for storageName in storageAccounts: {

- ...

-}]

-```

-For more information, see [Deploy in batches](loops.md#deploy-in-batches).

-You can now use multiple lines in function, array and object declarations. This feature requires [Bicep CLI version 0.7.X or higher](./install.md).

-* User-defined functions are not supported at the moment. However, an experimental feature is currently accessible. For more information, see [User-defined functions in Bicep](./user-defined-functions.md).

-Don't use secret data in the `settings` object because it uses clear text. For more information, see [Microsoft.Compute virtualMachines/extensions](/azure/templates/microsoft.compute/virtualmachines/extensions), [Custom Script Extension for Windows](../../virtual-machines/extensions/custom-script-windows.md), and [Use the Azure Custom Script Extension Version 2 with Linux virtual machines](../../virtual-machines/extensions/custom-script-linux.md).

-This rule finds parameters whose names look like secrets but without the [secure decorator](./parameters.md#decorators), for example: a parameter name contains the following keywords:

-Use the [secure decorator](./parameters.md#decorators) for the parameters that contain secrets. The secure decorator marks the parameter as secure. The value for a secure parameter isn't saved to the deployment history and isn't logged.

-To serially deploy instances of a resource, add the [batchSize decorator](./file.md#resource-and-module-decorators). Set its value to the number of instances to deploy concurrently. A dependency is created on earlier instances in the loop, so it doesn't start one batch until the previous batch completes.

-## Definition syntax

-The file for the module can be either a local file or an external file. The external file can be in template spec or a Bicep module registry.

-[Azure Verified Modules](https://azure.github.io/Azure-Verified-Modules/) are prebuilt, pretested, and preverified modules for deploying resources on Azure. Created and owned by Microsoft employees, these modules are designed to simplify and accelerate the deployment process for common Azure resources and configurations whilst also aligning to best practices; such as the Well-Architected Framework.

-This article describes how to define output values in a Bicep file. You use outputs when you need to return values from the deployed resources. You are limited to 64 outputs in a Bicep file. For more information, see [Template limits](../templates/best-practices.md#template-limits).

-## Define output values

-## Declaration

-For more information, see [User-defined data types](./user-defined-data-types.md#syntax).

-## Default value

-## Decorators

-Parameters use decorators for constraints or metadata. The decorators are in the format `@expression` and are placed above the parameter's declaration. You can mark a parameter as secure, specify allowed values, set the minimum and maximum length for a string, set the minimum and maximum value for an integer, and provide a description of the parameter.

-The following example shows two common uses for decorators.

-```bicep

-@secure()

-param demoPassword string

-@description('Must be at least Standard_A3 to support 2 NICs.')

-param virtualMachineSize string = 'Standard_DS1_v2'

-```

-The following table describes the available decorators and how to use them.

-The available decorators are described in the following sections.

-### Secure parameters

-You can mark string or object parameters as secure. The value of a secure parameter isn't saved to the deployment history and isn't logged.

-```bicep

-@secure()

-param demoPassword string

-@secure()

-param demoSecretObject object

-```

-### Length constraints

-You can specify minimum and maximum lengths for string and array parameters. You can set one or both constraints. For strings, the length indicates the number of characters. For arrays, the length indicates the number of items in the array.

-The following example declares two parameters. One parameter is for a storage account name that must have 3-24 characters. The other parameter is an array that must have from 1-5 items.

-```bicep

-@minLength(3)

-@maxLength(24)

-param storageAccountName string

-@minLength(1)

-@maxLength(5)

-param appNames array

-```

-### Integer constraints

-You can set minimum and maximum values for integer parameters. You can set one or both constraints.

-```bicep

-@minValue(1)

-@maxValue(12)

-param month int

-```

-## Use parameter

-## Objects as parameters

-If you're creating reusable Bicep code, you should consider defining names as [parameters](parameters.md). Use a [default parameter value](parameters.md#default-value) to define a default name that can be overridden. Default values help to make your Bicep files more reusable, ensuring that users of the file can define their own names if they need to follow a different naming convention.

-This article describes the syntax you use to add a resource to your Bicep file. You are limited to 800 resources in a Bicep file. For more information, see [Template limits](../templates/best-practices.md#template-limits).

-## Declaration

-## Location

-## Tags

-## Managed identities for Azure resources

-## Syntax

-The [`@allowed`](./parameters.md#decorators) decorator is only permitted on [`param` statements](./parameters.md). To declare a type with a set of predefined values in a `type`, use [union type syntax](./data-types.md#union-types).

- Decorators can be used on properties. `*` can be used to make all values require a constraint. Additional properties can still be defined when using `*`. This example creates an object that requires a key of type `int` named _id_, and that all other entries in the object must be a string value at least 10 characters long.

-The warning informs you that the _anObject_ type doesn't include a property named _otionalProperty_. While no errors occur during deployment, the Bicep compiler assumes _otionalProperty_ is a typo, that you intended to use _optionalProperty_ but misspelled it, and alert you to the inconsistency.

-## Import types between Bicep files

-Only user-defined data types that bear the `@export()` decorator can be imported to other templates.

-The following example enables you to import the two user-defined data types from other templates:

-```bicep

-@export()

-type myStringType = string

-@export()

-type myOtherStringType = myStringType

-```

-For more information, see [Import user-defined data types](./bicep-import-providers.md#import-user-defined-data-types-preview).

-When defining a user function, there are some restrictions:

-## Define the function

-You are limited to 256 variables in a Bicep file. For more information, see [Template limits](../templates/best-practices.md#template-limits).

-## Define variable

-## Use variable

-| Microsoft.Compute | [Virtual Machines](../../virtual-machines/index.yml)<br />[Virtual Machine Scale Sets](../../virtual-machine-scale-sets/index.yml) |

-| Microsoft.HanaOnAzure | [SAP HANA on Azure Large Instances](../../virtual-machines/workloads/sap/hana-overview-architecture.md) |

-| Microsoft.Maintenance | [Azure Maintenance](../../virtual-machines/maintenance-configurations.md) |

-| Microsoft.ServiceFabric | [Service Fabric](../../service-fabric/index.yml) |

-| Microsoft.VirtualMachineImages | [Azure Image Builder](../../virtual-machines/image-builder-overview.md) |

-| Microsoft.ContainerInstance | [Container Instances](../../container-instances/index.yml) |

-| Microsoft.RedHatOpenShift | [Azure Red Hat OpenShift](../../virtual-machines/linux/openshift-get-started.md) |

-For more information, see [Virtual machine sizes](../../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json).

-For more information, see [VM Applications](../../virtual-machines/vm-applications.md).

-[Linux](../../virtual-machines/disk-encryption.md#restrictions) or

-[Windows](../../virtual-machines/disk-encryption.md#restrictions) virtual machines. If you

-* [Virtual machine and cloud service sizes for Azure](../../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json)

-1. [Technical deep dive on platform-supported migration from classic to Azure Resource Manager](../../virtual-machines/migration-classic-resource-manager-deep-dive.md)

-2. [Platform supported migration of IaaS resources from Classic to Azure Resource Manager](../../virtual-machines/migration-classic-resource-manager-overview.md)

-3. [Migrate IaaS resources from classic to Azure Resource Manager by using Azure PowerShell](../../virtual-machines/migration-classic-resource-manager-ps.md)

-4. [Migrate IaaS resources from classic to Azure Resource Manager by using Azure CLI](../../virtual-machines/migration-classic-resource-manager-cli.md)

-* Virtual machines using scheduled patching can't be moved across resource groups or subscriptions. For a potential workaround, see [Managing VM updates with Maintenance Configurations](../../../virtual-machines/maintenance-configurations.md#guest)

-A virtual machine that is integrated with a key vault to implement [Azure Disk Encryption for Linux VMs](../../../virtual-machines/linux/disk-encryption-overview.md) or [Azure Disk Encryption for Windows VMs](../../../virtual-machines/windows/disk-encryption-overview.md) can be moved to another resource group when it is in deallocated state.

-1. In the destination subscription, recreate the virtual machine from the cloned OS disk using PowerShell, CLI, or an Azure Resource Manager template. Include the marketplace plan that's attached to the disk. The information about the plan should match the plan you purchased in the new subscription. For more information, see [Create the VM](../../../virtual-machines/marketplace-images.md#create-the-vm).

-For more information, see [Move a Marketplace Azure Virtual Machine to another subscription](../../../virtual-machines/azure-cli-change-subscription-marketplace.md).

- * [How to connect and sign on to an Azure virtual machine running Windows](../../virtual-machines/windows/connect-rdp.yml)

- * [Setting up WinRM access for Virtual Machines in Azure Resource Manager](../../virtual-machines/windows/connect-winrm.md)

- * [Connect to a Linux VM](../../virtual-machines/linux-vm-connect.md)

-Getting a parameter value from a file is helpful when you need to provide configuration values. For example, you can provide [cloud-init values for a Linux virtual machine](../../virtual-machines/linux/using-cloud-init.md).

-Getting a parameter value from a file is helpful when you need to provide configuration values. For example, you can provide [cloud-init values for a Linux virtual machine](../../virtual-machines/linux/using-cloud-init.md).

-Learn how to create a development environment for developing and testing ARM template deployment scripts with a deployment script image. You can either create an [Azure container instance](../../container-instances/container-instances-overview.md) or use [Docker](https://docs.docker.com/get-docker/). Both options are covered in this article.

-The deployment script resource is only available in the regions where Azure Container Instance is available. See [Resource availability for Azure Container Instances in Azure regions](../../container-instances/container-instances-region-availability.md). Currently, deployment script only uses public networking.

-Setting environment variables (EnvironmentVariable) in your container instances allows you to provide dynamic configuration of the application or script run by the container. Deployment script handles nonsecured and secured environment variables in the same way as Azure Container Instance. For more information, see [Set environment variables in container instances](../../container-instances/container-instances-environment-variables.md#secure-values). For an example, see [Sample templates](#sample-templates).

-The script service creates a [storage account](../../storage/common/storage-account-overview.md) (unless you specify an existing storage account) and a [container instance](../../container-instances/container-instances-overview.md) for script execution. If these resources are automatically created by the script service, both resources have the `azscripts` suffix in the resource names.

-| DeploymentScriptContainerGroupNameInvalid | The Azure container instance name (ACI) specified doesn't meet the ACI requirements. See [Troubleshoot common issues in Azure Container Instances](../../container-instances/container-instances-troubleshooting.md#issues-during-container-group-deployment).|

-Azure Resource Manager template (ARM template) extensions are small applications that provide post-deployment configuration and automation tasks on Azure resources. The most popular one is virtual machine extensions. See [Virtual machine extensions and features for Windows](../../virtual-machines/extensions/features-windows.md), and [Virtual machine extensions and features for Linux](../../virtual-machines/extensions/features-linux.md).

-The same changes also apply [data disks](../../virtual-machines/using-managed-disks-template-deployments.md).

-Another consideration for cloud consistency is the use of [virtual machine extensions](../../virtual-machines/extensions/features-windows.md) to configure the resources inside a VM. Not all VM extensions are available in Azure Stack. A template can specify the resources dedicated to the VM extension, creating dependencies and conditions within the template.

-/azure/virtual-machines/extensions/custom-script-windows), or [Linux](../../virtual-machines/extensions/custom-script-linux.md).

-Learn how to use [Azure virtual machine extensions](../../virtual-machines/extensions/features-windows.md) to perform post-deployment configuration and automation tasks on Azure VMs. Many different VM extensions are available for use with Azure VMs. In this tutorial, you deploy a Custom Script extension from an Azure Resource Manager template (ARM template) to run a PowerShell script on a Windows VM. The script installs Web Server on the VM.

- Verify that the generated password meets the VM password requirements. Each Azure service has specific password requirements. For the VM password requirements, see [What are the password requirements when you create a VM?](../../virtual-machines/windows/faq.yml#what-are-the-password-requirements-when-creating-a-vm-).

-| BadRequest | You sent deployment values that don't match what is expected by Resource Manager. Check the inner status message for help with troubleshooting. <br><br> Validate the template's syntax to resolve deployment errors when using a template that was exported from an existing Azure resource. | [Template reference](/azure/templates/) <br><br> [Resource location in ARM template](../templates/resource-location.md) <br><br> [Resource location in Bicep file](../bicep/resource-declaration.md#location) <br><br> [Resolve invalid template](error-invalid-template.md)|

-| LocationRequired | Provide a location for the resource. | [Resource location in ARM template](../templates/resource-location.md) <br><br> [Resource location in Bicep file](../bicep/resource-declaration.md#location) |

-| SubnetsNotInSameVnet | A virtual machine can only have one virtual network. When deploying several NICs, make sure they belong to the same virtual network. | [Windows VM multiple NICs](../../virtual-machines/windows/multiple-nics.md) <br><br> [Linux VM multiple NICs](../../virtual-machines/linux/multiple-nics.md) |

-1. Select **Overview**, select the VM, and then connect to it through RDP. If you need help with connecting, see [connect to a virtual machine](../virtual-machines/windows/connect-logon.md#connect-to-the-virtual-machine) for details.

-1. Use a [virtual machine](../virtual-machines/windows/quick-create-portal.md#create-virtual-machine) within the Azure Virtual Network where the Azure VMware Solution ExpressRoute terminates. For more information, see [Connect to Azure Virtual Network with ExpressRoute](#connect-to-azure-virtual-network-with-expressroute).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-Follow the instructions in the [Create your first Windows VM in the Azure portal](../virtual-machines/windows/quick-create-portal.md) tutorial. You created the VM in the virtual network that you created in the previous step. Start with a gallery image of Windows Server 2019 Datacenter to run the Azure Backup Server.

-To learn how to attach a new managed data disk to an existing Azure VM, see [Attach a managed data disk to a Windows VM by using the Azure portal](../virtual-machines/windows/attach-managed-disk-portal.yml).

- If you need help with connecting to the VM, see [connect to a virtual machine](../virtual-machines/windows/connect-logon.md#connect-to-the-virtual-machine) for details.

-| [Restore an encrypted VM](./backup-azure-vms-encryption.md) | From the portal, restore the disks and then use PowerShell to create the VM | <ul><li> [Encrypted VM with Microsoft Entra ID](../virtual-machines/windows/disk-encryption-windows-aad.md) <li> [Encrypted VM without Microsoft Entra ID](../virtual-machines/windows/disk-encryption-windows.md) <li> [Encrypted VM *with Microsoft Entra ID* migrated to *without Microsoft Entra ID*](../virtual-machines/windows/disk-encryption-faq.yml#can-i-migrate-vms-that-were-encrypted-with-a-microsoft-entra-app-to-encryption-without-a-microsoft-entra-app-)</ul> |

-Refer to [Azure Disk Encryption documentation](../virtual-machines/disk-encryption-overview.md).

-Refer to the [Azure VM documentation](../virtual-machines/index.yml).

-Refer to the [Managed disks documentation](../virtual-machines/managed-disks-overview.md).

-A snapshot is a full, read-only copy of a virtual hard drive (VHD) or an Azure File share. Learn more about [disk snapshots](../virtual-machines/windows/snapshot-copy-managed-disk.md) and [file snapshots](../storage/files/storage-snapshots-files.md).

-This tutorial describes how to back up [Azure Managed Disk](../virtual-machines/managed-disks-overview.md) from the Azure portal.

- Azure Disk Backup offers multiple backups per day. If you require more frequent backups, choose the **Hourly** backup frequency with the ability to take backups with intervals of every *1*, *2*, *4*, *6*, *8*, or *12* hours. The backups are scheduled based on the **Time** interval selected. For example, if you select **Every 4 hours**, then the backups are taken at approximately in the interval of every 4 hours so the backups are distributed equally across the day. If a once a day backup is sufficient, then choose the **Daily** backup frequency. In the daily backup frequency, you can specify the time of the day when your backups are taken. It's important to note that the time of the day indicates the backup start time and not the time when the backup completes. The time required for completing the backup operation is dependent on various factors including size of the disk, and churn rate between consecutive backups. However, Azure Disk backup is an agentless backup that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md), which doesn't impact the production application performance.

- >Azure Backup for Managed Disks uses incremental snapshots which are limited to 500 snapshots per disk. At a point in time you can have 500 snapshots for a disk. Thus, to prevent backup failure the retention duration is limited by the snapshot limit. To allow you to take on-demand backups aside from scheduled backups, backup policy limits the total backups to 450. Learn more about [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md#restrictions) for managed disk.

- >- Azure Backup uses [_incremental snapshots_](../virtual-machines/disks-incremental-snapshots.md#restrictions) of managed disks, which store only the delta changes to the disk as the last snapshot on Standard HDD storage, regardless of the storage type of the parent disk. For additional reliability, incremental snapshots are stored on Zone Redundant Storage (ZRS) by default in the ZRS supported regions. Currently, Azure Disk Backup supports operational backup of managed disks that doesn't copy backups to the Backup vault storage. So, the backup storage redundancy setting of the Backup vault doesnΓÇÖt apply to the recovery points.

- - You can't create an incremental snapshot for a particular disk outside of that disk's subscription. So, choose the resource group within the same subscription where the disk needs to be backed up. [Learn more](../virtual-machines/disks-incremental-snapshots.md#restrictions) about incremental snapshot for managed disks.

-**Replace existing** | You can restore a disk, and use it to replace a disk on the existing VM.<br/><br/> The current VM must exist. If it's been deleted, this option can't be used.<br/><br/> Azure Backup takes a snapshot of the existing VM before replacing the disk. The snapshot is copied to the vault and retained in accordance with the retention policy.　<br/><br/> When you choose a Vault-Standard recovery point, a VHD file with the content of the chosen recovery point is also created in the staging location you specify. Existing disks connected to the VM are replaced with the selected restore point. <br/><br/> After the replace disk operation, the original disk is retained in the resource group. You can choose to manually delete the original disks if they aren't needed. <br/><br/>Replace existing is supported for unencrypted managed VMs, including VMs [created using custom images](https://azure.microsoft.com/resources/videos/create-a-custom-virtual-machine-image-in-azure-resource-manager-with-powershell/). It's unsupported for classic VMs, unmanaged VMs, and [generalized VMs](../virtual-machines/windows/upload-generalized-managed.md).<br/><br/> If the restore point has more or less disks than the current VM, then the number of disks in the restore point will only reflect the VM configuration.<br><br> Replace existing is also supported for VMs with linked resources, like [user-assigned managed-identity](../active-directory/managed-identities-azure-resources/overview.md) or [Key Vault](/azure/key-vault/general/overview).

->- For information on managed/premium disk performance after restored via Azure Backup, see the [Latency](../virtual-machines/premium-storage-performance.md#latency) section.

-[Azure zone pinned VMs](../virtual-machines/windows/create-portal-availability-zone.md) can be restored in any [availability zones](../availability-zones/az-overview.md) of the same region.

-You're provided with an option to restore [unmanaged disks](../storage/common/storage-disaster-recovery-guidance.md#azure-unmanaged-disks) as [managed disks](../virtual-machines/managed-disks-overview.md) during restore. By default, the unmanaged VMs / disks are restored as unmanaged VMs / disks. However, if you choose to restore as managed VMs / disks, it's now possible to do so. These restore operations aren't triggered from the snapshot phase but only from the vault phase. This feature isn't available for unmanaged encrypted VMs.

-**Restore VMs using Hybrid Use Benefit** | If a Windows VM uses [Hybrid Use Benefit (HUB) licensing](../virtual-machines/windows/hybrid-use-benefit-licensing.md), restore the disks, and create a new VM using the provided template (with **License Type** set to **Windows_Server**), or PowerShell. This setting can also be applied after creating the VM.

-**Restore VMs with special network configurations** | Special network configurations include VMs using internal or external load balancing, using multiple NICS, or multiple reserved IP addresses. You restore these VMs by using the [restore disk option](#restore-disks). This option makes a copy of the VHDs into the specified storage account, and you can then create a VM with an [internal](../load-balancer/quickstart-load-balancer-standard-internal-powershell.md) or [external](../load-balancer/quickstart-load-balancer-standard-public-powershell.md) load balancer, [multiple NICS](../virtual-machines/windows/multiple-nics.md), or [multiple reserved IP addresses](../virtual-network/ip-services/virtual-network-multiple-ip-addresses-powershell.md), in accordance with your configuration.

-**Restore VM in any availability set** | When you restore a VM from the portal, there's no option to choose an availability set. A restored VM doesn't have an availability set. If you use the restore disk option, then you can [specify an availability set](../virtual-machines/windows/tutorial-availability-sets.md) when you create a VM from the disk using the provided template or PowerShell.

-**Linux** | Install by using an RPM or a DEB package from your distribution's package repository. This is the preferred method for installing and upgrading the Azure Linux agent. All the [endorsed distribution providers](../virtual-machines/linux/endorsed-distros.md) integrate the Azure Linux agent package into their images and repositories. The agent is available on [GitHub](https://github.com/Azure/WALinuxAgent), but we don't recommend installing from there.<br/><br/> If you're updating the agent, make sure no backup operations are running, and update the binaries.

-The time required for completing the backup operation depends on various factors including size of the disk, and churn rate between consecutive backups. However, Azure Disk Backup is an agentless backup that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md), which doesn't impact the production application performance.

-When choosing a server for running Azure Backup Server, it's recommended you start with a gallery image of Windows Server 2022 Datacenter or Windows Server 2019 Datacenter. The article, [Create your first Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md?toc=/azure/virtual-machines/windows/toc.json), provides a tutorial for getting started with the recommended virtual machine in Azure, even if you've never used Azure before. The recommended minimum requirements for the server virtual machine (VM) should be: Standard_A4_v2 with four cores and 8-GB RAM.

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-This article assumes that you already have an SQL database installed on an Azure VM. (You can also [create a VM using Azure CLI](../virtual-machines/linux/quick-create-cli.md)).

-Create a Recovery Services vault with the [az backup vault create](/cli/azure/backup/vault#az-backup-vault-create) command. Use the resource group and location as that of the VM you want to protect. Learn how to create a VM using Azure CLI with [this VM quickstart](../virtual-machines/linux/quick-create-cli.md).

- - The Azure VM Agent is installed by default on any Windows VM deployed from an Azure Marketplace image from the portal, PowerShell, Command Line Interface, or an Azure Resource Manager template. A [manual installation of the Agent](../virtual-machines/extensions/agent-windows.md#manual-installation) may be necessary when you create a custom VM image that's deployed to Azure.

- - Ensure the Azure VM Guest Agent service is running by executing the command `ps -e`. Also, ensure the [latest version](../virtual-machines/extensions/update-linux-agent.md) is installed. To learn more, see [Linux VM guest agent issues](backup-azure-troubleshoot-vm-backup-fails-snapshot-timeout.md#the-agent-installed-in-the-vm-is-out-of-date-for-linux-vms).

- - Ensure the [Linux VM agent dependencies on system packages](../virtual-machines/extensions/agent-linux.md#requirements) have the supported configuration. For example: Supported Python version is 2.6 and above.

- - If you've blocked access to `168.63.129.16` using the guest firewall or with a proxy, extensions will fail regardless of the above. Ports 80, 443, and 32526 are required, [Learn more](../virtual-machines/extensions/features-windows.md#network-access).

-This error occurs when one of the extension failures puts the VM into provisioning failed state.<br>**Open  Azure portal > VM > Settings > Extensions > Extensions status** and check if all extensions are in **provisioning succeeded** state. To learn more, see [Provisioning states](../virtual-machines/states-billing.md).

-1. Follow the instructions for [updating the Linux VM agent](../virtual-machines/extensions/update-linux-agent.md).

-Azure Backup supports agentless VM backups by using multi-disk [crash-consistent restore points](../virtual-machines/virtual-machines-create-restore-points.md) (preview). The [Enhanced VM backup policy](backup-azure-vms-enhanced-policy.md) now enables you to configure the consistency type of the backups (application-consistent restore points or crash-consistent restore points preview) for Azure VMs. This feature also enables Azure VM backup to retry the backup operation with *crash-consistent snapshots* (for *supported VMs*) if the application-consistent snapshot fails.

-You can restore [Azure zone pinned VMs](../virtual-machines/windows/create-portal-availability-zone.md) in any [availability zones](../availability-zones/az-overview.md) of the same region.

-* Step 2: [Detach data disk using PowerShell](../virtual-machines/windows/detach-disk.yml#detach-a-data-disk-using-powershell)

-* Step 3: [Attach data disk to Windows VM with PowerShell](../virtual-machines/windows/attach-disk-ps.md)

- * **Managed and non-encrypted VMs** - For managed non-encrypted VMs, attach the restored managed disks. For in-depth information, see [Attach a data disk to a Windows VM using PowerShell](../virtual-machines/windows/attach-disk-ps.md).

- * **Managed and encrypted VMs with Microsoft Entra ID (BEK only)** - For managed encrypted VMs with Microsoft Entra ID (encrypted using BEK only), attach the restored managed disks. For in-depth information, see [Attach a data disk to a Windows VM using PowerShell](../virtual-machines/windows/attach-disk-ps.md).

- * **Managed and encrypted VMs with Microsoft Entra ID (BEK and KEK)** - For managed encrypted VMs with Microsoft Entra ID (encrypted using BEK and KEK), attach the restored managed disks. For in-depth information, see [Attach a data disk to a Windows VM using PowerShell](../virtual-machines/windows/attach-disk-ps.md).

- After the secrets are available and the encryption details are set on the OS disk, to attach the restored managed disks, see [Attach a data disk to a Windows VM using PowerShell](../virtual-machines/windows/attach-disk-ps.md).

- After the key/secrets are available and the encryption details are set on the OS disk, to attach the restored managed disks, see [Attach a data disk to a Windows VM using PowerShell](../virtual-machines/windows/attach-disk-ps.md).

-By default, all the disks in your VMs are automatically encrypted-at-rest using platform-managed keys (PMK) that use [storage service encryption](../storage/common/storage-service-encryption.md). You can back up these VMs using Azure Backup without any specific actions required to support encryption on your end. For more information about encryption with platform-managed keys, [see this article](../virtual-machines/disk-encryption.md#platform-managed-keys).

-For more information about encryption of managed disks with customer-managed keys, see [this article](../virtual-machines/disk-encryption.md#customer-managed-keys).

-1. Make sure you have one or more [Windows](../virtual-machines/linux/disk-encryption-overview.md) or [Linux](../virtual-machines/linux/disk-encryption-overview.md) VMs with ADE enabled.

-This article explains how to use _Enhanced policy_ to configure _Multiple Backups Per Day_ and back up [Trusted Launch VMs](../virtual-machines/trusted-launch.md) with Azure Backup service.

-Azure Backup now supports _Enhanced policy_ that's needed to support new Azure offerings. For example, [Trusted Launch VM](../virtual-machines/trusted-launch.md) is supported with _Enhanced policy_ only.

->- Backups for VMs having [data access authentication enabled disks](../virtual-machines/windows/download-vhd.md?tabs=azure-portal#secure-downloads-and-uploads-with-azure-ad) will fail.

-**SSE** | With SSE, Azure Storage provides encryption at rest by automatically encrypting data before storing it. Azure Storage also decrypts data before retrieving it. Azure Backup supports backups of VMs with two types of Storage Service Encryption:<li> **SSE with platform-managed keys**: This encryption is by default for all disks in your VMs. See more [here](../virtual-machines/disk-encryption.md#platform-managed-keys).<li> **SSE with customer-managed keys**. With CMK, you manage the keys used to encrypt the disks. See more [here](../virtual-machines/disk-encryption.md#customer-managed-keys). | Azure Backup uses SSE for at-rest encryption of Azure VMs.

-* Upgrade the Azure disks to support higher IOPs. [Learn more here](../virtual-machines/disks-types.md)

-**Step 3:** Try [increasing the size of the VM](../virtual-machines/resize-vm.md) and retry the operation

-This issue can occur if disks are mounted to a directory using the device name (e.g., /dev/sdc1) instead of UUID. When the VM reboots or when it is recovered as a new VM, the device names are assigned in a random order. To ensure that the right drive is mounted to your directory, always mount drives using UUID obtained from the `blkid` utility. [Learn more](../virtual-machines/linux/attach-disk-portal.yml).

-* To update the Linux VM Agent, follow the instructions in the article [Updating the Linux VM Agent](../virtual-machines/extensions/update-linux-agent.md?toc=/azure/virtual-machines/linux/toc.json).

-3. Set up the VM in accordance with the [Windows](../virtual-machines/windows/quick-create-portal.md) or [Linux](../virtual-machines/linux/quick-create-portal.md) instructions.

-| **Encryption specific to the workload being backed-up** | - **Azure virtual machine backup**: Azure Backup supports backup of VMs with disks encrypted using [platform-managed keys](../virtual-machines/disk-encryption.md#platform-managed-keys), as well as [customer-managed keys](../virtual-machines/disk-encryption.md#customer-managed-keys) owned and managed by you. In addition, you can also back up your Azure Virtual machines that have their OS or data disks encrypted using [Azure Disk Encryption](backup-azure-vms-encryption.md#encryption-support-using-ade). ADE uses BitLocker for Windows VMs, and DM-Crypt for Linux VMs, to perform in-guest encryption. <br><br> - **TDE - enabled database backup is supported**. To restore a TDE-encrypted database to another SQL Server, you need to first [restore the certificate to the destination server](/sql/relational-databases/security/encryption/move-a-tde-protected-database-to-another-sql-server). The backup compression for TDE-enabled databases for SQL Server 2016 and newer versions is available, but at lower transfer size as explained [here](https://techcommunity.microsoft.com/t5/sql-server/backup-compression-for-tde-enabled-databases-important-fixes-in/ba-p/385593). |

-When choosing a server for Azure Backup Server, start with a Windows Server 2022 Datacenter or Windows Server 2019 Datacenter gallery image. The article, [Create your first Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md?toc=/azure/virtual-machines/windows/toc.json), provides a tutorial for getting started with the recommended virtual machine. The recommended minimum requirements for the server virtual machine (VM) should be: A2 Standard with two cores and 3.5-GB RAM. Use DPM\MABS [capacity planner](https://www.microsoft.com/download/details.aspx?id=54301) to get the appropriate RAM size and accordingly choose the IaaS VM size.

-| Linux | Linux running as [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) or [VMware](backup-azure-backup-server-vmware.md) or [Stack](backup-mabs-install-azure-stack.md) guest | Physical server, On-premises Hyper-V VM, Stack VM or VMware VM running Windows Server. | V4 | Hyper-V must be running on Windows Server 2016, Windows Server 2019, or Windows Server 2022. Protect: Entire virtual machine <br><br> Recover: Entire virtual machine <br><br> Only file-consistent snapshots are supported. <br><br> For a complete list of supported Linux distributions and versions, see the article, [Linux on distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md). |

-This article describes how to back up [Azure Managed Disk](../virtual-machines/managed-disks-overview.md) using Azure CLI.

-The time required for completing the backup operation depends on various factors including size of the disk, and churn rate between consecutive backups. However, Azure Disk Backup is an agentless backup that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md), which doesn't impact the production application performance.

-This article explains how to back up [Azure Managed Disk](../virtual-machines/managed-disks-overview.md) using Azure PowerShell.

-Azure Disk Backup offers multiple backups per day. If you require more frequent backups, choose the **Hourly** backup frequency with the ability to take backups with intervals of every 4, 6, 8 or 12 hours. The backups are scheduled based on the **Time** interval selected. For example, if you select **Every 4 hours**, then the backups are taken at approximately in the interval of every 4 hours so the backups are distributed equally across the day. If a once a day backup is sufficient, then choose the **Daily** backup frequency. In the daily backup frequency, you can specify the time of the day when your backups are taken. It's important to note that the time of the day indicates the backup start time and not the time when the backup completes. The time required for completing the backup operation is dependent on various factors including size of the disk, and churn rate between consecutive backups. However, Azure Disk backup is an agentless backup that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md), which doesn't impact the production application performance.

-This article explains how to back up [Azure Managed Disk](../virtual-machines/managed-disks-overview.md) from the Azure portal.

- Azure Disk Backup offers multiple backups per day. If you require more frequent backups, choose the **Hourly** backup frequency with the ability to take backups with intervals of every *1*, *2*, *4*, *6*, *8*, or *12* hours. The backups are scheduled based on the **Time** interval selected. For example, if you select **Every 4 hours**, then the backups are taken at approximately in the interval of every 4 hours so the backups are distributed equally across the day. If a once a day backup is sufficient, then choose the **Daily** backup frequency. In the daily backup frequency, you can specify the time of the day when your backups are taken. It's important to note that the time of the day indicates the backup start time and not the time when the backup completes. The time required for completing the backup operation is dependent on various factors including size of the disk, and churn rate between consecutive backups. However, Azure Disk backup is an agentless backup that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md), which doesn't impact the production application performance.

- >Azure Backup for Managed Disks uses incremental snapshots which are limited to 500 snapshots per disk. At a point in time you can have 500 snapshots for a disk. Thus, to prevent backup failure the retention duration is limited by the snapshot limit. To allow you to take on-demand backups aside from scheduled backups, backup policy limits the total backups to 450. Learn more about [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md#restrictions) for managed disk.

- >- Azure Backup uses [_incremental snapshots_](../virtual-machines/disks-incremental-snapshots.md#restrictions) of managed disks, which store only the delta changes to the disk as the last snapshot on Standard HDD storage, regardless of the storage type of the parent disk. For additional reliability, incremental snapshots are stored on Zone Redundant Storage (ZRS) by default in the ZRS supported regions. Currently, Azure Disk Backup supports operational backup of managed disks that doesn't copy backups to the Backup vault storage. So, the backup storage redundancy setting of the Backup vault doesnΓÇÖt apply to the recovery points.

- - You can't create an incremental snapshot for a particular disk outside of that disk's subscription. So, choose the resource group within the same subscription where the disk needs to be backed up. [Learn more](../virtual-machines/disks-incremental-snapshots.md#restrictions) about incremental snapshot for managed disks.

-Azure Disk Backup offers a turnkey solution that provides snapshot lifecycle management for [Azure Managed Disks](../virtual-machines/managed-disks-overview.md) by automating periodic creation of snapshots and retaining it for a configured duration using backup policy. You can manage the disk snapshots with zero infrastructure cost and without the need for custom scripting or any management overhead. This is a crash-consistent backup solution that takes point-in-time backup of a managed disk using [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md) with support for multiple backups per day. It's also an agent-less solution and doesn't impact production application performance. It supports backup and restore of both OS and data disks (including shared disks), whether or not they're currently attached to a running Azure virtual machine.

-| Azure Backup service uses Azure VM guest agent for doing backup but guest agent is not available on the target server. | The guest agent isn't enabled or is unhealthy. | [Install the VM guest agent](../virtual-machines/extensions/agent-windows.md) manually. |

-1. Make sure that the [Azure Virtual Machine Agent](../virtual-machines/extensions/agent-windows.md) is installed on the VM.

-Direct backup of Azure VMs | Back up the entire VM. | For application or file-system consistent backups, no additional agent is needed on the Azure VM. Azure Backup installs and uses an extension to the [Azure VM agent](../virtual-machines/extensions/agent-windows.md) that's running on the VM. <br><br> You can also use agentless crash-consistent backups. [Learn more](backup-azure-vms-agentless-multi-disk-crash-consistent-overview.md). | Restore as follows:<br/><br/> - **Create a basic VM**. This is useful if the VM has no special configuration, such as multiple IP addresses.<br/><br/> - **Restore the VM disk**. Restore the disk. Then attach it to an existing VM, or create a new VM from the disk by using PowerShell.<br/><br/> - **Replace the VM disk**. If a VM exists and it uses managed disks (unencrypted), you can restore a disk and use it to replace an existing disk on the VM.<br/><br/> - **Restore specific files or folders**. You can restore files or folders from a VM instead of restoring the entire VM.

-For Linux VM backups using the Linux Azure VM agent, Azure Backup supports the list of [Linux distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md). Note the following:

-|Oracle in Azure VMs | [Oracle Linux](../virtual-machines/linux/endorsed-distros.md) | Oracle 12.x or later |

-**Replace existing** | You can restore a disk and use it to replace a disk on the existing VM.<br/><br/> The current VM must exist. If it has been deleted, you can't use this option.<br/><br/> Azure Backup takes a snapshot of the existing VM before replacing the disk, and it stores the snapshot in the staging location that you specify. Existing disks connected to the VM are replaced with the selected restore point.<br/><br/> The snapshot is copied to the vault and retained in accordance with the retention policy. <br/><br/> After the replace-disk operation, the original disk is retained in the resource group. You can choose to manually delete the original disks if they aren't needed. <br/><br/>This option is supported for unencrypted managed VMs and for VMs [created from custom images](https://azure.microsoft.com/resources/videos/create-a-custom-virtual-machine-image-in-azure-resource-manager-with-powershell/). It's not supported for unmanaged disks and VMs, classic VMs, and [generalized VMs](../virtual-machines/capture-image-resource.yml).<br/><br/> If the restore point has more or fewer disks than the current VM, the number of disks in the restore point will only reflect the VM configuration.<br><br> This option is also supported for VMs with linked resources, like [user-assigned managed identity](../active-directory/managed-identities-azure-resources/overview.md) and [Azure Key Vault](/azure/key-vault/general/overview).

-Back up VMs of a certain size |You can back up any Azure VM that has at least two CPU cores and 1 GB of RAM.<br/><br/> [Learn more](../virtual-machines/sizes.md).

-Back up VMs in [availability sets](../virtual-machines/availability.md#availability-sets) | Supported.<br/><br/> You can't restore a VM in an availability set by using the option to quickly create a VM. Instead, when you restore the VM, restore the disk and use it to deploy a VM, or restore a disk and use it to replace an existing disk.

-Back up VMs that are deployed with [Azure Hybrid Benefit](../virtual-machines/windows/hybrid-use-benefit-licensing.md) | Supported.

-[Restore spot VMs](../virtual-machines/spot-vms.md) | Not supported. <br><br/> Azure Backup restores spot VMs as regular Azure VMs.

-[Restore VMs in an Azure dedicated host](../virtual-machines/dedicated-hosts.md) | Supported.<br></br>When you're restoring an Azure VM through the [Create new](backup-azure-arm-restore-vms.md#create-a-vm) option, the VM can't be restored in the dedicated host, even when the restore is successful. To achieve this, we recommend that you [restore as disks](backup-azure-arm-restore-vms.md#restore-disks). While you're restoring as disks by using the template, create a VM in a dedicated host, and then attach the disks.<br></br>This is not applicable in a secondary region while you're performing [cross-region restore](backup-azure-arm-restore-vms.md#cross-region-restore).

-[Restore Virtual Machine Scale Sets](../virtual-machine-scale-sets/virtual-machine-scale-sets-orchestration-modes.md#scale-sets-with-flexible-orchestration) | Supported for the flexible orchestration model to back up and restore a single Azure VM.

-<a name="tvm-backup">Back up trusted launch VMs</a> | Backup is supported. <br><br> Backup of trusted launch VMs is supported through [Enhanced policy](backup-azure-vms-enhanced-policy.md). You can enable backup through a [Recovery Services vault](./backup-azure-arm-vms-prepare.md), the [pane for managing a VM](./backup-during-vm-creation.md#start-a-backup-after-creating-the-vm), and the [pane for creating a VM](backup-during-vm-creation.md#create-a-vm-with-backup-configured). <br><br> **Feature details** <br><br> - Backup is supported in all regions where trusted launch VMs are available. <br><br> - Configuration of backups, alerts, and monitoring for trusted launch VMs is supported through the backup center. <br><br> - Migration of an existing [Gen2 VM](../virtual-machines/generation-2.md) (protected with Azure Backup) to a trusted launch VM is currently not supported. [Learn how to create a trusted launch VM](../virtual-machines/trusted-launch-portal.md?tabs=portal#deploy-a-trusted-launch-vm). <br><br> - Item-level restore is supported for the scenarios mentioned [here](backup-support-matrix-iaas.md#support-for-file-level-restore). <br><br> Note that if the trusted launch VM was created by converting a Standard VM, ensure that you remove all the recovery points created using Standard policy before enabling the backup operation for the VM.

-Storage type | Standard HDD, Standard SSD, Premium SSD. <br><br> Backup and restore of [zone-redundant storage disks](../virtual-machines/disks-redundancy.md#zone-redundant-storage-for-managed-disks) is supported.

-[Shared disks](../virtual-machines/disks-shared-enable.md) | Not supported. <br><br> - You can exclude shared disk with Enhanced policy and backup the other supported disks in the VM. <br><br> - You can use S2D to create a shared disk or standalone volumes by combining capacities from disks in different VMs. Azure Backup doesn't support backup of a shared volume (between VMs for database cluster or cluster Configuration) created using S2D.

-<a name="ultra-disk-backup">Ultra disks</a> | Supported with [Enhanced policy](backup-azure-vms-enhanced-policy.md). <br><br> [Supported regions](../virtual-machines/disks-types.md#ultra-disk-limitations). <br><br> - Configuration of Ultra disk protection is supported via Recovery Services vault and via virtual machine blade. <br><br> - Cross-region restore is currently not supported for machines using Ultra disks. <br><br> - GRS type vaults cannot be used for enabling backup. <br><br> - File-level restore is currently not supported for machines using Ultra disks.

-<a name="premium-ssd-v2-backup">Premium SSD v2</a> | Supported with [Enhanced policy](backup-azure-vms-enhanced-policy.md). <br><br> [Supported regions](../virtual-machines/disks-types.md#regional-availability). <br><br> - Configuration of Premium SSD v2 disk protection is supported via Recovery Services vault and via virtual machine blade. <br><br> - Cross-region restore is currently not supported for machines using Premium v2 disks and GRS type vaults cannot be used for enabling backup. <br><br> - File-level restore is currently not supported for machines using Premium SSD v2 disks.

-[Temporary disks](../virtual-machines/managed-disks-overview.md#temporary-disk) | Azure Backup doesn't back up temporary disks.

-NVMe/[ephemeral disks](../virtual-machines/ephemeral-os-disks.md) | Not supported.

-[Performance-plus disks](../virtual-machines/disks-enable-performance.md) | Recovering from snapshot tier (instant restore) will recover disks as performance-plus disks. Recovering from vault does not respect the performance-plus flag and will restore disks as normal disks.

-Azure Disk Backup offers a turnkey solution that provides snapshot lifecycle management for managed disks by automating periodic creation of snapshots and retaining it for configured duration using backup policy. You can manage the disk snapshots with zero infrastructure cost and without the need for custom scripting or any management overhead. This is a crash-consistent backup solution that takes point-in-time backup of a managed disk using [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md) with support for multiple backups per day. It's also an agent-less solution and doesn't impact production application performance. It supports backup and restore of both OS and data disks (including shared disks), whether or not they're currently attached to a running Azure virtual machine.

-Azure Disk backup is an agentless and crash consistent solution that uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md) and offers the following advantages:

-Azure Backup uses [incremental snapshots](../virtual-machines/disks-incremental-snapshots.md) of the managed disk. Incremental snapshots are charged per GiB of the storage occupied by the delta changes since the last snapshot. For example, if you're using a managed disk with a provisioned size of 128 GiB, with 100 GiB used, the first incremental snapshot is billed only for the used size of 100 GiB. 20 GiB of data is added on the disk before you create the second snapshot. Now, the second incremental snapshot is billed for only 20 GiB.

->- *On-demand copy only full* accepts any v0alue for retaintion.

-For more information, see [Troubleshoot backups on a SQL Server database](backup-sql-server-azure-troubleshoot.md).

-| Linux | Linux running as [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) or [VMware](backup-azure-backup-server-vmware.md) guest | Physical server, On-premises Hyper-V VM, Windows VM in VMware | V3 UR1 and V3 UR2 | Hyper-V must be running on Windows Server 2012 R2, Windows Server 2016, or Windows Server 2019. Protect: Entire virtual machine <br><br> Recover: Entire virtual machine <br><br> Only file-consistent snapshots are supported. <br><br> For a complete list of supported Linux distributions and versions, see the article, [Linux on distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md). |

-|Linux|Linux running as [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) or [VMware](backup-azure-backup-server-vmware.md) guest|Physical server, <br/>On-premises Hyper-V VM, <br/> Windows VM in VMware|V3, V2|Hyper-V must be running on Windows Server 2012 R2 or Windows Server 2016. Protect: Entire virtual machine<br /><br />Recover: Entire virtual machine <br/><br/> Only file-consistent snapshots are supported. <br/><br/> For a complete list of supported Linux distributions and versions, see the article, [Linux on distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md).|

-This quickstart enables backup on an existing Azure VM. If you need to create a VM, you can [create a VM with the Azure CLI](../virtual-machines/linux/quick-create-cli.md).

-Create a Recovery Services vault with [az backup vault create](/cli/azure/backup/vault#az-backup-vault-create). Specify the same resource group and location as the VM you wish to protect. If you used the [VM quickstart](../virtual-machines/linux/quick-create-cli.md), then you created:

-This quickstart describes how to enable backup on an existing Azure VM by using the Azure portal. If you need to create a VM, you can [create a VM with the Azure portal](../virtual-machines/windows/quick-create-portal.md).

-**Linux** | Install by using an RPM or a DEB package from your distribution's package repository. This is the preferred method for installing and upgrading the Azure Linux agent. All the [endorsed distribution providers](../virtual-machines/linux/endorsed-distros.md) integrate the Azure Linux agent package into their images and repositories. The agent is available on [GitHub](https://github.com/Azure/WALinuxAgent), but we don't recommend installing from there. <br><br> If you're updating the agent, make sure no backup operations are running, and update the binaries.</li><ul>

-> This article is applicable to virtual machines encrypted with Azure Disk encryption. For more information on ADE and how it differs from other disk encryption types in Azure, see [Disk Encryption Overview](../virtual-machines/disk-encryption-overview.md).

-This article describes how to restore [Azure Managed Disks](../virtual-machines/managed-disks-overview.md) from a restore point created by Azure Backup using Azure CLI.

-This article explains how to restore [Azure Managed Disks](../virtual-machines/managed-disks-overview.md) from a restore point created by Azure Backup.

-This article explains how to restore [Azure Managed Disks](../virtual-machines/managed-disks-overview.md) from a restore point created by Azure Backup.

-1. If the disk to be restored is encrypted with [customer-managed keys (CMK)](../virtual-machines/disks-enable-customer-managed-keys-portal.yml) or using [double encryption using platform-managed keys and customer-managed keys](../virtual-machines/disks-enable-double-encryption-at-rest-portal.md), then assign the **Reader** role permission to the Backup VaultΓÇÖs managed identity on the **Disk Encryption Set** resource.

-Azure Backup is [Backint certified](https://www.sap.com/dmc/exp/2013_09_adpd/enEN/#/solutions?id=s:e062231e-9fb7-4ea8-b7d2-e6fe448c592d) by SAP, to provide native backup support by taking advantage of the SAP HANA native APIs. This offering from Azure Backup aligns with the Azure Backup mantra of *zero-infrastructure backups*, which eliminate the need to deploy and manage the backup infrastructure. You can now seamlessly back up and restore SAP HANA databases that are running on Azure VMs ([M series VMs](../virtual-machines/m-series.md) are also supported now!) and apply the enterprise management capabilities that Azure Backup provides.

-Since the streams primarily deal with disks, you need to understand the disk performance for read and network performance to transfer backup data to gauge the backup and restore performance. Refer to [this article](../virtual-machines/disks-performance.md) for an in-depth understanding of disk/network throughput and performance in Azure VMs. These are also applicable to backup and restore performance.

-You can restore [Azure zone pinned VMs](../virtual-machines/windows/create-portal-availability-zone.md) in any [availability zones](../availability-zones/az-overview.md) of the same region.

-This document assumes that you already have an SAP HANA database installed on an Azure VM. (You can also [create a VM using Azure CLI](../virtual-machines/linux/quick-create-cli.md)).

-Create a Recovery Services vault with [az backup vault create](/cli/azure/backup/vault#az-backup-vault-create). Specify the same resource group and location as the VM you wish to protect. Learn how to create a VM using Azure CLI with this [VM quickstart](../virtual-machines/linux/quick-create-cli.md).

-Also on the right side, you'll find the [Azure proximity placement group's](../virtual-machines/co-location.md) name. The placement group's name is created automatically for each deployed BareMetal instance. Reference the proximity placement group when you deploy the Azure VMs that host the application layer. Use the proximity placement group associated with the BareMetal instance to ensure the Azure VMs are deployed close to the BareMetal instance.

->To locate the application layer in the same Azure datacenter as Revision 4.x, see [Azure proximity placement groups for optimal network latency](../virtual-machines/workloads/sap/sap-proximity-placement-scenarios.md).

-> [What is SAP HANA on Azure (Large Instances)?](../virtual-machines/workloads/sap/hana-overview-architecture.md)

-* To use RDP with a Linux virtual machine, you must also ensure that you have xrdp installed and configured on the Linux VM. To learn how to do this, see [Use xrdp with Linux](../virtual-machines/linux/use-remote-desktop.md).

-This article shows you how to securely and seamlessly connect to your virtual machine scale set instance in an Azure virtual network directly from the Azure portal using Azure Bastion. When you use Azure Bastion, VMs don't require a client, agent, or additional software. For more information about Azure Bastion, see the [Overview](bastion-overview.md). For more information about virtual machine scale sets, see [What are virtual machine scale sets?](../virtual-machine-scale-sets/overview.md)

- * Store your SSH private key as a secret in Azure Key Vault using the **PowerShell** or **Azure CLI** experience. Storing your private key via the Azure Key Vault portal experience interferes with the formatting and result in unsuccessful login. If you did store your private key as a secret using the portal experience and no longer have access to the original private key file, see [Update SSH key](../virtual-machines/extensions/vmaccess-linux.md#update-ssh-key) to update access to your target VM with a new SSH key pair.

- * Store your SSH private key as a secret in Azure Key Vault using the **PowerShell** or **Azure CLI** experience. Storing your private key via the Azure Key Vault portal experience interferes with the formatting and result in unsuccessful login. If you did store your private key as a secret using the portal experience and no longer have access to the original private key file, see [Update SSH key](../virtual-machines/extensions/vmaccess-linux.md#update-ssh-key) to update access to your target VM with a new SSH key pair.

- > Please store your SSH private key as a secret in Azure Key Vault using the **PowerShell** or **Azure CLI** experience. Storing your private key via the Azure Key Vault portal experience will interfere with the formatting and result in unsuccessful login. If you did store your private key as a secret using the portal experience and no longer have access to the original private key file, see [Update SSH key](../virtual-machines/extensions/vmaccess-linux.md#update-ssh-key) to update access to your target VM with a new SSH key pair.

- > Please store your SSH private key as a secret in Azure Key Vault using the **PowerShell** or **Azure CLI** experience. Storing your private key via the Azure Key Vault portal experience will interfere with the formatting and result in unsuccessful login. If you did store your private key as a secret using the portal experience and no longer have access to the original private key file, see [Update SSH key](../virtual-machines/extensions/vmaccess-linux.md#update-ssh-key) to update access to your target VM with a new SSH key pair.

-You can create a VM using the [Quickstart: Create a VM using PowerShell](../virtual-machines/windows/quick-create-powershell.md) or [Quickstart: Create a VM using the portal](../virtual-machines/windows/quick-create-portal.md) articles. Be sure you deploy the VM to the same virtual network to which you deployed Bastion. The VM you create in this section isn't a part of the Bastion configuration and doesn't become a bastion host. You connect to this VM later in this tutorial via Bastion.

-If you don't already have VMs in your virtual network, you can create a VM using [Quickstart: Create a Windows VM](../virtual-machines/windows/quick-create-portal.md), or [Quickstart: Create a Linux VM](../virtual-machines/linux/quick-create-portal.md)

- * If you don't already have a VM in a virtual network, create one using [Quickstart: Create a Windows VM](../virtual-machines/windows/quick-create-portal.md), or [Quickstart: Create a Linux VM](../virtual-machines/linux/quick-create-portal.md).

-> [Quickstart: Create a Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md)

- * If you don't already have a VM in a virtual network, create a VM by using [Quickstart: Create a Windows VM](../virtual-machines/windows/quick-create-portal.md) or [Quickstart: Create a Linux VM](../virtual-machines/linux/quick-create-portal.md).

-* A virtual machine in the virtual network. This VM isn't a part of the Bastion configuration and doesn't become a bastion host. You connect to this VM later in this tutorial via Bastion. If you don't have a VM, create one by using [Quickstart: Create a Windows VM](../virtual-machines/windows/quick-create-portal.md) or [Quickstart: Create a Linux VM](../virtual-machines/linux/quick-create-portal.md).

-1. Verify that you have configured [VNets](../virtual-network/quick-create-portal.md), and [virtual machines](../virtual-machines/windows/quick-create-portal.md) within the VNets.

- Yes. See the documentation for [Windows](../virtual-machines/extensions/key-vault-windows.md) and [Linux](../virtual-machines/extensions/key-vault-linux.md).

-[creating an image definition and version for Compute Gallery](../virtual-machines/image-version.md#create-an-image).

- See the [guide](../virtual-machines/create-gallery.md#create-a-private-gallery) for Compute Gallery creation.

-For more information, see [Azure Compute Gallery](../virtual-machines/azure-compute-gallery.md).

-> You can't use a third-party image that has additional license and purchase terms as your base image. For information about these Marketplace images, see the guidance for [Linux](../virtual-machines/linux/cli-ps-findimage.md#check-the-purchase-plan-information) or [Windows](../virtual-machines/windows/cli-ps-findimage.md#view-purchase-plan-properties) VMs.

-A snapshot is a full, read-only copy of a VHD. To create a snapshot of a VMs OS or data disks, you can use the Azure portal or command-line tools. For steps and options to create a snapshot, see the guidance for [VMs](../virtual-machines/snapshot-copy-managed-disk.md).

-For more information on using Packer to create a VM, see [Build a Linux image with Packer](../virtual-machines/linux/build-image-with-packer.md) or [Build a Windows image with Packer](../virtual-machines/windows/build-image-with-packer.md).

-When you create a pool of compute nodes in Batch, you have two options from which to select the node size and operating system: Cloud Services Configuration and Virtual Machine Configuration. [Virtual Machine Configuration](nodes-and-pools.md#virtual-machine-configuration) pools are composed of Azure VMs, which may be created from either Linux or Windows images. When you create a pool with Virtual Machine Configuration, you specify an [available compute node size](../virtual-machines/sizes.md), the virtual machine image reference to be installed on the nodes,and the Batch node agent SKU (a program that runs on each node and provides an interface between the node and the Batch service).

-The Batch service uses [virtual machine scale sets](../virtual-machine-scale-sets/overview.md) to provide compute nodes in the Virtual Machine Configuration. You can specify an image from the [Azure Marketplace](https://azuremarketplace.microsoft.com/marketplace/apps/category/compute?filters=virtual-machine-images&page=1), or [use the Azure Compute Gallery to prepare a custom image](batch-sig-images.md).

-> You can learn more about these properties and how to specify Marketplace images in [Find Linux VM images in the Azure Marketplace with the Azure CLI](../virtual-machines/linux/cli-ps-findimage.md). Note that some Marketplace images are not currently compatible with Batch.

-When you choose an [RDMA-capable size](../virtual-machines/sizes-hpc.md?toc=/azure/virtual-machines/windows/toc.json) such as A9 for the compute nodes in your Batch pool, your MPI application can take advantage of Azure's high-performance, low-latency remote direct memory access (RDMA) network.

-Look for the sizes specified as "RDMA capable" in [Sizes for virtual machines in Azure](../virtual-machines/sizes.md) (for VirtualMachineConfiguration pools) or [Sizes for Cloud Services](../cloud-services/cloud-services-sizes-specs.md) (for CloudServicesConfiguration pools).

-* High performance compute VM sizes ([Linux](../virtual-machines/sizes-hpc.md), [Windows](../virtual-machines/sizes-hpc.md))

-* GPU-enabled VM sizes ([Linux](../virtual-machines/sizes-gpu.md), [Windows](../virtual-machines/sizes-gpu.md))

-| [H16r, H16mr](../virtual-machines/sizes-hpc.md)<br/>[NC24r, NC24rs_v2, NC24rs_v3, ND24rs^*](../virtual-machines/linux/n-series-driver-setup.md#rdma-network-connectivity) | RDMA | Ubuntu 22.04 LTS <br/> (Azure Marketplace) | Intel MPI 5<br/><br/>Linux RDMA drivers | Enable inter-node communication, disable concurrent task execution |

-| [NCv3, NDv2, NDv4, NDv5 series](../virtual-machines/linux/n-series-driver-setup.md) | NVIDIA Tesla GPU (varies by series) | Ubuntu 22.04 LTS <br/> (Azure Marketplace) | NVIDIA CUDA or CUDA Toolkit drivers | N/A |

-| [NVv3, NVv4, NVv5 series](../virtual-machines/linux/n-series-driver-setup.md) | Accelerated Visualization GPU | Ubuntu 22.04 LTS <br/> (Azure Marketplace) | NVIDIA GRID drivers (if required) | N/A |

-| [H16r, H16mr](../virtual-machines/sizes-hpc.md)<br/>[NC24r, NC24rs_v2, NC24rs_v3, ND24rs^*](../virtual-machines/windows/n-series-driver-setup.md#rdma-network-connectivity) | RDMA | Windows Server 2016, 2012 R2, or<br/>2012 (Azure Marketplace) | Microsoft MPI 2012 R2 or later, or<br/> Intel MPI 5<br/><br/>Windows RDMA drivers | Enable inter-node communication, disable concurrent task execution |

-| [NC, NCv2, NCv3, ND, NDv2 series](../virtual-machines/windows/n-series-driver-setup.md) | NVIDIA Tesla GPU (varies by series) | Windows Server 2016 or <br/>2012 R2 (Azure Marketplace) | NVIDIA CUDA or CUDA Toolkit drivers| N/A |

-| [NV, NVv2, NVv4 series](../virtual-machines/windows/n-series-driver-setup.md) | NVIDIA Tesla M60 GPU | Windows Server 2016 or<br/>2012 R2 (Azure Marketplace) | NVIDIA GRID drivers | N/A |

-| [H16r, H16mr](../virtual-machines/sizes-hpc.md) | RDMA | Windows Server 2016, 2012 R2, 2012, or<br/>2008 R2 (Guest OS family) | Microsoft MPI 2012 R2 or later, or<br/>Intel MPI 5<br/><br/>Windows RDMA drivers | Enable inter-node communication,<br/> disable concurrent task execution |

-2. Add the [NVIDIA GPU Drivers extension](../virtual-machines/extensions/hpccompute-gpu-linux.md) to the VM by using the Azure portal, a client computer that connects to the Azure subscription, or Azure Cloud Shell. Alternatively, follow the steps to connect to the VM and [install CUDA drivers](../virtual-machines/linux/n-series-driver-setup.md) manually.

-2. Add the HpcVmDrivers extension to the VM by [running an Azure PowerShell command](../virtual-machines/sizes-hpc.md) from a client computer that connects to your Azure subscription, or using Azure Cloud Shell.

-|`vmSize`|String|The size of the virtual machines in the pool. All virtual machines in a pool are the same size. <br/><br/> For information about available sizes of virtual machines for Cloud Services pools (pools created with cloudServiceConfiguration), see [Sizes for Cloud Services](../cloud-services/cloud-services-sizes-specs.md). Batch supports all Cloud Services VM sizes except `ExtraSmall`.<br/><br/> For information about available VM sizes for pools using images from the Virtual Machines Marketplace (pools created with virtualMachineConfiguration) see [Sizes for Virtual Machines](../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json) (Linux) or [Sizes for Virtual Machines](../virtual-machines/sizes.md?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json) (Windows). Batch supports all Azure VM sizes except `STANDARD_A0` and those with premium storage (`STANDARD_GS`, `STANDARD_DS`, and `STANDARD_DSV2` series).|

-Batch pools in the Virtual Machine configuration support almost all [VM sizes](../virtual-machines/sizes.md) available in Azure.

-For example, Batch supports [ddv4 and ddsv4](../virtual-machines/ddv4-ddsv4-series.md), but does not support

-[dv4 and dsv4](../virtual-machines/dv4-dsv4-series.md).

-Some VM series, such as [FX](../virtual-machines/fx-series.md) and [Mv2](../virtual-machines/mv2-series.md), can only be used

-with [generation 2 VM images](../virtual-machines/generation-2.md). Generation 2 VM images are specified like any VM image,

-As with other workloads and types of application, a custom VM image can be created with the required rendering applications and plug-ins. The custom VM image is placed in the [Azure Compute Gallery](../virtual-machines/shared-image-galleries.md) and [can be used to create Batch Pools](batch-sig-images.md).

-For a description of available VM families and VM sizes, [see VM types and sizes](../virtual-machines/sizes.md).

-When you create an Azure Batch pool using the Virtual Machine Configuration, you specify a VM image that provides the operating system for each compute node in the pool. You can create a pool of virtual machines either with a supported Azure Marketplace image or create a custom image with an [Azure Compute Gallery image](../virtual-machines/shared-image-galleries.md).

-You need to create an Azure Compute Gallery to make your custom image available. Select this gallery when creating image in the following steps. To learn how to create an Azure Compute Gallery for your images, see [Create an Azure Compute Gallery](../virtual-machines/create-gallery.md).

-To create an image from a VM in the portal, see [Capture an image of a VM](../virtual-machines/capture-image-portal.md).

-To create an image using a source other than a VM, see [Create an image](../virtual-machines/image-version.md).

-> If the base image has purchase plan information, ensure that the gallery image has identical purchase plan information as the base image. For more information on creating image which has purchase plan, refer to [Supply Azure Marketplace purchase plan information when creating images](../virtual-machines/marketplace-images.md).

-> For the purchase plan information about these Marketplace images, see the guidance for [Linux](../virtual-machines/linux/cli-ps-findimage.md#check-the-purchase-plan-information) or [Windows](../virtual-machines/windows/cli-ps-findimage.md#view-purchase-plan-properties) VMs.

-| **Available regions** | All regions that support [Spot VMs](../virtual-machines/spot-vms.md) | All regions except Microsoft Azure operated by 21Vianet |

-| **Customer eligibility** | Not available for some subscription offer types. See more about [Spot limitations](../virtual-machines/spot-vms.md#limitations). | Available for all Batch customers |

-You can also use Batch to [run tightly coupled workloads](batch-mpi.md), where the applications you run need to communicate with each other, rather than running independently. Tightly coupled applications normally use the Message Passing Interface (MPI) API. You can run your tightly coupled workloads with Batch using [Microsoft MPI](/message-passing-interface/microsoft-mpi) or Intel MPI. Improve application performance with specialized [HPC](../virtual-machines/sizes-hpc.md) and [GPU-optimized](../virtual-machines/sizes-gpu.md) VM sizes.

-When upgrading images, VMs in Azure Batch Pool will follow roughly the same work flow as VirtualMachineScaleSets. To learn more about the detailed steps involved in the Auto OS Upgrade process for VirtualMachineScaleSets, you can refer to [VirtualMachineScaleSet page](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#how-does-automatic-os-image-upgrade-work).

-Only certain OS platform images are currently supported for automatic upgrade. For detailed images list, you can get from [VirtualMachineScaleSet page](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#supported-os-images).

-> To connect to a node running the Linux virtual machine configuration via SSH, see [Install and configure xrdp to use Remote Desktop with Ubuntu](../virtual-machines/linux/use-remote-desktop.md). To connect to nodes running Windows via RDP, see [How to connect and sign on to an Azure virtual machine running Windows](../virtual-machines/windows/connect-logon.md).

-[Virtual Machine Scale Sets Auto OS upgrade Support Matrix](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#supported-os-images).

-Consider using isolated VM sizes for workloads with compliance or regulatory requirements. Supported isolated sizes in virtual machine configuration mode include `Standard_E80ids_v4`, `Standard_M128ms`, `Standard_F72s_v2`, `Standard_G5`, `Standard_GS5`, and `Standard_E64i_v3`. For more information about isolated VM sizes, see [Virtual machine isolation in Azure](../virtual-machines/isolation.md).

-see this [article](../virtual-machines/linux/add-disk.md). Ensure that there are no dependencies or races as described by the Tip

-[article](../virtual-machines/windows/attach-disk-ps.md). Ensure any sample scripts are validated for idempotency before

-Also note that you can't create a pool with a zonal policy if it has inter-node communication enabled and uses a [VM SKU that supports InfiniBand](../virtual-machines/extensions/enable-infiniband.md).

-Some Azure virtual machine (VM) series support the use of [ephemeral OS disks](../virtual-machines/ephemeral-os-disks.md), which create the OS disk on the node virtual machine local storage. The default Batch pool configuration uses [Azure managed disks](../virtual-machines/managed-disks-overview.md) for the node OS disk, where the managed disk is like a physical disk, but virtualized and persisted in remote Azure Storage.

-To determine whether a VM series supports ephemeral OS disks, check the documentation for each VM instance. For example, the [Ddv4 and Ddsv4-series](../virtual-machines/ddv4-ddsv4-series.md) supports ephemeral OS disks.

-Alternately, you can programmatically query to check the 'EphemeralOSDiskSupported' capability. An example PowerShell cmdlet to query this capability is provided in the [ephemeral OS disk frequently asked questions](../virtual-machines/ephemeral-os-disks-faq.md).

-> If you are creating your pool with a Linux [custom image](batch-sig-images.md), you can only enable disk encryption only if your pool is using an [Encryption At Host Supported VM size](../virtual-machines/disk-encryption.md#supported-vm-sizes).

-> Encryption At Host is not currently supported on User Subscription Pools until the feature becomes [publicly available in Azure](../virtual-machines/disks-enable-host-based-encryption-portal.md#prerequisites).

-Some disk encryption configurations require that the VM family of the pool supports encryption at host. See [End-to-end encryption using encryption at host](../virtual-machines/disks-enable-host-based-encryption-portal.md) to determine which VM families support encryption at host.

- For more information about using spot VMs, see [Spot Virtual Machines](../virtual-machines/spot-vms.md).

-Within the Batch nodes, you can get managed identity tokens and use them to authenticate through Microsoft Entra authentication via the [Azure Instance Metadata Service](../virtual-machines/windows/instance-metadata-service.md).

-When you create an Azure Batch pool, you can choose from among almost all the VM families and sizes available in Azure. Azure offers a range of VM sizes for different workloads, including specialized [HPC](../virtual-machines/sizes-hpc.md) or [GPU-enabled](../virtual-machines/sizes-gpu.md) VM sizes. Note that node sizes can only be chosen at the time a pool is created. In other words, once a pool is created, its node size cannot be changed.

-1. Notice that Azure Batch is a free service and that the costs associated with Azure Batch are for the underlying resources that run your workloads. When adding Azure Batch to your estimate, the pricing calculator automatically creates a selection for **Cloud Services** and **Virtual machines**. You can read more about [Azure Cloud Services](../cloud-services/cloud-services-choose-me.md) and [Azure Virtual Machines (VMs)](../virtual-machines/overview.md) in each product's documentation. What you need to know for estimated the cost of Azure Batch is that virtual machines are the most significant resource.

- Select options from the drop-downs. There are various options available to choose from. The options that have the largest impact on your estimate total are your virtual machine's operating system, the operating system license if applicable, the [VM size](../virtual-machines/sizes.md) you select under **INSTANCE**, the number of instances you choose, and the amount of time your month your instances to run.

- You can learn more about the cost of running virtual machines from the [Plan to manage costs for virtual machines documentation](../virtual-machines/cost-optimization-plan-to-manage-costs.md).

- - To learn more about the costs associated with virtual machines, see the [How you're charged for virtual machines section of Plan to manage costs for virtual machines](../virtual-machines/cost-optimization-plan-to-manage-costs.md#how-youre-charged-for-virtual-machines).

- - By default, the OS disk is deleted with the VM, but it can be [set not to during the VM's creation](../virtual-machines/delete.md)

-If you intend to use Batch for a long period of time, you can reduce the cost of VMs by using [Azure Reservations](../cost-management-billing/reservations/save-compute-costs-reservations.md) for your workloads. A reservation rate is considerably lower than a pay-as-you-go rate. Virtual machine instances used without a reservation are charged at the pay-as-you-go rate. When you purchase a reservation, the reservation discount is applied. When you commit to one-year or three-year plans for [VM instances](../virtual-machines/prepay-reserved-vm-instances.md), significant discounts are applied to VM usage, including [VMs consumed via Batch pools](../virtual-machines/prepay-reserved-vm-instances.md#determine-the-right-vm-size-before-you-buy).

-[Trusted Launch](../virtual-machines/trusted-launch.md) (requires Gen2 VM images and a compatible VM size),

-[Virtual Machine Scale Sets Auto OS upgrade Support Matrix](../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md#supported-os-images).

-This section applies to the `Microsoft.Compute/virtualMachines` resource type. [Learn more about Virtual Machines](../virtual-machines/overview.md).

-This section applies to the `Microsoft.Compute/virtualMachineScaleSets` resource type. [Learn more about Virtual Machine Scale Sets](../virtual-machine-scale-sets/overview.md).

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode, optional otherwise. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

-Virtual machines have two target types. One target type enables service-direct faults (where no agent is required). The other target type enables agent-based faults (which requires the installation of an agent). The chaos agent is an application installed on your VM as a [VM extension](../virtual-machines/extensions/overview.md). You use it to inject faults in the guest operating system.

-Virtual machines have two target types. One target type enables service-direct faults (where no agent is required). Another target type enables agent-based faults (which requires the installation of an agent). The chaos agent is an application installed on your VM as a [VM extension](../virtual-machines/extensions/overview.md). You use it to inject faults in the guest operating system.

-For more information, see [Bring VMs up to date with the latest scale set model](../virtual-machine-scale-sets/virtual-machine-scale-sets-upgrade-policy.md).

-|[Av2](../virtual-machines/av2-series.md) | 100 |

-|[D](../virtual-machines/sizes-previous-gen.md?bc=%2fazure%2fvirtual-machines%2flinux%2fbreadcrumb%2ftoc.json&toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json#d-series) | 160 |

-|[Dv2](../virtual-machines/dv2-dsv2-series.md) | 210 - 250* |

-|[Dv3](../virtual-machines/dv3-dsv3-series.md) | 160 - 190* |

-|[Dav4](../virtual-machines/dav4-dasv4-series.md) | 230 - 260 |

-|[Eav4](../virtual-machines/eav4-easv4-series.md) | 230 - 260 |

-|[Ev3](../virtual-machines/ev3-esv3-series.md) | 160 - 190* |

-|[G](../virtual-machines/sizes-previous-gen.md?bc=%2fazure%2fvirtual-machines%2flinux%2fbreadcrumb%2ftoc.json&toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json#g-series) | 180-240* |

-|[H](../virtual-machines/h-series.md) | 290 - 300* |

-> Microsoft Azure has introduced newer generations of high-performance computing (HPC), general purpose, and memory-optimized virtual machines (VMs). For this reason, we recommend that you migrate workloads from the original H-series and H-series Promo VMs to our newer offerings by August 31, 2022. Azure [HC](../virtual-machines/hc-series.md), [HBv2](../virtual-machines/hbv2-series.md), [HBv3](../virtual-machines/hbv3-series.md), [Dv4](../virtual-machines/dv4-dsv4-series.md), [Dav4](../virtual-machines/dav4-dasv4-series.md), [Ev4](../virtual-machines/ev4-esv4-series.md), and [Eav4](../virtual-machines/eav4-easv4-series.md) VMs have greater memory bandwidth, improved networking capabilities, and better cost and performance across various HPC workloads.

-The Key Vault VM extension provides automatic refresh of certificates stored in an Azure key vault. Specifically, the extension monitors a list of observed certificates stored in key vaults. When the extension detects a change, it retrieves and installs the corresponding certificates. For more information, see [Key Vault VM extension for Windows](../virtual-machines/extensions/key-vault-windows.md).

-This article provides a feature analysis of Cloud Services (extended support) and Virtual Machine Scale Sets. For more information on Virtual Machine Scale Sets, visit the documentation [here](../virtual-machine-scale-sets/overview.md)

-When evaluating migration plans from Cloud Services (classic) to Cloud Services (extended support), you may want to investigate other Azure services such as: [Virtual Machine Scale Sets](../virtual-machine-scale-sets/overview.md), [App Service](../app-service/overview.md), [Azure Kubernetes Service](/azure/aks/intro-kubernetes), and [Azure Service Fabric](../service-fabric/overview-managed-cluster.md). These services continue to feature other capabilities, while Cloud Services (extended support) maintains feature parity with Cloud Services (classic).

-The migration tool utilizes the same APIs and has the same experience as the [Virtual Machine (classic) migration](../virtual-machines/migration-classic-resource-manager-overview.md).

-For more information, see [Overview of Platform-supported migration of IaaS resources from classic to Azure Resource Manager](../virtual-machines/migration-classic-resource-manager-overview.md)

-Planning is the most important step for a successful migration experience. Review the [Cloud Services (extended support) overview](overview.md) and [Planning for migration of IaaS resources from classic to Azure Resource Manager](../virtual-machines/migration-classic-resource-manager-plan.md) before beginning any migration steps.

-When evaluating migration plans from Cloud Services (classic) to Cloud Services (extended support), you may want to investigate other Azure services such as: [Virtual Machine Scale Sets](../virtual-machine-scale-sets/overview.md), [App Service](../app-service/overview.md), [Azure Kubernetes Service](/azure/aks/intro-kubernetes), and [Azure Service Fabric](../service-fabric/service-fabric-overview.md). These services continue to feature additional capabilities, while Cloud Services (extended support) maintains feature parity with Cloud Services (classic.)

-| upgradeDomainCount | Optional. Specifies the number of upgrade domains across which roles in this service are allocated. Role instances are allocated to an upgrade domain when the service is deployed. For more information, see [Update a Cloud Service role or deployment](sample-update-cloud-service.md) and [Manage the availability of virtual machines](../virtual-machines/availability.md) You can specify up to 20 upgrade domains. If not specified, the default number of upgrade domains is 5.|

-[Quick steps: Create and use an SSH public-private key pair for Linux VMs in Azure](../virtual-machines/linux/mac-create-ssh-keys.md?toc=%2fazure%2fvirtual-machines%2flinux%2ftoc.json) describes how to create certificates with SSH.

-* To learn how to enable the diagnostics extension for Virtual Machines, see [Create a Windows Virtual machine with monitoring and diagnostics using Azure Resource Manager Template](../virtual-machines/extensions/diagnostics-template.md)

-> This article references CentOS, a Linux distribution that is End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the [CentOS End Of Life guidance](~/articles/virtual-machines/workloads/centos/centos-end-of-life.md).

-[execution model-vms]:../virtual-machines/windows/overview.md

-* The A8-A11 and H-series sizes are also known as *compute-intensive instances*. The hardware that runs these sizes is designed and optimized for compute-intensive and network-intensive applications, including high-performance computing (HPC) cluster applications, modeling, and simulations. The A8-A11 series uses Intel Xeon E5-2670 @ 2.6 GHz and the H-series uses Intel Xeon E5-2667 v3 @ 3.2 GHz. For detailed information and considerations about using these sizes, see [High performance compute virtual machine (VM) sizes](../virtual-machines/sizes-hpc.md?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json).

-For information and considerations about using these sizes, see [High performance compute VM sizes](../virtual-machines/sizes-hpc.md?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json).

-> Microsoft Azure has introduced newer generations of high-performance computing (HPC), general purpose, and memory-optimized virtual machines (VMs). For this reason, we recommend that you migrate workloads from the original H-series and H-series Promo VMs to our newer offerings by August 31, 2022. Azure [HC](../virtual-machines/hc-series.md), [HBv2](../virtual-machines/hbv2-series.md), [HBv3](../virtual-machines/hbv3-series.md), [Dv4](../virtual-machines/dv4-dsv4-series.md), [Dav4](../virtual-machines/dav4-dasv4-series.md), [Ev4](../virtual-machines/ev4-esv4-series.md), and [Eav4](../virtual-machines/eav4-easv4-series.md) VMs have greater memory bandwidth, improved networking capabilities, and better cost and performance across various HPC workloads.

-* Learn more [about high performance compute VM sizes](../virtual-machines/sizes-hpc.md?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json) for HPC workloads.

-| upgradeDomainCount | Optional. Specifies the number of upgrade domains across which roles in this service are allocated. Role instances are allocated to an upgrade domain when the service is deployed. For more information, see [Update a cloud service role or deployment](cloud-services-how-to-manage-portal.md#update-a-cloud-service-role-or-deployment), [Manage the availability of virtual machines](../virtual-machines/availability.md) and [What is a Cloud Service Model](./cloud-services-model-and-package.md).<br /><br /> You can specify up to 20 upgrade domains. If not specified, the default number of upgrade domains is 5.|

-You can find more details at [Getting started with Confidential Containers on ACI](../container-instances/container-instances-confidential-overview.md).

-Azure confidential computing offers [DCsv2-series](../virtual-machines/dcv2-series.md) and [DCsv3/DCdsv3-series](../virtual-machines/dcv3-series.md) virtual machines (VMs). These VMs have support for [Intel® Software Guard Extensions (SGX)](https://intel.com/sgx).

-Each enclave has an encrypted page cache (EPC) with a set size. The EPC determines the amount of memory that an enclave can hold. [DCsv2-series](../virtual-machines/dcv2-series.md) VMs hold up to 168 MiB. [DCsv3/DCdsv3-series](../virtual-machines/dcv3-series.md) VMs hold up to 256 GB for more memory-intensive workloads.

-By default, AKS all workloads share the same kernel and the same cluster admin. With the preview of Pod Sandboxing on AKS, the isolation grew a notch higher with the ability to provide kernel isolation for workloads on the same AKS node. You can read more about the feature [here](/azure/aks/use-pod-sandboxing). Confidential Containers are the next step of this isolation and it uses the memory encryption capabilities of the underlying AMD SEV-SNP virtual machine sizes. These virtual machines are the [DCa_cc](../virtual-machines/dcasccv5-dcadsccv5-series.md) and [ECa_cc](../virtual-machines/ecasccv5-ecadsccv5-series.md) sizes with the capability of surfacing the hardwareΓÇÖs root of trust to the pods deployed on it.

-[Confidential containers on ACI](../container-instances/container-instances-confidential-overview.md) enables fast and easy deployment of containers natively in Azure and with the ability to protect data and code in use thanks to AMD EPYC™ processors with confidential computing capabilities. This is because your container(s) runs in a hardware-based and attested Trusted Execution Environment (TEE) without the need to adopt a specialized programming model and without infrastructure management overhead. With this launch you get:

-Run the following command to add a user node pool of `Standard_DC4s_v3` size with three nodes to the AKS cluster. You can choose another larger sized SKU from the [list of supported DCsv2/DCsv3 SKUs and regions](../virtual-machines/dcv3-series.md).

-[Intel SGX Confidential VMs - DCsv2 SKU List](../virtual-machines/dcv2-series.md)

-[Intel SGX Confidential VMs - DCsv3 SKU List](../virtual-machines/dcv3-series.md)

-1. If you want to create a generalized image, [remove machine-specific information](../virtual-machines/generalize.yml) before you create the image.

-1. In the **Create an image** page that opens, [create your image definition and version](../virtual-machines/image-version.md?tabs=portal#create-an-image).

-1. If you want to create a generalized image, [remove machine-specific information](../virtual-machines/generalize.yml) for the disk or snapshot before you create the image.

- 1. For **Operating system state**, select either **Generalized** or **Specialized** depending on your use case. If you're using a managed image as the source, always select **Generalized**. If you're using a storage blob (VHD) and want to select **Generalized**, follow the steps to [generalize a Linux VHD](../virtual-machines/linux/create-upload-generic.md) or [generalize a Windows VHD](../virtual-machines/windows/upload-generalized-managed.md) before you continue.

-| [**NCCadsH100v5-series**](../virtual-machines/sizes/gpu-accelerated/nccadsh100v5-series.md) | AMD SEV-SNP and NVIDIA H100 Tensor Core GPUs | CVM with Confidential GPU. |

-For more information about supported and unsupported VM scenarios, see [support for generation 2 VMs on Azure](../virtual-machines/generation-2.md).

-Choose `VMGuestStateOnly` for no OS disk confidential encryption. Or, choose `DiskWithVMGuestState` for OS disk confidential encryption with a platform-managed key. Secure Boot is enabled by default, but is optional for `VMGuestStateOnly`. For more information, see [secure boot and vTPM](../virtual-machines/trusted-launch.md). For more information on disk encryption and encryption at host, see [confidential OS disk encryption](confidential-vm-overview.md) and [encryption at host](/azure/virtual-machines/linux/disks-enable-host-based-encryption-cli).

-To create a confidential [disk encryption set](../virtual-machines/linux/disks-enable-customer-managed-keys-cli.md), you have two options: Using [Azure Key Vault](/azure/key-vault/general/quick-create-cli) or [Azure Key Vault managed Hardware Security Module (HSM)](/azure/key-vault/managed-hsm/quick-create-cli). Based on your security and compliance needs you can choose either option. However, it is important to note that the standard SKU is not supported. The following example uses Azure Key Vault Premium.

-8. Create a VM with the [az vm create](/cli/azure/vm) command. Choose `DiskWithVMGuestState` for OS disk confidential encryption with a customer-managed key. Enabling secure boot is optional, but recommended. For more information, see [secure boot and vTPM](../virtual-machines/trusted-launch.md). For more information on disk encryption, see [confidential OS disk encryption](confidential-vm-overview.md).

- h. Toggle [Generation 2](../virtual-machines/generation-2.md) images. Confidential VMs only run on Generation 2 images. To ensure, under **Image**, select **Configure VM generation**. In the pane **Configure VM generation**, for **VM generation**, select **Generation 2**. Then, select **Apply**.

- j. For **Authentication type**, if you're creating a Linux VM, select **SSH public key** . If you don't already have SSH keys, [create SSH keys for your Linux VMs](../virtual-machines/linux/mac-create-ssh-keys.md).

-To connect to a confidential VM with a Windows OS, see [How to connect and sign on to an Azure virtual machine running Windows](../virtual-machines/windows/connect-logon.md).

- For more information about connecting to Linux VMs, see [Quickstart: Create a Linux virtual machine in the Azure portal](../virtual-machines/linux/quick-create-portal.md).

- > You have the choice of using an SSH public key or a Password for authentication. SSH is more secure. For instructions on how to generate an SSH key, see [Create SSH keys on Linux and Mac for Linux VMs in Azure](../virtual-machines/linux/mac-create-ssh-keys.md).

-> Proceed to the next section and continue with this tutorial if you deployed a Linux VM. If you deployed a Windows VM, [follow these steps to connect to your Windows VM](../virtual-machines/windows/connect-logon.md)

- > You have the choice of using an SSH public key or a Password for authentication. SSH is more secure. For instructions on how to generate an SSH key, see [Create SSH keys on Linux and Mac for Linux VMs in Azure](../virtual-machines/linux/mac-create-ssh-keys.md).

-> Proceed to the next section and continue with this tutorial if you deployed a Linux VM. If you deployed a Windows VM, [follow these steps to connect to your Windows VM](../virtual-machines/windows/connect-logon.md) and then [install the OE SDK on Windows](https://github.com/openenclave/openenclave/blob/master/docs/GettingStartedDocs/install_oe_sdk-Windows.md).

-For more information about connecting to Linux VMs, see [Create a Linux VM on Azure using the Portal](../virtual-machines/linux/quick-create-portal.md).

-Secure Key Release (SKR) flow with Azure Key Vault (AKV) with confidential container offerings can implement in couple of ways. Confidential containers run a guest enlightened exposting AMD SEV-SNP device through a Linux Kernel that uses an in guest firmware with necessary Hyper-V related patches that we refer as Direct Linux Boot (DLB). This platform doesn't use vTPM and HCL based that Confidential VMs with AMD SEV-SNP support. This concept document assumes you plan to run the containers in [Azure Container Support choosing a confidential computing SKU](../container-instances/container-instances-tutorial-deploy-confidential-containers-cce-arm.md)

-1. **Step 2:** Set up a managed identity with Microsoft Entra ID and attach that to AKV. More [here](../container-instances/container-instances-managed-identity.md)

-1. **Step 3:** Deploy your container application with required parameters within ACI by setting up a confidential computing enforcement policy. More [here](../container-instances/container-instances-tutorial-deploy-confidential-containers-cce-arm.md)

-1. **Step 7:** Send the MAA token and the managed identity token generated by ACI to AKV for key release. More [here](../container-instances/container-instances-managed-identity.md)

-[ACI with Confidential container deployments](../container-instances/container-instances-tutorial-deploy-confidential-containers-cce-arm.md)

-For more information about supported and unsupported VM scenarios, see [support for generation 2 VMs on Azure](../virtual-machines/generation-2.md).

-For these VM sizes, the physical host server allocates all available hardware resources, including EPC memory, to your virtual machine only. This deployment isn't the same as the [Azure Dedicated Host](../virtual-machines/dedicated-hosts.md) service in other VM families.

-To provide Intel SGX support on confidential compute instances, all deployments must run on Generation 2 images. Azure confidential computing supports workloads running on **Ubuntu 20.04 Gen 2**, **Windows Server 2019 Gen 2** and **Ubuntu 22.04 Gen 2**. For more information about supported and unsupported scenarios, see [support for Generation 2 VMs on Azure](../virtual-machines/generation-2.md).

-Azure confidential computing doesn't support zone-redundancy through Azure availability zones at this time. For the highest availability and redundancy for confidential computing, use [Availability Sets](../virtual-machines/availability-set-overview.md). Because of hardware restrictions, Availability Sets for confidential computing instances can only have a maximum of 10 update domains.

-To deploy using ARM templates, see [Virtual machines in an Azure Resource Manager template](../virtual-machines/windows/template-description.md). Make sure to specify the correct properties for **vmSize** and your **imageReference**.

-With Azure Logic Apps and the Azure Container Instance connector, you can set up automated tasks and workflows that deploy and manage [container groups](../container-instances/container-instances-container-groups.md). The Container Instance connector supports the following actions:

- * [Create and manage container instances](../container-instances/container-instances-quickstart.md)

-review the connector's [reference page](/connectors/aci/) or container group [YAML reference](../container-instances/container-instances-reference-yaml.md).

-[Azure Container Instances (ACI)](../container-instances/index.yml) provides a single pod of Hyper-V isolated containers on demand. It can be thought of as a lower-level "building block" option compared to Container Apps. Concepts like scale, load balancing, and certificates aren't provided with ACI containers. For example, to scale to five container instances, you create five distinct container instances. Azure Container Apps provide many application-specific concepts on top of containers, including certificates, revisions, scale, and environments. Users often interact with Azure Container Instances through other services. For example, Azure Kubernetes Service can layer orchestration and scale on top of ACI through [virtual nodes](/azure/aks/virtual-nodes). If you need a less "opinionated" building block that doesn't align with the scenarios Azure Container Apps is optimizing for, Azure Container Instances is an ideal option.

- --name "otlp1" \

- --name "otlp2" \

-| `name` | A name you select to identify your OTLP-configured endpoint. |

-description: full attestation of container groups in confidential containers on Azure Container Instances

-# Attestation in Confidential containers on Azure Container Instances

-Attestation is an essential part of confidential computing and appears in the definition by the Confidential Computing Consortium ΓÇ£Confidential Computing is the protection of data in use by performing computation in a hardware-based, attested Trusted Execution Environment."

-According to the [Remote ATtestation procedureS (RATS) Architecture](https://www.ietf.org/rfc/rfc9334.html) In remote attestation, ΓÇ£one peer (the "Attester") produces believable information about itself ("Evidence") to enable a remote peer (the "Relying Party") to decide whether to consider that Attester a trustworthy peer. Remote attestation procedures are facilitated by an additional vital party (the "Verifier").ΓÇ¥ In simpler terms, attestation is a way of proving that a computer system is trustworthy.

-In Confidential Containers on ACI you can use an attestation token to verify that the container group

-## Full attestation

-Expanding upon this concept of attestation. Full attestation captures all the components that are part of the Trusted Execution Environment that is remotely verifiable. To achieve full attestation, in Confidential Containers, we have introduced the notion of a CCE policy, which defines a set of rules, which is enforced in the utility VM. The security policy is encoded in the attestation report as an SHA-256 digest stored in the HostData attribute, as provided to the AMD SEV-SNP hardware by the host operating system during the VM boot-up. This means that the security policy enforced by the utility VM is immutable throughout the lifetime of the utility VM.

-The exhaustive list of attributes that are part of the SEV-SNP attestation can be found [here](https://www.amd.com/system/files/TechDocs/56860.pdf).

-Some important fields to consider in an attestation token returned by [Microsoft Azure Attestation ( MAA )](/azure/attestation/overview)

-| Claim | Sample value | Description |

-|::|:-:|:--:|

-| x-ms-attestation-type | sevsnpvm | String value that describes the attestation type. For example, in this scenario sevsnp hardware |

-| x-ms-compliance-status | azure-compliant-uvm | Compliance status of the utility VM that runs the container group. |

-| x-ms-sevsnpvm-hostdata | 670fff86714a650a49b58fadc1e90fedae0eb32dd51e34931c1e7a1839c08f6f | Hash of the CCE policy that was generated using tooling during deployment. |

-| x-ms-sevsnpvm-is-debuggable | false | Flag to indicate whether the underlying hardware is running in debug mode |

-## Sample attestation token generated by MAA

-```json

-{

- "header": {

- "alg": "RS256",

- "jku": "https://sharedeus2.eus2.test.attest.azure.net/certs",

- "kid": "3bdCYJabzfhISFtb3J8yuEESZwufV7hhh08N3ZflAuE=",

- "typ": "JWT"

- },

- "payload": {

- "exp": 1680259997,

- "iat": 1680231197,

- "iss": "https://sharedeus2.eus2.test.attest.azure.net",

- "jti": "d288fef5880b1501ea70be1b9366840fd56f74e666a23224d6de113133cbd8d5",

- "nbf": 1680231197,

- "nonce": "3413764049005270139",

- "x-ms-attestation-type": "sevsnpvm",

- "x-ms-compliance-status": "azure-compliant-uvm",

- "x-ms-policy-hash": "9NY0VnTQ-IiBriBplVUpFbczcDaEBUwsiFYAzHu_gco",

- "x-ms-runtime": {

- "keys": [

- {

- "e": "AQAB",

- "key_ops": [

- "encrypt"

- ],

- "kid": "Nvhfuq2cCIOAB8XR4Xi9Pr0NP_9CeMzWQGtW_HALz_w",

- "kty": "RSA",

- "n": "v965SRmyp8zbG5eNFuDCmmiSeaHpujG2bC_keLSuzvDMLO1WyrUJveaa5bzMoO0pA46pXkmbqHisozVzpiNDLCo6d3z4TrGMeFPf2APIMu-RSrzN56qvHVyIr5caWfHWk-FMRDwAefyNYRHkdYYkgmFK44hhUdtlCAKEv5UQpFZjvh4iI9jVBdGYMyBaKQLhjI5WIh-QG6Za5sSuOCFMnmuyuvN5DflpLFz595Ss-EoBIY-Nil6lCtvcGgR-IbjUYHAOs5ajamTzgeO8kx3VCE9HcyKmyUZsiyiF6IDRp2Bpy3NHTjIz7tmkpTHx7tHnRtlfE2FUv0B6i_QYl_ZA5Q"

- }

- ]

- },

- "x-ms-sevsnpvm-authorkeydigest": "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",

- "x-ms-sevsnpvm-bootloader-svn": 3,

- "x-ms-sevsnpvm-familyId": "01000000000000000000000000000000",

- "x-ms-sevsnpvm-guestsvn": 2,

- "x-ms-sevsnpvm-hostdata": "670fff86714a650a49b58fadc1e90fedae0eb32dd51e34931c1e7a1839c08f6f",

- "x-ms-sevsnpvm-idkeydigest": "cf7e12541981e6cafd150b5236785f4364850e2c4963825f9ab1d8091040aea0964bb9a8835f966bdc174d9ad53b4582",

- "x-ms-sevsnpvm-imageId": "02000000000000000000000000000000",

- "x-ms-sevsnpvm-is-debuggable": false,

- "x-ms-sevsnpvm-launchmeasurement": "a1e1a4b64e8de5c664ceee069010441f74cf039065b5b847e82b9d1a7629aaf33d5591c6b18cee48a4dde481aa88d0fb",

- "x-ms-sevsnpvm-microcode-svn": 115,

- "x-ms-sevsnpvm-migration-allowed": false,

- "x-ms-sevsnpvm-reportdata": "7ab000a323b3c873f5b81bbe584e7c1a26bcf40dc27e00f8e0d144b1ed2d14f10000000000000000000000000000000000000000000000000000000000000000",

- "x-ms-sevsnpvm-reportid": "a489c8578fb2f54d895fc8d000a85b2ff4855c015e4fb7216495c4dba4598345",

- "x-ms-sevsnpvm-smt-allowed": true,

- "x-ms-sevsnpvm-snpfw-svn": 8,

- "x-ms-sevsnpvm-tee-svn": 0,

- "x-ms-sevsnpvm-uvm-endorsement": {

- "x-ms-sevsnpvm-guestsvn": "100",

- "x-ms-sevsnpvm-launchmeasurement": "a1e1a4b64e8de5c664ceee069010441f74cf039065b5b847e82b9d1a7629aaf33d5591c6b18cee48a4dde481aa88d0fb"

- },

- "x-ms-sevsnpvm-vmpl": 0,

- "x-ms-ver": "1.0"

- }

-}

-```

-## Generating an attestation token

-We have open-sourced sidecar container implementations that provide an easy rest interface to get a raw SNP (Secure Nested Paging) report produced by the hardware or a MAA token. The sidecar is available at this [repository](https://github.com/microsoft/confidential-sidecar-containers) and can be deployed with your container group.

-## Next steps

-description: Create a container group in a virtual network and use an Azure application gateway to expose a static frontend IP address to a containerized web app.

-# Expose a static IP address for a container group

-This article shows one way to expose a static, public IP address for a [container group](container-instances-container-groups.md) by using an Azure [application gateway](../application-gateway/overview.md). Follow these steps when you need a static entry point for an external-facing containerized app that runs in Azure Container Instances.

-In this article, you use the Azure CLI to create the resources for this scenario:

-* An Azure virtual network

-* A container group deployed [in the virtual network](container-instances-vnet.md) that hosts a small web app

-* An application gateway with a public frontend IP address, a listener to host a website on the gateway, and a route to the backend container group

-As long as the application gateway runs and the container group exposes a stable private IP address in the network's delegated subnet, the container group is accessible at this public IP address.

-> [!NOTE]

-> Azure Application Gateway [supports HTTP, HTTPS, HTTP/2, and WebSocket protocols](../application-gateway/application-gateway-faq.yml).

->

-> Azure charges for an application gateway based on the amount of time that the gateway is provisioned and available, as well as the amount of data it processes. See [pricing](https://azure.microsoft.com/pricing/details/application-gateway/).

-## Create virtual network

-In a typical case, you might already have an Azure virtual network. If you don't have one, create one as shown with the following example commands. The virtual network needs separate subnets for the application gateway and the container group.

-If you need one, create an Azure resource group. For example:

-```azurecli

-az group create --name myResourceGroup --location eastus

-```

-Create a virtual network with the [az network vnet create][az-network-vnet-create] command. This command creates the *myAGSubnet* subnet in the network.

-```azurecli

-az network vnet create \

- --name myVNet \

- --resource-group myResourceGroup \

- --location eastus \

- --address-prefix 10.0.0.0/16 \

- --subnet-name myAGSubnet \

- --subnet-prefix 10.0.1.0/24

-```

-Use the [az network vnet subnet create][az-network-vnet-subnet-create] command to create a subnet for the backend container group. Here, its name is *myACISubnet*.

-```azurecli

-az network vnet subnet create \

- --name myACISubnet \

- --resource-group myResourceGroup \

- --vnet-name myVNet \

- --address-prefix 10.0.2.0/24

-```

-Use the [az network public-ip create][az-network-public-ip-create] command to create a static public IP resource. In a later step, this address is configured as the front end of the application gateway.

-```azurecli

-az network public-ip create \

- --resource-group myResourceGroup \

- --name myAGPublicIPAddress \

- --allocation-method Static \

- --sku Standard

-```

-## Create container group

-Run the following [az container create][az-container-create] to create a container group in the virtual network you configured in the previous step.

-The group is deployed in the *myACISubnet* subnet and contains a single instance named *appcontainer* that pulls the `aci-helloworld` image. As shown in other articles in the documentation, this image packages a small web app written in Node.js that serves a static HTML page.

-```azurecli

-az container create \

- --name appcontainer \

- --resource-group myResourceGroup \

- --image mcr.microsoft.com/azuredocs/aci-helloworld \

- --vnet myVNet \

- --subnet myACISubnet

-```

-When successfully deployed, the container group is assigned a private IP address in the virtual network. For example, run the following [az container show][az-container-show] command to retrieve the group's IP address:

-```azurecli

-az container show \

- --name appcontainer --resource-group myResourceGroup \

- --query ipAddress.ip --output tsv

-```

-Output is similar to: `10.0.2.4`.

-For use in a later step, save the IP address in an environment variable:

-```azurecli

-ACI_IP=$(az container show \

- --name appcontainer \

- --resource-group myResourceGroup \

- --query ipAddress.ip --output tsv)

-```

-> [!IMPORTANT]

-> If the container group is stopped, started, or restarted, the container group's private IP is subject to change. If this happens, you will need to update the application gateway configuration.

-## Create application gateway

-Create an application gateway in the virtual network, following the steps in the [application gateway quickstart](../application-gateway/quick-create-cli.md). The following [az network application-gateway create][az-network-application-gateway-create] command creates a gateway with a public frontend IP address and a route to the backend container group. See the [Application Gateway documentation](../application-gateway/index.yml) for details about the gateway settings.

-```azurecli

-az network application-gateway create \

- --name myAppGateway \

- --location eastus \

- --resource-group myResourceGroup \

- --capacity 2 \

- --sku Standard_v2 \

- --http-settings-protocol http \

- --public-ip-address myAGPublicIPAddress \

- --vnet-name myVNet \

- --subnet myAGSubnet \

- --servers "$ACI_IP" \

- --priority 100

-```

-It can take up to 15 minutes for Azure to create the application gateway.

-## Test public IP address

-Now you can test access to the web app running in the container group behind the application gateway.

-Run the [az network public-ip show][az-network-public-ip-show] command to retrieve the frontend public IP address of the gateway:

-```azurecli

-az network public-ip show \

-```

-Output is a public IP address, similar to: `52.142.18.133`.

-To view the running web app when successfully configured, navigate to the gateway's public IP address in your browser. Successful access is similar to:

-

-## Next steps

-* See a [quickstart template](https://github.com/Azure/azure-quickstart-templates/tree/master/application-workloads/wordpress/aci-wordpress-vnet) to create a container group with a WordPress container instance as a backend server behind an application gateway.

-* You can also configure an application gateway with a certificate for SSL termination. See the [overview](../application-gateway/ssl-overview.md) and the [tutorial](../application-gateway/create-ssl-portal.md).

-* Depending on your scenario, consider using other Azure load-balancing solutions with Azure Container Instances. For example, use [Azure Traffic Manager](../traffic-manager/traffic-manager-overview.md) to distribute traffic across multiple container instances and across multiple regions. See this [blog post](https://aaronmsft.com/posts/azure-container-instances/).

-[az-network-vnet-create]: /cli/azure/network/vnet#az_network_vnet_create

-[az-network-vnet-subnet-create]: /cli/azure/network/vnet/subnet#az_network_vnet_subnet_create

-[az-network-public-ip-create]: /cli/azure/network/public-ip#az_network_public_ip_create

-[az-network-public-ip-show]: /cli/azure/network/public-ip#az_network_public_ip_show

-[az-network-application-gateway-create]: /cli/azure/network/application-gateway#az_network-application-gateway-create

-[az-container-create]: /cli/azure/container#az_container_create

-[az-container-show]: /cli/azure/container#az_container_show

-description: Best practices and considerations for customers to account for in their Container Instances workloads.

-# Best practices and considerations for Azure Container Instances

-Azure Container Instances allow you to package, deploy and manage cloud applications without having to manage the underlying infrastructure. Common scenarios that run on ACI include burst workloads, task automation, and build jobs. You can use ACI by defining the resources they need per container group, including vCPU and memory. ACI is a great solution for any scenario that can operate in isolated container and provides fast start up times, hyper-visor level security, custom container sizes, and more. The information below will help you determine if Azure Container Instances is best for your scenario.

-## What to consider

-UserΓÇÖs credentials passed via command line interface (CLI) are stored as plain text in the backend. Storing credentials in plain text is a security risk; Microsoft advises customers to store user credentials in CLI environment variables to ensure they are encrypted/transformed when stored in the backend.

-There are default limits that may require quota increases. For more details: [Resource availability & quota limits for ACI - Azure Container Instances | Microsoft Learn](./container-instances-resource-and-quota-limits.md)

-Container Images can't be larger than 15 GB, any images above this size may cause unexpected behavior: [How large can my container image be?](./container-instances-faq.yml)

-If your container image is larger than 15 GB, you can [mount an Azure Fileshare](container-instances-volume-azure-files.md) to store the image.

-If a container group restarts, the container groupΓÇÖs IP may change. We advise against using a hard coded IP address in your scenario. If you need a static public IP address, use Application Gateway: [Static IP address for container group - Azure Container Instances | Microsoft Learn](./container-instances-application-gateway.md).

-There are ports that are reserved for service functionality. We advise you not to use these ports, because their use will lead to unexpected behavior: [Does the ACI service reserve ports for service functionality?](./container-instances-faq.yml).

-Your container groups may restart due to platform maintenance events. These maintenance events are done to ensure the continuous improvement of the underlying infrastructure: [Container had an isolated restart without explicit user input](./container-instances-faq.yml).

-ACI doesn't allow [privileged container operations](./container-instances-faq.yml). We advise you not to depend on using the root directory for your scenario

-## Best practices

-We advise running container groups in multiple regions so your workloads can continue to run if there's an issue in one region.

-We advise against using a hard coded IP address in your scenario since a container group's IP address isn't guaranteed. To mitigate connectivity issues, we recommend configuring a gateway. If your container is behind a public IP address and you need a static public IP address, use [Application Gateway](./container-instances-application-gateway.md). If your container is behind a virtual network and you need a static IP address, we recommend using [NAT Gateway](./container-instances-nat-gateway.md).

-## Other Azure Container options

-### Azure Container Apps

-Azure Container Apps enables you to build serverless microservices based on containers. Azure Container Apps doesn't provide direct access to the underlying Kubernetes APIs. If you require access to the Kubernetes APIs and control plane, you should use Azure Kubernetes Service. However, if you would like to build Kubernetes-style applications and don't require direct access to all the native Kubernetes APIs and cluster management, Container Apps provides a fully managed experience based on best-practices. For these reasons, many teams may prefer to start building container microservices with Azure Container Apps.

-### Azure App Service

-Azure App Service provides fully managed hosting for web applications including websites and web APIs. These web applications may be deployed using code or containers. Azure App Service is optimized for web applications. Azure App Service is integrated with other Azure services including Azure Container Apps or Azure Functions. If you plan to build web apps, Azure App Service is an ideal option.

-### Azure Container Instances

-Azure Container Instances (ACI) provides a single pod of Hyper-V isolated containers on demand. It can be thought of as a lower-level "building block" option compared to Container Apps. Concepts like scale, load balancing, and certificates aren't provided with ACI containers. For example, to scale to five container instances, you create five distinct container instances. Azure Container Apps provide many application-specific concepts on top of containers, including certificates, revisions, scale, and environments. Users often interact with Azure Container Instances through other services. For example, Azure Kubernetes Service can layer orchestration and scale on top of ACI through virtual nodes. If you need a less "opinionated" building block that doesn't align with the scenarios Azure Container Apps is optimizing for, Azure Container Instances is an ideal option.

-### Azure Kubernetes Service

-Azure Kubernetes Service (AKS) provides a fully managed Kubernetes option in Azure. It supports direct access to the Kubernetes API and runs any Kubernetes workload. The full cluster resides in your subscription, with the cluster configurations and operations within your control and responsibility. Teams looking for a fully managed version of Kubernetes in Azure, Azure Kubernetes Service is an ideal option.

-### Azure Functions

-Azure Functions is a serverless Functions-as-a-Service (FaaS) solution. It's optimized for running event-driven applications using the functions programming model. It shares many characteristics with Azure Container Apps around scale and integration with events, but optimized for ephemeral functions deployed as either code or containers. The Azure Functions programming model provides productivity benefits for teams looking to trigger the execution of your functions on events and bind to other data sources. If you plan to build FaaS-style functions, Azure Functions is the ideal option. The Azure Functions programming model is available as a base container image, making it portable to other container based compute platforms allowing teams to reuse code as environment requirements change.

-### Azure Spring Apps

-Azure Spring Apps is a fully managed service for Spring developers. If you want to run Spring Boot, Spring Cloud or any other Spring applications on Azure, Azure Spring Apps is an ideal option. The service manages the infrastructure of Spring applications so developers can focus on their code. Azure Spring Apps provides lifecycle management using comprehensive monitoring and diagnostics, configuration management, service discovery, CI/CD integration, blue-green deployments, and more.

-### Azure Red Hat OpenShift

-Azure Red Hat OpenShift is jointly engineered, operated, and supported by Red Hat and Microsoft to provide an integrated product and support experience for running Kubernetes-powered OpenShift. With Azure Red Hat OpenShift, teams can choose their own registry, networking, storage, and CI/CD solutions, or use the built-in solutions for automated source code management, container and application builds, deployments, scaling, health management, and more from OpenShift. If your team or organization is using OpenShift, Azure Red Hat OpenShift is an ideal option.

-## Next steps

-Learn how to deploy a multi-container container group with an Azure Resource Manager template:

-> [!div class="nextstepaction"]

-> [Deploy a container group][resource-manager template]

-<!-- LINKS - Internal -->

-[resource-manager template]: container-instances-multi-container-group.md

-description: Learn more about confidential container groups.

-# Confidential containers on Azure Container Instances

-This article introduces how confidential containers on Azure Container Instances can enable you to secure your workloads running in the cloud. This article provides background about the feature set, scenarios, limitations, and resources.

-Confidential containers on Azure Container Instances enable customers to run Linux containers within a hardware-based and attested Trusted Execution Environment (TEE). Customers can lift and shift their containerized Linux applications or build new confidential computing applications without needing to adopt any specialized programming models to achieve the benefits of confidentiality in a TEE. Confidential containers on Azure Container Instances protect data-in-use and encrypts data being used in memory. Azure Container Instances extends this capability through verifiable execution policies, and verifiable hardware root of trust assurances through guest attestation.

- 

-## Features of confidential containers on Azure Container Instances

-### Lift and shift applications

-Customers can lift and shift their containerized Linux applications or build new confidential computing applications without needing to adopt specialized programming models to achieve the benefits of confidentiality in a TEE.

-### Hardware based trusted execution environment

-Confidential containers on Azure Container Instances are deployed in a container group with a Hyper-V isolated TEE, which includes a memory encryption key that is generated and managed by an AMD SEV-SNP capable processor. Data in use in memory is encrypted with this key to help provide protection against data replay, corruption, remapping, and aliasing-based attacks.

-### Verifiable execution policies

-Confidential containers on Azure Container Instances can run with verifiable execution policies that enable customers to have control over what software and actions are allowed to run within the TEE. These execution policies help to protect against bad actors creating unexpected application modifications that could potentially leak sensitive data. Execution policies are authored by the customer through provided [tooling](https://github.com/Azure/azure-cli-extensions/blob/main/src/confcom/azext_confcom/README.md) and are verified through cryptographic proof.

-### Remote guest attestation

-Confidential containers on ACI provide support for remote guest attestation which is used to verify the trustworthiness of your container group before creating a secure channel with a relying party. Container groups can generate an SNP hardware attestation report, which is signed by the hardware and includes information about the hardware and software. This generated hardware attestation report can then be verified by the Microsoft Azure Attestation service via an [open-source sidecar application](https://github.com/microsoft/confidential-sidecar-containers) or by another attestation service before any sensitive data is released to the TEE.

-## Confidential computing enforcement policies

-Confidential containers support container-level integrity and attestation via confidential computing enforcement (CCE) policies. Confidential computing enforcement policies prescribe the components that are permitted to run within the container group and will be enforced by the container runtime.

-### Azure CLI confcom extension

-The Azure CLI confcom extension enables customers to generate confidential computing enforcement policies using an ARM template as an input and providing a base 64 string policy as an output. This output is included in the definition of the container group to enforce which components are permitted to run. For more details on authoring confidential computing execution policies, see [Azure CLI confcom extension](https://github.com/Azure/azure-cli-extensions/blob/main/src/confcom/azext_confcom/README.md).

-## Secure key release and encrypted file system sidecars

-Confidential containers on Azure Container Instances integrates with two open source sidecars to support confidential functionality within the container group. You can find these sidecars and more information in the [confidential sidecar repository](https://github.com/microsoft/confidential-sidecar-containers).

-### Secure key release sidecar

-Confidential containers on Azure Container Instances provide a sidecar open source container for attestation and secure key release. This sidecar instantiates a web server, which exposes a REST API so that other containers can retrieve a hardware attestation report or a Microsoft Azure Attestation token via the POST method. The sidecar integrates with Azure Key vault for releasing a key to the container group after validation has been completed.

-### Encrypted file system sidecar

-Confidential containers on Azure Container Instances provide a sidecar container to mount a remote encrypted filesystem previously uploaded to Azure Blob Storage. The sidecar container transparently retrieves the hardware attestation and the certificate chain endorsing the attestationΓÇÖs signing key. It then requests Microsoft Azure Attestation to authorize an attestation token, which is required for securely releasing the filesystemΓÇÖs encryption key from the managed HSM. The key is released to the sidecar container only if the attestation token is signed by the expected authority and the attestation claims match the keyΓÇÖs release policy. The sidecar container transparently uses the key to mount the remote encrypted filesystem; this process will preserve the confidentiality and integrity of the filesystem upon any operation from a container that is running within the container group.

-## Scenarios

-### Data clean rooms for multi-party data analytics and machine learning training

-Business transactions and project collaborations often require sharing confidential data amongst multiple parties. This data may include personal information, financial information, and medical records which need to be protected from unauthorized access. Confidential containers on Azure Container Instances provide the necessary features (hardware-based TEEs, remote attestation) for customers to process training data from multiple sources without exposing input data to other parties. This enables organizations to get more value from their or their partners' datasets while maintaining control over access to their sensitive information. This makes confidential containers on Azure Container Instances ideal for multi-party data analytics scenarios such as confidential machine learning.

- 

-### Confidential inference

-ACI provides fast and easy deployments, flexible resource allocation, and pay per use pricing, which positions it as a great platform for confidential inference workloads. With confidential containers on Azure Container Instances, model developers and data owners can collaborate while protecting the intellectual property of the model developer and keeping the data used for inferencing secure and private. Check out a [sample](https://aka.ms/ccaciinference) deployment of confidential inference using confidential containers on Azure Container Instances.

- 

-## Unsupported Scenarios

-* Confidential computing enforcement policies must be generated by the Azure CLI confcom extension and can't be manually created.

-## Resources

-* [Azure CLI confcom extension](https://github.com/Azure/azure-cli-extensions/blob/main/src/confcom/azext_confcom/README.md)

-* [Confidential sidecar containers](https://github.com/microsoft/confidential-sidecar-containers)

-* [Confidential hello world application](https://aka.ms/ccacihelloworld)

-* [Machine learning inference demo](https://aka.ms/ccaciinference)

-## Next Steps

-* For a deployment example, see [Deploy a confidential container group with Azure Resource Manager](./container-instances-tutorial-deploy-confidential-containers-cce-arm.md)

-description: This guide describes how Caddy can be used as a reverse proxy to enhance your application with automatic HTTPS

-# Enable automatic HTTPS with Caddy in a sidecar container

-This article describes how Caddy can be used as a sidecar container in a [container group](container-instances-container-groups.md) acting as a reverse proxy to provide an automatically managed HTTPS endpoint for your application.

-Caddy is a powerful, enterprise-ready, open source web server with automatic HTTPS written in Go and represents an alternative to Nginx.

-The automatization of certificates is possible because Caddy supports the ACMEv2 API ([RFC 8555](https://www.rfc-editor.org/rfc/rfc8555)) that interacts with [Let's Encrypt](https://letsencrypt.org/) to issue certificates.

-In this example, only the Caddy container gets exposed on ports 80/TCP and 443/TCP. The application behind the reverse proxy remains private. The network communication between Caddy and your application happens via localhost.

-> [!NOTE]

-> This stands in contrast to the intra container group communication known from docker compose, where containers can be referenced by name.

-The example mounts the [Caddyfile](https://caddyserver.com/docs/caddyfile), which is required to configure the reverse proxy, from a file share hosted on an Azure Storage account.

-> [!NOTE]

-> For production deployments, most users will want to bake the Caddyfile into a custom docker image based on [caddy](https://hub.docker.com/_/caddy). This way, there is no need to mount files into the container.

-## Prepare the Caddyfile

-Create a file called `Caddyfile` and paste the following configuration. This configuration creates a reverse proxy configuration, pointing to your application container listening on 5000/TCP.

-```console

-my-app.westeurope.azurecontainer.io {

- reverse_proxy http://localhost:5000

-}

-```

-It's important to note, that the configuration references a domain name instead of an IP address. Caddy needs to be reachable by this URL to carry out the challenge step required by the ACME protocol and to successfully retrieve a certificate from Let's Encrypt.

-> [!NOTE]

-> For production deployment, users might want to use a domain name they control, e.g., `api.company.com` and create a CNAME record pointing to e.g. `my-app.westeurope.azurecontainer.io`. If so, it needs to be ensured, that the custom domain name is also used in the Caddyfile, instead of the one assigned by Azure (e.g., `*.westeurope.azurecontainer.io`). Further, the custom domain name, needs to be referenced in the ACI YAML configuration described later in this example.

-## Prepare storage account

-Create a storage account

-```azurecli

-az storage account create \

- --name <storage-account> \

- --resource-group <resource-group> \

- --location westeurope

-```

-Store the connection string to an environment variable

-```azurecli

-AZURE_STORAGE_CONNECTION_STRING=$(az storage account show-connection-string --name <storage-account> --resource-group <resource-group> --output tsv)

-```

-Create the file shares required to store the container state and caddy configuration.

-```azurecli

-az storage share create \

- --name proxy-caddyfile \

- --account-name <storage-account>

-az storage share create \

- --name proxy-config \

- --account-name <storage-account>

-

- az storage share create \

- --name proxy-data \

- --account-name <storage-account>

-```

-Retrieve the storage account keys and make a note for later use

-```azurecli

-az storage account keys list -g <resource-group> -n <storage-account>

-```

-## Deploy container group

-### Create YAML file

-Create a file called `ci-my-app.yaml` and paste the following content. Ensure to replace `<account-key>` with one of the access keys previously received and `<storage-account>` accordingly.

-This YAML file defines two containers `reverse-proxy` and `my-app`. The `reverse-proxy` container mounts the three previously created file shares. The configuration also exposes port 80/TCP and 443/TCP of the `reverse-proxy` container. The communication between both containers happens on localhost only.

->[!NOTE]

-> It's important to note, that the `dnsNameLabel` key, defines the public DNS name, under which the container instance group will be reachable, it needs to match the FQDN defined in the `Caddyfile`

-```yml

-name: ci-my-app

-apiVersion: "2021-10-01"

-location: westeurope

-properties:

- containers:

- - name: reverse-proxy

- properties:

- image: caddy:2.6

- ports:

- - protocol: TCP

- port: 80

- - protocol: TCP

- port: 443

- resources:

- requests:

- memoryInGB: 1.0

- cpu: 1.0

- limits:

- memoryInGB: 1.0

- cpu: 1.0

- volumeMounts:

- - name: proxy-caddyfile

- mountPath: /etc/caddy

- - name: proxy-data

- mountPath: /data

- - name: proxy-config

- mountPath: /config

- - name: my-app

- properties:

- image: mcr.microsoft.com/azuredocs/aci-helloworld

- ports:

- - port: 5000

- protocol: TCP

- environmentVariables:

- - name: PORT

- value: 5000

- resources:

- requests:

- memoryInGB: 1.0

- cpu: 1.0

- limits:

- memoryInGB: 1.0

- cpu: 1.0

- ipAddress:

- ports:

- - protocol: TCP

- port: 80

- - protocol: TCP

- port: 443

- type: Public

- dnsNameLabel: my-app

- osType: Linux

- volumes:

- - name: proxy-caddyfile

- azureFile:

- shareName: proxy-caddyfile

- storageAccountName: "<storage-account>"

- storageAccountKey: "<account-key>"

- - name: proxy-data

- azureFile:

- shareName: proxy-data

- storageAccountName: "<storage-account>"

- storageAccountKey: "<account-key>"

- - name: proxy-config

- azureFile:

- shareName: proxy-config

- storageAccountName: "<storage-account>"

- storageAccountKey: "<account-key>"

-```

-### Deploy the container group

-Create a resource group with the [az group create](/cli/azure/group#az-group-create) command:

-```azurecli

-az group create --name <resource-group> --location westeurope

-```

-Deploy the container group with the [az container create](/cli/azure/container#az-container-create) command, passing the YAML file as an argument.

-```azurecli

-az container create --resource-group <resource-group> --file ci-my-app.yaml

-```

-### View the deployment state

-To view the state of the deployment, use the following [az container show](/cli/azure/container#az-container-show) command:

-```azurecli

-az container show --resource-group <resource-group> --name ci-my-app --output table

-```

-### Verify TLS connection

-Before verifying if everything went well, give the container group some time to fully start and for Caddy to request a certificate.

-#### OpenSSL

-We can use the `s_client` subcommand of OpenSSL for that purpose.

-```bash

-echo "Q" | openssl s_client -connect my-app.westeurope.azurecontainer.io:443

-```

-```console

-CONNECTED(00000188)

-Certificate chain

- 0 s:CN = my-app.westeurope.azurecontainer.io

- i:C = US, O = Let's Encrypt, CN = R3

- 1 s:C = US, O = Let's Encrypt, CN = R3

- i:C = US, O = Internet Security Research Group, CN = ISRG Root X1

- 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1

- i:O = Digital Signature Trust Co., CN = DST Root CA X3

-Server certificate

-MIIEgTCCA2mgAwIBAgISAxxidSnpH4vVuCZk9UNG/pd2MA0GCSqGSIb3DQEBCwUA

-MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD

-EwJSMzAeFw0yMzA0MDYxODAzMzNaFw0yMzA3MDUxODAzMzJaMC4xLDAqBgNVBAMT

-I215LWFwcC53ZXN0ZXVyb3BlLmF6dXJlY29udGFpbmVyLmlvMFkwEwYHKoZIzj0C

-AQYIKoZIzj0DAQcDQgAEaaN/wGyFcimM+1O4WzbFgO6vIlXxXqp9vgmLZHpFrNwV

-aO8JbaB7hE+M5EAg34LDY80RyHgY+Ff4vTh2Z96rVqOCAl4wggJaMA4GA1UdDwEB

-/wQEAwIHgDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/

-BAIwADAdBgNVHQ4EFgQUoL5DP+4PWiyE79hL5o+v8uymHdAwHwYDVR0jBBgwFoAU

-FC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzAB

-hhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5p

-LmxlbmNyLm9yZy8wLgYDVR0RBCcwJYIjbXktYXBwLndlc3RldXJvcGUuYXp1cmVj

-b250YWluZXIuaW8wTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEw

-KDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEEBgor

-BgEEAdZ5AgQCBIH1BIHyAPAAdgC3Pvsk35xNunXyOcW6WPRsXfxCz3qfNcSeHQmB

-Je20mQAAAYdX8+CQAAAEAwBHMEUCIQC9Ztqd3DXoJhOIHBW+P7ketGrKlVA6nPZl

-9CiOrn6t8gIgXHcrbBqItemndRMv+UJ3DaBfTkYOqECecOJCgLhSYNUAdgDoPtDa

-PvUGNTLnVyi8iWvJA9PL0RFr7Otp4Xd9bQa9bgAAAYdX8+CAAAAEAwBHMEUCIBJ1

-24z44vKFUOLCi1a7ymVuWErkmLb/GtysvcxILaj0AiEAr49hyKfen4BbSTwC8Fg4

-/LgZnn2F3uHI+9p+ZMO9xTAwDQYJKoZIhvcNAQELBQADggEBACqxa21eiW3JrZwk

-FHgpd6SxhUeecrYXxFNva1Y6G//q2qCmGeKK3GK+ZGPqDtcoASH5t5ghV4dIT4WU

-auVDLFVywXzR8PT6QUu3W8QxU+W7406twBf23qMIgrF8PIWhStI5mn1uCpeqlnf5

-HpRaj2f5/5n19pcCZcrRx94G9qhPYdMzuy4mZRhxXRqrpIsabqX3DC2ld8dszCvD

-pkV61iuARgm3MIQz1yL/x5Bn4nywjnhYZA4KFktC0Ti55cPRh1mkzGQAsYQDdWrq

-dVav+U9dOLQ4Sq4suaDmzDzApr+hpQSJhwgRN16+tLMyZ6INAU2JWKDxiyDTdOuH

-jz456og=

-subject=CN = my-app.westeurope.azurecontainer.io

-issuer=C = US, O = Let's Encrypt, CN = R3

-No client certificate CA names sent

-Peer signing digest: SHA256

-Peer signature type: ECDSA

-Server Temp Key: X25519, 253 bits

-SSL handshake has read 4208 bytes and written 401 bytes

-Verification error: unable to get local issuer certificate

-New, TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256

-Server public key is 256 bit

-Secure Renegotiation IS NOT supported

-Compression: NONE

-Expansion: NONE

-No ALPN negotiated

-Early data was not sent

-Verify return code: 20 (unable to get local issuer certificate)

-Post-Handshake New Session Ticket arrived:

-SSL-Session:

- Protocol : TLSv1.3

- Cipher : TLS_AES_128_GCM_SHA256

- Session-ID: 85F1A4290F99A0DD28C8CB21EF4269E7016CC5D23485080999A8548057729B24

- Session-ID-ctx:

- Resumption PSK: 752D438C19A5DBDBF10781F863D5E5D9A8859230968A9EAFFF7BBA86937D004F

- PSK identity: None

- PSK identity hint: None

- SRP username: None

- TLS session ticket lifetime hint: 604800 (seconds)

- TLS session ticket:

- 0000 - 2f 25 98 90 9d 46 9b 01-03 78 db bd 4d 64 b3 a6 /%...F...x..Md..

- 0010 - 52 c0 7a 8a b6 3d b8 4b-c0 d7 fc 04 e8 63 d4 bb R.z..=.K.....c..

- 0020 - 15 b3 25 b7 be 64 3d 30-2b d7 dc 7a 1a d1 22 63 ..%..d=0+..z.."c

- 0030 - 42 30 90 65 6b b5 e1 83-a3 6c 76 c8 f6 ae e9 31 B0.ek....lv....1

- 0040 - 45 91 33 57 8e 9f 4b 6a-2e 2c 9b f9 87 5f 71 1d E.3W..Kj.,..._q.

- 0050 - 5a 84 59 50 17 31 1f 62-2b 0e 1e e5 70 03 d9 e9 Z.YP.1.b+...p...

- 0060 - 50 1c 5d 1f a4 3c 8a 0e-f4 c5 7d ce 9e 5c 98 de P.]..<....}..\..

- 0070 - e5 .

- Start Time: 1680808973

- Timeout : 7200 (sec)

- Verify return code: 20 (unable to get local issuer certificate)

- Extended master secret: no

- Max Early Data: 0

-read R BLOCK

-```

-#### Chrome browser

-Navigate to ``` https://my-app.westeurope.azurecontainer.io ``` and verify the certificate by clicking on the padlock next to the URL.

-To see the certificate details, click on "Connection is secure" followed by "certificate is valid".

-## Next steps

-description: Create an SSL or TLS endpoint for a container group running in Azure Container Instances by running Nginx in a sidecar container

-# Enable a TLS endpoint in a sidecar container

-This article shows how to create a [container group](container-instances-container-groups.md) with an application container and a sidecar container running a TLS/SSL provider. By setting up a container group with a separate TLS endpoint, you enable TLS connections for your application without changing your application code.

-You set up an example container group consisting of two containers:

-* An application container that runs a simple web app using the public Microsoft aci-helloworld image.

-* A sidecar container running the public [Nginx](https://hub.docker.com/_/nginx) image, configured to use TLS.

-In this example, the container group only exposes port 443 for Nginx with its public IP address. Nginx routes HTTPS requests to the companion web app, which listens internally on port 80. You can adapt the example for container applications that listen on other ports.

-See [Next steps](#next-steps) for other approaches to enabling TLS in a container group.

-## Create a self-signed certificate

-To set up Nginx as a TLS provider, you need a TLS/SSL certificate. This article shows how to create and set up a self-signed TLS/SSL certificate. For production scenarios, you should obtain a certificate from a certificate authority.

-To create a self-signed TLS/SSL certificate, use the [OpenSSL](https://www.openssl.org/) tool available in Azure Cloud Shell and many Linux distributions, or use a comparable client tool in your operating system.

-First create a certificate request (.csr file) in a local working directory:

-```console

-openssl req -new -newkey rsa:2048 -nodes -keyout ssl.key -out ssl.csr

-```

-Follow the prompts to add the identification information. For Common Name, enter the hostname associated with the certificate. When prompted for a password, press Enter without typing, to skip adding a password.

-Run the following command to create the self-signed certificate (.crt file) from the certificate request. For example:

-```console

-openssl x509 -req -days 365 -in ssl.csr -signkey ssl.key -out ssl.crt

-```

-You should now see three files in the directory: the certificate request (`ssl.csr`), the private key (`ssl.key`), and the self-signed certificate (`ssl.crt`). You use `ssl.key` and `ssl.crt` in later steps.

-## Configure Nginx to use TLS

-### Create Nginx configuration file

-In this section, you create a configuration file for Nginx to use TLS. Start by copying the following text into a new file named `nginx.conf`. In Azure Cloud Shell, you can use Visual Studio Code to create the file in your working directory:

-```console

-code nginx.conf

-```

-In `location`, be sure to set `proxy_pass` with the correct port for your app. In this example, we set port 80 for the `aci-helloworld` container.

-```console

-# nginx Configuration File

-# https://wiki.nginx.org/Configuration

-# Run as a less privileged user for security reasons.

-user nginx;

-worker_processes auto;

-events {

- worker_connections 1024;

-}

-pid /var/run/nginx.pid;

-http {

- #Redirect to https, using 307 instead of 301 to preserve post data

- server {

- listen [::]:443 ssl;

- listen 443 ssl;

- server_name localhost;

- # Protect against the BEAST attack by not using SSLv3 at all. If you need to support older browsers (IE6) you may need to add

- # SSLv3 to the list of protocols below.

- ssl_protocols TLSv1.2;

- # Ciphers set to best allow protection from Beast, while providing forwarding secrecy, as defined by Mozilla - https://wiki.mozilla.org/Security/Server_Side_TLS#Nginx

- ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:AES128:AES256:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK;

- ssl_prefer_server_ciphers on;

- # Optimize TLS/SSL by caching session parameters for 10 minutes. This cuts down on the number of expensive TLS/SSL handshakes.

- # The handshake is the most CPU-intensive operation, and by default it is re-negotiated on every new/parallel connection.

- # By enabling a cache (of type "shared between all Nginx workers"), we tell the client to re-use the already negotiated state.

- # Further optimization can be achieved by raising keepalive_timeout, but that shouldn't be done unless you serve primarily HTTPS.

- ssl_session_cache shared:SSL:10m; # a 1mb cache can hold about 4000 sessions, so we can hold 40000 sessions

- ssl_session_timeout 24h;

- # Use a higher keepalive timeout to reduce the need for repeated handshakes

- keepalive_timeout 300; # up from 75 secs default

- # remember the certificate for a year and automatically connect to HTTPS

- add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains';

- ssl_certificate /etc/nginx/ssl.crt;

- ssl_certificate_key /etc/nginx/ssl.key;

- location / {

- proxy_pass http://localhost:80; # TODO: replace port if app listens on port other than 80

- proxy_set_header Connection "";

- proxy_set_header Host $host;

- proxy_set_header X-Real-IP $remote_addr;

- proxy_set_header X-Forwarded-For $remote_addr;

- }

- }

-}

-```

-### Base64-encode secrets and configuration file

-Base64-encode the Nginx configuration file, the TLS/SSL certificate, and the TLS key. In the next section, you enter the encoded contents in a YAML file used to deploy the container group.

-```console

-cat nginx.conf | base64 > base64-nginx.conf

-cat ssl.crt | base64 > base64-ssl.crt

-cat ssl.key | base64 > base64-ssl.key

-```

-## Deploy container group

-Now deploy the container group by specifying the container configurations in a [YAML file](container-instances-multi-container-yaml.md).

-### Create YAML file

-Copy the following YAML into a new file named `deploy-aci.yaml`. In Azure Cloud Shell, you can use Visual Studio Code to create the file in your working directory:

-```console

-code deploy-aci.yaml

-```

-Enter the contents of the base64-encoded files where indicated under `secret`. For example, `cat` each of the base64-encoded files to see its contents. During deployment, these files are added to a [secret volume](container-instances-volume-secret.md) in the container group. In this example, the secret volume is mounted to the Nginx container.

-```yaml

-api-version: 2019-12-01

-location: westus

-name: app-with-ssl

-properties:

- containers:

- - name: nginx-with-ssl

- properties:

- image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine

- ports:

- - port: 443

- protocol: TCP

- resources:

- requests:

- cpu: 1.0

- memoryInGB: 1.5

- volumeMounts:

- - name: nginx-config

- mountPath: /etc/nginx

- - name: my-app

- properties:

- image: mcr.microsoft.com/azuredocs/aci-helloworld

- ports:

- - port: 80

- protocol: TCP

- resources:

- requests:

- cpu: 1.0

- memoryInGB: 1.5

- volumes:

- - secret:

- ssl.crt: <Enter contents of base64-ssl.crt here>

- ssl.key: <Enter contents of base64-ssl.key here>

- nginx.conf: <Enter contents of base64-nginx.conf here>

- name: nginx-config

- ipAddress:

- ports:

- - port: 443

- protocol: TCP

- type: Public

- osType: Linux

-tags: null

-type: Microsoft.ContainerInstance/containerGroups

-```

-### Deploy the container group

-Create a resource group with the [az group create](/cli/azure/group#az-group-create) command:

-```azurecli-interactive

-az group create --name myResourceGroup --location westus

-```

-Deploy the container group with the [az container create](/cli/azure/container#az-container-create) command, passing the YAML file as an argument.

-```azurecli

-az container create --resource-group <myResourceGroup> --file deploy-aci.yaml

-```

-### View deployment state

-To view the state of the deployment, use the following [az container show](/cli/azure/container#az-container-show) command:

-```azurecli

-az container show --resource-group <myResourceGroup> --name app-with-ssl --output table

-```

-For a successful deployment, output is similar to the following:

-```console

-Name ResourceGroup Status Image IP:ports Network CPU/Memory OsType Location

- -- - - -- -

-app-with-ssl myresourcegroup Running nginx, mcr.microsoft.com/azuredocs/aci-helloworld 52.157.22.76:443 Public 1.0 core/1.5 gb Linux westus

-```

-## Verify TLS connection

-Use your browser to navigate to the public IP address of the container group. The IP address shown in this example is `52.157.22.76`, so the URL is **https://52.157.22.76**. You must use HTTPS to see the running application, because of the Nginx server configuration. Attempts to connect over HTTP fail.

-

-> [!NOTE]

-> Because this example uses a self-signed certificate and not one from a certificate authority, the browser displays a security warning when connecting to the site over HTTPS. You might need to accept the warning or adjust browser or certificate settings to proceed to the page. This behavior is expected.

->

-## Next steps

-This article showed you how to set up an Nginx container to enable TLS connections to a web app running in the container group. You can adapt this example for apps that listen on ports other than port 80. You can also update the Nginx configuration file to automatically redirect server connections on port 80 (HTTP) to use HTTPS.

-While this article uses Nginx in the sidecar, you can use another TLS provider such as [Caddy](https://caddyserver.com/).

-If you deploy your container group in an [Azure virtual network](container-instances-vnet.md), you can consider other options to enable a TLS endpoint for a backend container instance, including:

-* [Azure Functions Proxies](../azure-functions/functions-proxies.md)

-* [Azure API Management](../api-management/api-management-key-concepts.md)

-* [Azure Application Gateway](../application-gateway/overview.md) - see a sample [deployment template](https://github.com/Azure/azure-quickstart-templates/tree/master/application-workloads/wordpress/aci-wordpress-vnet).

-description: Learn about container groups in Azure Container Instances, a collection of instances that share a lifecycle and resources such as CPUs, storage, and network

-# Container groups in Azure Container Instances

-The top-level resource in Azure Container Instances is the *container group*. This article describes what container groups are and the types of scenarios they enable.

-## What is a container group?

-A container group is a collection of containers that get scheduled on the same host machine. The containers in a container group share a lifecycle, resources, local network, and storage volumes. It's similar in concept to a *pod* in [Kubernetes][kubernetes-pod].

-The following diagram shows an example of a container group that includes multiple containers:

-![Container groups diagram][container-groups-example]

-This example container group:

-* Is scheduled on a single host machine.

-* Is assigned a DNS name label.

-* Exposes a single public IP address, with one exposed port.

-* Consists of two containers. One container listens on port 80, while the other listens on port 5000.

-* Includes two Azure file shares as volume mounts, and each container mounts one of the shares locally.

-> [!NOTE]

-> Multi-container groups currently support only Linux containers. For Windows containers, Azure Container Instances only supports deployment of a single container instance. While we are working to bring all features to Windows containers, you can find current platform differences in the service [Overview](container-instances-overview.md#linux-and-windows-containers).

-## Deployment

-Here are two common ways to deploy a multi-container group: use a [Resource Manager template][resource-manager template] or a [YAML file][yaml-file]. A Resource Manager template is recommended when you need to deploy additional Azure service resources (for example, an [Azure Files share][azure-files]) when you deploy the container instances. Due to the YAML format's more concise nature, a YAML file is recommended when your deployment includes only container instances. For details on properties you can set, see the [Resource Manager template reference](/azure/templates/microsoft.containerinstance/containergroups) or [YAML reference](container-instances-reference-yaml.md) documentation.

-To preserve a container group's configuration, you can export the configuration to a YAML file by using the Azure CLI command [az container export][az-container-export]. Export allows you to store your container group configurations in version control for "configuration as code." Or, use the exported file as a starting point when developing a new configuration in YAML.

-## Resource allocation

-Azure Container Instances allocates resources such as CPUs, memory, and optionally [GPUs][gpus] (preview) to a multi-container group by adding the [resource requests][resource-requests] of the instances in the group. Taking CPU resources as an example, if you create a container group with two container instances, each requesting 1 CPU, then the container group is allocated 2 CPUs.

-### Resource usage by container instances

-Each container instance in a group is allocated the resources specified in its resource request. However, the maximum resources used by a container instance in a group could be different if you configure its optional [resource limit][resource-limits] property. The resource limit of a container instance must be greater than or equal to the mandatory [resource request][resource-requests] property.

-* If you don't specify a resource limit, the container instance's maximum resource usage is the same as its resource request.

-* If you specify a limit for a container instance, the instance's maximum usage could be greater than the request, up to the limit you set. Correspondingly, resource usage by other container instances in the group could decrease. The maximum resource limit you can set for a container instance is the total resources allocated to the group.

-

-For example, in a group with two container instances each requesting 1 CPU, one of your containers might run a workload that requires more CPUs to run than the other.

-In this scenario, you could set a resource limit of up to 2 CPUs for the container instance. This configuration allows the container instance to use up to 2 CPUs if available.

-> [!NOTE]

-> A small amount of a container group's resources is used by the service's underlying infrastructure. Your containers will be able to access most but not all of the resources allocated to the group. For this reason, plan a small resource buffer when requesting resources for containers in the group.

-### Minimum and maximum allocation

-* Allocate a **minimum** of 1 CPU and 1 GB of memory to a container group. Individual container instances within a group can be provisioned with less than 1 CPU and 1 GB of memory.

-* For the **maximum** resources in a container group, see the [resource availability][region-availability] for Azure Container Instances in the deployment region.

-## Networking

-Container groups can share an external-facing IP address, one or more ports on that IP address, and a DNS label with a fully qualified domain name (FQDN). To enable external clients to reach a container within the group, you must expose the port on the IP address and from the container. A container group's IP address and FQDN are released when the container group is deleted.

-Within a container group, container instances can reach each other via localhost on any port, even if those ports aren't exposed externally on the group's IP address or from the container.

-Optionally deploy container groups into an [Azure virtual network][virtual-network] to allow containers to communicate securely with other resources in the virtual network.

-## Storage

-You can specify external volumes to mount within a container group. Supported volumes include:

-* [Azure file share][azure-files]

-* [Secret][secret]

-* [Empty directory][empty-directory]

-* [Cloned git repo][volume-gitrepo]

-You can map those volumes into specific paths within the individual containers in a group.

-## Common scenarios

-Multi-container groups are useful in cases where you want to divide a single functional task into a small number of container images. These images can then be delivered by different teams and have separate resource requirements.

-Example usage could include:

-* A container serving a web application and a container pulling the latest content from source control.

-* An application container and a logging container. The logging container collects the logs and metrics output by the main application and writes them to long-term storage.

-* An application container and a monitoring container. The monitoring container periodically makes a request to the application to ensure that it's running and responding correctly, and raises an alert if it's not.

-* A front-end container and a back-end container. The front end might serve a web application, with the back end running a service to retrieve data.

-## Next steps

-Learn how to deploy a multi-container container group with an Azure Resource Manager template:

-> [!div class="nextstepaction"]

-> [Deploy a container group][resource-manager template]

-<!-- IMAGES -->

-[container-groups-example]: ./media/container-instances-container-groups/container-groups-example.png

-<!-- LINKS - External -->

-[dcos-pod]: https://dcos.io/docs/1.10/deploying-services/pods/

-[kubernetes-pod]: https://kubernetes.io/docs/concepts/workloads/pods/

-<!-- LINKS - Internal -->

-[resource-manager template]: container-instances-multi-container-group.md

-[yaml-file]: container-instances-multi-container-yaml.md

-[region-availability]: container-instances-resource-and-quota-limits.md

-[resource-requests]: /rest/api/container-instances/2022-09-01/container-groups/create-or-update#resourcerequests

-[resource-limits]: /rest/api/container-instances/2022-09-01/container-groups/create-or-update#resourcelimits

-[resource-requirements]: /rest/api/container-instances/2022-09-01/container-groups/create-or-update#resourcerequirements

-[azure-files]: container-instances-volume-azure-files.md

-[virtual-network]: container-instances-virtual-network-concepts.md

-[secret]: container-instances-volume-secret.md

-[volume-gitrepo]: container-instances-volume-gitrepo.md

-[gpus]: container-instances-gpu.md

-[empty-directory]: container-instances-volume-emptydir.md

-[az-container-export]: /cli/azure/container#az_container_export

-description: Configure a public or private DNS configuration for a container group

-# Deploy a container group with custom DNS settings

-In [Azure Virtual Network](../virtual-network/virtual-networks-overview.md), you can deploy container groups using the `az container create` command in the Azure CLI. You can also provide advanced configuration settings to the `az container create` command using a YAML configuration file.

-This article demonstrates how to deploy a container group with custom DNS settings using a YAML configuration file.

-For more information on deploying container groups to a virtual network, see the [Deploy in a virtual network article](container-instances-vnet.md).

-> [!IMPORTANT]

-> Previously, the process of deploying container groups on virtual networks used [network profiles](./container-instances-virtual-network-concepts.md#network-profile) for configuration. However, network profiles have been retired as of the `2021-07-01` API version. We recommend you use the latest API version, which relies on [subnet IDs](../virtual-network/subnet-delegation-overview.md) instead.

-## Prerequisites

-* An **active Azure subscription**. If you don't have an active Azure subscription, create a [free account](https://azure.microsoft.com/free) before you begin.

-* **Azure CLI**. The command-line examples in this article use the [Azure CLI](/cli/azure/) and are formatted for the Bash shell. You can [install the Azure CLI](/cli/azure/install-azure-cli) locally or use the [Azure Cloud Shell][cloud-shell-bash].

-* A **resource group** to manage all the resources you use in this how-to guide. We use the example resource group name **ACIResourceGroup** throughout this article.

- ```azurecli-interactive

- az group create --name ACIResourceGroup --location westus

- ```

-## Limitations

-For networking scenarios and limitations, see [Virtual network scenarios and resources for Azure Container Instances](container-instances-virtual-network-concepts.md).

-> [!IMPORTANT]

-> Container group deployment to a virtual network is available for Linux containers in most regions where Azure Container Instances is available. For details, see [Regions and resource availability](container-instances-region-availability.md).

-Examples in this article are formatted for the Bash shell. For PowerShell or command prompt, adjust the line continuation characters accordingly.

-## Create your virtual network

-You'll need a virtual network to deploy a container group with a custom DNS configuration. This virtual network will require a subnet with permissions to create Azure Container Instances resources and a linked private DNS zone to test name resolution.

-This guide uses a virtual network named `aci-vnet`, a subnet named `aci-subnet`, and a private DNS zone named `private.contoso.com`. We use **Azure Private DNS Zones**, which you can learn about in the [Private DNS Overview](../dns/private-dns-overview.md).

-If you have an existing virtual network that meets these criteria, you can skip to [Deploy your container group](#deploy-your-container-group).

-> [!TIP]

-> You can modify the following commands with your own information as needed.

-1. Create the virtual network using the [az network vnet create][az-network-vnet-create] command. Enter address prefixes in Classless Inter-Domain Routing (CIDR) format (for example: `10.0.0.0/16`).

- ```azurecli-interactive

- az network vnet create \

- --name aci-vnet \

- --resource-group ACIResourceGroup \

- --location westus \

- --address-prefix 10.0.0.0/16

- ```

-1. Create the subnet using the [az network vnet subnet create][az-network-vnet-subnet-create] command. The following command creates a subnet in your virtual network with a delegation that permits it to create container groups. For more information about working with subnets, see the [Add, change, or delete a virtual network subnet](../virtual-network/virtual-network-manage-subnet.md). For more information about subnet delegation, see the [Virtual Network Scenarios and Resources article section on delegated subnets](container-instances-virtual-network-concepts.md#subnet-delegated).

- ```azurecli-interactive

- az network vnet subnet create \

- --name aci-subnet \

- --resource-group ACIResourceGroup \

- --vnet-name aci-vnet \

- --address-prefixes 10.0.0.0/24 \

- --delegations Microsoft.ContainerInstance/containerGroups

- ```

-1. Record the subnet ID key-value pair from the output of this command. You'll use this in your YAML configuration file later. It will take the form `"id"`: `"/subscriptions/<subscription-ID>/resourceGroups/ACIResourceGroup/providers/Microsoft.Network/virtualNetworks/aci-vnet/subnets/aci-subnet"`.

-1. Create the private DNS Zone using the [az network private-dns zone create][az-network-private-dns-zone-create] command.

- ```azurecli-interactive

- az network private-dns zone create -g ACIResourceGroup -n private.contoso.com

- ```

-1. Link the DNS zone to your virtual network using the [az network private-dns link vnet create][az-network-private-dns-link-vnet-create] command. The DNS server is only required to test name resolution. The `-e` flag enables automatic hostname registration, which is unneeded, so we set it to `false`.

- ```azurecli-interactive

- az network private-dns link vnet create \

- -g ACIResourceGroup \

- -n aciDNSLink \

- -z private.contoso.com \

- -v aci-vnet \

- -e false

- ```

-Once you've completed the steps above, you should see an output with a final key-value pair that reads `"virtualNetworkLinkState"`: `"Completed"`.

-## Deploy your container group

-> [!NOTE]

-> Custom DNS settings are not currently available in the Azure portal for container group deployments. They must be provided with YAML file, Resource Manager template, [REST API](/rest/api/container-instances/2022-09-01/container-groups/create-or-update), or an [Azure SDK](https://azure.microsoft.com/downloads/).

-Copy the following YAML into a new file named *custom-dns-deploy-aci.yaml*. Edit the following configurations with your values:

-* `dnsConfig`: DNS settings for your containers within your container group.

- * `nameServers`: A list of name servers to be used for DNS lookups.

- * `searchDomains`: DNS suffixes to be appended for DNS lookups.

-* `ipAddress`: The private IP address settings for the container group.

- * `ports`: The ports to open, if any.

- * `protocol`: The protocol (TCP or UDP) for the opened port.

-* `subnetIDs`: Network settings for the subnet(s) in the virtual network.

- * `id`: The full Resource Manager resource ID of the subnet, which you obtained earlier.

-> [!NOTE]

-> The DNS config fields aren't automatically queried at this time, so these fields must be explicitly filled out.

-```yaml

-apiVersion: '2021-07-01'

-location: westus

-name: pwsh-vnet-dns

-properties:

- containers:

- - name: pwsh-vnet-dns

- properties:

- command:

- - /bin/bash

- - -c

- - echo hello; sleep 10000

- environmentVariables: []

- image: mcr.microsoft.com/powershell:latest

- ports:

- - port: 80

- resources:

- requests:

- cpu: 1.0

- memoryInGB: 2.0

- dnsConfig:

- nameServers:

- - 10.0.0.10 # DNS Server 1

- - 10.0.0.11 # DNS Server 2

- searchDomains: contoso.com # DNS search suffix

- ipAddress:

- type: Private

- ports:

- - port: 80

- subnetIds:

- - id: /subscriptions/<subscription-ID>/resourceGroups/ACIResourceGroup/providers/Microsoft.Network/virtualNetworks/aci-vnet/subnets/aci-subnet

- osType: Linux

-tags: null

-type: Microsoft.ContainerInstance/containerGroups

-```

-Deploy the container group with the [az container create][az-container-create] command, specifying the YAML file name with the `--file` parameter:

-```azurecli-interactive

-az container create --resource-group ACIResourceGroup \

- --file custom-dns-deploy-aci.yaml

-```

-Once the deployment is complete, run the [az container show][az-container-show] command to display its status. Sample output:

-```azurecli-interactive

-az container show --resource-group ACIResourceGroup --name pwsh-vnet-dns -o table

-```

-```output

-Name ResourceGroup Status Image IP:ports Network CPU/Memory OsType Location

-pwsh-vnet-dns ACIResourceGroup Running mcr.microsoft.com/powershell 10.0.0.5:80 Private 1.0 core/2.0 gb Linux westus

-```

-After the status shows `Running`, execute the [az container exec][az-container-exec] command to obtain bash access within the container.

-```azurecli-interactive

-az container exec --resource-group ACIResourceGroup --name pwsh-vnet-dns --exec-command "/bin/bash"

-```

-Validate that DNS is working as expected from within your container. For example, read the `/etc/resolv.conf` file to ensure it's configured with the DNS settings provided in the YAML file.

-```bash

-root@wk-caas-81d609b206c541589e11058a6d260b38-90b0aff460a737f346b3b0:/# cat /etc/resolv.conf

-nameserver 10.0.0.10

-nameserver 10.0.0.11

-search contoso.com

-```

-## Clean up resources

-### Delete container instances

-When you're finished with the container instance you created, delete it with the [az container delete][az-container-delete] command:

-```azurecli-interactive

-az container delete --resource-group ACIResourceGroup --name pwsh-vnet-dns -y

-```

-### Delete network resources

-If you don't plan to use this virtual network again, you can delete it with the [az network vnet delete][az-network-vnet-delete] command:

-```azurecli-interactive

-az network vnet delete --resource-group ACIResourceGroup --name aci-vnet

-```

-### Delete resource group

-If you don't plan to use this resource group outside of this guide, you can delete it with [az group delete][az-group-delete] command:

-```azurecli-interactive

-az group delete --name ACIResourceGroup

-```

-Enter `y` when prompted if you're sure you wish to perform the operation.

-## Next steps

-See the Azure quickstart template [Create an Azure container group with VNet](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.containerinstance/aci-vnet), to deploy a container group within a virtual network.

-<!-- LINKS - Internal -->

-[az-network-vnet-create]: /cli/azure/network/vnet#az-network-vnet-create

-[az-network-vnet-subnet-create]: /cli/azure/network/vnet/subnet#az-network-vnet-subnet-create

-[az-network-private-dns-zone-create]: /cli/azure/network/private-dns/zone#az-network-private-dns-zone-create

-[az-network-private-dns-link-vnet-create]: /cli/azure/network/private-dns/link/vnet#az-network-private-dns-link-vnet-create

-[az-container-create]: /cli/azure/container#az-container-create

-[az-container-show]: /cli/azure/container#az-container-show

-[az-container-exec]: /cli/azure/container#az-container-exec

-[az-container-delete]: /cli/azure/container#az-container-delete

-[az-network-vnet-delete]: /cli/azure/network/vnet#az-network-vnet-delete

-[az-group-delete]: /cli/azure/group#az-group-create

-[cloud-shell-bash]: ../cloud-shell/overview.md

-description: Use a dedicated host to achieve true host-level isolation for your Azure Container Instances workloads

-# Deploy on dedicated hosts

-"Dedicated" is an Azure Container Instances (ACI) sku that provides an isolated and dedicated compute environment for securely running containers. Using the dedicated sku results in each container group having a dedicated physical server in an Azure datacenter, ensuring full workload isolation to help meet your organization's security and compliance requirements.

-The dedicated sku is appropriate for container workloads that require workload isolation from a physical server perspective.

-A dedicated host for Azure Container Instances provides [double encryption at rest](../virtual-machines/windows/disk-encryption.md#double-encryption-at-rest) for your container data when it is persisted by the service to the cloud. This encryption protects your data to help meet your organization's security and compliance requirements. ACI also gives you the option to [encrypt this data with your own key](container-instances-encrypt-data.md), giving you greater control over the data related to your ACI deployments.

-## Prerequisites

-> [!NOTE]

-> Due to some current limitations, not all limit increase requests are guaranteed to be approved.

-* The default limit for any subscription to use the dedicated sku is 0. If you would like to use this sku for your production container deployments, create an [Azure Support request][azure-support] to increase the limit.

-## Use the dedicated sku

-> [!IMPORTANT]

-> Using the dedicated sku is only available in **API version 2019-12-01 or later**. Specify this API version or a more recent one in your deployment template.

-Starting with API version 2019-12-01, there is a `sku` property under the container group properties section of a deployment template, which is required for an ACI deployment. Currently, you can use this property as part of an Azure Resource Manager deployment template for ACI. Learn more about deploying ACI resources with a template in the [Tutorial: Deploy a multi-container group using a Resource Manager template](./container-instances-multi-container-group.md).

-The `sku` property can have one of the following values:

-* `Standard` - the standard ACI deployment choice, which still guarantees hypervisor-level security

-* `Dedicated` - used for workload level isolation with dedicated physical hosts for the container group

-## Modify your JSON deployment template

-In your deployment template, modify or add the following properties:

-* Under `resources`, set `apiVersion` to `2019-12-01`.

-* Under the container group properties, add a `sku` property with value `Dedicated`.

-Here is an example snippet for the resources section of a container group deployment template that uses the dedicated sku:

-```json

-[...]

-"resources": [

- {

- "name": "[parameters('containerGroupName')]",

- "type": "Microsoft.ContainerInstance/containerGroups",

- "apiVersion": "2019-12-01",

- "location": "[resourceGroup().location]",

- "properties": {

- "sku": "Dedicated",

- "containers": {

- [...]

- }

- }

- }

-]

-```

-Following is a complete template that deploys a sample container group running a single container instance:

-```json

-{

- "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",

- "contentVersion": "1.0.0.0",

- "parameters": {

- "containerGroupName": {

- "type": "string",

- "defaultValue": "myContainerGroup",

- "metadata": {

- "description": "Container Group name."

- }

- }

- },

- "resources": [

- {

- "name": "[parameters('containerGroupName')]",

- "type": "Microsoft.ContainerInstance/containerGroups",

- "apiVersion": "2019-12-01",

- "location": "[resourceGroup().location]",

- "properties": {

- "sku": "Dedicated",

- "containers": [

- {

- "name": "container1",

- "properties": {

- "image": "nginx",

- "command": [

- "/bin/sh",

- "-c",

- "while true; do echo `date`; sleep 1000000; done"

- ],

- "ports": [

- {

- "protocol": "TCP",

- "port": 80

- }

- ],

- "environmentVariables": [],

- "resources": {

- "requests": {

- "memoryInGB": 1.0,

- "cpu": 1

- }

- }

- }

- }

- ],

- "restartPolicy": "Always",

- "ipAddress": {

- "ports": [

- {

- "protocol": "TCP",

- "port": 80

- }

- ],

- "type": "Public"

- },

- "osType": "Linux"

- },

- "tags": {}

- }

- ]

-}

-```

-## Deploy your container group

-If you created and edited the deployment template file on your desktop, you can upload it to your Cloud Shell directory by dragging the file into it.

-Create a resource group with the [az group create][az-group-create] command.

-```azurecli-interactive

-az group create --name myResourceGroup --location eastus

-```

-Deploy the template with the [az deployment group create][az-deployment-group-create] command.

-```azurecli-interactive

-az deployment group create --resource-group myResourceGroup --template-file deployment-template.json

-```

-Within a few seconds, you should receive an initial response from Azure. A successful deployment takes place on a dedicated host.

-<!-- LINKS - Internal -->

-[az-group-create]: /cli/azure/group#az-group-create

-[az-deployment-group-create]: /cli/azure/deployment/group#az-deployment-group-create

-<!-- LINKS - External -->

-[azure-support]: https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest

-description: Configure Azure firewall and user-defined routes for Azure Container Instances workloads that use the firewall's public IP address for ingress and egress

-# Configure a single public IP address for outbound and inbound traffic to a container group

-Setting up a [container group](container-instances-container-groups.md) with an external-facing IP address allows external clients to use the IP address to access a container in the group. For example, a browser can access a web app running in a container. However, currently a container group uses a different IP address for outbound traffic. This egress IP address isn't exposed programmatically, which makes container group monitoring and configuration of client firewall rules more complex.

-This article provides steps to configure a container group in a [virtual network](container-instances-virtual-network-concepts.md) integrated with [Azure Firewall](../firewall/overview.md). By setting up a user-defined route to the container group and firewall rules, you can route and identify traffic to and from the container group. Container group ingress and egress use the public IP address of the firewall. A single egress IP address can be used by multiple container groups deployed in the virtual network's subnet delegated to Azure Container Instances.

-In this article, you use the Azure CLI to create the resources for this scenario:

-* Container groups deployed on a delegated subnet [in the virtual network](container-instances-vnet.md)

-* An Azure firewall deployed in the network with a static public IP address

-* A user-defined route on the container groups' subnet

-* A NAT rule for firewall ingress and an application rule for egress

-You then validate ingress and egress from example container groups through the firewall.

-> [!NOTE]

-> To download the complete script, go to [full script](https://github.com/Azure-Samples/azure-cli-samples/blob/master/container-instances/egress-ip-address.sh).

-## Get started

-This tutorial makes use of a randomized variable. If you are using an existing resource group, modify the value of this variable appropriately.

-**Azure resource group**: If you don't have an Azure resource group already, create a resource group with the [az group create][az-group-create] command. Modify the location value as appropriate.

-## Deploy ACI in a virtual network

-In a typical case, you might already have an Azure virtual network in which to deploy a container group. For demonstration purposes, the following commands create a virtual network and subnet when the container group is created. The subnet is delegated to Azure Container Instances.

-The container group runs a small web app from the `aci-helloworld` image. As shown in other articles in the documentation, this image packages a small web app written in Node.js that serves a static HTML page.

-Create the container group with the [az container create][az-container-create] command:

-> [!TIP]

-> Adjust the value of `--subnet address-prefix` for the IP address space you need in your subnet. The smallest supported subnet is /29, which provides eight IP addresses. Some IP addresses are reserved for use by Azure.

-For use in a later step, get the private IP address of the container group by running the [az container show][az-container-show] command:

-## Deploy Azure Firewall in network

-In the following sections, use the Azure CLI to deploy an Azure firewall in the virtual network. For background, see [Tutorial: Deploy and configure Azure Firewall using the Azure portal](../firewall/deploy-cli.md).

-First, use the [az network vnet subnet create][az-network-vnet-subnet-create] to add a subnet named AzureFirewallSubnet for the firewall. AzureFirewallSubnet is the *required* name of this subnet.

-Use the following [Azure CLI commands](../firewall/deploy-cli.md) to create a firewall in the subnet.

-If not already installed, add the firewall extension to the Azure CLI using the [az extension add][az-extension-add] command:

-Create the firewall resources using the [az network firewall create][az-network-firewall-create] command:

-Update the firewall configuration using the [az network firewall update][az-network-firewall-update] command:

-Get the firewall's private IP address using the [az network firewall ip-config list][az-network-firewall-ip-config-list] command. This private IP address is used in a later command.

-Get the firewall's public IP address using the [az network public-ip show][az-network-public-ip-show] command. This public IP address is used in a later command.

-## Define user-defined route on ACI subnet

-Define a use-defined route on the ACI subnet, to divert traffic to the Azure firewall. For more information, see [Route network traffic](../virtual-network/tutorial-create-route-table-cli.md).

-### Create route table

-First, run the following [az network route-table create][az-network-route-table-create] command to create the route table. Create the route table in the same region as the virtual network.

-### Create route

-Run [az network-route-table route create][az-network-route-table-route-create] to create a route in the route table. To route traffic to the firewall, set the next hop type to `VirtualAppliance`, and pass the firewall's private IP address as the next hop address.

-### Associate route table to ACI subnet

-Run the [az network vnet subnet update][az-network-vnet-subnet-update] command to associate the route table with the subnet delegated to Azure Container Instances.

-## Configure rules on firewall

-By default, Azure Firewall denies (blocks) inbound and outbound traffic.

-### Configure NAT rule on firewall to ACI subnet

-Create a [NAT rule](../firewall/rule-processing.md) on the firewall to translate and filter inbound internet traffic to the application container you started previously in the network. For details, see [Filter inbound Internet traffic with Azure Firewall DNAT](../firewall/tutorial-firewall-dnat.md)

-Create a NAT rule and collection by using the [az network firewall nat-rule create][az-network-firewall-nat-rule-create] command:

-Add NAT rules as needed to filter traffic to other IP addresses in the subnet. For example, other container groups in the subnet could expose IP addresses for inbound traffic, or other internal IP addresses could be assigned to the container group after a restart.

-### Create outbound application rule on the firewall

-Run the following [az network firewall application-rule create][az-network-firewall-application-rule-create] command to create an outbound rule on the firewall. This sample rule allows access from the subnet delegated to Azure Container Instances to the FQDN `checkip.dyndns.org`. HTTP access to the site is used in a later step to confirm the egress IP address from Azure Container Instances.

-## Test container group access through the firewall

-The following sections verify that the subnet delegated to Azure Container Instances is properly configured behind the Azure firewall. The previous steps routed both incoming traffic to the subnet and outgoing traffic from the subnet through the firewall.

-### Test ingress to a container group

-Test inbound access to the `appcontainer` running in the virtual network by browsing to the firewall's public IP address. Previously, you stored the public IP address in variable $FW_PUBLIC_IP:

-Output is similar to:

-```console

-52.142.18.133

-```

-If the NAT rule on the firewall is configured properly, you see the following when you enter the firewall's public IP address in your browser:

-### Test egress from a container group

-Deploy the following sample container into the virtual network. When it runs, it sends a single HTTP request to `http://checkip.dyndns.org`, which displays the IP address of the sender (the egress IP address). If the application rule on the firewall is configured properly, the firewall's public IP address is returned.

-View the container logs to confirm the IP address is the same as the public IP address of the firewall.

-```azurecli

-az container logs \

- --resource-group $resourceGroup \

- --name testegress

-```

-Output is similar to:

-```console

-<html><head><title>Current IP Check</title></head><body>Current IP Address: 52.142.18.133</body></html>

-```

-## Clean up resources

-When no longer needed, you can use [az group delete](/cli/azure/group) to remove the resource group and all related resources as follows. The `--no-wait` parameter returns control to the prompt without waiting for the operation to complete. The `--yes` parameter confirms that you wish to delete the resources without an additional prompt to do so.

-```azurecli-interactive

-az group delete --name $resourceGroup --yes --no-wait

-```

-## Next steps

-In this article, you set up container groups in a virtual network behind an Azure firewall. You configured a user-defined route and NAT and application rules on the firewall. By using this configuration, you set up a single, static IP address for ingress and egress from Azure Container Instances.

-For more information about managing traffic and protecting Azure resources, see the [Azure Firewall](../firewall/index.yml) documentation.

-[az-group-create]: /cli/azure/group#az_group_create

-[az-container-create]: /cli/azure/container#az_container_create

-[az-network-vnet-subnet-create]: /cli/azure/network/vnet/subnet#az_network_vnet_subnet_create

-[az-extension-add]: /cli/azure/extension#az_extension_add

-[az-network-firewall-create]: /cli/azure/network/firewall#az_network_firewall_create

-[az-network-firewall-update]: /cli/azure/network/firewall#az_network_firewall_update

-[az-network-public-ip-show]: /cli/azure/network/public-ip/#az_network_public_ip_show

-[az-network-route-table-create]:/cli/azure/network/route-table/#az_network_route_table_create

-[az-network-route-table-route-create]: /cli/azure/network/route-table/route#az_network_route_table_route_create

-[az-network-firewall-ip-config-list]: /cli/azure/network/firewall/ip-config#az_network_firewall_ip_config_list

-[az-network-vnet-subnet-update]: /cli/azure/network/vnet/subnet#az_network_vnet_subnet_update

-[az-container-exec]: /cli/azure/container#az_container_exec

-[az-vm-create]: /cli/azure/vm#az_vm_create

-[az-vm-open-port]: /cli/azure/vm#az_vm_open_port

-[az-vm-list-ip-addresses]: /cli/azure/vm#az_vm_list_ip_addresses

-[az-network-firewall-application-rule-create]: /cli/azure/network/firewall/application-rule#az_network_firewall_application_rule_create

-[az-network-firewall-nat-rule-create]: /cli/azure/network/firewall/nat-rule#az_network_firewall_nat_rule_create

-description: Learn about encryption of data persisted for your container instance resources and how to encrypt the data with a customer-managed key

-# Encrypt deployment data

-When running Azure Container Instances (ACI) resources in the cloud, the ACI service collects and persists data related to your containers. ACI automatically encrypts this data when it is persisted in the cloud. This encryption protects your data to help meet your organization's security and compliance commitments. ACI also gives you the option to encrypt this data with your own key, giving you greater control over the data related to your ACI deployments.

-## About ACI data encryption

-Data in ACI is encrypted and decrypted using 256-bit AES encryption. It is enabled for all ACI deployments, and you don't need to modify your deployment or containers to take advantage of this encryption. This includes metadata about the deployment, environment variables, keys being passed into your containers, and logs persisted after your containers are stopped so you can still see them. Encryption does not affect your container group performance, and there is no additional cost for encryption.

-You can rely on Microsoft-managed keys for the encryption of your container data, or you can manage the encryption with your own keys. The following table compares these options: