+## November 2023

+### Updated articles

+- [Set up a password reset flow in Azure Active Directory B2C](add-password-reset-policy.md) - Editorial updates

+- [Enrich tokens with claims from external sources using API connectors](add-api-connector-token-enrichment.md) - Editorial updates

+- [Enable custom domains for Azure Active Directory B2C](custom-domain.md) - Editorial updates

+- [Set up sign-in for multitenant Microsoft Entra ID using custom policies in Azure Active Directory B2C](identity-provider-azure-ad-multi-tenant.md) - Editorial updates

+- [Manage Azure AD B2C with Microsoft Graph](microsoft-graph-operations.md) - Editorial updates

+- [Enable multifactor authentication in Azure Active Directory B2C](multi-factor-authentication.md) - Editorial updates

+- [What is Azure Active Directory B2C?](overview.md) - Editorial updates

+- [Technical and feature overview of Azure Active Directory B2C](technical-overview.md) - Editorial updates

+- [Tutorial: Create user flows and custom policies in Azure Active Directory B2C](tutorial-create-user-flows.md) - Editorial updates

+- [User flows and custom policies overview](user-flow-overview.md) - Editorial updates

+- [OAuth 2.0 authorization code flow in Azure Active Directory B2C](authorization-code-flow.md) - Editorial updates

+- [Create and read a user account by using Azure Active Directory B2C custom policy](custom-policies-series-store-user.md) - Editorial updates

+- [Define a Microsoft Entra multifactor authentication technical profile in an Azure AD B2C custom policy](multi-factor-auth-technical-profile.md) - Editorial updates

+* See the sample for how to use the exported model [(VAIDK/OpenVino)](https://github.com/Azure-Samples/customvision-export-samples)

+* [PowerShell](../use-your-data-quickstart.md?tabs=command-line%2Cpowershell&pivots=programming-language-powershell#example-powershell-commands)

+## Virtual network support & private endpoint support

+* For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)

+* Azure OpenAI, Azure AI Search, and Azure Storage Accounts can be protected under private endpoints and virtual private networks.

+## Document-level access control

+> [!NOTE]

+> Document-level access control is supported for Azure AI search only.

+## Schedule automatic index refreshes

+> [!NOTE]

+> Automatic index refreshing is supported for Azure Blob storage only.

+## Token usage estimation for Azure OpenAI on your data

+| Model | Total tokens available | Max tokens for system message | Max tokens for model response |

+|-||||

+| ChatGPT Turbo (0301) 8k | 8000 | 400 | 1500 |

+| ChatGPT Turbo 16k | 16000 | 1000 | 3200 |

+| GPT-4 (8k) | 8000 | 400 | 1500 |

+| GPT-4 32k | 32000 | 2000 | 6400 |

+The table above shows the total number of tokens available for each model type. It also determines the maximum number of tokens that can be used for the [system message](#system-message) and the model response. Additionally, the following also consume tokens:

+* The meta prompt (MP): if you limit responses from the model to the grounding data content (`inScope=True` in the API), the maximum number of tokens is 4036 tokens. Otherwise (for example if `inScope=False`) the maximum is 3444 tokens. This number is variable depending on the token length of the user question and conversation history. This estimate includes the base prompt as well as the query rewriting prompts for retrieval.

+* User question and history: Variable but capped at 2000 tokens.

+* Retrieved documents (chunks): The number of tokens used by the retrieved document chunks depends on multiple factors. The upper bound for this is the number of retrieved document chunks multiplied by the chunk size. It will, however, be truncated based on the tokens available tokens for the specific model being used after counting the rest of fields.

+ 20% of the available tokens are reserved for the model response. The remaining 80% of available tokens include the meta prompt, the user question and conversation history, and the system message. The remaining token budget is used by the retrieved document chunks.

+```python

+import tiktoken

+class TokenEstimator(object):

+ GPT2_TOKENIZER = tiktoken.get_encoding("gpt2")

+ def estimate_tokens(self, text: str) -> int:

+ return len(self.GPT2_TOKENIZER.encode(text))

+

+token_output = TokenEstimator.estimate_tokens(input_text)

+```

+1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes), the ingestion process starts at `[3]`.

+You can protect Azure OpenAI resources in [virtual networks and private endpoints](/azure/ai-services/cognitive-services-virtual-networks) the same way as any Azure AI service.

+Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for the Azure OpenAI on your data resource (as a multitenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](https://aka.ms/applyacsvpnaoaioyd). The application will be reviewed in ten business days and you will be contacted via email about the results. If you are eligible, we will send a private endpoint request to your search service, and you will need to approve the request.

+Learn more about the [manual approval workflow](/azure/private-link/private-endpoint-overview#access-to-a-private-link-resource-using-approval-workflow).

+| `filter` | string | Optional | null | The filter pattern used for [restricting access to sensitive documents](./concepts/use-your-data.md#document-level-access-control)

+# [OpenAI Python 0.28.1](#tab/python)

+# [OpenAI Python 1.x](#tab/python-new)

+```cmd

+pip install openai json requests os tiktoken time

+```

+# [OpenAI Python 0.28.1](#tab/python)

+openai.api_version = '2023-10-01-preview' # This API version or later is required to access fine-tuning for turbo/babbage-002/davinci-002

+# [OpenAI Python 1.x](#tab/python-new)

+```python

+# Upload fine-tuning files

+import os

+from openai import AzureOpenAI

+client = AzureOpenAI(

+ azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),

+ api_key=os.getenv("AZURE_OPENAI_KEY"),

+ api_version="2023-10-01-preview" # This API version or later is required to access fine-tuning for turbo/babbage-002/davinci-002

+)

+training_file_name = 'training_set.jsonl'

+validation_file_name = 'validation_set.jsonl'

+# Upload the training and validation dataset files to Azure OpenAI with the SDK.

+training_response = client.files.create(

+ file=open(training_file_name, "rb"), purpose="fine-tune"

+)

+training_file_id = training_response.id

+validation_response = client.files.create(

+ file=open(validation_file_name, "rb"), purpose="fine-tune"

+)

+validation_file_id = validation_response.id

+print("Training file ID:", training_file_id)

+print("Validation file ID:", validation_file_id)

+```

+# [OpenAI Python 0.28.1](#tab/python)

+# [OpenAI Python 1.x](#tab/python-new)

+```python

+response = client.fine_tuning.jobs.create(

+ training_file=training_file_id,

+ validation_file=validation_file_id,

+ model="gpt-35-turbo-0613", # Enter base model name. Note that in Azure OpenAI the model name contains dashes and cannot contain dot/period characters.

+)

+job_id = response.id

+# You can use the job ID to monitor the status of the fine-tuning job.

+# The fine-tuning job will take some time to start and complete.

+print("Job ID:", response.id)

+print("Status:", response.id)

+print(response.model_dump_json(indent=2))

+```

+# [OpenAI Python 0.28.1](#tab/python)

+# [OpenAI Python 1.x](#tab/python-new)

+```python

+# Track training status

+from IPython.display import clear_output

+import time

+start_time = time.time()

+# Get the status of our fine-tuning job.

+response = client.fine_tuning.jobs.retrieve(job_id)

+status = response.status

+# If the job isn't done yet, poll it every 10 seconds.

+while status not in ["succeeded", "failed"]:

+ time.sleep(10)

+

+ response = client.fine_tuning.jobs.retrieve(job_id)

+ print(response.model_dump_json(indent=2))

+ print("Elapsed time: {} minutes {} seconds".format(int((time.time() - start_time) // 60), int((time.time() - start_time) % 60)))

+ status = response.status

+ print(f'Status: {status}')

+ clear_output(wait=True)

+print(f'Fine-tuning job {job_id} finished with status: {status}')

+# List all fine-tuning jobs for this resource.

+print('Checking other fine-tune jobs for this resource.')

+response = client.fine_tuning.jobs.list()

+print(f'Found {len(response.data)} fine-tune jobs.')

+```

+# [OpenAI Python 0.28.1](#tab/python)

+# [OpenAI Python 1.x](#tab/python-new)

+```python

+#Retrieve fine_tuned_model name

+response = client.fine_tuning.jobs.retrieve(job_id)

+print(response.model_dump_json(indent=2))

+fine_tuned_model = response.fine_tuned_model

+```

+# [OpenAI Python 0.28.1](#tab/python)

+# [OpenAI Python 1.x](#tab/python-new)

+```python

+import os

+from openai import AzureOpenAI

+client = AzureOpenAI(

+ azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),

+ api_key=os.getenv("AZURE_OPENAI_KEY"),

+ api_version="2023-05-15"

+)

+response = client.chat.completions.create(

+ model="gpt-35-turbo-ft", # model = "Custom deployment name you chose for your fine-tuning model"

+ messages=[

+ {"role": "system", "content": "You are a helpful assistant."},

+ {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},

+ {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},

+ {"role": "user", "content": "Do other Azure AI services support this too?"}

+ ]

+)

+print(response.choices[0].message.content)

+```

+- [Azure OpenAI on your data](./concepts/use-your-data.md#virtual-network-support--private-endpoint-support) now supports private endpoints.

+- Ability to [filter access to sensitive documents](./concepts/use-your-data.md#document-level-access-control).

+- [Automatically refresh your index on a schedule](./concepts/use-your-data.md#schedule-automatic-index-refreshes).

+You must have the latest aks-preview Azure CLI extension installed and register the `Microsoft.ContainerService` `AzureOverlayPreview` feature flag.

+> [!NOTE]

+5. Create an *azure-disk-pod.yaml* file to reference your *PersistentVolumeClaim*.

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+> [!NOTE]

+> With the retirement of [Open Service Mesh (OSM)](https://docs.openservicemesh.io/) by the Cloud Native Computing Foundation (CNCF), we recommend identifying your OSM configurations and migrating them to an equivalent Istio configuration. For information about migrating from OSM to Istio, see [Migration guidance for Open Service Mesh (OSM) configurations to Istio](open-service-mesh-istio-migration-guidance.md).

+[WebAssembly (WASM)][wasm] is a binary format that is optimized for fast download and maximum execution speed in a WASM runtime. A WASM runtime is designed to run on a target architecture and execute WebAssemblies in a sandbox, isolated from the host computer, at near-native performance. By default, WebAssemblies can't access resources on the host outside of the sandbox unless it is explicitly allowed, and they can't communicate over sockets to access things like environment variables or HTTP traffic. The [WebAssembly System Interface (WASI)][wasi] standard defines an API for WASM runtimes to provide access to WebAssemblies to the environment and resources outside the host using a capabilities model.

+API Management platform migration from `stv1` to `stv2` involves updating the underlying compute alone and has no impact on the service/API configuration persisted in the storage layer.

+> [!NOTE]

+> The following script is written for the bash shell. To run the script in PowerShell, prefix the variable names with the `$` character. Example: `$APIM_NAME`.

+* The local cache contains a one-time copy of the _/site_ and _/siteextensions_ folders of the shared content store, at _D:\home\site_ and _D:\home\siteextensions_, respectively. The files are copied to the local cache when the app starts. The size of the two folders for each app is limited to 1 GB by default, but can be increased to 2 GB. Note that as the cache size increases, it will take longer to load the cache. If you've increased local cache limit to 2 GB and the copied files exceed the maximum size of 2 GB, App Service silently ignores local cache and reads from the remote file share.

+> [!IMPORTANT]

+> When the copied files exceed the defined Local Cache size limit or when no limit is defined, deployment and swapping operations may fail with an error. See the [FAQ](#frequently-asked-questions-faq) for more information.

+>

+### What if Local Cache size limit is exceeded?

+When the copied files exceed the Local Cache size limit, the app will read from the remote share. However, deployment and swap operations may fail with an error. See the table below for size limits and results.

+| **Local Cache size** | **Coped files** | **Result** |

+| | | |

+|Γëñ 2 GB|Γëñ Local Cache size|Reads from local cache.|

+|Γëñ 2 GB|> Local Cache size|Reads from remote share.<br/> **Note:** Deployment and swap operations may fail with an error.|

+In this tutorial, you'll deploy a data-driven Python web app (**[Django](https://www.djangoproject.com/)** or **[Flask](https://flask.palletsprojects.com/)**) to **[Azure App Service](./overview.md#app-service-on-linux)** with the **[Azure Database for PostgreSQL](../postgresql/index.yml)** relational database service. Azure App Service supports [Python](https://www.python.org/downloads/) in a Linux server environment.

+**To complete this tutorial, you'll need:**

+* An Azure account with an active subscription. If you don't have an Azure account, you [can create one for free](https://azure.microsoft.com/free/python).

+* Knowledge of Python with Flask development or [Python with Django development](/training/paths/django-create-data-driven-websites/)

+* [Azure Developer CLI](/azure/developer/azure-developer-cli/install-azd) installed. You can follow the steps with the [Azure Cloud Shell](https://shell.azure.com) because it already has Azure Developer CLI installed.

+## Skip to the end

+### [Flask](#tab/flask)

+With [Azure Developer CLI](/azure/developer/azure-developer-cli/install-azd) installed, you can deploy a fully configured sample app shown in this tutorial and see it running in Azure. Just running the following commands in an empty working directory:

+```bash

+azd auth login

+azd init --template msdocs-flask-postgresql-sample-app

+azd up

+```

+### [Django](#tab/django)

+With [Azure Developer CLI](/azure/developer/azure-developer-cli/install-azd) installed, you can skip to the end of the tutorial by running the following commands in an empty working directory:

+```bash

+azd auth login

+azd init --template msdocs-django-postgresql-sample-app

+azd up

+```

+--

+To run the application locally, make sure you have [Python 3.7 or higher](https://www.python.org/downloads/) and [PostgreSQL](https://www.postgresql.org/download/) installed locally. Then, clone the sample repository's `starter-no-infra` branch and change to the repository root.

+git clone -b starter-no-infra https://github.com/Azure-Samples/msdocs-flask-postgresql-sample-app

+Create an *.env* file as shown below using the *.env.sample* file as a guide. Set the value of `DBNAME` to the name of an existing database in your local PostgreSQL instance. Set the values of `DBHOST`, `DBUSER`, and `DBPASS` as appropriate for your local PostgreSQL instance.

+```

+DBNAME=<database name>

+DBHOST=<database-hostname>

+DBUSER=<db-user-name>

+DBPASS=<db-password>

+```

+git clone -b starter-no-infra https://github.com/Azure-Samples/msdocs-django-postgresql-sample-app

+--

+### [Flask](#tab/flask)

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-1.png" alt-text="A screenshot showing how to use the search box in the top tool bar to find the Web App + Database creation wizard (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-1.png":::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-2.png" alt-text="A screenshot showing how to configure a new app and database in the Web App + Database wizard (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-2.png":::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-3.png" alt-text="A screenshot showing the deployment process completed (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-3.png":::

+ :::column-end:::

+### [Django](#tab/django)

+ :::column span="2":::

+ **Step 1:** In the Azure portal:

+ 1. Enter "web app database" in the search bar at the top of the Azure portal.

+ 1. Select the item labeled **Web App + Database** under the **Marketplace** heading.

+ You can also navigate to the [creation wizard](https://portal.azure.com/?feature.customportal=false#create/Microsoft.AppServiceWebAppDatabaseV3) directly.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-1.png" alt-text="A screenshot showing how to use the search box in the top tool bar to find the Web App + Database creation wizard (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-1.png":::

+ :::column span="2":::

+ **Step 2:** In the **Create Web App + Database** page, fill out the form as follows.

+ 1. *Resource Group* &rarr; Select **Create new** and use a name of **msdocs-python-postgres-tutorial**.

+ 1. *Region* &rarr; Any Azure region near you.

+ 1. *Name* &rarr; **msdocs-python-postgres-XYZ** where *XYZ* is any three random characters. This name must be unique across Azure.

+ 1. *Runtime stack* &rarr; **Python 3.12**.

+ 1. *Database* &rarr; **PostgreSQL - Flexible Server** is selected by default as the database engine. The server name and database name are also set by default to appropriate values.

+ 1. *Add Azure Cache for Redis* &rarr; **Yes**.

+ 1. *Hosting plan* &rarr; **Basic**. When you're ready, you can [scale up](manage-scale-up.md) to a production pricing tier later.

+ 1. Select **Review + create**.

+ 1. After validation completes, select **Create**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-2-django.png" alt-text="A screenshot showing how to configure a new app and database in the Web App + Database wizard (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-2-django.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3:** The deployment takes a few minutes to complete. Once deployment completes, select the **Go to resource** button. You're taken directly to the App Service app, but the following resources are created:

+ - **Resource group** &rarr; The container for all the created resources.

+ - **App Service plan** &rarr; Defines the compute resources for App Service. A Linux plan in the *Basic* tier is created.

+ - **App Service** &rarr; Represents your app and runs in the App Service plan.

+ - **Virtual network** &rarr; Integrated with the App Service app and isolates back-end network traffic.

+ - **Azure Database for PostgreSQL flexible server** &rarr; Accessible only from within the virtual network. A database and a user are created for you on the server.

+ - **Azure Cache for Redis** &rarr; Accessible only from within the virtual network.

+ - **Private DNS zones** &rarr; Enables DNS resolution of the PostgreSQL server and the Redis server in the virtual network.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-3.png" alt-text="A screenshot showing the deployment process completed (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-create-app-postgres-3.png":::

+ :::column-end:::

+--

+### [Flask](#tab/flask)

+ **Step 1:** In the App Service page, in the left menu, select **Configuration**.

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-1.png" alt-text="A screenshot showing how to open the configuration page in App Service (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-1.png":::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-2.png" alt-text="A screenshot showing how to see the autogenerated connection string (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3:** In a terminal or command prompt, run the following Python script to generate a unique secret: `python -c 'import secrets; print(secrets.token_hex())'`. Copy the output value to use in the next step.

+ :::column-end:::

+ :::column:::

+ :::column-end:::

+### [Django](#tab/django)

+ :::column span="2":::

+ **Step 1:** In the App Service page, in the left menu, select **Configuration**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-1.png" alt-text="A screenshot showing how to open the configuration page in App Service (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2:** In the **Application settings** tab of the **Configuration** page, verify that `AZURE_POSTGRESQL_CONNECTIONSTRING` and `AZURE_REDIS_CONNECTIONSTRING` are present. They will be injected into the runtime environment as an environment variable.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-2-django.png" alt-text="A screenshot showing how to see the autogenerated connection string (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-get-connection-string-2.png":::

+ **Step 4:** Back in the **Configuration** page, select **New application setting**. Name the setting `SECRET_KEY`. Paste the value from the previous value. Select **OK**.

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-app-service-app-setting.png" alt-text="A screenshot showing how to set the SECRET_KEY app setting in the Azure portal (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-app-service-app-setting.png":::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-app-service-app-setting-save.png" alt-text="A screenshot showing how to save the SECRET_KEY app setting in the Azure portal (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-app-service-app-setting-save.png":::

+--

+ 1. Under **Authentication type**, select **User-assigned identity**.

+ **Step 1:** Back in the App Service page, in the left menu,

+ 1. Select **SSH**.

+ **Step 1:** Back in the App Service page, in the left menu,

+ 1. Select **SSH**.

+### [Flask](#tab/flask)

+ :::column span="2":::

+ **Step 1:** In the App Service page:

+ 1. From the left menu, select **Overview**.

+ 1. Select the URL of your app. You can also navigate directly to `https://<app-name>.azurewebsites.net`.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-1.png" alt-text="A screenshot showing how to launch an App Service from the Azure portal (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2:** Add a few restaurants to the list.

+ Congratulations, you're running a web app in Azure App Service, with secure connectivity to Azure Database for PostgreSQL.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png" alt-text="A screenshot of the Flask web app with PostgreSQL running in Azure showing restaurants and restaurant reviews (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png":::

+ :::column-end:::

+### [Django](#tab/django)

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-1.png" alt-text="A screenshot showing how to launch an App Service from the Azure portal (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-1.png":::

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2-django.png" alt-text="A screenshot of the Flask web app with PostgreSQL running in Azure showing restaurants and restaurant reviews (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png":::

+The home page shows that the name of the last visited restaurant, and the data is saved to and retrieved from the Azure cache. Remember that the sample app uses the connection string `AZURE_REDIS_CONNECTIONSTRING`, which was created for you by the wizard.

+--

+ ### [Flask](#tab/flask)

+

+ ### [Django](#tab/django)

+

+ ```bash

+ git clone -b starter-no-infra https://github.com/Azure-Samples/msdocs-django-postgresql-sample-app

+ cd msdocs-django-postgresql-sample-app

+ ```

+ --

+ This azd template contains files (*azure.yaml* and the *infra* directory) that generate a secure-by-default architecture with the following Azure resources:

+ ### [Flask](#tab/flask)

+

+ - **Resource group** &rarr; The container for all the created resources.

+ - **App Service plan** &rarr; Defines the compute resources for App Service. A Linux plan in the *B1* tier is specified.

+ - **App Service** &rarr; Represents your app and runs in the App Service plan.

+ - **Virtual network** &rarr; Integrated with the App Service app and isolates back-end network traffic.

+ - **Azure Database for PostgreSQL flexible server** &rarr; Accessible only from within the virtual network. A database and a user are created for you on the server.

+ - **Private DNS zone** &rarr; Enables DNS resolution of the PostgreSQL server in the virtual network.

+ - **Log Analytics workspace** &rarr; Acts as the target container for your app to ship its logs, where you can also query the logs.

+ ### [Django](#tab/django)

+

+ - **Resource group** &rarr; The container for all the created resources.

+ - **App Service plan** &rarr; Defines the compute resources for App Service. A Linux plan in the *B1* tier is specified.

+ - **App Service** &rarr; Represents your app and runs in the App Service plan.

+ - **Virtual network** &rarr; Integrated with the App Service app and isolates back-end network traffic.

+ - **Azure Database for PostgreSQL flexible server** &rarr; Accessible only from within the virtual network. A database and a user are created for you on the server.

+ - **Azure Cache for Redis** &rarr; Accessible only from within the virtual network.

+ - **Private DNS zone** &rarr; Enables DNS resolution of the PostgreSQL server in the virtual network.

+ - **Log Analytics workspace** &rarr; Acts as the target container for your app to ship its logs, where you can also query the logs.

+ --

+1. In the azd output, find the app settings and find the settings `AZURE_POSTGRESQL_CONNECTIONSTRING` and `AZURE_REDIS_CONNECTIONSTRING`. To keep secrets safe, only the setting names are displayed. They look like this in the azd output:

+ - AZURE_REDIS_CONNECTIONSTRING

+1. `AZURE_POSTGRESQL_CONNECTIONSTRING` contains the connection string to the Postgres database in Azure, and `AZURE_REDIS_CONNECTIONSTRING` contains the connection string to the Redis cache in Azure. You need to use them your code to connect to it. Open *azureproject/production.py*, uncomment the following lines, and save the file:

+ ### [Flask](#tab/flask)

+

+ ### [Django](#tab/django)

+

+ ```python

+ conn_str = os.environ['AZURE_POSTGRESQL_CONNECTIONSTRING']

+ conn_str_params = {pair.split('=')[0]: pair.split('=')[1] for pair in conn_str.split(' ')}

+ DATABASES = {

+ 'default': {

+ 'ENGINE': 'django.db.backends.postgresql',

+ 'NAME': conn_str_params['dbname'],

+ 'HOST': conn_str_params['host'],

+ 'USER': conn_str_params['user'],

+ 'PASSWORD': conn_str_params['password'],

+ }

+ }

+ CACHES = {

+ "default": {

+ "BACKEND": "django_redis.cache.RedisCache",

+ "LOCATION": os.environ.get('AZURE_REDIS_CONNECTIONSTRING'),

+ "OPTIONS": {

+ "CLIENT_CLASS": "django_redis.client.DefaultClient",

+ "COMPRESSOR": "django_redis.compressors.zlib.ZlibCompressor",

+ },

+ }

+ }

+ ```

+ Your application code is now configured to connect to the PostgreSQL database and Redis cache in Azure.

+ --

+

+1. In the terminal, run `azd deploy`.

+### [Flask](#tab/flask)

+### [Django](#tab/django)

+1. In the azd output, find the URL for the SSH session and navigate to it in the browser. It looks like this in the output:

+ <pre>

+ Open SSH session to App Service container at: https://&lt;app-name>.scm.azurewebsites.net/webssh/host

+ </pre>

+1. In the SSH terminal, run `python manage.py migrate`. If it succeeds, App Service is [connecting successfully to the database](#i-get-an-error-when-running-database-migrations).

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-generate-db-schema-django-2.png" alt-text="A screenshot showing the commands to run in the SSH shell and their output (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-generate-db-schema-django-2.png":::

+ > [!NOTE]

+ > Only changes to files in `/home` can persist beyond app restarts. Changes outside of `/home` aren't persisted.

+ >

+

+--

+ ### [Flask](#tab/flask)

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png" alt-text="A screenshot of the Flask web app with PostgreSQL running in Azure showing restaurants and restaurant reviews (Flask)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png":::

+ ### [Django](#tab/django)

+

+ :::image type="content" source="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2-django.png" alt-text="A screenshot of the Django web app with PostgreSQL running in Azure showing restaurants and restaurant reviews (Django)." lightbox="./media/tutorial-python-postgresql-app/azure-portal-browse-app-2.png":::

+ --

+### [Flask](#tab/flask)

+### [Django](#tab/django)

+--

+In the azd output, find the link to stream App Service logs and navigate to it in the browser. The link looks like this in the azd output:

+<pre>

+Stream App Service logs at: https://portal.azure.com/#@/resource/subscriptions/&lt;subscription-guid>/resourceGroups/&lt;group-name>/providers/Microsoft.Web/sites/&lt;app-name>/logStream

+</pre>

+|[Hitachi UCP with Red Hat OpenShift](https://www.hitachivantara.com/en-us/solutions/hybrid-cloud-infrastructure.html)|1.25.11|1.25.0_2023-11-14|16.0.5100.7246|Not validated|

+|[Hitachi UCP with VMware Tanzu](https://www.hitachivantara.com/en-us/solutions/hybrid-cloud-infrastructure.html)|1.23.8 |1.16.0_2023-02-14 |16.0.937.6221 |14.5 (Ubuntu 20.04)|

+Use the following commands to delete your Flux configurations and, if desired, the Flux extension itself.

+#### Delete the Flux configurations

+> [!IMPORTANT]

+> Be sure to delete all Flux configurations in the cluster before you delete the Flux extension. Deleting the extension without first deleting the Flux configurations may leave your cluster in an unstable condition.

+> [!IMPORTANT]

+> Be sure to delete all Flux configurations in the cluster before you delete the Flux extension. Deleting the extension without first deleting the Flux configurations may leave your cluster in an unstable condition.

+| `Arc` | `his.arc.azure.com`</br>`guestconfiguration.azure.com` |

+[Manage VM extensions to use Azure management services for your SCVMM VMs](../servers/manage-vm-extensions.md).

+[Manage VM extensions to use Azure management services for your SCVMM VMs](../servers/manage-vm-extensions.md).

+- [Browse and enable SCVMM resources through Azure RBAC](enable-scvmm-inventory-resources.md).

+- [Create a VM using Azure Arc-enabled SCVMM](create-virtual-machine.md).

+Download the [deboarding script](https://aka.ms/arcvmwaredeboard) to do a full cleanup of all the Arc-enabled VMware resources. The script removes all the Azure resources, including vCenter, custom location, virtual machines, virtual templates, hosts, clusters, resource pools, datastores, virtual networks, Azure Resource Manager (ARM) resource of Appliance, and the appliance VM running on vCenter.

+The following optional settings can be configured for the SQL trigger:

+| Setting | Description|

+When you develop your functions locally, you need to take trigger and binding behaviors into consideration. For HTTP triggers, you can simply call the HTTP endpoint on the local computer, using `http://localhost/`. For non-HTTP triggered functions, there are several options to run locally:

+During local testing, you must be running the host provided by Core Tools (func.exe) locally. For more information, see [Azure Functions Core Tools](functions-run-local.md).

+| Creator - Conversion, Dataset, Feature State, Features, Map Configuration, Style, Routeset, Wayfinding | 50 | Not Available | Not Available |

+| Red Hat Enterprise Linux Server 6.7+ | | | |

+ "Microsoft-Event"

+ "userId (?<redactedUserId>[\\da-zA-Z]+)"

+We recommend you use the OpenTelemetry APIs whenever possible, but there might be some scenarios when you have to use the Application Insights [Classic API](api-custom-events-metrics.md).

+##### Events

+1. Add `Microsoft.ApplicationInsights` to your application.

+2. Create a `TelemetryClient` instance.

+> [!NOTE]

+> It's important to only create once instance of the TelemetryClient per application.

+```csharp

+var telemetryConfiguration = new TelemetryConfiguration { ConnectionString = "" };

+var telemetryClient = new TelemetryClient(telemetryConfiguration);

+```

+3. Use the client to send custom telemetry.

+```csharp

+telemetryClient.TrackEvent("testEvent");

+```

+##### Events

+1. Add `Microsoft.ApplicationInsights` to your application.

+2. Create a `TelemetryClient` instance.

+> [!NOTE]

+> It's important to only create once instance of the TelemetryClient per application.

+```csharp

+var telemetryConfiguration = new TelemetryConfiguration { ConnectionString = "" };

+var telemetryClient = new TelemetryClient(telemetryConfiguration);

+```

+3. Use the client to send custom telemetry.

+```csharp

+telemetryClient.TrackEvent("testEvent");

+```

+* [Log4j 2.0 MapMessage](https://logging.apache.org/log4j/2.0/javadoc/log4j-api/org/apache/logging/log4j/message/MapMessage.html) (a `MapMessage` key of `"message"` is captured as the log message)

+| ***Data Collection Rules*** | |

+> Syslog collection is now GA. However due to slower rollouts towards the year end, the agent version with the GA changes will not be in all regions until the end of January 2024. Agent versions 3.1.16 and above have Syslog GA changes. Please check agent version before enabling in production.

+This script installs VM extensions for Log Analytics/Azure Monitoring Agent (AMA) and Dependency Agent if needed for VM Insights. If AMA is onboarded, a Data Collection Rule (DCR) and a User Assigned Managed Identity (UAMI) is also associated with the virtual machines and virtual machine scale sets.

+- See [Supported operating systems](./vminsights-enable-overview.md#supported-operating-systems) to ensure that the operating system of the virtual machine or virtual machine scale set you're enabling is supported.

+To enable VM insights for multiple VMs or virtual machine scale set, use the PowerShell script [Install-VMInsights.ps1](https://www.powershellgallery.com/packages/Install-VMInsights). The script is available from the Azure PowerShell Gallery. This script iterates through the virtual machines or virtual machine scale sets according to the parameters that you specify. The script can be used to enable VM insights for:

+- The scoped resource group that's specified by `-ResourceGroup`.

+- A single VM or virtual machine scale set that's specified by `-Name`.

+Verify that you're using Az PowerShell module version 1.0.0 or later with `Enable-AzureRM` compatibility aliases enabled. Run `Get-Module -ListAvailable Az` to find the version. If you need to upgrade, see [Install Azure PowerShell module](/powershell/azure/install-azure-powershell). If you're running PowerShell locally, you also need to run `Connect-AzAccount` to create a connection with Azure.

+For a list of the script's argument details and example usage, run `Get-Help`.

+Get-Help Install-VMInsights.ps1 -Detailed

+```

+Use the script to enable VM insights using Azure Monitoring Agent and Dependency Agent, or Log Analytics Agent.

+### [Azure Monitor Agent](#tab/AMA)

+AMA Onboarding

+If AMA is onboarded, a Data Collection Rule (DCR) and a User Assigned Managed Identity (UAMI) is also associated to the VM/VMSS and UAMI settings are passed over to AMA extension.

+```powershell

+Install-VMInsights.ps1 -SubscriptionId <SubscriptionId> `

+[-ResourceGroup <ResourceGroup>] `

+[-ProcessAndDependencies ] `

+[-Name <MV or VMSS name>] `

+-DcrResourceId <DataCollectionRuleResourceId> `

+-UserAssignedManagedIdentityName <UserAssignedIdentityName> `

+-UserAssignedManagedIdentityResourceGroup <UserAssignedIdentityResourceGroup>

+```

+Required Arguments:

+ + `-SubscriptionId <String>` Azure subscription ID.

+ + `-DcrResourceId <String> ` Data Collection Rule (DCR) Azure resource ID identifier.

+ + `-UserAssignedManagedIdentityResourceGroup <String> ` Name of User Assigned Managed Identity (UAMI) resource group.

+ + `-UserAssignedManagedIdentityName <String> ` Name of User Assigned Managed Identity (UAMI).

+Optional Arguments:

+ + `-ProcessAndDependencies` Set this flag to onboard the Dependency Agent with Azure Monitoring Agent (AMA) settings. If not specified, only Azure Monitoring Agent (AMA) will be onboarded.

+ + ` - Name <String>` Name of the VM or VMSS to be onboarded. If not specified, all VMs and VMSS in the subscription or resource group will be onboarded.

+ + `- ResourceGroup <String>` Name of the resource group containing the VM or VMSS to be onboarded. If not specified, all VMs and VMSS in the subscription will be onboarded.

+Example:

+```azurepowershell

+Install-VMInsights.ps1 -SubscriptionId 12345678-abcd-abcd-1234-12345678 `

+-ResourceGroup rg-AMAPowershell `

+-ProcessAndDependencies `

+-Name vmAMAPowershellWindows `

+-DcrResourceId /subscriptions/12345678-abcd-abcd-1234-12345678/resourceGroups/rg-AMAPowershell/providers/Microsoft.Insights/dataCollectionRules/MSVMI-ama-vmi-default-dcr `

+-UserAssignedManagedIdentityName miamatest1 `

+-UserAssignedManagedIdentityResourceGroup amapowershell

+```

+The output has the following format:

+```powershell

+Name Account SubscriptionName Environment TenantId

+- - - -- --

+AzMon001 12345678-abcd-123… MSI@9876 AzMon001 AzureCloud abcd1234-9876-abcd-1234-1234abcd5648

+Getting list of VMs or VM Scale Sets matching specified criteria.

+VMs and VMSS matching selection criteria :

+ResourceGroup : rg-AMAPowershell

+ vmAMAPowershellWindows

+Confirm

+Continue?

+[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"):

+(rg-AMAPowershell) : Assigning roles

+(rg-AMAPowershell) vmAMAPowershellWindows : Assigning User Assigned Managed Identity edsMIAMATest

+(rg-AMAPowershell) vmAMAPowershellWindows : Successfully assigned User Assigned Managed Identity edsMIAMATest

+(rg-AMAPowershell) vmAMAPowershellWindows : Data Collection Rule Id /subscriptions/12345678-abcd-abcd-1234-12345678/resourceGroups/rg-AMAPowershell/providers/Microsoft.Insights/dataCollectionRules/MSVMI-ama-vmi-default-dcr already associated with the VM.

+(rg-AMAPowershell) vmAMAPowershellWindows : Extension AzureMonitorWindowsAgent, type = Microsoft.Azure.Monitor.AzureMonitorWindowsAgent already installed. Provisioning State : Succeeded

+(rg-AMAPowershell) vmAMAPowershellWindows : Installing/Updating extension AzureMonitorWindowsAgent, type = Microsoft.Azure.Monitor.AzureMonitorWindowsAgent

+(rg-AMAPowershell) vmAMAPowershellWindows : Successfully installed/updated extension AzureMonitorWindowsAgent, type = Microsoft.Azure.Monitor.AzureMonitorWindowsAgent

+(rg-AMAPowershell) vmAMAPowershellWindows : Installing/Updating extension DA-Extension, type = Microsoft.Azure.Monitoring.DependencyAgent.DependencyAgentWindows

+(rg-AMAPowershell) vmAMAPowershellWindows : Successfully installed/updated extension DA-Extension, type = Microsoft.Azure.Monitoring.DependencyAgent.DependencyAgentWindows

+(rg-AMAPowershell) vmAMAPowershellWindows : Successfully onboarded VM insights

+Summary :

+Total VM/VMSS to be processed : 1

+Succeeded : 1

+Skipped : 0

+Failed : 0

+VMSS Instance Upgrade Failures : 0

+```

+### [Log Analytics Agent](#tab/LogAnalyticsAgent)

+Use the following command to enable VM insights using Log Analytics Agent and Dependency Agent.

+Install-VMInsights.ps1 -WorkspaceId $WorkspaceId `

+-WorkspaceKey $WorkspaceKey `

+-SubscriptionId $SubscriptionId `

+-WorkspaceRegion <region>

+```

+The output has the following format:

+```powershell

+Check your VM/VMSS in Azure portal to see if the extensions are installed or use the following command:

+```powershell

+az vm extension list --resource-group <resource group> --vm-name <VM name> -o table

+Name ProvisioningState Publisher Version AutoUpgradeMinorVersion

+ - -

+AzureMonitorWindowsAgent Succeeded Microsoft.Azure.Monitor 1.16 True

+DA-Extension Succeeded Microsoft.Azure.Monitoring.DependencyAgent 9.10 True

+```

+* Australia Central

+* Australia Central 2

+* Switzerland North

+* Switzerland West

+* UAE North

+This quickstart guides you through the steps to create a [Bicep file](overview.md) with Visual Studio Code. You create a storage account and a virtual network. You also learn how the Bicep extension simplifies development by providing type safety, syntax validation, and autocompletion.

+To set up your environment for Bicep development, see [Install Bicep tools](install.md). After completing those steps, you have [Visual Studio Code](https://code.visualstudio.com/) and the [Bicep extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-bicep). You also have either the latest [Azure CLI](/cli/azure/) or the latest [Azure PowerShell module](/powershell/azure/new-azureps-module-az).

+VS Code with the Bicep extension simplifies development by providing predefined snippets. In this quickstart, you add a snippet that creates a virtual network.

+Launch Visual Studio Code and create a new file named **main.bicep**.

+In *main.bicep*, type **vnet**, and then select **res-vnet** from the list, and then press [TAB] or [ENTER].

+> If you don't see those intellisense options in VS Code, make sure you've installed the Bicep extension as specified in [Prerequisites](#prerequisites). If you have installed the extension, give the Bicep language service some time to start after opening your Bicep file. It usually starts quickly, but you don't have intellisense options until it starts. A notification in the lower right corner indicates that the service is starting. When that notification disappears, the service is running.

+ location: location

+Within this snippet, you find all the necessary values for defining a virtual network. You may notice two curly underlines. A yellow one denotes a warning related to an outdated API version, while a red curly underline signals an error caused by a missing parameter definition.

+Remove `@2019-11-01`, and replace it with `@`. Select the latest API version.

+You'll fix the missing parameter definition error in the next section.

+You can also modify this code to meet your requirements. For example, `name` isn't a great name for the virtual network. Change the `name` property to `examplevnet`.

+name: 'exampleVNet'

+The code snippet you added in the last section misses a parameter definition.

+At the top of the file, add:

+param location

+When you add a space after **location**, notice that intellisense offers the data types that are available for the parameter. Select **string**.

+Give the parameter a default value:

+param location string = resourceGroup().location

+For more information about the function used in the default value, see [resourceGroup()](./bicep-functions-scope.md#resourcegroup).

+Add another parameter for the storage account name with a default value:

+```bicep

+param storageAccountName string = 'store${uniqueString(resourceGroup().id)}'

+```

+For more information, see [Interpolation](./data-types.md#strings) and [uniqueString()](./bicep-functions-string.md#uniquestring).

+This parameter works fine, but storage accounts have limits on the length of the name. The name must have at least three characters and no more than 24 characters. You can specify those requirements by adding decorators to the parameter.

+Add both decorators and specify the character limits:

+param storageAccountName string = 'store${uniqueString(resourceGroup().id)}'

+param storageAccountName string = 'store${uniqueString(resourceGroup().id)}'

+Your parameters are ready to use.

+Instead of using a snippet to define the storage account, you use intellisense to set the values. Intellisense makes this step easier than having to manually type the values.

+When you add a space after the symbolic name, a list of resource types is displayed. Continue typing **storageacc** until you can select it from the available options.

+After selecting **Microsoft.Storage/storageAccounts**, you're presented with the available API versions. Select the latest version. For the following screenshot, it is **2023-01-01**.

+After the single quote for the resource type, add **=** and a space. You're presented with options for adding properties to the resource. Select **required-properties**.

+resource exampleStorage 'Microsoft.Storage/storageAccounts@2023-01-01' = {

+Again, intellisense helps you. Set `name` to `storageAccountName`, which is the parameter that contains a name for the storage account. For `location`, set it to `location`, which is a parameter you created earlier. When adding `sku.name` and `kind`, intellisense presents the valid options.

+When finished, you have:

+param storageAccountName string = 'store${uniqueString(resourceGroup().id)}'

+ name: 'exampleVNet'

+ name: storageAccountName

+1. Right-click the Bicep file inside the VS Code, and then select **Deploy Bicep file**.

+1. In the **Please enter name for deployment** text box, type **deployStorageAndVNet**, and then press **[ENTER]**.

+1. Select a location for the resource group, select **Central US** or a location of your choice, and then press **[ENTER]**.

+It takes a few moments to create the resources. For more information, see [Deploy Bicep files with Visual Studio Code](./deploy-vscode.md).

+az deployment group create --resource-group exampleRG --template-file main.bicep --parameters storageAccountName=uniquename

+New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile ./main.bicep -storageAccountName "uniquename"

+> [!NOTE]

+| `JitterBufferSizeAvg` | The average size of jitter buffer over the duration of each media stream. A jitter buffer is a shared data area where voice packets can be collected, stored, and sent to the voice processor in evenly spaced intervals. Jitter buffer is used to counter the effects of jitter. <br><br> Jitter buffers can be either static or dynamic. Static jitter buffers are set to a fixed size, while dynamic jitter buffers can adjust their size based on network conditions. The goal of the jitter buffer is to provide a smooth and uninterrupted stream of audio and video data to the user. <br><br> In the web SDK, this 'JitterBufferSizeAvg' is the average value of the 'jitterBufferDelay' during the call, the 'jitterBufferDelay' is the duration of an audio sample or a video frame that stays in the jitter buffer. <br><br> Normally when 'JitterBufferSizeAvg' value is greater than 200 ms, it will cause a negative quality impact.

+| `JitterBufferSizeMax` | The maximum jitter buffer size measured during the duration of each media stream. <br><br> Normally when this value is greater than 200 ms, it will cause a negative quality impact.

+| `HealedDataRatioAvg` | The average percentage of lost or damaged data packets that are successfully reconstructed or recovered by the healer over the duration of audio stream. Healed data ratio is a measure of the effectiveness of error correction techniques used in VoIP systems. <br><br> When this value is greater than 0.1 (10%), we consider the stream as bad quality.

+| `HealedDataRatioMax` | The maximum healed data ratio measured during the duration of each media stream. <br><br> When this value is greater than 0.1 (10%), we consider the stream as bad quality.

+| `VideoFrameRateAvg` | The average number of video frames that are transmitted per second during a video/screensharing call. The video frame rate can impact the quality and smoothness of the video stream, with higher frame rates generally resulting in smoother and more fluid motion. The standard frame rate for WebRTC video is typically 30 frames per second (fps), although this can vary depending on the specific implementation and network conditions. <br><br> The stream quality is considered poor when this value is less than 7 for video stream, or less than 1 for screensharing stream.

+| `RecvResolutionHeight` | The average of vertical size of the incoming video stream that is transmitted during a video/screensharing call. It's measured in pixels and is one of the factors that determines the overall resolution and quality of the video stream. The specific resolution used may depend on the capabilities of the devices and network conditions involved in the call. <br><br> The stream quality is considered poor when this value is less than 240 for video stream, or less than 768 for screensharing stream.

+| `RecvFreezeDurationPerMinuteInMs` | The average freeze duration in milliseconds per minute for incoming video/screensharing stream. Freezes are typically due to bad network condition and can degrade the stream quality. <br><br> The stream quality is considered poor when this value is greater than 6,000 ms for video stream, or greater than 25,000 ms for screensharing stream.

+Azure AI services can be easily integrated into any application regardless of the programming language. When creating an Azure Resource in Azure portal, enable the option and provide the URL to the Azure AI services. This simple experience allows developers to meet their needs, scale, and avoid investing time and resources into designing and maintaining a custom solution.

+You will need to connect your Azure Communication Services resource with the Azure AI resource through the Azure portal. There are two ways you can accomplish this step:

+- By navigating through the steps of the Cognitive Services tab in your Azure Communication Services (recommended).

+- Manually adding the Managed Identity to your Azure Communication Services resource. This step is more advanced and requires a little more effort to connect your Azure Communication Services to your Azure AI services.

+- An [Azure AI Services resource](../../../../articles/ai-services/multi-service-resource.md) .

+2. If system-assigned managed identity isn't enabled, you will need to enable it.

+3. In the Cognitive Services tab, click on "Enable Managed Identity" button.

+

+4. Enable system assigned identity. This action begins the creation of the identity; A pop-up notification appears notifying you that the request is being processed.

+5. Once the identity is enabled, you should see something similar.

+6. When managed identity is enabled the Cognitive Service tab should show a button 'Connect cognitive service' to connect the two services.

+7. Click on 'Connect cognitive service', select the Subscription, Resource Group and Resource and click 'Connect' in the context pane that opens up.

+8. If connection is successful, you should see a green banner confirming successful connection.

+9. Now in the Cognitive Service tab you should see your connected services showing up.

+### Advanced option: Manually adding Managed Identity to Azure Communication Services resource

+- eastus

+- eastus2

+- westus

+- westus2

+- westus3

+- canadacentral

+- northeurope

+## Known limitation

+- In-band DTMF is not supported, use RFC 2833 DTMF instead.

+- For Teams Interop scenarios, it is the number of Azure Communication Services users, not Teams users, that must be below 20 for the typing indicator feature to be supported.

+- For Teams Interop scenarios, the typing indicator event might contain a blank display name when sent from Teams user.

+- For Teams Interop scenarios, read receipts aren't supported for Teams users.

+- Known issue: Read receipts aren't supported for Teams users.

+- Known issue: Editing of messages by the Teams user isn't supported.

+|**Supported Destinations**| United States, Canada, Puerto Rico| United States, Canada, United Kingdom | Germany, Netherlands, United Kingdom, Australia, France, Switzerland, Sweden, Italy, Spain, Denmark, Ireland, Portugal, Poland, Austria, Lithuania, Latvia, Estonia| Norway, Finland, Slovakia, Slovenia, Czech Republic|

+ Title: Azure Communication Services Call Diagnostics

+description: Use Call Diagnostics to diagnose call issues with Azure Communication Services

+# Call Diagnostics

+Understanding your call quality and reliability is foundational to

+delivering a great customer calling experience. There are various

+issues that can affect the quality of your calls, such as poor internet

+connectivity, software compatibility issues, and technical difficulties

+with devices. These issues can be frustrating for all call participants,

+whether they're a patient checking in for a doctorΓÇÖs call, or a student

+taking a lesson with their teacher. As a developer, diagnosing and

+fixing these issues can be time-consuming and frustrating.

+Call Diagnostics acts as a detective for your calls. It helps developers

+using Azure Communication Services investigate events that happened in a call to

+identify likely causes of poor call quality and reliability. Just like a

+real conversation, many things happen simultaneously in a call that may

+or may not affect your communication. Call DiagnosticsΓÇÖ timeline makes

+it easier to visualize what happened in a call by showing you rich data

+visualizations of call events and providing insights into issues that

+commonly affect calls.

+## How to enable Call Diagnostics

+Azure Communication Services collects call data in the form of metrics

+and events. You must enable a Diagnostic Setting in Azure Monitor to

+send these data to a Log Analytics workspace for Call Diagnostics to

+analyze new call data.

+> [!IMPORTANT]

+> Call Diagnostics canΓÇÖt query data from data that wasnΓÇÖt sent to a Log Analytics workspace. Diagnostic Settings will only begin collect data by single Azure Communications Services Resource ID once enabled. See our Frequently Asked Question on enabling Call Diagnostics [here](#frequently-asked-questions)

+Since Call Diagnostics is an application layer on top of data for your

+Azure Communications Service Resource you can query these call data and

+[build workbook reports on top of your data](../../../azure-monitor/logs/data-platform-logs.md#what-can-you-do-with-azure-monitor-logs)

+You can access Call Diagnostics from any Azure Communication Services

+Resource in your Azure portal. When you open your Azure Communications

+Services Resource, just look for the ΓÇ£MonitoringΓÇ¥ section on the left

+side of the screen and select "Call Diagnostics."

+Once you have setup Call Diagnostics for your Azure Communication Services Resource you can search for calls using valid callIDs that took place in that resource. Data can take several hours after call completion to appear in your resource and populate in Call Diagnostics.

+**Call Diagnostics has four main sections:**

+- [Call Search](#call-search)

+- [Call Overview](#call-overview)

+- [Call Issues](#call-issues)

+- [Call Timeline](#call-timeline)

+## Call Search

+The search section lets you find individual calls, or filter calls to explore calls with issues. Clicking on a call takes you to a detail screen where you

+see three sections, **Overview**, **Issues**, and **Timeline** for the

+selected call.

+The search field allows you to search by callID. See our documentation to [access your client call ID.](../troubleshooting-info.md#access-your-client-call-id)

+ <!-- (**insert image)** -->

+> [!NOTE]

+> You can explore information icons and links within Call Diagnostics to learn functionality, definitions, and helpful tips.

+## Call Overview

+Once you select a call from the Call Search page, your call details will

+display in the Call Overview tab. YouΓÇÖll see a call summary highlighting

+the participants in the call and key metrics for their call quality. You

+can select a participant to drill into their call timeline details

+directly or navigate to the Call Issues tab for further analysis.

+<!-- (**<u>TODO insert image)</u>** -->

+<!-- > [!NOTE]

+> You can explore information icons and links on each page within Call Diagnostics to learn functionality, definitions, and helpful tips. -->

+<!-- (**insert image)** -->

+## Call Issues

+The Call Issues tab gives you a high-level analysis of any media quality

+and reliability issues that were detected during the call.

+Call Issues highlights detected issues commonly known to affect userΓÇÖs call

+quality such as poor network conditions, speaking while muted, or device

+failures during a call. If you want to explore a detected issue, select

+the highlighted item and you'll see a pre-populated view of the

+related events in the Timeline tab.

+<!-- (**<u>TODO insert image)</u>** -->

+<!-- > [!NOTE]

+> You can explore information icons and links on each page within Call Diagnostics to learn functionality, definitions, and helpful tips. -->

+## Call Timeline

+When call issues are difficult to troubleshoot, you can explore the

+timeline tab to see a detailed sequence of events that occurred during

+the call.

+The timeline view is complex and designed for developers who need

+explore details of a call and interpret detailed debugging data. In

+large calls the timeline view can present an overwhelming amount of

+information, we recommend relying on filtering to narrow your search

+results and reduce complexity.

+You can view detailed call logs for each participant within a call. Call information may not be present, this can be due to various reasons such as privacy constraints between different calling resources. See frequently asked questions to learn more.

+<!-- (**<u>TODO insert image)</u>** -->

+<!-- > [!NOTE]

+> You can explore information icons and links on each page within Call Diagnostics to learn functionality, definitions, and helpful tips. -->

+<!-- # Common issues

+Issue categories can include:

+- Azure Communication Services issue

+- Calling deployment issue

+- Network issue

+- User actions or inactions (e.g. not allowing device permissions),

+ driving through a tunnel.

+To help you get started, you will find below the steps to triage common

+issues using Call Diagnostics.

+***ΓÇ£Other participants couldnΓÇÖt hear me on the callΓÇ¥***

+Dive into the audio section for the participant to see if there are any

+issues detected. In the case below, we see that the microphone was muted

+unexpectedly. In other cases, we might see errors with the deviceΓÇÖs set

+up and permissions.

+(**<u>TODO insert image)</u>**

+***ΓÇ£My video was choppy and pixelatedΓÇ¥***

+Explore the video section for the participant to see if a poor network

+connection in a call may have caused the issue.

+(**<u>TODO insert image)</u>**

+***ΓÇ£My call unexpectedly droppedΓÇ¥***

+**<u>TODO -</u>** Show how you might drill down to show the end-user

+lost connection.

+(**<u>TODO insert image)</u>**

+***ΓÇ£Other participants couldnΓÇÖt see me on the callΓÇ¥***

+Show how you might drill down to show the status of the camera in the

+call and any detected failures.

+(**<u>TODO insert image)</u>**

+## Call quality resources

+Ensuring good call quality starts with your calling setup, please

+explore our documentation to learn how you can use the UI Library to

+benefit from our quality and reliability tools \<[link to manage call

+quality](https://learn.microsoft.com/azure/communication-services/concepts/voice-video-calling/manage-call-quality)\>. -->

+## Frequently asked questions:

+- How do I setup Call Diagnostics?

+ - Follow instructions to add diagnostic settings for your resource here [Enable logs via Diagnostic Settings in Azure Monitor.](../analytics/enable-logging.md) When prompted to select [select logs](../analytics/enable-logging.md#adding-a-diagnostic-setting) select "**allLogs**".

+ - If you have multiple Azure Communications Services Resource IDs you must enable these settings for each resource ID and query call details for participants within their respective Azure Communications Services Resource ID. Your data volume, retention, and CDC query usage in Log Analytics is billed through existing Azure data meters, monitor your data usage and retention policies for [cost considerations as needed](../../../azure-monitor/logs/cost-logs.md)

+- If Azure Communication Services participants join from different Azure Communication Services Resources, how will they display in Call Diagnostics?

+ - If all the participants are from the same Azure subscription, they'll appear as "remote participants". However, Call Diagnostics wonΓÇÖt show any participant details for Azure Communication Services participants from another resource. You need to review that same call ID from the specific Azure Communication Services Resource the participant belongs to.

+ <!-- 2. If that ACS resource isn't part of **<u>your Azure subscription

+ and / or hasn't enabled Diagnostics Settings to store call logs,

+ there will not be any data available</u>** for Call Diagnostics. -->

+<!-- 1. If Teams participants join a call, how will they display in Call

+ Diagnostics?

+ 1. If a Teams participant organized the call through Microsoft

+ Teams, that participant will appear as a participant in Call

+ Diagnostics, however they'll have fewer call details populated.

+ 2. If there were other Teams participants besides the Teams meeting

+ organizer, those participants won't appear in Call

+ Diagnostics. -->

+<!-- 1. How do I find a Call ID?

+ a. Link -->

+<!-- 1. My call ID should be here?

+ a. It could no longer be stored by your Log Analytics workspace, you may need to ensure you retain your call data in diagnostics settings. It's possible your callID is incorrect. (**ENG add details on which call ID to specifically pull in the event of multiple callIDs.**)

+ a. Maybe itΓÇÖs not the ACS call ID, check ΓÇ£how do I find a callID?ΓÇ¥ to learn more. -->

+<!-- 1. My call had issues, but Call Diagnostics doesnΓÇÖt show any issues.

+ a. Call Diagnostics relies on several common call issues to help diagnose calls. Issues can still occur outside of the existing telemetry or can be caused by unlisted call participants you arenΓÇÖt allowed to view due to privacy restrictions. -->

+<!-- 1. What types of calls are visible in Call Diagnostics?

+ a. Call types included.

+ 1. Includes call data for Web JS SDK, Native SKD, PSTN, Call Automation.

+ 1. Includes some Call Automation Bot data edges

+ a. Partial data.

+ a. Different SDKs, privacy considerations may prevent you from receiving those data. -->

+<!-- 1. What are limits of what our data reaches.

+ 1. Privacy restrictions may prevent you from seeing the full call roster.

+1. What are bots?

+1. What capabilities does Search have?

+1. What capabilities does Overview have?

+1. What capabilities does Issues have?

+1. What capabilities does Timeline have?

+ 1. You can zoom within the timeline by using SHIFT+mouse-scroll wheel and pan left and right by clicking and dragging within the timeline itself. -->

+<!-- 1. What types of issues might I find?

+ a. Participant’s call issues generally fall into these categories: 

+ 1. They can’t join a call. 

+ 1. They can’t do something in a call (mute, start video, etc.). 

+ 1. They get dropped from a call. 

+ 1. They have a poor call experience (audio/video quality).  -->

+<!-- FAQ - Clear cache - Ask Nan.

+People need to do X, in case your cache is stale or causing issues,

+choose credential A vs. B

+Clear your cache to ensure X, you may need clear your cache occasionally if you experience issues using Call Diagnostics. -->

+## Next steps

+- Learn how to manage call quality, see: [Improve and manage call quality](manage-call-quality.md)

+- Continue to learn other quality best practices, see: [Best practices: Azure Communication Services calling SDKs](../best-practices.md)

+- Learn how to use the Log Analytics workspace, see: [Log Analytics Tutorial](../../../../articles/azure-monitor/logs/log-analytics-tutorial.md)

+- Create your own queries in Log Analytics, see: [Get Started Queries](../../../../articles/azure-monitor/logs/get-started-queries.md)

+- Explore known call issues, see: [Known issues in the SDKs and APIs](../known-issues.md)

+<!-- added to the toc.yml file at row 583.

+ - name: Monitor and manage call quality

+ items:

+ - name: Manage call quality

+ href: concepts/voice-video-calling/manage-call-quality.md

+ displayName: diagnostics, Survey, feedback, quality, reliability, users, end, call, quick

+ - name: End of Call Survey

+ href: concepts/voice-video-calling/end-of-call-survey-concept.md

+ displayName: diagnostics, Survey, feedback, quality, reliability, users, end, call, quick

+ -->

+- Explore known issues, see: [Known issues in the SDKs and APIs](../known-issues.md)

+ Title: Azure Communication Services Call Automation how-to for passing call contextual data in Call Automation

+description: Provides a how-to guide for passing contextual information with Call Automation.

+# How to pass contextual data between calls

+Call Automation allows developers to pass along custom contextual information when routing calls. Developers can pass metadata about the call, callee or any other information that is relevant to their application or business logic. This allows businesses to manage, and route calls across networks without having to worry about losing context.

+Passing context is supported by specifying custom headers. These are an optional list of key-value pairs that can be included as part of `AddParticipant` or `Transfer` actions. The context can be later retrieved as part of the `IncomingCall` event payload.

+Custom call context is also forwarded to the SIP protocol, this includes both the freeform custom headers as well as the standard User-to-User Information (UUI) SIP header. When routing an inbound call from your telephony network, the data set from your SBC in the custom headers and UUI is similarly included in the `IncomingCall` event payload.

+All custom context data is opaque to Call Automation or SIP protocols and its content is unrelated to any basic functions.

+Below are samples on how to get started using custom context headers in Call Automation.

+As a prerequisite, we recommend you to read these articles to make the most of this guide:

+- Call Automation [concepts guide](../../concepts/call-automation/call-automation.md#call-actions) that describes the action-event programming model and event callbacks.

+- Learn about [user identifiers](../../concepts/identifiers.md#the-communicationidentifier-type) like CommunicationUserIdentifier and PhoneNumberIdentifier used in this guide.

+For all the code samples, `client` is CallAutomationClient object that can be created as shown and `callConnection` is the CallConnection object obtained from Answer or CreateCall response. You can also obtain it from callback events received by your application.

+## Technical parameters

+Call Automation supports up to 5 custom SIP headers and 1000 custom VOIP headers. Additionally, developers can include a dedicated User-To-User header as part of SIP headers list.

+The custom SIP header key must start with a mandatory ΓÇÿX-MS-Custom-ΓÇÖ prefix. The maximum length of a SIP header key is 64 chars, including the X-MS-Custom prefix. The maximum length of SIP header value is 256 chars. The same limitations apply when configuring the SIP headers on your SBC.

+The maximum length of a VOIP header key is 64 chars. These headers can be sent without ΓÇÿx-MS-CustomΓÇÖ prefix. The maximum length of VOIP header value is 1024 chars.

+## Adding custom context when inviting a participant

+### [csharp](#tab/csharp)

+```csharp

+// Invite a communication services user and include one VOIP header

+var addThisPerson = new CallInvite(new CommunicationUserIdentifier("<user_id>"));

+addThisPerson.CustomCallingContext.AddVoip("myHeader", "myValue");

+AddParticipantsResult result = await callConnection.AddParticipantAsync(addThisPerson);

+// Invite a PSTN user and set UUI and custom SIP headers

+var callerIdNumber = new PhoneNumberIdentifier("+16044561234");

+var addThisPerson = new CallInvite(new PhoneNumberIdentifier("+16041234567"), callerIdNumber);

+// Set custom UUI header. This key is sent on SIP protocol as User-to-User

+addThisPerson.CustomCallingContext.AddSipUui("value");

+// This provided key will be automatically prefixed with X-MS-Custom on SIP protocol, such as 'X-MS-Custom-{key}'

+addThisPerson.CustomCallingContext.AddSipX("header1", "customSipHeaderValue1");

+AddParticipantsResult result = await callConnection.AddParticipantAsync(addThisPerson);

+```

+### [Java](#tab/java)

+```java

+// Invite a communication services user and include one VOIP header

+CallInvite callInvite = new CallInvite(new CommunicationUserIdentifier("<user_id>"));

+callInvite.getCustomCallingContext().addVoip("voipHeaderName", "voipHeaderValue");

+AddParticipantOptions addParticipantOptions = new AddParticipantOptions(callInvite);

+Response<AddParticipantResult> addParticipantResultResponse = callConnectionAsync.addParticipantWithResponse(addParticipantOptions).block();

+// Invite a PSTN user and set UUI and custom SIP headers

+PhoneNumberIdentifier callerIdNumber = new PhoneNumberIdentifier("+16044561234");

+CallInvite callInvite = new CallInvite(new PhoneNumberIdentifier("+16041234567"), callerIdNumber);

+callInvite.getCustomCallingContext().addSipUui("value");

+callInvite.getCustomCallingContext().addSipX("header1", "customSipHeaderValue1");

+AddParticipantOptions addParticipantOptions = new AddParticipantOptions(callInvite);

+Response<AddParticipantResult> addParticipantResultResponse = callConnectionAsync.addParticipantWithResponse(addParticipantOptions).block();

+```

+### [JavaScript](#tab/javascript)

+```javascript

+// Invite a communication services user and include one VOIP header

+const customCallingContext: CustomCallingContext = [];

+customCallingContext.push({ kind: "voip", key: "voipHeaderName", value: "voipHeaderValue" })

+const addThisPerson = {

+ targetParticipant: { communicationUserId: "<acs_user_id>" },

+ customCallingContext: customCallingContext,

+};

+const addParticipantResult = await callConnection.addParticipant(addThisPerson);

+// Invite a PSTN user and set UUI and custom SIP headers

+const callerIdNumber = { phoneNumber: "+16044561234" };

+const customCallingContext: CustomCallingContext = [];

+customCallingContext.push({ kind: "sipuui", key: "", value: "value" });

+customCallingContext.push({ kind: "sipx", key: "headerName", value: "headerValue" })

+const addThisPerson = {

+ targetParticipant: { phoneNumber: "+16041234567" },

+ sourceCallIdNumber: callerIdNumber,

+ customCallingContext: customCallingContext,

+};

+const addParticipantResult = await callConnection.addParticipant(addThisPerson);

+```

+### [Python](#tab/python)

+```python

+#Invite a communication services user and include one VOIP header

+voip_headers = {"voipHeaderName", "voipHeaderValue"}

+target = CommunicationUserIdentifier("<acs_user_id>")

+result = call_connection_client.add_participant(

+ target,

+ voip_headers=voip_headers

+)

+#Invite a PSTN user and set UUI and custom SIP headers

+caller_id_number = PhoneNumberIdentifier("+16044561234")

+sip_headers = {}

+sip_headers.add("User-To-User", "value")

+sip_headers.add("X-MS-Custom-headerName", "headerValue")

+target = PhoneNumberIdentifier("+16041234567")

+result = call_connection_client.add_participant(

+ target,

+ sip_headers=sip_headers,

+ source_caller_id_number=caller_id_number

+)

+```

+--

+## Adding custom context during call transfer

+### [csharp](#tab/csharp)

+```csharp

+//Transfer to communication services user and include one VOIP header

+var transferDestination = new CommunicationUserIdentifier("<user_id>");

+var transferOption = new TransferToParticipantOptions(transferDestination);

+var transferOption = new TransferToParticipantOptions(transferDestination) {

+ OperationContext = "<Your_context>",

+ OperationCallbackUri = new Uri("<uri_endpoint>") // Sending event to a non-default endpoint.

+};

+transferOption.CustomCallingContext.AddVoip("customVoipHeader1", "customVoipHeaderValue1");

+TransferCallToParticipantResult result = await callConnection.TransferCallToParticipantAsync(transferOption);

+//Transfer a PSTN call to phone number and set UUI and custom SIP headers

+var transferDestination = new PhoneNumberIdentifier("<target_phoneNumber>");

+var transferOption = new TransferToParticipantOptions(transferDestination);

+transferOption.CustomCallingContext.AddSipUui("uuivalue");

+transferOption.CustomCallingContext.AddSipX("header1", "headerValue");

+TransferCallToParticipantResult result = await callConnection.TransferCallToParticipantAsync(transferOption)

+```

+### [Java](#tab/java)

+```java

+//Transfer to communication services user and include one VOIP header

+CommunicationIdentifier transferDestination = new CommunicationUserIdentifier("<user_id>");

+TransferCallToParticipantOptions options = new TransferCallToParticipantOptions(transferDestination);

+options.getCustomCallingContext().addVoip("voipHeaderName", "voipHeaderValue");

+Response<TransferCallResult> transferResponse = callConnectionAsync.transferToParticipantCallWithResponse(options).block();

+//Transfer a PSTN call to phone number and set UUI and custom SIP headers

+CommunicationIdentifier transferDestination = new PhoneNumberIdentifier("<taget_phoneNumber>");

+TransferCallToParticipantOptions options = new TransferCallToParticipantOptions(transferDestination);

+options.getCustomCallingContext().addSipUui("UUIvalue");

+options.getCustomCallingContext().addSipX("sipHeaderName", "value");

+Response<TransferCallResult> transferResponse = callConnectionAsync.transferToParticipantCallWithResponse(options).block();

+```

+### [JavaScript](#tab/javascript)

+```javascript

+//Transfer to communication services user and include one VOIP header

+const transferDestination = { communicationUserId: "<user_id>" };

+const transferee = { communicationUserId: "<transferee_user_id>" };

+const options = { transferee: transferee, operationContext: "<Your_context>", operationCallbackUrl: "<url_endpoint>" };

+const customCallingContext: CustomCallingContext = [];

+customCallingContext.push({ kind: "voip", key: "customVoipHeader1", value: "customVoipHeaderValue1" })

+options.customCallingContext = customCallingContext;

+const result = await callConnection.transferCallToParticipant(transferDestination, options);

+//Transfer a PSTN call to phone number and set UUI and custom SIP headers

+const transferDestination = { phoneNumber: "<taget_phoneNumber>" };

+const transferee = { phoneNumber: "<transferee_phoneNumber>" };

+const options = { transferee: transferee, operationContext: "<Your_context>", operationCallbackUrl: "<url_endpoint>" };

+const customCallingContext: CustomCallingContext = [];

+customCallingContext.push({ kind: "sipuui", key: "", value: "uuivalue" });

+customCallingContext.push({ kind: "sipx", key: "headerName", value: "headerValue" })

+options.customCallingContext = customCallingContext;

+const result = await callConnection.transferCallToParticipant(transferDestination, options);

+```

+### [Python](#tab/python)

+```python

+#Transfer to communication services user and include one VOIP header

+transfer_destination = CommunicationUserIdentifier("<user_id>")

+transferee = CommnunicationUserIdentifer("transferee_user_id")

+voip_headers = {"customVoipHeader1", "customVoipHeaderValue1"}

+result = call_connection_client.transfer_call_to_participant(

+ target_participant=transfer_destination,

+ transferee=transferee,

+ voip_headers=voip_headers,

+ opration_context="Your context",

+ operationCallbackUrl="<url_endpoint>"

+)

+#Transfer a PSTN call to phone number and set UUI and custom SIP headers

+transfer_destination = PhoneNumberIdentifer("<target_phoneNumber>")

+transferee = PhoneNumberIdentifer("transferee_phoneNumber")

+sip_headers={}

+sip_headers.add("X-MS-Custom-headerName", "headerValue")

+sip_headers.add("User-To-User","uuivale")

+result = call_connection_client.transfer_call_to_participant(

+ target_participant=transfer_destination,

+ transferee=transferee,

+ sip_headers=sip_headers,

+ opration_context="Your context",

+ operationCallbackUrl="<url_endpoint>"

+)

+```

+Transfer of a VoIP call to a phone number is currently not supported.

+--

+## Reading custom context from an incoming call event

+### [csharp](#tab/csharp)

+```csharp

+AcsIncomingCallEventData incomingEvent = <incoming call event from Event Grid>;

+// Retrieve incoming call custom context

+AcsIncomingCallCustomContext callCustomContext = incomingEvent.CustomContext;

+// Inspect dictionary with key/value pairs

+var voipHeaders = callCustomContext.VoipHeaders;

+var sipHeaders = callCustomContext.SipHeaders;

+// Proceed to answer or reject call as usual

+```

+### [Java](#tab/java)

+```java

+AcsIncomingCallEventData incomingEvent = <incoming call event from Event Grid>;

+// Retrieve incoming call custom context

+AcsIncomingCallCustomContext callCustomContext = incomingEvent.getCustomContext();

+// Inspect dictionary with key/value pairs

+Map<String, String> voipHeaders = callCustomContext.getVoipHeaders();

+Map<String, String> sipHeaders = callCustomContext.getSipHeaders();

+// Proceed to answer or reject call as usual

+```

+### [JavaScript](#tab/javascript)

+```javascript

+// Retrieve incoming call custom context

+const callCustomContext = incomingEvent.customContext;

+// Inspect dictionary with key/value pairs

+const voipHeaders = callCustomContext.voipHeaders;

+const sipHeaders = callCustomContext.sipHeaders;

+// Proceed to answer or reject call as usual

+```

+### [Python](#tab/python)

+```python

+# Retrieve incoming call custom context

+callCustomContext = incomingEvent.customContext

+# Inspect dictionary with key/value pairs

+voipHeaders = callCustomContext.voipHeaders

+sipHeaders = callCustomContext.sipHeaders

+```

+--

+## Additional resources

+- For a sample payload of the incoming call, refer to this [guide](../../../event-grid/communication-services-voice-video-events.md#microsoftcommunicationincomingcall).

+- Learn more about [SIP protocol details for direct routing](../../concepts/telephony/direct-routing-sip-specification.md).

+## Known limitations

+- In-band DTMF is not supported, use RFC 2833 DTMF instead.

+> [Use serverless containers](start-serverless-containers.md)

+ Title: Using serverless containers on Azure

+description: Get started using serverless containers on Azure with Azure Container Apps

+# Use serverless containers on Azure

+| [Build the app](quickstart-portal.md) | Deploy your first app, then create an event driven app to process a message queue. |

+ - [Troubleshoot apps faster with App Service](troubleshoot-app-service.md)

+## Sample prompts

+ Title: Troubleshoot your apps faster with App Service using Microsoft Copilot for Azure (preview)

+description: Learn how Microsoft Copilot for Azure (preview) can help you troubleshoot your web apps hosted with App Service.

+# Troubleshoot your apps faster with App Service using Microsoft Copilot for Azure (preview)

+Microsoft Copilot for Azure (preview) can act as your expert companion for [Azure App Service](/azure/app-service/overview) diagnostics and solutions.

+App Service offers more than sixty troubleshooting tools for different types of issues. Rather than figure out which tool to use, you can ask Microsoft Copilot for Azure (preview) about the problem you're experiencing. Microsoft Copilot for Azure (preview) will determine which tool is best suited to your question, whether it's related to high CPU usage, networking issues, getting a memory dump, or more. You'll see relevant diagnostics to help you resolve any problems you're experiencing.

+When you ask Microsoft Copilot for Azure (preview) for App Service troubleshooting help, it automatically pulls context when possible, based on the current conversation or the app you're viewing in the Azure portal. If the context isn't clear, you'll be prompted to specify the resource for which you want information.

+## Sample prompts

+Here are a few examples of the kinds of prompts you can use to get help with App Service troubleshooting. Modify these prompts based on your real-life scenarios, or try additional prompts to get help with different types of issues.

+- "My web app is down"

+- "My web app is slow"

+- "Enable auto heal"

+- "Take a memory dump"

+## Examples

+You can tell Microsoft Copilot for Azure (preview) "my web app is down." After you select the resource that you want to troubleshoot, Microsoft Copilot for Azure opens the **App Service - Web App Down** tool so you can view diagnostics.

+When you say "Take a memory dump" to Microsoft Copilot for Azure (preview), Microsoft Copilot for Azure (preview) suggests opening the **Collect a Memory Dump** tool so that you can take a snapshot of the app's current state. In this example, Microsoft Copilot for Azure (preview) continues to work with the resource selected earlier in the conversation.

+## Next steps

+- Explore [capabilities](capabilities.md) of Microsoft Copilot for Azure (preview).

+- Learn more about [Azure Monitor](/azure/azure-monitor/).

+- [Request access](https://aka.ms/MSCopilotforAzurePreview) to Microsoft Copilot for Azure (preview).

+> [!WARNING]

+> In the event of a write region outage, where the Azure Cosmos DB account promotes a secondary region to be the new primary write region via *service-managed failover*, the original write region will **not be be promoted back as the write region automatically** once it is recovered. It is your responsibility to switch back to the recovered region as the write region using [PowerShell, the Azure CLI, or the Azure portal](how-to-manage-database-account.md#manual-failover) (once safe to do so, as described above).

+ Title: Availability zone (AZ) outage resiliency ΓÇô Azure Cosmos DB for PostgreSQL

+description: Disaster recovery using Azure availability zones (AZ) concepts

+# Availability zone outage resiliency in Azure Cosmos DB for PostgreSQL

+Many Azure regions have availability zones. Availability zones (AZs) are separated groups of datacenters within a region. Availability zones are close enough to have low-latency connections to other availability zones within their region. They're connected by a high-performance network with a round-trip latency of less than 2 milliseconds.

+At the same time, availability zones are far enough apart to reduce the likelihood that more than one will be affected by local outages or weather. Availability zones have independent power, cooling, and networking infrastructure. They're designed so that if one zone experiences an outage, then regional services are supported by the remaining zones across various Azure services.

+Azure Cosmos DB for PostgreSQL supports availability zones for improved reliability and disaster recovery. Advantages of availability zones vary depending on whether [high availability](./concepts-high-availability.md) is enabled on an Azure Cosmos DB for PostgreSQL cluster.

+## Availability zone outage resiliency for regional service components

+There are many Azure Cosmos DB for PostgreSQL service components in each supported Azure region that don't belong to individual clusters but are rather critical parts of running the managed service. These components allow ongoing execution of all management operations such as new cluster provisioning and scaling existing clusters and all internal operations such as monitoring node health.

+When Azure region supports availability zones, all of these service components are configured to be AZ redundant. It means that all Azure Cosmos DB for PostgreSQL service components can sustain outage of an AZ, or in other words are resilient to a single AZ outage.

+Whether a cluster is configured with high availability or not, its ongoing operations depend on these service components. AZ redundancy of the service components is a critical element of availability zone outage resiliency in Azure Cosmos DB for PostgreSQL.

+## Availability zone outage impact on clusters with and without high availability

+All nodes in a cluster are provisioned into one availability zone. Preferred AZ setting allows you to put all cluster nodes in the same availability zone where the application is deployed. Having all nodes in the same AZ ensures lower latency between the nodes thus improving overall cluster performance.

+When high availability (HA) is enabled on a cluster, all primary nodes are created in one AZ and all standby nodes are provisioned into another AZ. Nodes can move between availability zones during the following events:

+- A failure occurs on a primary HA-enabled node. In this case primary node's standby is going to become a new primary and standby node's AZ is going to be the new AZ for that primary node.

+- A [scheduled maintenance](./concepts-maintenance.md) event happens on cluster. At the end of maintenance all primary nodes in a cluster are going to be in the same AZ.

+If high availability *is* enabled, cluster continues to be available throughout AZ outage with a possible failover on those primary nodes that are in the impacted AZ.

+If high availability *is not* enabled on a cluster, only outage in the AZ where nodes are deployed would impact cluster availability.

+You can always check availability zone for each primary node using the [Azure portal](./concepts-cluster.md#node-availability-zone) or using programmatic methods such as [REST APIs](/rest/api/postgresqlhsc/servers/get).

+To get resiliency benefits of availability zones, your cluster needs to be in [one of the Azure regions](./resources-regions.md) where Azure Cosmos DB for PostgreSQL is configured for AZ outage resiliency.

+## Next steps

+- Check out [regions that are configured for AZ outage resiliency](./resources-regions.md) in Azure Cosmos DB for PostgreSQL

+- Learn about [availability zones in Azure](../../reliability/availability-zones-overview.md)

+- Learn how to [enable high availability](howto-high-availability.md) in a cluster

+Azure Cosmos DB for PostgreSQL displays the [availability zone](./concepts-availability-zones.md) of each node

+assigned to a zone. (Only [certain regions](./resources-regions.md)

+If [high availability](./concepts-high-availability.md) is enabled for the cluster, and a node fails

+over to a standby, you may see its availability

+into the same availability zone together during the next [maintenance event](./concepts-maintenance.md).

+All primary nodes in a cluster are provisioned into one [availability zone](./concepts-availability-zones.md)

+zone of each primary node in a cluster. You can also check availability zone of each node in a cluster using one of the programmatic methods such as [REST APIs](/rest/api/postgresqlhsc/servers/get).

+- Learn how to [enable high availability](howto-high-availability.md) in a cluster.

+- Learn about [availability zones](./concepts-availability-zones.md) in Azure Cosmos DB for PostgreSQL.

+ Title: Configure and view availability zones in Azure Cosmos DB for PostgreSQL

+description: How to set preferred availability zone and check AZs for nodes

+# Use availability zones in Azure Cosmos DB for PostgreSQL

+Azure Cosmos DB for PostgreSQL provisions all nodes of a cluster in a single [availability zone](./concepts-availability-zones.md) (AZ) for better performance and availability. If cluster has [high availability](./concepts-high-availability.md) enabled, all standby nodes are provisioned into another availability zone to make sure all nodes in cluster continue to be available, with a possible failover, if there is an AZ outage.

+To get resiliency benefits of availability zones, your cluster needs to be in [one of the Azure regions](./resources-regions.md) where Azure Cosmos DB for PostgreSQL is configured for AZ outage resiliency.

+In this article, you learn how to specify preferred availability zone for your Azure Cosmos DB for PostgreSQL cluster. You will also learn how to check availability zone for each node once cluster is provisioned.

+## Specify preferred availability zone for new cluster

+By default preferred availability zone isn't set for a new cluster. In that case Azure Cosmos DB for PostgreSQL service would randomly select an availability zone for primary nodes.

+Selecting preferred AZ is possible during cluster creation on the **Scale** page in the **Availability zones** section.

+## Change preferred availability zone

+Once cluster is provisioned, select AZ in the **Preferred availability zone** drop-down list on the **Scale** page for your cluster in the Azure portal. Click the **Save** button to apply your selection.

+To avoid disruption, change of the availability zone isn't applied immediately. Rather all nodes are going to be moved to preferred availability zone during the next [maintenance](./concepts-maintenance.md) event.

+## Check availability zone for each node

+The **Overview** tab for the cluster lists all nodes along with the **Availability zone** column that shows actual availability zone for each primary cluster node.

+## Next steps

+- Learn more about [availability zones](./concepts-availability-zones.md) in Azure Cosmos DB for PostgreSQL.

+- Learn more about [availability zones in Azure](/azure/reliability/availability-zones-overview).

+- Use [REST APIs](/rest/api/postgresqlhsc/clusters/update), [Terraform](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/cosmosdb_postgresql_cluster), or [Azure CLI](/cli/azure/cosmosdb/postgres/cluster#az-cosmosdb-postgres-cluster-update) to perform operations with availability zones in Azure Cosmos DB for PostgreSQL.

+## Next steps

+- Learn more about [high availability](concepts-high-availability.md).

+- Learn more about [availability zones](./concepts-availability-zones.md) in Azure Cosmos DB for PostgreSQL.

+> [!IMPORTANT]

+> Data Factory doesn't support private endpoints for Azure Cosmos DB for PostgreSQL at this time.

+* General availability: [Availability zone (AZ) outage resiliency](./concepts-availability-zones.md) is now supported in [select regions](./resources-regions.md)

+| Region | HA | AZ outage resiliency | Geo-redundant backup stored in |

+| | | | |

+| Australia Central | :heavy_check_mark: | N/A | :x: |

+| Australia East | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Brazil South | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Canada Central | :heavy_check_mark: | :heavy_check_mark: | Canada East |

+| Canada East | :heavy_check_mark: | N/A | Canada Central |

+| Central India | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Central US | :heavy_check_mark: | :heavy_check_mark: | East US 2 |

+| East Asia | :heavy_check_mark: | :heavy_check_mark: | Southeast Asia |

+| East US | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| East US 2 | :heavy_check_mark: | :x: | Central US |

+| France Central | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Germany West Central | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Japan East | :heavy_check_mark: | :heavy_check_mark: | Japan West |

+| Japan West | :heavy_check_mark: | :x: | Japan East |

+| Korea Central | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| North Central US | :heavy_check_mark: | N/A | South Central US |

+| North Europe | :heavy_check_mark: | :heavy_check_mark: | West Europe |

+| Qatar Central | :heavy_check_mark: | :x: | :x: |

+| South Central US | :heavy_check_mark: | :heavy_check_mark: | North Central US |

+| South India | :heavy_check_mark: | N/A | :x: |

+| Southeast Asia | :heavy_check_mark: | :x:| East Asia |

+| Sweden Central | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| Switzerland North | :heavy_check_mark: | :heavy_check_mark: | Switzerland West |

+| Switzerland West ΓÇá | :heavy_check_mark: | N/A | Switzerland North |

+| UK South | :heavy_check_mark: | :heavy_check_mark: | :x: |

+| West Central US | :heavy_check_mark: | N/A | West US 2 |

+| West Europe | :heavy_check_mark: | :x: | North Europe |

+| West US | :heavy_check_mark: | :x: | East US |

+| West US 2 | :heavy_check_mark: | :heavy_check_mark: | West Central US |

+| West US 3 | :heavy_check_mark: | :heavy_check_mark: | :x: |

+- Learn how to [create a cluster in the portal](./quickstart-create-portal.md).

+- See details about [availability zone outage resiliency](./concepts-availability-zones.md) in Azure Cosmos DB for PostgreSQL.

+- Check out [backup redundancy options](./concepts-backup.md#backup-redundancy).

+Exports at the management group scope support only usage charges. Purchases, including reservations and savings plans aren't supported. Amortized cost reports are also not supported. When you create an export from the Azure portal for a management group scope, the metric field isn't shown because it defaults to the usage type. When you create a management group scope export using the REST API, choose [ExportType](/rest/api/cost-management/exports/create-or-update#exporttype) as `Usage`.

+| servicePrincipalCredentialType | The credential type to use for service-principal authentication. Valid values are "ServicePrincipalKey" and "ServicePrincipalCert". <br/><br/>Note: It's recommended to use ServicePrincipalKey. There's known limitation for ServicePrincipalCert credential type where the service may encounter transient issue of failing to retrieve secret from the key vault.| Yes when authentication is "AADServicePrincipal" |

+description: Learn how to create and access a dev center for Azure Deployment Environments project using the Azure CLI.

+This quickstart guide shows you how to create and configure a dev center in Azure Deployment Environments.

+A platform engineering team typically sets up a dev center, attaches external catalogs to the dev center, creates projects, and provides access to development teams. Development teams can then create [environments](concept-environments-key-concepts.md#environments) by using [environment definitions](concept-environments-key-concepts.md#environment-definitions), connect to individual resources, and deploy applications.

+- Install the [Azure CLI devcenter extension](how-to-install-devcenter-cli-extension.md).

+- A GitHub account and a [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) with repo access.

+To create and configure a dev center in Azure Deployment Environments:

+1. Install the Azure CLI *devcenter* extension.

+ az account set --subscription <subscriptionName>

+1. Configure the default location where you want to create the dev center. Make sure to choose an [available region for Azure Deployment Environments](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?products=deployment-environments&regions=all):

+ az group create -n <resourceGroupName>

+ az config set defaults.group=<resourceGroupName>

+ az devcenter admin devcenter create -n <devcenterName>

+ After a few minutes, the output indicates that it was created:

+ "id": "/subscriptions/.../<devcenterName>",

+ "resourceGroup": "<resourceGroupName>",

+## Add a personal access token to Azure Key Vault

+You need an Azure Key Vault to store the GitHub personal access token (PAT) that's used to grant Azure access to your GitHub repository.

+1. Create a key vault:

+ az keyvault create -n <keyvaultName>

+ > You might get the following error:

+1. Add the GitHub PAT to Key Vault as a secret:

+ az keyvault secret set --vault-name <keyvaultName> --name GHPAT --value <personalAccessToken>

+### Attach a system-assigned managed identity

+ az devcenter admin devcenter update -n <devcenterName> --identity-type SystemAssigned

+### Give the system-assigned managed identity access to the key vault secret

+Make sure that the identity has access to the key vault secret that contains the GitHub PAT to access your repository. Key Vaults support two methods of access; Azure role-based access control or vault access policy. In this quickstart, you use a vault access policy.

+1. Retrieve the Object ID of your dev center's identity:

+ OID=$(az ad sp list --display-name <devcenterName> --query [].id -o tsv)

+1. Add a Key Vault policy to allow the dev center to get secrets from Key Vault:

+ az keyvault set-policy -n <keyvaultName> --secret-permissions get --object-id $OID

+Azure Deployment Environments supports attaching Azure DevOps repositories and GitHub repositories. You can store a set of curated IaC templates in a repository. Attaching the repository to a dev center as a catalog gives your development teams access to the templates and allows them to quickly create consistent environments.

+You can use this [sample catalog](https://github.com/Azure/deployment-environments) as your repository. Make a fork of the repository for the following steps.

+> If you're attaching an Azure DevOps repository, use these steps: [Get the clone URL of an Azure DevOps repository](how-to-configure-catalog.md#get-the-clone-url-for-your-azure-devops-repository).

+1. Navigate to your repository, select **<> Code**, and then copy the clone URL.

+1. Make a note of the branch that you're working in.

+ :::image type="content" source="media/how-to-create-configure-dev-center/github-info.png" alt-text="Screenshot that shows the GitHub repo with branch, copy URL, and folder highlighted." lightbox="media/how-to-create-configure-dev-center/github-info.png":::

+ SECRETID=$(az keyvault secret show --vault-name <keyvaultName> --name GHPAT --query id -o tsv)

+1. Add the catalog.

+ # Sample catalog example

+ az devcenter admin catalog create --git-hub path="/Environments" branch="main" secret-identifier=$SECRETID uri=$REPO_URL -n <catalogName> -d <devcenterName>

+1. Confirm that the catalog was successfully added and synced:

+ az devcenter admin catalog list -d <devcenterName> -o table

+1. Create an environment type:

+ az devcenter admin environment-type create -d <devcenterName> -n <environmentTypeName>

+1. Confirm that the environment type was created:

+ az devcenter admin environment-type list -d <devcenterName> -o table

+> [Create and configure a project by using the Azure CLI](how-to-create-configure-projects.md)

+description: Learn how to create a project in Azure Deployment Environments and associate the project with a dev center using the Azure CLI.

+This quickstart guide shows you how to create a project in Azure Deployment Environments. Then, you associate the project with the dev center you created in [Create and configure a dev center by using the Azure CLI](how-to-create-configure-dev-center.md).

+1. Install the Azure CLI *devcenter* extension.

+ az account set --subscription <subscriptionName>

+ az configure --defaults group=<resourceGroupName>

+ DEVCID=$(az devcenter admin devcenter show -n <devcenterName> --query id -o tsv)

+ az devcenter admin project create -n <projectName> \

+ az devcenter admin project show -n <projectName>

+### Assign the Owner role to a managed identity

+ SUBID=$(az account show --name <subscriptionName> --query id -o tsv)

+1. Retrieve the Object ID of the dev center's identity using the name of the dev center resource:

+ OID=$(az ad sp list --display-name <devcenterName> --query [].id -o tsv)

+ echo $OID

+1. Assign the role of Owner to the dev center on the subscription:

+1. Retrieve the Role ID for the Owner of the subscription:

+ az configure --defaults group=<resourceGroupName>

+1. Show allowed environment type for the project:

+ az devcenter admin project-allowed-environment-type list --project <projectName> --query [].name

+ az devcenter admin project-environment-type create -n <availableEnvironmentType> \

+ --project <projectName> \

+> At least one identity (system-assigned or user-assigned) must be enabled for deployment identity. The identity is used to perform the environment deployment on behalf of the developer. Additionally, the identity attached to the dev center should be [assigned the Owner role](how-to-configure-managed-identity.md) for access to the deployment subscription for each environment type.

+1. Optionally, you can assign the Dev Environment User role:

+In this quickstart, you created a project and granted project access to your development team. To learn how your development team members can create environments, advance to the next quickstart.

+> [Create and access an environment by using the Azure CLI](how-to-create-access-environments.md)

+5. For service-to-service (S2S) communication, ADME uses MSI (Microsoft Service Identity).

+ 1. The first Service Principal is used for API access. It can also manage infrastructure resources.

+ 2. The second Service Principal is used for service-to-service (S2S) communications.

+The entitlements service of Azure Data Manager for Energy allows you to create groups and manage memberships of the groups. An entitlement group defines permissions on services/data sources for a given data partition in your Azure Data Manager for Energy instance. Users added to a given group obtain the associated permissions.

+Please note that different groups and associated user entitlements need to be set for every **new data partition** even in the same Azure Data Manager for Energy instance.

+1. You have two ways to get the list of data partitions in your Azure Data Manager for Energy instance.

+2. The value to be sent for the param `email` is the `Object_ID` (OID) of the user and not the user's email.

+ "customContext": {

+ "voipHeaders": {

+ "voipHeaderName": "value"

+ }

+ },

+# Receive Microsoft Graph API change events through Azure Event Grid (preview)

+This article describes steps to subscribe to events published by Microsoft Graph API. The following table lists the event sources for which events are available through Graph API. For most resources, events announcing its creation, update, and deletion are supported. For detailed information about the resources for which events are raised for event sources, see [supported resources by Microsoft Graph API change notifications](/graph/webhooks#supported-resources)

+.

+> Microsoft Graph API's ability to send events to Azure Event Grid is currently in **public preview**. If you have questions or need support, email us at [ask-graph-and-grid@microsoft.com](mailto:ask-graph-and-grid@microsoft.com?subject=Support%20Request).

+|Microsoft event source |Available event types |

+|: | :-|

+|Microsoft Entra ID| [Microsoft Entra event types](azure-active-directory-events.md) |

+|Microsoft Outlook| [Microsoft Outlook event types](outlook-events.md) |

+|Microsoft 365 group conversations ||

+|Microsoft Teams| [Microsoft Teams event types](teams-events.md) |

+|Microsoft SharePoint and OneDrive| |

+|Microsoft SharePoint| |

+|Security alerts| |

+|Microsoft Conversations| |

+|Microsoft Universal Print||

+## Why should I subscribe to events from Microsoft Graph API sources via Event Grid?

+- You require to route events to different downstream applications, webhooks, or Azure services depending on some of the properties in the event. For example, you might want to route event types such as `Microsoft.Graph.UserCreated` and `Microsoft.Graph.UserDeleted` to a specialized application that processes users' onboarding and off-boarding. You might also want to send `Microsoft.Graph.UserUpdated` events to another application that syncs contacts information, for example. You can achieve that using a single Graph API subscription when using Event Grid as a notification destination. For more information, see [event filtering](event-filtering.md) and [event handlers](event-handlers.md).

+- Interoperability is important to you. You want to forward and handle events in a standard way using CNCF's [CloudEvents](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/spec.md) specification standard.

+- You like the extensibility support that CloudEvents provides. For example, if you want to trace events across compliant systems, use CloudEvents extension [Distributed Tracing](https://github.com/cloudevents/spec/blob/v1.0.1/extensions/distributed-tracing.md). Learn more about more [CloudEvents extensions](https://github.com/cloudevents/spec/blob/v1.0.1/documented-extensions.md).

+- You want to use proven event-driven approaches adopted by the industry.

+## Enable Graph API events to flow to your partner topic

+You request Microsoft Graph API to forward events to an Event Grid partner topic by creating a Graph API subscription using the Microsoft Graph API SDKs and **following the steps in the links to samples provided** in this section. See [Supported languages for Microsoft Graph API SDK](/graph/sdks/sdks-overview#supported-languages.md) for available SDK support.

+### General prerequisites

+You should meet these general prerequisites before implementing your application to create and renew Microsoft Graph API subscriptions:

+- Become familiar with the [high-level steps to subscribe to partner events](subscribe-to-partner-events.md#high-level-steps). As described in that article, prior to creating a Graph API subscription you should follow the instructions in:

+ - [Register the Event Grid resource provider](subscribe-to-partner-events.md#register-the-event-grid-resource-provider) with your Azure subscription.

+ - [Authorize partner](subscribe-to-partner-events.md#authorize-partner-to-create-a-partner-topic) to create a partner topic in your resource group.

+- Have a working knowledge of [Microsoft Graph API notifications](/graph/api/resources/webhooks). As part of your learning, you could use the [Graph API Explorer](https://developer.microsoft.com/en-us/graph/graph-explorer) to create Graph API subscriptions.

+- Understand [Partner Events concepts](partner-events-overview.md).

+- Identify the Microsoft Graph API resource from which you want to receive system state change events. See [Microsoft Graph API change notifications](/graph/webhooks#supported-resources) for more information. For example, for tracking changes to users in Microsoft Entra ID you should use the [user](/graph/api/resources/user) resource. Use [group](/graph/api/resources/group) for tracking changes to user groups.

+- Have a tenant administrator account on a Microsoft 365 tenant. You can get a development tenant for free by joining the [Microsoft 365 Developer Program](https://developer.microsoft.com/microsoft-365/dev-program).

+You'll find other prerequisites specific to the programming language of choice and the development environment you use in the Microsoft Graph API samples links found in a coming section.

+> [!IMPORTANT]

+> While detailed instructions to implement your application are found in the [samples with detailed instructions](#samples-with-detailed-instructions) section, you should read all sections in this article as they contain additional, important information related to forwarding Microsoft Graph API events using Event Grid.

+### How to create a Microsoft Graph API subscription

+When you create a Graph API subscription, a partner topic is created for you. You pass the following information in parameter *notificationUrl* to specify what partner topic to create and be associated to the new Graph API subscription:

+- partner topic name

+- resource group name in which the partner topic is created

+- region (location)

+- Azure subscription

+These code samples show you how to create a Graph API subscription. They show examples for creating a subscription to receive events from all users in a Microsoft Entra ID tenant when they're created, updated, or deleted.

+# [HTTP](#tab/http)

+<!-- {

+ "blockType": "request",

+ "name": "create_subscription_from_subscriptions"

+}-->

+```http

+POST https://graph.microsoft.com/v1.0/subscriptions

+Content-type: application/json

+ "notificationUrl": "EventGrid:?azuresubscriptionid=8A8A8A8A-4B4B-4C4C-4D4D-12E12E12E12E&resourcegroup=yourResourceGroup&partnertopic=yourPartnerTopic&location=theNameOfAzureRegionFortheTopic",

+ "lifecycleNotificationUrl": "EventGrid:?azuresubscriptionid=8A8A8A8A-4B4B-4C4C-4D4D-12E12E12E12E&resourcegroup=yourResourceGroup&partnertopic=yourPartnerTopic&location=theNameOfAzureRegionFortheTopic",

+ "expirationDateTime": "2024-03-31T00:00:00Z",

+ "clientState": "secretClientValue"

+# [C#](#tab/csharp)

+# [CLI](#tab/cli)

+# [Go](#tab/go)

+# [Java](#tab/java)

+# [JavaScript](#tab/javascript)

+# [PHP](#tab/php)

+# [PowerShell](#tab/powershell)

+# [Python](#tab/python)

+- `notificationUrl`: a URI used to define the partner topic to which events are sent. It must conform to the following pattern: `EventGrid:?azuresubscriptionid=<you-azure-subscription-id>&resourcegroup=<your-resource-group-name>&partnertopic=<the-name-for-your-partner-topic>&location=<the-Azure-region-name-where-you-want-the-topic-created>`. The location (also known as Azure region) `name` can be obtained by executing the **az account list-locations** command. Don't use a location display name. For example, don't use "West Central US". Use `westcentralus` instead.

+- `lifecycleNotificationUrl`: a URI used to define the partner topic to which `microsoft.graph.subscriptionReauthorizationRequired`events are sent. This event signals your application that the Graph API subscription is expiring soon. The URI follows the same pattern as *notificationUrl* described above if using Event Grid as destination to lifecycle events. In that case, the partner topic should be the same as the one specified in *notificationUrl*.

+- resource: the resource that generates events that announce state changes.

+- expirationDateTime: the expiration time at which the subscription expires and the flow of events stop. It must conform to the format specified in [RFC 3339](https://tools.ietf.org/html/rfc3339). You must specify an expiration time that is within the [maximum subscription length allowable per resource type](/graph/api/resources/subscription#subscription-lifetime).

+- client state. This property is optional. It is used for verification of calls to your event handler application during event delivery. For more information, see [Graph API subscription properties](/graph/api/resources/subscription#properties).

+> [!IMPORTANT]

+>

+> - The partner topic name must be unique within the same Azure region. Each tenant-application ID combination can create up to 10 unique partner topics.

+>

+> - Be mindful of certain [Graph API resources' service limits](/graph/webhooks#azure-ad-resource-limitations) when developing your solution.

+> - Existing Graph API subscriptions without a `lifecycleNotificationUrl` property don't receive lifecycle events. To add the lifecycleNotificationUrl property, you should delete the existing subscription and create a new subscription specifying the property during subscription creation.

+> [!NOTE]

+> If your application uses the header `x-ms-enable-features` with your request to create a Graph API subscription during **private preview**, you should remove it as it is no longer necessary.

+After creating a Graph API subscription, you have a partner topic created on Azure.

+### Renew a Microsoft Graph API subscription

+A Graph API subscription must be renewed by your application before it expires to avoid stopping the flow of events. To help you automate the renewal process, Microsoft Graph API supports **lifecycle notifications events** to which your application can subscribe. Currently, all type of Microsoft Graph API resources support the `microsoft.graph.subscriptionReauthorizationRequired`, which is sent when any of the following conditions occur:

+- Access token is about to expire.

+- Graph API subscription is about to expire.

+- A tenant administrator has revoked your app's permissions to read a resource.

+If you didn't renew your Graph API subscription after it has been expired, you need to create a new Graph API subscription. You could refer to the same partner topic you used in your expired subscription as long as it has been expired for less than 30 days. If the Graph API subscription has expired for more than 30 days, you can't reuse your existing partner topic. In this case, you'll need to either specify another partner topic name. Alternatively, you can delete the existing partner topic to create a new partner topic with the same name during the Graph API subscription creation.

+#### How to renew a Microsoft Graph API subscription

+Upon receiving a `microsoft.graph.subscriptionReauthorizationRequired` event your application should renew the Graph API subscription by doing these actions:

+1. If you provided a client secret in the *clientState* property when you created the Graph API subscription, that client secret in included with the event. Validate that the event's clientState matches the value used when you created the Graph API subscription.

+1. Ensure that the app has a valid access token to take the next step. More information is provided in the coming [samples with detailed instructions](#samples-with-detailed-instructions) section.

+1. Call either of the following two APIs. If the API call succeeds, the change notification flow resumes.

+ - Call the `/reauthorize` action to reauthorize the subscription without extending its expiration date.

+

+ <!-- {

+ "blockType": "request",

+ "name": "change-notifications-lifecycle-notifications-reauthorize"

+ }-->

+ ```http

+ POST https://graph.microsoft.com/beta/subscriptions/{id}/reauthorize

+ ```

+ - Perform a regular "renew" action to reauthorize *and* renew the subscription at the same time.

+ <!-- {

+ "blockType": "request",

+ "name": "change-notifications-lifecycle-notifications-renew"

+ }-->

+ ```http

+ PATCH https://graph.microsoft.com/beta/subscriptions/{id}

+ Content-Type: application/json

+ {

+ "expirationDateTime": "2024-04-30T11:00:00.0000000Z"

+ }

+ ```

+ Renewing might fail if the app is no longer authorized to access to the resource. It might then be necessary for the app to obtain a new access token to successfully reauthorize a subscription.

+Authorization challenges don't replace the need to renew a subscription before it expires. The lifecycles of access tokens and subscription expiration are not the same. Your access token may expire before your subscription. It is important to be prepared to regularly reauthorize your endpoint to refresh your access token. Reauthorizing your endpoint will not renew your subscription. However, renewing your subscription will also reauthorize your endpoint.

+When renewing and/or reauthorizing your Graph API subscription the same partner topic specified when the subscription was created is used.

+When Specifying a new *expirationDateTime*, it must be at least three hours from the current time. Otherwise, your application may receive `microsoft.graph.subscriptionReauthorizationRequired` events soon after renewal.

+For examples about how to reauthorize your Graph API subscription using any of the supported languages, see [subscription reauthorize request](/graph/api/subscription-reauthorize#request).

+For examples about how to renew and reauthorize your Graph API subscription using any of the supported languages, see [update subscription request.](/graph/api/subscription-update#request).

+### Samples with detailed instructions

+Microsoft Graph API documentation provides code samples with instructions to:

+- Set up your development environment with specific instructions according to the language you use. Instructions also include how to get a Microsoft 365 tenant for development purposes.

+- Create a Graph API subscriptions. To renew a subscription, you can call the Graph API using the code snippets in [How to renew a Graph API subscription](#how-to-renew-a-microsoft-graph-api-subscription) above.

+- Get authentication tokens to use them when calling Microsoft Graph API.

+>[!NOTE]

+> It is possible to create your Graph API subscription using the [Microsoft Graph API Explorer](https://developer.microsoft.com/en-us/graph/graph-explorer). You should still use the samples for other important aspects of your solution such as authentication and receiving events.

+Web application samples are available for the following languages:

+- [C# sample](https://github.com/microsoftgraph/aspnetcore-webhooks-sample)

+- [Java sample](https://github.com/microsoftgraph/java-spring-webhooks-sample)

+ - [GraphAPIController](https://github.com/jfggdl/event-grid-ms-graph-api-java-snippet) contains sample code to create, delete, and renew a Graph API subscription. It must be used along with the Java sample application above.

+- [NodeJS sample](https://github.com/microsoftgraph/nodejs-webhooks-sample).

+> [!IMPORTANT]

+> You need to activate your partner topic that is created as part of your Graph API subscription creation. You also need to create an Event Grid event subscription to your web application to receive events. To that end, you use the URL configured in your web application to receive events as a webhook endpoint in your event subscription. [Next steps](#next-steps) for more information.

+> [!IMPORTANT]

+> Do you need sample code for another language or have questions? Please email us at [ask-graph-and-grid@microsoft.com](mailto:ask-graph-and-grid@microsoft.com?subject=Need%20support%20for%20sample%20in%20other%20language).

+Follow the instructions in the following two steps to complete set-up to receive Microsoft Graph API events using Event Grid:

+- [Activate the partner topic](subscribe-to-partner-events.md#activate-a-partner-topic) created as part of the Microsoft Graph API creation.

+- [Subscribe to events](subscribe-to-partner-events.md#subscribe-to-events) by creating an event subscription to your partner topic.

+Other useful links:

+

+- [Information on Microsoft Graph API](https://developer.microsoft.com/graph/rest-api).

+- [Microsoft Graph API tutorials](/graph/tutorials), which shows how to use Graph API. This article doesn't necessarily include examples for sending events to Event Grid.

+* DNS Private Resolver: Azure ExpressRoute FastPath does not support connectivity to [DNS Private Resolver](../dns/dns-private-resolver-overview.md).

+ Title: Azure Data Factory Managed Airflow with Apache Spark® on HDInsight on AKS

+description: Learn how to perform Apache Spark® job orchestration using Azure Data Factory Managed Airflow

+# Apache Spark® job orchestration using Azure Data Factory Managed Airflow

+This article covers managing a Spark job using [Apache Spark Livy API](https://livy.incubator.apache.org/docs/latest/rest-api.html) and orchestration data pipeline with Azure Data Factory Managed Airflow. [Azure Data Factory Managed Airflow](/azure/data-factory/concept-managed-airflow) service is a simple and efficient way to create and manage [Apache Airflow](https://airflow.apache.org/) environments, enabling you to run data pipelines at scale easily.

+Apache Airflow is an open-source platform that programmatically creates, schedules, and monitors complex data workflows. It allows you to define a set of tasks, called operators that can be combined into directed acyclic graphs (DAGs) to represent data pipelines.

+The following diagram shows the placement of Airflow, Key Vault, and HDInsight on AKS in Azure.

+Multiple Azure Service Principals are created based on the scope to limit the access it needs and to manage the client credential life cycle independently.

+It is recommended to rotate access keys or secrets periodically (you can use various [design patternΓÇÖs](/azure/key-vault/secrets/tutorial-rotation-dual?tabs=azure-cli) to rotate secrets).

+## Setup steps

+1. [Setup Spark Cluster](create-spark-cluster.md)

+1. Upload your Apache Spark Application jar to the storage account. It can be the primary storage account associated with the Spark cluster or any other storage account, where you should assign the "Storage Blob Data Owner" role to the user-assigned MSI used for the cluster in this storage account.

+1. Azure Key Vault - You can follow [this tutorial to create a new Azure Key Vault](/azure/key-vault/general/quick-create-portal/) in case, if you don't have one.

+1. Create [Microsoft Entra service principal](/cli/azure/ad/sp/) to access Key Vault – Grant permission to access Azure Key Vault with the “Key Vault Secrets Officer” role, and make a note of ‘appId’ ‘password’, and ‘tenant’ from the response. We need to use the same for Airflow to use Key Vault storage as backends for storing sensitive information.

+ ```

+ az ad sp create-for-rbac -n <sp name> --role ΓÇ£Key Vault Secrets OfficerΓÇ¥ --scopes <key vault Resource ID>

+ ```

+1. Create Managed Airflow enable with [Azure Key Vault](/azure/data-factory/enable-azure-key-vault-for-managed-airflow) to store and manage your sensitive information in a secure and centralized manner. By doing this, you can use variables and connections, and they automatically be stored in Azure Key Vault. The name of connections and variables need to be prefixed by variables_prefix  defined in AIRFLOW__SECRETS__BACKEND_KWARGS. For example, If variables_prefix has a value as  hdinsight-aks-variables then for a variable key of hello, you would want to store your Variable at hdinsight-aks-variable -hello.

+ - Add the following settings for the Airflow configuration overrides in integrated runtime properties:

+ - AIRFLOW__SECRETS__BACKEND:

+ `"airflow.providers.microsoft.azure.secrets.key_vault.AzureKeyVaultBackend"`

+ - AIRFLOW__SECRETS__BACKEND_KWARGS:

+ `"{"connections_prefix": "airflow-connections", "variables_prefix": "hdinsight-aks-variables", "vault_url":ΓÇ»<your keyvault uri>}ΓÇ¥`

+ - Add the following setting for the Environment variables configuration in the Airflow integrated runtime properties:

+ - AZURE_CLIENT_IDΓÇ»= `<App Id from Create Azure AD Service Principal>`

+ - AZURE_TENANT_IDΓÇ»= `<Tenant from Create Azure AD Service Principal> `

+ - AZURE_CLIENT_SECRETΓÇ»= `<Password from Create Azure AD Service Principal> `

+ Add Airflow requirements - [apache-airflow-providers-microsoft-azure](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/https://docsupdatetracker.net/index.html)

+ :::image type="content" source="./media/spark-job-orchestration/airflow-configuration-environment-variable.png" alt-text="Screenshot shows airflow configuration and environment variables." lightbox="./media/spark-job-orchestration/airflow-configuration-environment-variable.png":::

+

+1. Create [Microsoft Entra service principal](/cli/azure/ad/sp/) to access HDInsight on AKS cluster Azure – [Grant access to HDInsight AKS Cluster](/azure/hdinsight-aks/hdinsight-on-aks-manage-authorization-profile#how-to-grant-access), make a note of appId, password, and tenant from the response.

+ `az ad sp create-for-rbac -n <sp name>`

+1. Create the following secrets in your key vault with the value from the previous AD Service principal appId, password, and tenant, prefixed by property variables_prefix defined in AIRFLOW__SECRETS__BACKEND_KWARGS. The DAG code can access any of these variables without variables_prefix.

+ - hdinsight-aks-variables-api-client-id=`<App ID from previous step> `

+ - hdinsight-aks-variables-api-secret=`<Password from previous step> `

+ - hdinsight-aks-variables-tenant-id=`<Tenant from previous step> `

+ ```python

+ from airflow.models import Variable

+ def retrieve_variable_from_akv():

+ variable_value = Variable.get("client-id")

+ print(variable_value)

+ ```

+

+## DAG definition

+A DAG (Directed Acyclic Graph) is the core concept of Airflow, collecting Tasks together, organized with dependencies and relationships to say how they should run.

+There are three ways to declare a DAG:

+ - You can use a context manager, which adds the DAG to anything inside it implicitly

+ - You can use a standard constructor, passing the DAG into any operators you use

+ - You can use the @dag decorator to turn a function into a DAG generator (from airflow.decorators import dag)

+DAGs are nothing without Tasks to run, and those are come in the form of either Operators, Sensors or TaskFlow.

+You can read more details about DAGs, Control Flow, SubDAGs, TaskGroups, etc. directly fromΓÇ»[Apache Airflow](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html).ΓÇ»

+## DAG execution

+Example code is available on the [git](https://github.com/sethiaarun/hdinsight-aks/blob/spark-airflow-example/spark/Airflow/airflow-python-example-code.py); download the code locally on your computer and upload the wordcount.py to a blob storage. Follow the [steps](/azure/data-factory/how-does-managed-airflow-work#steps-to-import) to import DAG into your Managed Airflow created during setup.

+The airflow-python-example-code.py is an example of orchestrating a Spark job submission using Apache Spark with HDInsight on AKS. The example is based on [SparkPi](https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/SparkPi.scala) example provided on Apache Spark.

+The DAG has the following steps:

+1. get `OAuth Token`

+1. Invoke Apache Spark Livy Batch API to submit a new job

+The DAG expects to have setup for the Service Principal, as described during the setup process for the OAuth Client credential and pass the following input configuration for the execution.

+### Execution steps

+1. Execute the DAG from the [Airflow UI](https://airflow.apache.org/docs/apache-airflow/stable/ui.html), you can open the Azure Data Factory Managed Airflow UI by clicking on Monitor icon.

+ :::image type="content" source="./media/spark-job-orchestration/airflow-user-interface-step-1.png" alt-text="Screenshot shows open the Azure data factory managed airflow UI by clicking on monitor icon." lightbox="./media/spark-job-orchestration/airflow-user-interface-step-1.png":::

+1. Select the ΓÇ£SparkWordCountExampleΓÇ¥ DAG from the ΓÇ£DAGsΓÇ¥ page.

+ :::image type="content" source="./media/spark-job-orchestration/airflow-user-interface-step-2.png" alt-text="Screenshot shows select the Spark word count example." lightbox="./media/spark-job-orchestration/airflow-user-interface-step-2.png":::

+1. Click on the “execute” icon from the top right corner and select “Trigger DAG w/ config”.

+ :::image type="content" source="./media/spark-job-orchestration/airflow-user-interface-step-3.png" alt-text="Screenshot shows select execute icon." lightbox="./media/spark-job-orchestration/airflow-user-interface-step-3.png":::

+

+1. Pass required configuration JSON

+ ```JSON

+ {

+ "spark_cluster_fqdn":"<<domain name>>.eastus2.hdinsightaks.net",

+ "app_jar_path":"abfs://filesystem@<storageaccount>.dfs.core.windows.net",

+ "job_name":"<job_name>",

+ }

+ ```

+1. Click on ΓÇ£TriggerΓÇ¥ button, it starts the execution of the DAG.

+1. You can visualize the status of DAG tasks from the DAG run

+ :::image type="content" source="./media/spark-job-orchestration/dag-task-status.png" alt-text="Screenshot shows dag task status." lightbox="./media/spark-job-orchestration/dag-task-status.png":::

+1. Validate the job from ΓÇ£Apache Spark History ServerΓÇ¥

+ :::image type="content" source="./media/spark-job-orchestration/validate-job-execution.png" alt-text="Screenshot shows validate job execution." lightbox="./media/spark-job-orchestration/validate-job-execution.png":::

+## Example code

+This is an example of orchestrating data pipeline using Airflow with HDInsight on AKS

+

+The example is based on [SparkPi](https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/SparkPi.scala) example provided on Apache Spark.

+### Reference

+- Refer to the [sample code](https://github.com/Azure-Samples/hdinsight-aks/blob/main/spark/Airflow/airflow-python-example-code.py).

+- [Apache Spark Website](https://spark.apache.org/)

+- Apache, Apache Airflow, Airflow, Apache Spark, Spark, and associated open source project names are [trademarks](../trademarks.md) of the [Apache Software Foundation](https://www.apache.org/) (ASF).

+ `trino://$USER@$TRINO_CLUSTER_HOST_NAME.hdinsightaks.net:443/$DEFAULT_CATALOG`

+Note that it may take a few minutes for the instance to show up.

+ Note that it may take a few minutes for the instance to show up.

+ Title: How does Azure IoT Operations work in layered network?

+#

+description: Use the Layered Network Management service to enable Azure IoT Operations in industrial network environment.

+#CustomerIntent: As an operator, I want to learn about the architecture of Azure IoT Operations in a Purdue Network environment and how does Layered Network Managment support this scenario.

+# How does Azure IoT Operations work in layered network?

+## Industrial scenario for the Azure IoT Operations

+In the basic architecture described in [Azure IoT Operations Architecture Overview](../get-started/overview-iot-operations.md#architecture-overview), all the Azure IoT Operations components are deployed to a single internet-connected cluster. In this type of environment, component-to-component and component-to-Azure connections are enabled by default.

+However, in many industrial scenarios, computing units for different purposes are located in separate networks. For example:

+- Assets and servers on the factory floor

+- Data collecting and processing solutions in the data center

+- Business logic applications with information workers

+In some cases, the network design includes a single isolated network that is located behind the firewall or is physically disconnected from the internet. In other cases, a more complicated layered network topology is configured, such as the [ISA-95](https://www.isa.org/standards-and-publications/isa-standards/isa-standards-committees/isa95)/[Purdue Network architecture](https://en.wikipedia.org/wiki/Purdue_Enterprise_Reference_Architecture).

+Layered Network Management is designed for facilitating connections between Azure and clusters in different kinds of isolated network environments. Enabling Azure IoT Operations to function in top-level isolated layers and nested isolated layers as needed.

+## How does Layered Network Management work?

+The following diagram describes the mechanism to redirect traffic from an isolated network to Azure Arc. It explains the underlying logic. For information on specific steps to achieve this mechanism, see [Configure Azure IoT Layered Network Management](howto-configure-l4-cluster-layered-network.md).

+1. When an Arc agent or extension is attempting to connect to its corresponding cloud side service, it uses the DNS to resolve the domain name of the target service endpoint.

+1. The custom DNS returns the **IP address of the Layered Network Management instance** at the upper level instead of the real IP address of the service endpoint.

+1. The Arc extension initiates a connection to the Layered Network Management instance with its IP address.

+1. If the Layered Network Management instance is at the internet facing level, it forwards the traffic to the target Arc service endpoint. If the Layered Network Management instance isn't at the top level, it forwards the traffic to the next Layered Network Management instance, and so on.

+> [!NOTE]

+> Layered Network Management only forwards internet traffic when the destination is on the allowlist.

+

+## Example of Azure IoT Operations in layered network

+The following diagram is an example of Azure IoT Operations being deployed to multiple clusters in multiple network layers. Based on the Purdue Network paradigm, level 4 is the enterprise network, level 3 is the operation and control layer, and level 2 is the controller system layer. Moreover, in our prototypical network, only level 4 has direct internet access.

+In the pictured example, Azure IoT Operations is deployed to level 2 through 4. At level 3 and level 4, the **Layered Network Management services** are configured to receive and forward the network traffic from the layer that is one level below. With this forwarding mechanism, all the clusters illustrated in this deployment are able to connect to Azure and become Arc-enabled. The connection to Arc enables users to manage any Arc-enabled endpoint such as the servers, the cluster and the Arc-enabled service workloads from the cloud.

+With extra configurations, the Layered Network Management service can also direct east-west traffic. This route enables Azure IoT Operations components to send data to other components at upper level and form data pipelines from the bottom layer to the cloud.

+In a multi-layer network, the Azure IoT Operations components can be deployed across layers based on your architecture and data flow needs. This example provides some general ideas of where individual components will be placed.

+- The **OPC UA Broker** may locate at the lower layer that is closer to your assets and OPC UA servers. This is also true for the **Akri** agent.

+- The data shall be transferred towards the cloud side through the **MQ** components in each layer.

+- The **Data Processor** is generally placed at the top layer as the most likely layer to have significant compute capacity and as a final stop for the data to get prepared before being sent to the cloud.

+## Next steps

+- To understand how to set up a cluster in an isolated environment for Azure IoT Operations scenarios, see [Configure Layered Network Management service to enable Azure IoT Operations in an isolated network](howto-configure-aks-edge-essentials-layered-network.md).

+- A level 4 single-node cluster running on a host machine with direct access to the internet.

+- The level 3 cluster that is blocked from accessing internet. It connects to the Layered Network Management service as a proxy for all the Azure Arc related traffic.

+For more information, see [Example of logical segmentation with minimum hardware](howto-configure-layered-network.md#example-of-logical-segmentation-with-minimum-hardware).

+

+### Configure level 4 Kubernetes cluster and Layered Network Management

+- Set up a Windows 11 machine and configure AKS Edge Essentials or set up K3S Kubernetes on an Ubuntu machine.

+In the local network, you need to set up the mechanism to redirect all the network traffic to the Layered Network Management service. Use the steps in [Configure custom DNS](howto-configure-layered-network.md#configure-custom-dns). In the article:

+- If you choose the *CoreDNS* approach, you can skip to *Configure and Arc enable level 3 cluster* and configure the CoreDNS before your Arc-enable the level 3 cluster.

+- If you choose to use a *DNS server*, follow the steps to set up the DNS server before you move to the next section in this article.

+

+# [AKS Edge Essentials](#tab/aksee)

+There are few limitations for setting up AKS Edge Essentials as the level 3 cluster.

+- When configuring the custom DNS, you must use a DNS server. The CoreDNS approach is not applicable to AKS Edge Essentials cluster.

+- If you plan to access and manage the cluster remotely, you need to make a [full deployment](/azure/aks/hybrid/aks-edge-howto-multi-node-deployment) instead of a [single machine deployment](/azure/aks/hybrid/aks-edge-howto-single-node-deployment). Moreover, the full deployment can't be hosted on an Azure VM.

+## Prepare Windows 11

+You should complete this step in an *internet facing environment* outside of the isolated network. Otherwise, you need to prepare the offline installation package for the following required software.

+If you're using VM to create your Windows 11 machines, use the [VM image](https://developer.microsoft.com/windows/downloads/virtual-machines/) that includes Visual Studio preinstalled. Having Visual Studio ensures the required certificates needed by Arc onboarding are included.

+1. Install [Windows 11](https://www.microsoft.com/software-download/windows11) on your device.

+1. Install [Helm](https://helm.sh/docs/intro/install/) 3.8.0 or later.

+1. Install [Kubectl](https://kubernetes.io/docs/tasks/tools/)

+1. Download the [installer for the validated AKS Edge Essentials](https://aka.ms/aks-edge/msi-k3s-1.2.414.0) version.

+1. Install AKS Edge Essentials. Follow the steps in [Prepare your machines for AKS Edge Essentials](/azure/aks/hybrid/aks-edge-howto-setup-machine). Be sure to use the installer you downloaded in the previous step and not the most recent version.

+1. **Certificates:** For level 3 and lower, you ARC onboard the cluster that isn't connected to the internet. Therefore, you need to install certificates steps in [Prerequisites for AKS Edge Essentials offline installation](/azure/aks/hybrid/aks-edge-howto-offline-install).

+1. Install the following optional software if you plan to try IoT Operations quickstarts or MQTT related scenarios.

+ - [MQTTUI](https://github.com/EdJoPaTo/mqttui/releases) or other MQTT client

+ - [Mosquitto](https://mosquitto.org/)

+1. Install Azure CLI. You can install the Azure CLI directly onto the level 3 machine or on another *developer* or *jumpbox* machine if you plan to access the level 3 cluster remotely. If you choose to access the Kubernetes cluster remotely to keep the cluster host clean, you run the *kubectl* and *az* related commands from the developer machine for the rest of the steps in this article.

+ The *AKS Edge Essentials - Single machine deployment* does not support accessing Kubernetes remotely. If you want to enable remote kubectl access, you will need to create the [Full Kubernetes Deployment](/azure/aks/hybrid/aks-edge-howto-multi-node-deployment) instead. Additional configurations are needed when creating this type of Kubernetes cluster.

+ - Install Azure CLI. Follow the steps in [Install Azure CLI on Windows](/cli/azure/install-azure-cli-windows).

+ - Install *connectedk8s* and other extensions.

+ ```bash

+ az extension add --name connectedk8s

+ az extension add --name k8s-extension

+ az extension add --name customlocation

+ ```

+ - [Install Azure CLI extension](/cli/azure/iot/ops) using `az extension add --name azure-iot-ops`.

+## Create the AKS Edge Essentials cluster

+To create the AKS Edge Essentials cluster that's compatible with Azure IoT Operations:

+1. Complete the steps in [Create a single machine deployment](/azure/aks/hybrid/aks-edge-howto-single-node-deployment).

+ Create a [Full Kubernetes Deployment](/azure/aks/hybrid/aks-edge-howto-multi-node-deployment) instead if you plan to remotely access the kubernetes from another machine.

+ At the end of [Step 1: single machine configuration parameters](/azure/aks/hybrid/aks-edge-howto-single-node-deployment#step-1-single-machine-configuration-parameters), modify the following values in the *aksedge-config.json* file as follows:

+ - `Init.ServiceIPRangeSize` = 10

+ - `LinuxNode.DataSizeInGB` = 30

+ - `LinuxNode.MemoryInMB` = 8192

+ In the **Network** section, set the `SkipDnsCheck` property to **true**. Add and set the `DnsServers` to the address of the DNS server in the subnet.

+ ```json

+ "DnsServers": ["<IP ADDRESS OF THE DNS SERVER IN SUBNET>"],

+ "SkipDnsCheck": true,

+ ```

+1. Install **local-path** storage in the cluster by running the following command:

+ ```cmd

+ kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/storage/local-path-provisioner/local-path-storage.yaml

+ ```

+## Move the device to level 3 isolated network

+In your isolated network layer, the DNS server was configured in a prerequisite step using [Create sample network environment](./howto-configure-layered-network.md). Complete the step if you haven't done so.

+After the device is moved to level 3, configure the DNS setting using the following steps:

+1. In **Windows Control Panel** > **Network and Internet** > **Network and Sharing Center**, select the current network connection.

+1. In the network properties dialog, select **Properties** > **Internet Protocol Version 4 (TCP/IPv4)** > **Properties**.

+1. Select **Use the following DNS server addresses**.

+1. Enter the level 3 DNS server local IP address.

+ :::image type="content" source="./media/howto-configure-l3-cluster-layered-network/windows-dns-setting.png" alt-text="Screenshot that shows Windows DNS setting with the level 3 DNS server local IP address.":::

+- **AKS Edge Essentials** - *Arc-connected cluster and GitOps* category in [AKS Edge Essentials requirements and support matrix](/azure/aks/hybrid/aks-edge-system-requirements)

+- **K3S Kubernetes cluster** - [Azure Arc-enabled Kubernetes system requirements](/azure/azure-arc/kubernetes/system-requirements)

+The following steps for setting up [AKS Edge Essentials](/azure/aks/hybrid/aks-edge-overview) and [K3S](https://docs.k3s.io/) Kubernetes cluster are verified by Microsoft.

+# [K3S Cluster](#tab/k3s)

+## Prepare an Ubuntu machine

+1. Ubuntu 22.04 LTS is the recommended version for the host machine.

+1. Install [Helm](https://helm.sh/docs/intro/install/) 3.8.0 or later.

+1. Install [Kubectl](https://kubernetes.io/docs/tasks/tools/).

+1. Install the Azure CLI. You can install the Azure CLI directly onto the level 4 machine or on another *developer* or *jumpbox* machine if you plan to access the level 3 cluster remotely. If you choose to access the Kubernetes cluster remotely to keep the cluster host clean, you run the *kubectl* and *az*" related commands from the *developer* machine for the rest of the steps in this article.

+ - Install Azure CLI. Follow the steps in [Install Azure CLI on Linux](/cli/azure/install-azure-cli-linux).

+ - Install *connectedk8s* and other extensions.

+ ```bash

+ az extension add --name connectedk8s

+ az extension add --name k8s-extension

+ ```

+ - [Install Azure CLI extension](/cli/azure/iot/ops) using `az extension add --name azure-iot-ops`.

+## Create the K3S cluster

+1. Install K3S with the following command:

+

+ ```bash

+ curl -sfL https://get.k3s.io | sh -s - --disable=traefik --write-kubeconfig-mode 644

+ ```

+

+ > [!IMPORTANT]

+ > Be sure to use the `--disable=traefik` parameter to disable treafik. Otherwise, you might have an issue when you try to allocate public IP for the Layered Network Management service in later steps.

+1. Copy the K3s configuration yaml file to `.kube/config`.

+ ```bash

+ mkdir ~/.kube

+ cp ~/.kube/config ~/.kube/config.back

+ sudo KUBECONFIG=~/.kube/config:/etc/rancher/k3s/k3s.yaml kubectl config view --flatten > ~/.kube/merged

+ mv ~/.kube/merged ~/.kube/config

+ chmod 0600 ~/.kube/config

+ export KUBECONFIG=~/.kube/config

+ #switch to k3s context

+ kubectl config use-context default

+ ```

+# [AKS Edge Essentials](#tab/aksee)

+ > [!NOTE]

+ > This is a one-time configuration per subscription.

+ ```powershell

+ az provider register -n "Microsoft.ExtendedLocation"

+ az provider register -n "Microsoft.Kubernetes"

+ az provider register -n "Microsoft.KubernetesConfiguration"

+ az provider register -n "Microsoft.IoTOperationsOrchestrator"

+ az provider register -n "Microsoft.IoTOperationsMQ"

+ az provider register -n "Microsoft.IoTOperationsDataProcessor"

+ az provider register -n "Microsoft.DeviceRegistry"

+ ```

+ namespace: azure-iot-operations

+> [!IMPORTANT]

+> This step is for AKS Edge Essentials only.

+To use Azure IoT Layered Network Management service, you need to configure an isolated network environment. For example, the [ISA-95](https://www.isa.org/standards-and-publications/isa-standards/isa-standards-committees/isa95)/[Purdue Network architecture](http://www.pera.net/). This page provides few examples for setting up a test environment depends on how you want to achieve the isolation.

+- *Physical segmentation* - The networks are physically separated. In this case, the Layered Network Management needs to be deployed to a dual NIC (Network Interface Card) host to connect to both the internet-facing network and the isolated network.

+- *Logical segmentation* - The network is logically segmented with configurations such as VLAN, subnet or firewall. The Layered Network Management has a single endpoint and configured to be visible to its own network layer and the isolated layer.

+Both approaches require you to configure a custom DNS in the isolated network layer to direct the network traffic to the Layered Network Management instance in upper layer.

+> [!IMPORTANT]

+> The network environments outlined in Layered Network Management documentation are examples for testing the Layered Network Management. It's not a recommendation of how you build your network and cluster topology for production.

+

+The following diagram illustrates an isolated network environment where each level is logically segmented with subnets. In this test environment, there are multiple clusters one at each level. The clusters can be AKS Edge Essentials or K3S. The Kubernetes cluster in the level 4 network has direct internet access. The Kubernetes clusters in level 3 and below don't have internet access.

+

+- **Level 2 subnet (10.102.0.0/16)** - Like level 3, this subnet doesn't have access to the internet. It is configured to only have access to the IP address 10.103.0.33 in level 3. This subnet contains a Windows 11 machine with the IP address 10.102.0.28 and a Linux machine that hosts a DNS server. There's one Windows 11 machine (node) in this network with IP address 10.102.0.28. All the domains in the DNS configuration must be mapped to the address 10.103.0.33.

+Refer to the following examples for setup this type of network environment.

+### Example of logical segmentation with minimum hardware

+In this example, both machines are connected to an access point (AP) which connects to the internet. The level 4 host machine can access the internet. The level 3 host is blocked for accessing the internet with the AP's configuration. For example, firewall or client control. As both machines are in the same network, the Layered Network Management instance hosted on level 4 cluster is by default visible to the level 3 machine and cluster.

+An extra custom DNS needs to be set up in the local network to provide domain name resolution and point the traffic to Layered Network Management. For more information, see [Configure custom DNS](#configure-custom-dns).

+

+### Example of logical segmentation in Azure

+In this example, a test environment is created with a [virtual network](/azure/virtual-network/virtual-networks-overview) and a [Linux virtual machine](/azure/virtual-machines/linux/quick-create-portal) in Azure.

+> [!IMPORTANT]

+> Virtual environment is for exploration and evaluation only. For more information, see [validated environments](/azure/iot-operations/get-started/overview-iot-operations#validated-environments) for Azure IoT Operations.

+1. Create a virtual network in your Azure subscription. Create subnets for at least two layers (level 4 and level 3).

+1. It's optional to create an extra subnet for the *jumpbox* or *developer* machine to remotely access the machine or cluster across layers. This setup is convenient if you plan to create more than two network layers. Otherwise, you can connect the jumpbox machine to level 4 network.

+1. Create [network security groups](/azure/virtual-network/network-security-groups-overview) for each level and attach to the subnet accordingly.

+1. You can use the default value for level 4 security group.

+1. You need to configure additional inbound and outbound rules for level 3 (and lower level) security group.

+ - Add inbound and outbound security rules to deny all network traffic.

+ - With a higher priority, add inbound and outbound security rules to allow network traffic to and from the IP range of level 4 subnet.

+ - [Optional] If you create a *jumpbox* subnet, create inbound and outbound rules for allowing traffic to and from this subnet.

+1. Create Linux VMs in level 3 and level 4.

+ - Refer to [validated environments](/azure/iot-operations/get-started/overview-iot-operations#validated-environments) for specification of the VM.

+ - When creating the VM, connect the machine to the subnet that is created in earlier steps.

+ - Skip the security group creation for VM.

+A custom DNS is needed for level 3 and below. It ensures that DNS resolution for network traffic originating within the cluster is pointed to the parent level Layered Network Management instance. In an existing or production environment, incorporate the following DNS resolutions into your DNS design. If you want to set up a test environment for Layered Network Management service and Azure IoT Operations, you can refer to the following examples.

+While the DNS setup can be achieved many different ways, this example uses an extension mechanism provided by CoreDNS that is the default DNS server for K3S clusters. URLs on the allowlist, which need to be resolved are added to the CoreDNS.

+> [!IMPORTANT]

+> The CoreDNS approach is only applicable to K3S cluster on Ubuntu host at level 3.

+ Title: "Quickstart: Configure Layered Network Management to Arc-enable a cluster in Azure environment"

+description: Deploy Azure IoT Layered Network Management to an AKS cluster and Arc-enable a cluster on an Ubuntu VM.

+# Quickstart: Configure Layered Network Management to Arc-enable a cluster in Azure environment

+ namespace: azure-iot-operations

+Azure IoT Layered Network Management service is a component that facilitates the connection between Azure and clusters in isolated network environment. In industrial scenarios, the isolated network follows the *[ISA-95](https://www.isa.org/standards-and-publications/isa-standards/isa-standards-committees/isa95)/[Purdue Network architecture](https://en.wikipedia.org/wiki/Purdue_Enterprise_Reference_Architecture)*. The Layered Network Management service can route the network traffic from a non-internet facing layer through an internet facing layer and then to Azure. This service is deployed and managed as a component of Azure IoT Operations Preview on Arc-enabled Kubernetes clusters. Review the network architecture of your solution and use the Layered Network Management service if it's applicable and necessary for your scenarios. If you integrated other mechanisms of controlling internet access for the isolated network, you should compare the functionality with Layered Network Management service and choose the one that fits your needs the best. Layered Network Management is an optional component and it's not a dependency for any feature of Azure IoT Operations Preview.

+- Learn [How Does Azure IoT Operations Work in Layered Network?](concept-iot-operations-in-layered-network.md)

+- [Configure automated performance testing](./quickstart-add-load-test-cicd.md).

+- Learn how to [Set up automated regression testing with CI/CD](./quickstart-add-load-test-cicd.md).

+- Learn more about [configuring automated performance testing with CI/CD](./quickstart-add-load-test-cicd.md).

+- [Tutorial: automate regression tests](./quickstart-add-load-test-cicd.md)

+For more information about running a load test in a CI/CD workflow, see the [Automated regression testing quickstart](./quickstart-add-load-test-cicd.md).

+ testId="<test-id>"

+ > You can use the downloaded test configuration YAML file for [setting up automated load testing in a CI/CD pipeline](./how-to-configure-load-test-cicd.md).

+- [Set up automated load testing with CI/CD](./quickstart-add-load-test-cicd.md)

+- [Set up automated load testing with CI/CD](./quickstart-add-load-test-cicd.md)

+In this section, you configure test criteria for a load test, as part of a CI/CD workflow. Learn how to [set up automated performance testing with CI/CD](./quickstart-add-load-test-cicd.md).

+Learn how to [set up automated performance testing with CI/CD](./quickstart-add-load-test-cicd.md).

+- To learn about performance test automation, see [Configure automated performance testing](./quickstart-add-load-test-cicd.md).

+- To learn about performance test automation, see [Configure automated performance testing](./quickstart-add-load-test-cicd.md).

+- Learn how to [configure automated performance testing](./quickstart-add-load-test-cicd.md).

+- To learn about performance test automation, see [Configure automated performance testing](./quickstart-add-load-test-cicd.md).

+If you run a load test within your CI/CD workflow, you can add a CSV file to the test configuration YAML file. For more information about running a load test in a CI/CD workflow, see [how to add load testing to CI/CD](./how-to-configure-load-test-cicd.md).

+| `zipArtifacts` | array | | List of zip artifact files load test. For files other than JMeter scripts and user properties, if the file size exceeds 50MB, compress them into a ZIP file. Ensure that the ZIP file remains below 50 MB in size. Only 5 ZIP artifacts are allowed with a maximum of 1000 files in each and uncompressed size of 1 GB. |

+zipArtifacts:

+ - sampleArtifact.zip

+ - TestData.zip

+- Learn how to build [automated regression testing in your CI/CD workflow](./quickstart-add-load-test-cicd.md).

+- Learn how to [configure automated performance testing](./quickstart-add-load-test-cicd.md).

+ Title: 'Tutorial: Identify performance issues with load testing'

+description: In this tutorial, you learn how to identify performance bottlenecks in a web app by running a high-scale load test with Azure Load Testing. Use the dashboard to analyze client-side and server-side metrics.

+In this tutorial, you learn how to identify performance bottlenecks in a web application by using Azure Load Testing. You simulate load for a sample Node.js web application, and then use the load test dashboard to analyze client-side and server-side metrics.

+The sample application consists of a Node.js web API, which interacts with a NoSQL database. You deploy the web API to Azure App Service web apps and use Azure Cosmos DB as the database.

+In this tutorial, you learn how to:

+> * Add Azure app components to the load test.

+> * Identify performance bottlenecks by using the load test dashboard.

+* The [Azure CLI](/cli/azure/install-azure-cli) installed on your local computer.

+* Azure CLI version 2.2.0 or later. Run `az --version` to find the version that is installed on your computer. If you need to install or upgrade the Azure CLI, see [How to install the Azure CLI](/cli/azure/install-azure-cli).

+### Prerequisites check

+Before you start, validate your environment:

+* Sign in to the Azure portal and check that your subscription is active.

+* Check your version of the Azure CLI in a terminal or command window by running `az --version`. For the latest version, see the [latest release notes](/cli/azure/release-notes-azure-cli?tabs=azure-cli).

+ If you don't have the latest version, update your installation by following the [installation guide for your operating system or platform](/cli/azure/install-azure-cli).

+## Deploy the sample application

+In this tutorial, you're generating load against a sample web application that you deploy to Azure App Service. Use Azure CLI commands, Git commands, and PowerShell commands to deploy the sample application in your Azure subscription.

+Now that you have the sample application deployed and running, you can create an Azure load testing resource and a load test.

+## Create a load test

+In this tutorial, you're creating a load test with the Azure CLI by uploading a JMeter test script (`jmx` file). The sample application repository already contains a load test configuration file and JMeter test script.

+To create a load test by using the Azure portal, follow the steps in [Quickstart: create a load test with a JMeter script](./how-to-create-and-run-load-test-with-jmeter-script.md).

+Follow these steps to create an Azure load testing resource and a load test by using the Azure CLI:

+1. Open a terminal window and enter the following command to sign in to your Azure subscription.

+ ```azurecli

+ az login

+ ```

+1. Go to the sample application directory.

+ ```azurecli

+ cd nodejs-appsvc-cosmosdb-bottleneck

+ ```

+1. Create a resource group for the Azure load testing resource.

+ Optionally, you can also reuse the resource group of the sample application you deployed previously.

+ Replace the `<load-testing-resource-group-name>` text placeholder with the name of the resource group.

+ ```azurecli

+ resourceGroup="<load-testing-resource-group-name>"

+ location="East US"

+

+ az group create --name $resourceGroup --location $location

+ ```

+1. Create an Azure load testing resource with the [`az load create`](/cli/azure/load) command.

+ Replace the `<load-testing-resource-name>` text placeholder with the name of the load testing resource.

+ ```azurecli

+ # This script requires the following Azure CLI extensions:

+ # - load

+

+ loadTestResource="<load-testing-resource-name>"

+

+ az load create --name $loadTestResource --resource-group $resourceGroup --location $location

+ ```

+1. Create a load test for simulating load against your sample application with the [`az load test create`](/cli/azure/load/test) command.

+ Replace the `<web-app-hostname>` text placeholder with the App Service hostname of the sample application. This value is of the form `myapp.azurewebsites.net`. Don't include the `https://` part of the URL.

+ ```azurecli

+ testId="sample-app-test"

+ webappHostname="<web-app-hostname>"

+

+ az load test create --test-id $testId --load-test-resource $loadTestResource --resource-group $resourceGroup --load-test-config-file SampleApp.yaml --env webapp=$webappHostname

+ ```

+ This command uses the `Sampleapp.yaml` load test configuration file, which references the `SampleApp.jmx` JMeter test script. You use a command-line parameter to pass the sample application hostname to the load test.

+You now have an Azure load testing resource and a load test to generate load against the sample web application in your Azure subscription.

+## Add Azure app components to monitor the application

+Azure Load Testing enables you to monitor resource metrics for the Azure components of your application. By analyzing these *server-side metrics*, you can identify performance and stability issues in your application directly from the Azure Load Testing dashboard.

+In this tutorial, you add the Azure components for the sample application you deployed on Azure, such as the app service, Cosmos DB account, and more.

+To add the Azure app components for the sample application to your load test:

+1. In the [Azure portal](https://portal.azure.com), go to your Azure load testing resource.

+1. On the left pane, select **Tests** to view the list of load tests

+1. Select the checkbox next to your load test, and then select **Edit**.

+ :::image type="content" source="media/tutorial-identify-bottlenecks-azure-portal/edit-load-test.png" alt-text="Screenshot that shows the list of load tests in the Azure portal, highlighting how to select a test from the list and the Edit button to modify the load test configuration." lightbox="media/tutorial-identify-bottlenecks-azure-portal/edit-load-test.png":::

+1. Go to the **Monitoring** tab, and then select **Add/Modify**.

+1. Select the checkboxes for the sample application you deployed previously, and then select **Apply**.

+ :::image type="content" source="media/tutorial-identify-bottlenecks-azure-portal/configure-load-test-select-app-components.png" alt-text="Screenshot that shows how to add app components to a load test in the Azure portal." lightbox="media/tutorial-identify-bottlenecks-azure-portal/configure-load-test-select-app-components.png":::

+ > [!TIP]

+ > You can use the resource group filter to only view the Azure resources in the sample application resource group.

+1. Select **Apply** to save the changes to the load test configuration.

+You successfully added the Azure app components for the sample application to your load test to enable monitoring server-side metrics while the load test is running.

+## Run the load test

+You can now run the load test to simulate load against the sample application you deployed in your Azure subscription. In this tutorial, you run the load test from within the Azure portal. Alternately, you can [configure your CI/CD workflow to run your load test](./quickstart-add-load-test-cicd.md).

+To run your load test in the Azure portal:

+1. In the [Azure portal](https://portal.azure.com), go to your Azure load testing resource.

+1. On the left pane, select **Tests** to view the list of load tests

+1. Select the load test from the list to view the test details and list of test runs.

+1. Select **Run**, and then **Run** again to start the load test.

+ Optionally, you can enter a test run description.

+ :::image type="content" source="./media/tutorial-identify-bottlenecks-azure-portal/run-load-test-first-run.png" alt-text="Screenshot that shows how to start a load test in the Azure portal." lightbox="./media/tutorial-identify-bottlenecks-azure-portal/run-load-test-first-run.png":::

+ When you run a load test, Azure Load Testing deploys the JMeter test script and any extra files to the test engine instance(s), and then starts the load test.

+1. When the load test starts, you should see the load test dashboard.

+ If the dashboard doesn't show, you can select **Refresh** on then select the test run from the list.

+ The load test dashboard presents the test run details, such as the client-side metrics and server-side application metrics. The graphs on the dashboard refresh automatically.

+ :::image type="content" source="./media/tutorial-identify-bottlenecks-azure-portal/load-test-dashboard-client-metrics.png" alt-text="Screenshot that shows the client-side metrics graphs in the load test dashboard in the Azure portal." lightbox="./media/tutorial-identify-bottlenecks-azure-portal/load-test-dashboard-client-metrics.png":::

+## Use server-side metrics to identify performance bottlenecks

+In this section, you analyze the results of the load test to identify performance bottlenecks in the application. Examine both the client-side and server-side metrics to determine the root cause of the problem.

+1. First, look at the client-side metrics. You notice that the 90th percentile for the **Response time** metric for the `add` and `get` API requests is higher than it is for the `lasttimestamp` API.

+ Notice that the **Normalized RU Consumption** metric shows that the database was quickly running at 100% resource utilization. The high resource usage might cause database throttling errors. It also might increase response times for the `add` and `get` web APIs.

+In this section, you allocate more resources to the database to resolve the performance bottleneck.

+Now that you increased the database throughput, rerun the load test and verify that the performance results improved:

+ You can see a new test run entry with a status column that cycles through the **Provisioning**, **Executing**, and **Done** states. At any time, select the test run to monitor how the load test is progressing.

+1. Check the server-side metrics for Azure Cosmos DB and ensure that the performance improved.

+Now that you updated the scale settings of the database, you can see that:

+* The response time for the `add` and `get` APIs improved.

+As a result, the overall performance of your application improved.

+## Related content

+- Get more details about how to [diagnose failing tests](./how-to-diagnose-failing-load-test.md)

+- [Monitor server-side metrics](./how-to-monitor-server-side-metrics.md) to identify performance bottlenecks in your application

+- [Define load test fail criteria](./how-to-define-test-criteria.md) to validate test results against your service requirements

+- Learn more about the [key concepts for Azure Load Testing](./concept-load-testing-concepts.md).

+Sometimes your workflow must connect to an on-premises data source and can use only connectors that provide this access through an on-premises data gateway. To set up this on-premises data gateway, you have to complete the following tasks: install the local on-premises data gateway and create an on-premises data gateway resource in Azure for the local data gateway. When you add a trigger or action to your workflow from a connector that requires the data gateway, you can select the data gateway resource to use with your connection.

+>

+> To directly access on-premises resources in Azure virtual networks without having to use the data gateway,

+> consider creating a [Standard logic app workflow](create-single-tenant-workflows-azure-portal.md),

+> rather than a Consumption logic app workflow. In a Standard workflow, built-in connectors don't

+> require a data gateway to access on-premises data sources.

+This guide shows how to create the Azure data gateway resource after you [install the on-premises gateway on your local computer](logic-apps-gateway-install.md).

+In Azure Logic Apps, the on-premises data gateway supports [on-premises connectors](../connectors/managed.md#on-premises-connectors) for the following data sources:

+* You already [installed an on-premises data gateway on a local computer](logic-apps-gateway-install.md). This data gateway installation must exist before you can create a data gateway resource that links to this installation. You can install only one data gateway per local computer.

+ * When you create a data gateway resource in Azure, you select a data gateway installation to link with your gateway resource and only that gateway resource. Each gateway resource can link to only one gateway installation. You can't select a gateway installation that's already associated with another gateway resource.

+ > Currently, you can't share a data gateway resource or installation across multiple subscriptions.

+After you install the data gateway on a local computer, create the Azure resource for your data gateway.

+ :::image type="content" source="./media/logic-apps-gateway-connection/search-for-on-premises-data-gateway.png" alt-text="Screenshot shows Azure portal search box with the words, on-premises data gateway. The results list shows the selected option, On-premises data gateways.":::

+ :::image type="content" source="./media/logic-apps-gateway-connection/add-azure-data-gateway-resource.png" alt-text="Screenshot shows the page for On-premises data gateways with the selected option for Create.":::

+ :::image type="content" source="./media/logic-apps-gateway-connection/on-premises-data-gateway-create-connection.png" alt-text="Screenshot shows the page for Create a gateway. The Name, Region, and other boxes contain values. The button, Review + create, appears selected.":::

+1. On the validation page that appears, confirm all the information that you provided, and select **Create**.

+1. Add a trigger or action from a connector that supports on-premises connections through the data gateway.

+* In the Azure portal search box, enter **api connections**, and select **API Connections**.

+ :::image type="content" source="./media/logic-apps-gateway-connection/delete-on-premises-data-gateway.png" alt-text="Screenshot shows on-premises data gateway resource in the Azure portal. On the toolbar, Delete is selected.":::

+[Gated models](https://huggingface.co/docs/hub/models-gated) require users to agree to share their contact information and accept the model owners' terms and conditions in order to access the model. Attempting to deploy such models will fail with a `KeyError`.

+1. Select a running compute instance in the **Compute** dropdown list at the top of the dashboard. If you don't have a running compute, create a new compute instance by selecting the plus sign (**+**) next to the dropdown. Or you can select the **Start compute** button to start a stopped compute instance. Creating or starting a compute instance might take few minutes.

+Select **Dashboard configuration** to open a panel with a list of the components you've configured on your dashboard. You can hide components on your dashboard by selecting the **Trash** icon, as shown in the following image:

+By using the model explanation component, you can see which features were most important in your model's predictions. You can view what features affected your model's prediction overall on the **Aggregate feature importance** pane or view feature importances for individual data points on the **Individual feature importance** pane.

+- Incorrectly detecting an object class when a ground truth object doesn't exist

+- **View images per class label**: Identify successful and error image instances per selected class label(s), and the distribution of each class label in your dataset. If a class label has "10/120 examples", out of 120 total images in the dataset, 10 images belong to that class label.

+> - **These four methods are specific to AutoML image classification only** and will not work with other task types such as object detection, instance segmentation etc. Non-AutoML image classification models can leverage SHAP vision for model interpretability.

+>- **The explanations are only generated for the predicted class**. For multilabel classification, a threshold on confidence score is required, to select the classes for which the explanations are generated. See the [parameter list](how-to-responsible-ai-vision-insights.md#responsible-ai-vision-insights-component-parameter-automl-specific) for the parameter name.

+The wizard provides an interface for entering all the necessary parameters to create your Responsible AI dashboard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio UI. The studio presents a guided flow and instructional text to help contextualize the variety of choices about which Responsible AI components you'd like to populate your dashboard with.

+After you've selected a profile, the **Component parameters for model debugging** configuration pane for the corresponding components appears.

+Alternatively, if you select the **Real-life interventions** profile, you'll see the following screen generate a causal analysis. This will help you understand the causal effects of features you want to "treat" on a certain outcome you want to optimize.

+1. **Treatment features (required)**: Choose one or more features that you're interested in changing ("treating") to optimize the target outcome.

+1. **Name**: Give your dashboard a unique name so that you can differentiate it when you're viewing the list of dashboards for a given model.

+After you've finished configuring your experiment, select **Create** to start generating your Responsible AI dashboard. You'll be redirected to the experiment page to track the progress of your job with a link to the resulting Responsible AI dashboard from the job page when it's completed.

+- Select the registered model you'd like to create a scorecard for and select the **Responsible AI** tab.

+The wizard will allow you to customize your PDF scorecard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio to help contextualize the variety of choices of UI with a guided flow and instructional text to help you choose the components you'd like to populate your scorecard with. The wizard is divided into seven steps, with an eighth step (fairness assessment) that will only appear for models with categorical features:

+6. *The Causal analysis* section answers real-world "what if" questions about how changes of treatments would impact a real-world outcome. If the causal component is activated in the Responsible AI dashboard for which you're generating a scorecard, no more configuration is needed.

+ title="From Python",

+ 'image_path_1' : [

+ 'image_path_2': [

+ DataFrame({ 'image_path_1' : 'label_1', 'image_path_2' : 'label_2' ... })

+ classes: '["cat", "dog"]'

+ name="rai_vision_insights", label="latest"

+ title="From Python",

+| `dataset_type` | Whether the Images in the dataset are read from publicly available url or they're stored in the user's datastore. <br> For AutoML models, images are always read from User's workspace datastore, hence the dataset type for AutoML models is "private". <br> For private dataset type, we download the images on the compute before generating the explanations. | Enum <br> - Public <br> - Private |

+Foundation models are machine learning models that have been pre-trained on vast amounts of data, and that can be fine tuned for specific tasks with relatively small amount of domain specific data. These models serve as a starting point for custom models and accelerate the model building process for a variety of tasks including natural language processing, computer vision, speech and generative AI tasks. Azure Machine Learning provides the capability to easily integrate these pre-trained foundation models into your applications. **Foundation models in Azure Machine Learning** provides Azure Machine Learning native capabilities that enable customers to discover, evaluate, fine tune, deploy and operationalize open-source foundation models at scale.

+You can quickly test out any pre-trained model using the Sample Inference widget on the model card, providing your own sample input to test the result. Additionally, the model card for each model includes a brief description of the model and links to samples for code based inferencing, fine-tuning and evaluation of the model.

+1. Provide the Azure Machine Learning Compute cluster you would like to use for fine-tuning the model. Evaluation needs to run on GPU compute. Ensure that you have sufficient compute quota for the compute SKUs you wish to use.

+1. Select **Finish** in the Evaluate wizard to submit your evaluation job. Once the job completes, you can view evaluation metrics for the model. Based on the evaluation metrics, you might decide if you would like to fine-tune the model using your own training data. Additionally, you can decide if you would like to register the model and deploy it to an endpoint.

+## How to fine-tune foundation models using your own training data

+In order to improve model performance in your workload, you might want to fine tune a foundation model using your own training data. You can easily fine-tune these foundation models by using either the fine-tune settings in the studio or by using the code based samples linked from the model card.

+### Fine-tune using the studio

+You can invoke the fine-tune settings form by selecting on the **Fine-tune** button on the model card for any foundation model.

+**Fine-tune Settings:**

+**Fine-tuning task type**

+* Every pre-trained model from the model catalog can be fine-tuned for a specific set of tasks (For Example: Text classification, Token classification, Question answering). Select the task you would like to use from the drop-down.

+1. Pass in the training data you would like to use to fine-tune your model. You can choose to either upload a local file (in JSONL, CSV or TSV format) or select an existing registered dataset from your workspace.

+* Test data: Pass in the test data you would like to use to evaluate your fine-tuned model. Selecting **Automatic split** reserves an automatic split of training data for test.

+* Compute: Provide the Azure Machine Learning Compute cluster you would like to use for fine-tuning the model. Fine-tuning needs to run on GPU compute. We recommend using compute SKUs with A100 / V100 GPUs when fine tuning. Ensure that you have sufficient compute quota for the compute SKUs you wish to use.

+3. Select **Finish** in the fine-tune form to submit your fine-tuning job. Once the job completes, you can view evaluation metrics for the fine-tuned model. You can then register the fine-tuned model output by the fine-tuning job and deploy this model to an endpoint for inferencing.

+### Fine-tuning using code based samples

+Currently, Azure Machine Learning supports fine-tuning models for the following language tasks:

+To enable users to quickly get started with fine-tuning, we have published samples (both Python notebooks and CLI examples) for each task in the [azureml-examples git repo Finetune samples](https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune). Each model card also links to fine-tuning samples for supported fine-tuning tasks.

+You can deploy foundation models (both pre-trained models from the model catalog, and fine-tuned models, once they're registered to your workspace) to an endpoint that can then be used for inferencing. Deployment to both real time endpoints and batch endpoints is supported. You can deploy these models by using either the Deploy UI wizard or by using the code based samples linked from the model card.

+> [!IMPORTANT]

+> __Workspaces without public network access:__ Deploying foundational models to online endpoints without egress connectivity requires [packaging the models (preview)](how-to-package-models.md) first. By using model packaging, you can avoid the need for an internet connection, which Azure Machine Learning would otherwise require to dynamically install necessary Python packages for the MLflow models.

+#### Deployment settings

+##### Networking

+Curated models from the Azure Machine Learning are in MLflow format. If you are planning to deploy these models under an online endpoint without public internet network connectivity, you need to package the model first.

+##### Shared quota

+> Models from Hugging Face are subject to third-party license terms available on the Hugging Face model details page. It is your responsibility to comply with the model's license terms.

+You need to provide compute for the Model import to run. Running the Model Import results in the specified model being imported from Hugging Face and registered to your Azure Machine Learning workspace. You can then fine-tune this model or deploy it to an endpoint for inferencing.

+- [Secure Azure Kubernetes Service inferencing environment](../how-to-secure-kubernetes-inferencing-environment.md)

+- [Secure your managed online endpoints with network isolation](../how-to-secure-online-endpoint.md)

+When you use the feature store CRUD client to GET a feature set - for example, `fs_client.feature_sets.get(name, version)`"` - you might see this error:

+ VisitError(ExecutionError(StreamError(NotFound)))

+ ExecutionError(StreamError(NotFound)); Not able to find path: azureml://subscriptions/{sub_id}/resourcegroups/{rg}/workspaces/{ws}/datastores/workspaceblobstore/paths/LocalUpload/{guid}/feature_retrieval_spec.yaml

+ at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1203)

+ at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatus(AzureBlobFileSystem.java:408)

+ at org.apache.hadoop.fs.Globber.listStatus(Globber.java:128)

+ at org.apache.hadoop.fs.Globber.doGlob(Globber.java:291)

+ at org.apache.hadoop.fs.Globber.glob(Globber.java:202)

+ at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2124)

+ at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1203)

+ at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatus(AzureBlobFileSystem.java:408)

+ at org.apache.hadoop.fs.Globber.listStatus(Globber.java:128)

+ at org.apache.hadoop.fs.Globber.doGlob(Globber.java:291)

+ at org.apache.hadoop.fs.Globber.glob(Globber.java:202)

+ at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2124)

+ at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:306)

+ at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:293)

+ at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)

+ at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)

+ at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2410)

+ at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2380)

+ at com.google.common.cache.LocalCache$S

+When using the feature store CRUD client to stream materialization job results to notebook using `fs_client.jobs.stream("<job_id>")`, the SDK call fails with an error

+In this article, learn how to use a custom Docker image when you're training models with Azure Machine Learning. You'll use the example scripts in this article to classify pet images by creating a convolutional neural network.

+* Once Cassandra is done restarting on all nodes, check `nodetool status`. Both datacenters should appear in the list, with their nodes in the UN (Up/Normal) state.

+> [!TIP]

+> Auto-Replicate setting should be enabled via an arm template.

+> The arm template should include:

+> ```json

+> "properties":{

+> ...

+> "externalDataCenters": ["dc-name-1","dc-name-2"],

+> "autoReplicate": "AllKeyspaces",

+> ...

+> }

+> ```

+> [!INFO]

+>

+2. Update the new database with your new schema changes or updates needed for your database.

+3. Put the production database in a read-only state. It would be best if you didn't have write operations on the production database until deployment is completed.

+4. Test your application with the newly updated database from step 1.

+5. Deploy your application changes and make sure the application is now using the new database with the latest updates.

+6. Keep the old production database to roll back the changes. You can then evaluate to delete the old production database or export it on Azure Storage if needed.

+This page describes the Azure Database for MySQL versioning policy and applies to Azure Database for MySQL - Single Server and Azure Database for MySQL - Flexible Server deployment modes.

+Azure Database for MySQL was developed from [MySQL Community Edition](https://www.mysql.com/products/community/), using the InnoDB storage engine. The service supports the community's current major versions, namely MySQL 5.7, and 8.0. MySQL uses the X.Y.Z. naming scheme where X is the major version, Y is the minor version, and Z is the bug fix release. For more information about the scheme, see the [MySQL documentation](https://dev.mysql.com/doc/refman/5.7/en/which-version.html).

+The retirement details for MySQL major versions are listed in the following table. Dates shown follow the [MySQL versioning policy](https://www.mysql.com/support/eol-notice.html).

+| | | | | |

+| [MySQL 8](https://mysqlserverteam.com/whats-new-in-mysql-8-0-generally-available/) | [Features](https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-21.html) | December 11, 2019 | NA |April 2026|

+In response to the customer's requests, Microsoft decided to prolong the support for Azure Database for MySQL beyond __October 2023__. During the extended support period, which lasts until __September 2025__, Microsoft prioritizes the availability, reliability, and security of the service. While there are no specific guarantees regarding minor version upgrades, we implement essential modifications to ensure that the service remains accessible, dependable, and protected. Our plan includes:

+To summarize, creating Azure Database for MySQL flexible server based on v5.7 won't be available after __April 2024__. However, it's important to note that certain scenarios such as replica creation, point in time recovery, and migration from Azure Database for MySQL - Single Server or Azure Database for MariaDB to Azure Database for MySQL - Flexible Server, will allow you to create MySQL version 5.7 until the end of the extended support period.

+A: Azure Database for MySQL - Single Server doesn't offer built-in support for major version upgrade from v5.7 to v8.0. As Azure Database for MySQL - Single Server is on deprecation path, there are no investments planned to support major version upgrade from v5.7 to v8.0. The recommended path to upgrade from v5.7 of Azure Database for MySQL - Single Server to v8.0 is to first [migrate your v5.7 Azure Database for MySQL - Single Server to v5.7 of Azure Database for MySQL - Flexible Server](single-server/whats-happening-to-mysql-single-server.md#migrate-from-single-server-to-flexible-server). After the migration is completed and server is stabilized on Flexible Server, you can proceed with performing a [major version upgrade](flexible-server/how-to-upgrade.md) on the migrated Azure Database for MySQL - Flexible Server from v5.7 to v8.0. The extended support for v5.7 on Flexible Server will allow you to run on v5.7 longer and plan your upgrade to v8.0 on Flexible Server at a later point in time after migration from Single Server.

+A: Yes, it's expected that there will be some downtime during the upgrade process. The specific duration varies depending on factors such as the size and complexity of the database. We advise conducting a test upgrade on a nonproduction environment to assess the expected downtime and evaluate the potential performance impact. If you wish to minimize downtime for your applications during the upgrade, you can explore the option of [perform minimal downtime major version upgrade from MySQL 5.7 to MySQL 8.0 using read replica](flexible-server/how-to-upgrade.md#perform-minimal-downtime-major-version-upgrade-from-mysql-57-to-mysql-80-using-read-replicas).

+A: While your data will remain unaffected during the upgrade process, it's highly advisable to create a backup of your data before proceeding with the upgrade. This precautionary measure helps mitigate the risk of potential data loss in the event of unforeseen complications.

+__Q: What will happen to the server 5.7 after Sep 2025?__

+A: If there's MariaDB\Single server in your subscription, this subscription is still permitted to create Azure Database for MySQL ΓÇô Flexible Server v5.7 to migrate to Azure Database for MySQL ΓÇô Flexible Server.

+- In the extreme event of a serious threat to the service caused by the MySQL database engine vulnerability identified in, the retired database version, Azure may choose to stop the compute node of your database server from securing the service first. You're asked to upgrade the server before bringing the server online. During the upgrade process, your data is always protected using automatic backups performed on the service, which can be used to restore to the older version if desired.

+2. Use the admin account and password to connect to your database server. Use your preferred client tool, MySQL Workbench, mysql.exe, or HeidiSQL.

+3. Edit and run the following SQL code. Replace the placeholder value `db_user` with your intended new user name. Replace the placeholder value `testdb` with your database name.

+ Now that you have created the database, you can start by creating a nonadmin user by using the ```CREATE USER``` MySQL statement.

+To view the privileges allowed for user **db_user** on **testdb** database, run the ```SHOW GRANTS``` MySQL statement.

+Sign in to the server, specifying the designated database and using the new username and password. This example shows the MySQL command line. When you use this command, you're prompted for the user's password. Use your own server name, database name, and user name. See how to connect the single server and the flexible in the following table.

+To restrict the type of operations a user can run on the database, you must explicitly add the operations in the **GRANT** statement. See the following example:

+All Azure Database for MySQL servers are created with a user called "azure_superuser". Microsoft created a system account to manage the server to conduct monitoring, backups, and other regular maintenance. On-call engineers may also use this account to access the server during an incident with certificate authentication and must request access using just-in-time (JIT) processes.

+# Troubleshoot replication latency in Azure Database for MySQL - Flexible Server

+The [read replica](concepts-read-replicas.md) feature allows you to replicate data from an Azure Database for MySQL server to a read-only replica server. You can scale out workloads by routing read and reporting queries from the application to replica servers. This setup reduces the pressure on the source server and improves overall performance and latency of the application as it scales.

+In this article, you'll learn how to troubleshoot replication latency in Azure Database for MySQL. You'll also get a better idea of some common causes of increased replication latency on replica servers.

+Azure Database for MySQL provides the metric for replication lag in seconds in [Azure Monitor](concepts-monitoring.md). This metric is available only on read replica servers. It's calculated by the seconds_behind_master metric that's available in MySQL.

+If Slave_IO_Running is `Yes` and Slave_SQL_Running is `Yes`, then the replication is running fine.

+If you don't see high CPU utilization on the source server, the problem might be network latency. If network latency is suddenly abnormally high, check the [Azure status page](https://azure.status.microsoft/status) for known issues or outages.

+If you see the following values, then a heavy burst of transactions on the source server is likely causing the replication latency.

+2. Deploy the Bicep file using either Azure CLI or Azure PowerShell.

+- **Azure Database for MySQL**. This option falls into the industry category of PaaS, and represents a fully managed MySQL database engine based on the stable version of the MySQL community edition. This relational database as a service (DBaaS), hosted on the Azure cloud platform, falls into the industry category of PaaS. With a managed instance of MySQL on Azure, you can use built-in features viz automated patching, high availability, automated backups, elastic scaling, enterprise-grade security, compliance and governance, monitoring and alerting that require extensive configuration when MySQL Server is either on-premises or in an Azure VM. When using MySQL as a service, you pay-as-you-go, with options to scale up or out for greater control without interruption. [Azure Database for MySQL](flexible-server/overview.md), powered by the MySQL community edition, is available in two deployment modes:

+ - [Flexible Server](flexible-server/overview.md) is a fully managed production-ready database service designed for more granular control and flexibility over database management functions and configuration settings. The flexible server architecture allows users to opt for high availability within a single availability zone and across multiple availability zones. Flexible servers provide better cost optimization controls with the ability to stop/start the server and burstable compute tier, ideal for workloads that don't need full compute capacity continuously. Flexible Server also supports reserved instances allowing you to save up to 63% cost, which is ideal for production workloads with predictable compute capacity requirements. The service supports the community version of MySQL 5.7 and 8.0. The service is generally available today in various [Azure regions](flexible-server/overview.md#azure-regions). Flexible servers are best suited for all new developments and migration of production workloads to Azure Database for MySQL service.

+ - [Single Server](single-server/single-server-overview.md) is a fully managed database service designed for minimal customization. The single server platform is designed to handle most database management functions such as patching, backups, high availability, and security with minimal user configuration and control. The architecture is optimized for built-in high availability with 99.99% availability in a single availability zone. It supports the community version of MySQL 5.6 (retired), 5.7, and 8.0. The service is generally available today in various [Azure regions](https://azure.microsoft.com/global-infrastructure/services/). Single servers are best-suited **only for existing applications already leveraging single servers**. It's recommended to choose Flexible Server for all new developments or migrations.

+- **MySQL on Azure VMs**. This option falls into the industry category of IaaS. With this service, you can run MySQL Server inside a managed virtual machine on the Azure cloud platform. You can install all recent versions and editions of MySQL on a virtual machine.

+| Ability to restore to a different VNet | No | Yes | Yes |

+#### Billing

+## Azure portal method

+## API method

+Use the Spacecrafts REST Operation Group to [update a spacecraft's TLE](/rest/api/orbital/azureorbitalgroundstation/spacecrafts/create-or-update/) in the Azure Orbital Ground Station API.

+ Title: Scaling Resources in Azure Database for PostgreSQL - Flexible Server

+description: This article describes the resource scaling in Azure Database for PostgreSQL - Flexible Server.

+# Scaling Resources in Azure Database for PostgreSQL - Flexible Server

+Azure Database for PostgreSQL = Flexible Server supports both vertical and horizontal scaling options.

+You scale vertically by adding more resources to the Flexible server instance, such as increasing the instance-assigned number of CPUs and memory. Network throughput of your instance depends on the values you choose for CPU and memory. Once a Flexible server instance is created, you can independently change the CPU (vCores), the amount of storage, and the backup retention period. The number of vCores can be scaled up or down. The storage size however can only be increased. In addition, uou can scale the backup retention period up or down from 7 to 35 days. The resources can be scaled using multiple tools for instance [Azure portal](./quickstart-create-server-portal.md) or the [Azure CLI](./quickstart-create-server-cli.md).

+> [!NOTE]

+> After you increase the storage size, you can't go back to a smaller storage size.

+You scale horizontally by creating [read replicas](./concepts-read-replicas.md). Read replicas let you scale your read workloads onto separate flexible server instance without affecting the performance and availability of the primary instance.

+When you change the number of vCores or the compute tier, the instance is restarted for the new server type to take effect. During this time the system switches over to the new server type, no new connections can be established, and all uncommitted transactions will be rolled back. The overall time it takes to restart your server depends on the crash recovery process and database activity at the time of the restart. Restarts typically takes a minute or less but it can be higher and can take several minutes, depending on transactional activity at the time of the restart. Scaling the storage does not require a server restart in most cases. Similarly, backup retention period changes is an online operation. To improve the restart time, we recommend that you perform scale operations during off-peak hours. That approach reduces the time needed to restart the database server.

+## Near-zero downtime scaling

+Near-zero Downtime Scaling is a feature designed to minimize downtime when modifying storage and compute tiers. If you modify the number of vCores or change the compute tier, the server undergoes a restart to apply the new configuration. During this transition to the new server, no new connections can be established. Typically, this process with regular scaling could take anywhere between 2 to 10 minutes. However, with the new 'Near-zero downtime' Scaling feature this duration has been reduced to less than 30 seconds. This significant reduction in downtime during scaling resources, that greatly improves the overall availability of your database instance.

+### How it works

+When updating your Flexible server in scaling scenarios, we create a new copy of your server (VM) with the updated configuration, synchronize it with your current one, briefly switch to the new copy with a 30-second interruption, and retire the old server, all at no extra cost to you. This process allows for seamless updates while minimizing downtime and ensuring cost-efficiency. This scaling process is triggered when changes are made to the storage and compute tiers, and the experience remains consistent for both (HA) and non-HA servers. This feature is enabled in all Azure regions and there is **no customer action required** to use this capability.

+> [!NOTE]

+> Near-zero downtime scaling process is the _default_ operation. However, in cases where the following limitations are encountered, the system switches to regular scaling, which involves more downtime compared to the near-zero downtime scaling.

+#### Pre-requisites

+- In order for near-zero downtime scaling to work, you should enable all inbound/outbound connections between the IPs in the delegated subnet. If these are not enabled near zero downtime scaling process will not work and scaling will occur through the standard scaling workflow.

+

+#### Limitations

+- Near-zero Downtime Scaling will not work if there are regional capacity constraints or quota limits on customer subscriptions.

+- Near-zero Downtime Scaling doesn't work for replica server but supports the primary server. For replica server it will automatically go through regular scaling process.

+- Near-zero Downtime Scaling will not work if a VNET injected Server with delegated subnet does not have sufficient usable IP addresses. If you have a standalone server, one additional IP address is necessary, and for a HA-enabled server, two extra IP addresses are required.

+## Related content

+- [create a PostgreSQL server in the portal](how-to-manage-server-portal.md)

+- [service limits](concepts-limits.md)

+This article describes reliability support in Azure Private 5G Core. It covers both regional resiliency with [availability zones](#availability-zone-support) and [cross-region disaster recovery and business continuity](#cross-region-disaster-recovery-and-business-continuity). For an overview of reliability in Azure, see [Azure reliability](/azure/architecture/framework/resiliency/overview).

+### Prerequisites

+See [Products available by region](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?products=private-5g-core) for the Azure regions where Azure Private 5G Core is available.

+

+## Cross-region disaster recovery and business continuity

+Azure Private 5G Core is only available in multi-region (3+N) geographies. The service automatically replicates SIM credentials to a backup region in the same geography. This means that there's no loss of data in the event of region failure. Within four hours of the failure, all resources in the failed region are available to view through the Azure portal and ARM tools but will be read-only until the failed region is recovered. The packet core running at the Edge continues to operate without interruption and network connectivity will be maintained.

+### Outage detection, notification, and management

+#### Preparation

+#### Recovery

+#### Failed region restored

+- Use the operational backup region as the new primary region and use the recovered region as a backup. No further action is required.

+- Make the recovered region the new active primary region by following the instructions in [Move resources to a different region](./region-move-private-mobile-network-resources.md) to switch back to the recovered region.

+#### Testing

+- [Reliability in Azure](/azure/reliability/overview)

+Many of the offerings Azure provides require you to set up disaster recovery in multiple regions and aren't the responsibility of Microsoft. Not all Azure services automatically replicate data or automatically fall back from a failed region to cross-replicate to another enabled region. In these cases, you are responsible for configuring recovery and replication.

+Microsoft does ensure that the baseline infrastructure and platform services are available. But in some scenarios, usage demands that you duplicate your deployments and storage in a multi-region capacity, if you choose to. These examples illustrate the shared responsibility model. It's a fundamental pillar in your business continuity and disaster recovery strategy.

+A good example of the shared responsibility model is the deployment of virtual machines. If you want to set up *cross-region replication* for resiliency if there's region failure, you must deploy a duplicate set of virtual machines in an alternate enabled region. Azure doesn't automatically replicate these services over if there's a failure. It's your responsibility to deploy necessary assets. You must have a process to manually change primary regions, or you must use a traffic manager to detect and automatically fail over.

+ Testing is done in several ways, including self-test in a production or near-production environment, and as part of Azure full-region down drills in canary region sets. These enabled regions are identical to production regions but can be disabled without affecting your services. Testing is considered integrated because all services are affected simultaneously.

+- **Customer enablement**: When you are responsible for setting up disaster recovery, Azure is required to have public-facing documentation guidance. For all such services, links are provided to documentation and details about the process.

+Included in these tests are services where you are responsible for setting up disaster recovery following Microsoft public-facing documentation. Service teams create customer-like instances to show that customer-enabled disaster recovery works as expected and that the instructions provided are accurate.

+[Azure AI Search](../search/search-reliability.md?toc=/azure/reliability/toc.json&bc=/azure/reliability/breadcrumb/toc.json)|

+[Azure Image Builder](reliability-image-builder.md)|

+[Azure Operator Nexus](reliability-operator-nexus.md)|

+[Azure Storage - Blob Storage](../storage/common/storage-disaster-recovery-guidance.md?toc=/azure/reliability/toc.json&bc=/azure/reliability/breadcrumb/toc.json)|

+|[Azure Cosmos DB for MongoDB vCore](../cosmos-db/mongodb/vcore/failover-disaster-recovery.md?toc=/azure/reliability/toc.json&bc=/azure/reliability/breadcrumb/toc.json) |

+| [Azure Data Manager for Energy](./reliability-energy-data-services.md) |

+| [Azure Deployment Environments](reliability-deployment-environments.md)|

+|[Azure Private 5G Core](../private-5g-core/reliability-private-5g-core.md?toc=/azure/reliability/toc.json&bc=/azure/reliability/breadcrumb/toc.json)|

+| [Azure Spring Apps](reliability-spring-apps.md) |

+| [Azure Storage Mover](./reliability-azure-storage-mover.md)|

+> | [Role Based Access Control Administrator](#role-based-access-control-administrator) | Manage access to Azure resources by assigning roles using Azure RBAC. This role does not allow you to manage access using other ways, such as Azure Policy. | f58310d9-a9f6-439a-9e8d-f62e7b41a168 |

+> | [Azure Front Door Domain Contributor](#azure-front-door-domain-contributor) | Can manage Azure Front Door domains, but can't grant access to other users. | 0ab34830-df19-4f8c-b84e-aa85b8afa6e8 |

+> | [Azure Front Door Domain Reader](#azure-front-door-domain-reader) | Can view Azure Front Door domains, but can't make changes. | 0f99d363-226e-4dca-9920-b807cf8e1a5f |

+> | [Azure Front Door Profile Reader](#azure-front-door-profile-reader) | Can view AFD standard and premium profiles and their endpoints, but can't make changes. | 662802e2-50f6-46b0-aed2-e834bacc6d12 |

+> | [Azure Front Door Secret Contributor](#azure-front-door-secret-contributor) | Can manage Azure Front Door secrets, but can't grant access to other users. | 3f2eb865-5811-4578-b90a-6fc6fa0df8e5 |

+> | [Azure Front Door Secret Reader](#azure-front-door-secret-reader) | Can view Azure Front Door secrets, but can't make changes. | 0db238c4-885e-4c4f-a933-aa2cef684fca |

+> | [AzureML Compute Operator](#azureml-compute-operator) | Can access and perform CRUD operations on Machine Learning Services managed compute resources (including Notebook VMs). | e503ece1-11d0-4e8e-8e2c-7a6c3bf38815 |

+### Role Based Access Control Administrator

+ "roleName": "Role Based Access Control Administrator",

+### Azure Front Door Domain Contributor

+Can manage Azure Front Door domains, but can't grant access to other users.

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/profileresults/customdomainresults/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/customdomains/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/customdomains/write | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/customdomains/delete | |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/subscriptions/resourceGroups/read | Gets or lists resource groups. |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can manage Azure Front Door domains, but can't grant access to other users.",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/0ab34830-df19-4f8c-b84e-aa85b8afa6e8",

+ "name": "0ab34830-df19-4f8c-b84e-aa85b8afa6e8",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.Cdn/operationresults/profileresults/customdomainresults/read",

+ "Microsoft.Cdn/profiles/customdomains/read",

+ "Microsoft.Cdn/profiles/customdomains/write",

+ "Microsoft.Cdn/profiles/customdomains/delete",

+ "Microsoft.Resources/subscriptions/resourceGroups/read"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "Azure Front Door Domain Contributor",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+### Azure Front Door Domain Reader

+Can view Azure Front Door domains, but can't make changes.

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/profileresults/customdomainresults/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/customdomains/read | |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/subscriptions/resourceGroups/read | Gets or lists resource groups. |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can view Azure Front Door domains, but can't make changes.",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/0f99d363-226e-4dca-9920-b807cf8e1a5f",

+ "name": "0f99d363-226e-4dca-9920-b807cf8e1a5f",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.Cdn/operationresults/profileresults/customdomainresults/read",

+ "Microsoft.Cdn/profiles/customdomains/read",

+ "Microsoft.Resources/subscriptions/resourceGroups/read"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "Azure Front Door Domain Reader",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+### Azure Front Door Profile Reader

+Can view AFD standard and premium profiles and their endpoints, but can't make changes.

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.Authorization](resource-provider-operations.md#microsoftauthorization)/*/read | Read roles and role assignments |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/edgenodes/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/* | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/*/read | |

+> | [Microsoft.Insights](resource-provider-operations.md#microsoftinsights)/alertRules/* | Create and manage a classic metric alert |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/deployments/* | Create and manage a deployment |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/subscriptions/resourceGroups/read | Gets or lists resource groups. |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/profileresults/afdendpointresults/CheckCustomDomainDNSMappingStatus/action | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/queryloganalyticsmetrics/action | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/queryloganalyticsrankings/action | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/querywafloganalyticsmetrics/action | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/querywafloganalyticsrankings/action | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/afdendpoints/CheckCustomDomainDNSMappingStatus/action | |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can view AFD standard and premium profiles and their endpoints, but can't make changes.",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/662802e2-50f6-46b0-aed2-e834bacc6d12",

+ "name": "662802e2-50f6-46b0-aed2-e834bacc6d12",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.Authorization/*/read",

+ "Microsoft.Cdn/edgenodes/read",

+ "Microsoft.Cdn/operationresults/*",

+ "Microsoft.Cdn/profiles/*/read",

+ "Microsoft.Insights/alertRules/*",

+ "Microsoft.Resources/deployments/*",

+ "Microsoft.Resources/subscriptions/resourceGroups/read",

+ "Microsoft.Cdn/operationresults/profileresults/afdendpointresults/CheckCustomDomainDNSMappingStatus/action",

+ "Microsoft.Cdn/profiles/queryloganalyticsmetrics/action",

+ "Microsoft.Cdn/profiles/queryloganalyticsrankings/action",

+ "Microsoft.Cdn/profiles/querywafloganalyticsmetrics/action",

+ "Microsoft.Cdn/profiles/querywafloganalyticsrankings/action",

+ "Microsoft.Cdn/profiles/afdendpoints/CheckCustomDomainDNSMappingStatus/action"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "Azure Front Door Profile Reader",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+### Azure Front Door Secret Contributor

+Can manage Azure Front Door secrets, but can't grant access to other users.

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/profileresults/secretresults/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/secrets/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/secrets/write | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/secrets/delete | |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/subscriptions/resourceGroups/read | Gets or lists resource groups. |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can manage Azure Front Door secrets, but can't grant access to other users.",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/3f2eb865-5811-4578-b90a-6fc6fa0df8e5",

+ "name": "3f2eb865-5811-4578-b90a-6fc6fa0df8e5",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.Cdn/operationresults/profileresults/secretresults/read",

+ "Microsoft.Cdn/profiles/secrets/read",

+ "Microsoft.Cdn/profiles/secrets/write",

+ "Microsoft.Cdn/profiles/secrets/delete",

+ "Microsoft.Resources/subscriptions/resourceGroups/read"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "Azure Front Door Secret Contributor",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+### Azure Front Door Secret Reader

+Can view Azure Front Door secrets, but can't make changes.

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/operationresults/profileresults/secretresults/read | |

+> | [Microsoft.Cdn](resource-provider-operations.md#microsoftcdn)/profiles/secrets/read | |

+> | [Microsoft.Resources](resource-provider-operations.md#microsoftresources)/subscriptions/resourceGroups/read | Gets or lists resource groups. |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can view Azure Front Door secrets, but can't make changes.",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/0db238c4-885e-4c4f-a933-aa2cef684fca",

+ "name": "0db238c4-885e-4c4f-a933-aa2cef684fca",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.Cdn/operationresults/profileresults/secretresults/read",

+ "Microsoft.Cdn/profiles/secrets/read",

+ "Microsoft.Resources/subscriptions/resourceGroups/read"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "Azure Front Door Secret Reader",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+### AzureML Compute Operator

+Can access and perform CRUD operations on Machine Learning Services managed compute resources (including Notebook VMs). [Learn more](../machine-learning/how-to-assign-roles.md)

+> [!div class="mx-tableFixed"]

+> | Actions | Description |

+> | | |

+> | [Microsoft.MachineLearningServices](resource-provider-operations.md#microsoftmachinelearningservices)/workspaces/computes/* | |

+> | [Microsoft.MachineLearningServices](resource-provider-operations.md#microsoftmachinelearningservices)/workspaces/notebooks/vm/* | |

+> | **NotActions** | |

+> | *none* | |

+> | **DataActions** | |

+> | *none* | |

+> | **NotDataActions** | |

+> | *none* | |

+```json

+{

+ "assignableScopes": [

+ "/"

+ ],

+ "description": "Can access and perform CRUD operations on Machine Learning Services managed compute resources (including Notebook VMs).",

+ "id": "/providers/Microsoft.Authorization/roleDefinitions/e503ece1-11d0-4e8e-8e2c-7a6c3bf38815",

+ "name": "e503ece1-11d0-4e8e-8e2c-7a6c3bf38815",

+ "permissions": [

+ {

+ "actions": [

+ "Microsoft.MachineLearningServices/workspaces/computes/*",

+ "Microsoft.MachineLearningServices/workspaces/notebooks/vm/*"

+ ],

+ "notActions": [],

+ "dataActions": [],

+ "notDataActions": []

+ }

+ ],

+ "roleName": "AzureML Compute Operator",

+ "roleType": "BuiltInRole",

+ "type": "Microsoft.Authorization/roleDefinitions"

+}

+```

+Can perform all actions within an Azure Machine Learning workspace, except for creating or deleting compute resources and modifying the workspace itself. [Learn more](../machine-learning/how-to-assign-roles.md)

+- [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+1. Use the [Connect-AzAccount](/powershell/module/az.accounts/connect-azaccount) command and follow the instructions that appear to sign in to your directory as Role Based Access Control Administrator.

+1. Use the [az login](/cli/azure/reference-index#az-login) command and follow the instructions that appear to sign in to your directory as Role Based Access Control Administrator.

+| Use [custom security attributes on a principal](conditions-format.md#principal-attributes) in a condition | GA | November 2023 |

+Just like role assignments, to add or update conditions, you must be signed in to Azure with a user that has the `Microsoft.Authorization/roleAssignments/write` and `Microsoft.Authorization/roleAssignments/delete` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator).

+To create a custom role, you must have permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator).

+- Permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator)

+- Permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator)

+To update a custom role, use the [Role Definitions - Create Or Update](/rest/api/authorization/role-definitions/create-or-update) REST API. To call this API, you must be signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleDefinitions/write` permission on all the `assignableScopes`, such as [User Access Administrator](built-in-roles.md#user-access-administrator).

+- Permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator).

+This condition allows a delegate to add or remove role assignments for all roles except the [Owner](built-in-roles.md#owner), [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator), and [User Access Administrator](built-in-roles.md#user-access-administrator) roles.

+> | Roles | [Owner](built-in-roles.md#owner)<br/>[Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)<br/>[User Access Administrator](built-in-roles.md#user-access-administrator) |

+> | Roles | [Owner](built-in-roles.md#owner)<br/>[Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)<br/>[User Access Administrator](built-in-roles.md#user-access-administrator) |

+1. Alice assigns the Role Based Access Control Administrator role to Dara. Alice adds conditions so that Dara can only assign the Backup Contributor or Backup Reader roles to the Marketing and Sales groups.

+The [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) role is a built-in role that has been designed for delegating role assignment management to others. It has fewer permissions than [User Access Administrator](built-in-roles.md#user-access-administrator), which follows least privilege best practices. The Role Based Access Control Administrator role has following permissions:

+1. Select the [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) role

+ You can select any role that includes the `Microsoft.Authorization/roleAssignments/write` action, but Role Based Access Control Administrator has fewer permissions.

+1. Select the **Role Based Access Control Administrator** role.

+ You can select any role that includes the `Microsoft.Authorization/roleAssignments/write` or `Microsoft.Authorization/roleAssignments/delete` actions, such as [User Access Administrator](built-in-roles.md#user-access-administrator), but [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) has fewer permissions.

+- `Microsoft.Authorization/roleAssignments/write` and `Microsoft.Authorization/roleAssignments/delete` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator).

+- `Microsoft.Authorization/roleAssignments/write` and `Microsoft.Authorization/roleAssignments/delete` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+[Azure RBAC](overview.md) is an authorization system built on [Azure Resource Manager](../azure-resource-manager/management/overview.md) that provides fine-grained access management to Azure resources, such as compute and storage. Azure RBAC includes over 100 built-in roles. There are five fundamental Azure roles. The first three apply to all resource types:

+| [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) | <ul><li>Manage user access to Azure resources</li><li>Assign roles in Azure RBAC</li><li>Assign themselves or others the Owner role</li><li>Can't manage access using other ways, such as Azure Policy</li></ul> |

+- `Microsoft.Authorization/roleAssignments/write` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+- [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+- `Microsoft.Authorization/roleAssignments/write` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+[Azure role-based access control (Azure RBAC)](overview.md) is the authorization system you use to manage access to Azure resources. To remove access from an Azure resource, you remove a role assignment. This article describes how to remove roles assignments using the Azure portal, Azure PowerShell, Azure CLI, and REST API.

+- `Microsoft.Authorization/roleAssignments/delete` permissions, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator)

+To assign a role, use the [Role Assignments - Create](/rest/api/authorization/role-assignments/create) REST API and specify the security principal, role definition, and scope. To call this API, you must have access to the `Microsoft.Authorization/roleAssignments/write` action, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator).

+| [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) | <ul><li>Manage user access to Azure resources</li><li>Assign roles in Azure RBAC</li><li>Assign themselves or others the Owner role</li><li>Can't manage access using other ways, such as Azure Policy</li></ul> |

+To assign roles, you must be signed in with a user that is assigned a role that has role assignments write permission, such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) at the scope you are trying to assign the role. Similarly, to remove a role assignment, you must have the role assignments delete permission.

+- [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) role to add or remove role assignments.

+- [User Access Administrator](./built-in-roles.md#user-access-administrator) role to add role assignments, remove role assignments, or delete custom roles.

+Check that you're currently signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleAssignments/write` permission such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) at the scope you're trying to assign the role.

+Check that you are currently signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleAssignments/write` permission such as [Role Based Access Control Administrator](built-in-roles.md#role-based-access-control-administrator) at the scope you are trying to assign the role.

+Check that you're currently signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleDefinitions/write` permission such as [User Access Administrator](built-in-roles.md#user-access-administrator).

+- If you don't have permissions, ask your administrator to assign you a role that has the `Microsoft.Authorization/roleDefinitions/write` action, such as [User Access Administrator](built-in-roles.md#user-access-administrator), at the scope of the assignable scope.

+- Permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator)

+- Permissions to create custom roles, such as [User Access Administrator](built-in-roles.md#user-access-administrator)

+> | `management_subnet_nsg_name` | The name of the network security group | Optional | |

+> | `management_firewall_subnet_arm_id` | The Azure resource identifier for the Azure Firewall subnet | Mandatory | For brown-field deployments |

+> | `management_bastion_subnet_arm_id` | The Azure resource identifier for the Azure Bastion subnet | Mandatory | For brown-field deployments |

+> | | | | |

+> | `use_private_endpoint` | Use private endpoints. | Optional | |

+> | `use_service_endpoint` | Use service endpoints for subnets. | Optional | |

+### Web App parameters

+> [!div class="mx-tdCol2BreakAll "]

+> | Variable | Description | Type | Notes |

+> | -- | - | -- | |

+> | `use_webapp` | Boolean value indicating if a webapp should be deployed. | Optional | |

+> | `app_service_SKU_name` | The SKU of the App Service Plan. | Optional | |

+> | `app_registration_app_id` | The app registration id to be used for the webapp. | Optional | |

+> | `webapp_client_secret` | The SKU of the App Service Plan. | Optional | Will be persisted in Key Vault |

+> | Variable | Description | Type |

+> | - | - | - |

+> | `app_proximityplacementgroup_arm_ids` | Specifies the Azure resource identifiers of existing proximity placement groups for the app tier. | |

+> | `app_proximityplacementgroup_names` | Specifies the names of the proximity placement groups for the app tier. | |

+> | `custom_disk_sizes_filename` | Defines the disk sizing file name, See [Custom sizing](configure-extra-disks.md). | Optional |

+> | `disk_encryption_set_id` | The disk encryption key to use for encrypting managed disks by using customer-provided keys. | Optional |

+> | `proximityplacementgroup_arm_ids` | Specifies the Azure resource identifiers of existing proximity placement groups. | |

+> | `proximityplacementgroup_names` | Specifies the names of the proximity placement groups. | |

+> | `resource_offset` | Provides an offset for resource naming. | Optional |

+> | `scaleset_id` | Azure resource identifier for the virtual machine scale set | Optional |

+> | `use_app_proximityplacementgroups` | Controls if the app tier virtual machines are placed in a different ppg from the database. | Optional |

+> | `use_loadbalancers_for_standalone_deployments` | Controls if load balancers are deployed for standalone installations | Optional |

+> | `use_scalesets_for_deployment` | Use Flexible Virtual Machine Scale Sets for the deployment | Optional |

+> | `user_assigned_identity_id | User assigned identity to assign to the virtual machines | Optional |

+[SAP Deployment Automation Framework](https://github.com/Azure/sap-automation) is an open-source orchestration tool that can deploy, install, and maintain SAP environments. You can deploy the systems on any of the SAP-supported operating system versions and into any Azure region. You can create infrastructure for SAP landscapes based on SAP HANA and NetWeaver with AnyDB by using [Terraform](https://www.terraform.io/). The environments can be configured using [Ansible](https://www.ansible.com/).

+- Deployment infrastructure (control plane, typically deployed in the hub)

+- SAP infrastructure (SAP workload zone, typically deployed in a spoke.)

+The dependency between the control plane and the application plane is illustrated in the following diagram. In a typical deployment, a single control plane is used to manage multiple SAP deployments.

+- A Web application for configuration management

+As part of the download process, the application manifest and the supporting templates are also persisted in the storage account. The application manifest and the dependent manifests are aggregated into a single manifest file that is used by the installation process.

+- SAP workload zone which is used for the shared resources for the SAP systems

+The workload zone allows for partitioning of the deployments into different environments, such as development, test, and production. The workload zone provides the shared resources (networking and credentials management) to the SAP systems.

+- Virtual network

+- Azure Key Vault for system credentials (VMs and SAP accounts)

+> In order to use this functionality you need to add an additional disk named 'custom' to one or more of your Virtual machines. For more information, see [Custom disk sizing](configure-extra-disks.md).

+You can extend the SAP Deployment Automation Framework by mounting extra mount points in your installation.

+When you add the following section to the sap-parameters.yaml file, a filesystem '/usr/custom' is exported from the Central Services virtual machine and available via NFS.

+## Custom Stripe sizes (Linux)

+If you want to the stripe sizes used by the framework when creating the disks, you can add the following section to the sap-parameters.yaml file with the values you want.

+```yaml

+# User and group IDs

+hana_data_stripe_size: 256

+hana_log_stripe_size: 64

+db2_log_stripe_size: 64

+db2_data_stripe_size: 256

+db2_temp_stripe_size: 128

+sybase_data_stripe_size: 256

+sybase_log_stripe_size: 64

+sybase_temp_stripe_size: 128

+oracle_data_stripe_size: 256

+oracle_log_stripe_size: 128

+```

+## Custom volume sizes (Linux)

+If you want to the default volume sizes used by the framework, you can add the following section to the sap-parameters.yaml file with the values you want.

+```yaml

+sapmnt_volume_size: 32g

+usrsap_volume_size: 32g

+hanashared_volume_size: 32g

+```

+You can use Azure Repos to store your configuration files. Azure Pipelines provides pipelines, which can be used to deploy and configure the infrastructure and the SAP application.

+> API reference docs are now versioned. To get the right content, open a reference page and then apply the version-specific filter located above the table of contents.

+> Azure portal supports a one-click upgrade path for 2023-07-01-preview indexes. The portal detects 2023-07-01-preview indexes and provides a **Migrate** button. Before selecting **Migrate**, select **Edit JSON** to review the updated schema first. You should find a schema that conforms to the changes described in this section. Portal migration only handles indexes with one vector search algorithm configuration, creating a default profile that maps to the algorithm. Indexes with multiple configurations require manual migration.

+1. Modify vector field definitions, replacing `vectorSearchConfiguration` with `vectorSearchProfile`. Make sure the profile name resolves to a new vector profile definition, and not the algorithm configuration name. Other vector field properties remain unchanged. For example, they can't be filterable, sortable, or facetable, nor use analyzers or normalizers or synonym maps.

+* [BM25 ranking algorithm](index-ranking-similarity.md) replaces the previous ranking algorithm with newer technology. Services created after 2019 use this algorithm automatically. For older services, you must set parameters to use the new algorithm.

+description: Azure AI Search is an AI-powered information retrieval platform, helps developers build rich search experiences and generative AI apps that combine large language models with enterprise data.

+> Site Recovery won't be enabled if an unsupported VM is created within the scope of the policy.

+ if ($a.AccessControlType -eq "User" -and $a.DefaultScope -eq $false -and $a.EntityId -eq $id)

+Azure Storage automatically encrypts all data in a storage account at the service level using 256-bit AES with GCM mode encryption, one of the strongest block ciphers available, and is FIPS 140-2 compliant. Customers who require higher levels of assurance that their data is secure can also enable 256-bit AES with CBC encryption at the Azure Storage infrastructure level for double encryption. Double encryption of Azure Storage data protects against a scenario where one of the encryption algorithms or keys might be compromised. In this scenario, the additional layer of encryption continues to protect your data.

+Apache Spark pools in Azure Synapse use runtimes to tie together essential component versions such as Azure Synapse optimizations, packages, and connectors with a specific Apache Spark version. Each runtime is upgraded periodically to include new improvements, features, and patches. When you create a serverless Apache Spark pool, you have the option to select the corresponding Apache Spark version. Based on this, the pool comes pre-installed with the associated runtime components and packages. The runtimes have the following advantages:

+3. Long Term Support (LTS) runtime is patched with security fixes only.

+4. End of life announced (EOLA) runtime will not have bug and feature fixes. Security fixes are backported based on risk assessment.

+Question: What steps should be taken in migrating from 2.4 to 3.X?

+Question: I get an error when I try to upgrade Spark pool runtime using PowerShell commandlet when the Spark pool has attached libraries

+Answer: Do not use PowerShell Commandlet if you have custom libraries attached to the Spark pool. Instead follow these steps:

+* Recreate Spark Pool 3.3 from the ground up.

+* Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3

+Apache Spark pools in Azure Synapse use runtimes to tie together essential component versions such as Azure Synapse optimizations, packages, and connectors with a specific Apache Spark version. Each runtime is upgraded periodically to include new improvements, features, and patches.

+| Generally Available (GA) | 12 months* | Generally Available (GA) runtimes are open to all eligible customers and are ready for production use. <br/> A GA runtime may not be elected to move into an LTS stage at Microsoft discretion. |

+> * The above timelines are provided as examples based on current Apache Spark releases. If the Apache Spark project changes the lifecycle of a specific version affecting a Synapse runtime, changes to the stage dates are noted on the [release notes](./apache-spark-version-support.md).

+At the end of the Preview lifecycle for the runtime, Microsoft will assess if the runtime moves into a Generally Availability (GA) based on customer usage, security and stability criteria.

+If not eligible for GA stage, the Preview runtime moves into the retirement cycle.

+Once a runtime is Generally Available, only security fixes are backported. In addition, new components or features are introduced if they don't change underlying dependencies or component versions.

+At the end of the GA lifecycle for the runtime, Microsoft will assess if the runtime has an extended lifecycle (LTS) based on customer usage, security and stability criteria.

+If not eligible for LTS stage, the GA runtime moves into the retirement cycle.

+For runtimes that are covered by Long term support (LTS) customers are encouraged to expedite validation and migration of code base and workloads to the latest GA runtimes. We recommend that customers don't onboard new workloads using an LTS runtime. Security fixes and stability improvements may be backported, but no new components or features are introduced into the runtime at this stage.

+During the EOLA stage, existing Synapse Spark pools function as expected, and new pools of the same version can be created. The runtime version is listed on Azure Synapse Studio, Synapse API, or Azure portal. At the same time, we strongly recommend migrating your workloads to the latest General Availability (GA) runtimes.

+* It isn't possible to create new Spark pools using the retired version through Azure Synapse Studio, the Synapse API, or the Azure portal.

+* The retired runtime version won't be available in Azure Synapse Studio, the Synapse API, or the Azure portal.

+## Frequently asked questions

+### What is the expected frequency for the maintenance.

+Maintenance can happen more than once per month, because maintenance can include OS updates, security patches and drivers, internal Azure infrastructure updates, and DW patches and updates. Every customer has a twice-weekly schedule of maintenance cycles between through SaturdayΓÇôSunday, and TuesdayΓÇôThursday.

+### What changes have been made after the maintenance is completed, even though my dedicated SQL pool version remains the same?

+After a maintenance update is completed, the SQL pool version may remain unchanged. This is because maintenance can include OS updates, security patches and drivers, internal Azure infrastructure updates, and DW patches and updates. Only if a Synapse DW patch or update is included in the maintenance will you see a change to the SQL Dedicated Pool version.

+### Is it possible to upgrade the version of my dedicated SQL pool on demand?

+- No, scheduled maintenance handles the management of dedicated SQL pools. However, you might have some options to trigger the maintenance once the cycle started, depending on your situation. Verify [Skip or change maintenance schedule](#skip-or-change-maintenance-schedule)

+- It's important to keep in mind that the dedicated SQL Pool is a Platform as a Service (PaaS) feature. This implies that Microsoft Azure handles all kinds of tasks related to the service, such as infrastructure, maintenance, updates, and scalability. Scheduled maintenance can be tracked by setting an alert/notification so you stay informed of impending maintenance activity.

+### What changes, if any, should be made before or after the dedicated SQL pool maintenance is completed?

+- During maintenance, your service will be briefly taken offline, similar to what occurs during a pause, resume, or scale operation. Typically, the overall maintenance operation is completed in well under 30 minutes. However, it could take a little longer, depending on database activity during the maintenance window. We recommend pausing ETL, table updates, and especially transactional operations to avoid longer than normal maintenance. For example:

+- If your instance is extremely busy during the planned window, especially with frequent update and delete activity, the maintenance operation might take longer than the normal time. To reduce the chance of extended maintenance activity, we recommend limiting activity to mostly read-only queries against the database if possible, and especially avoiding long-running transactional queries (see the next item).

+- If there are active transactions when the maintenance begins, they are canceled and rolled back, potentially causing delays in restoring the online service. To prevent this situation, we recommend ensuring that there are no long-running transactions active at the start of your maintenance window.

+### We were notified about an upcoming dedicated SQL pool scheduled maintenance with tracking ID 0000-000, but it was subsequently canceled or rescheduled. What prompted the cancellation or rescheduling of the maintenance?

+There are various factors that could lead to the cancellation of scheduled maintenance, including actions such as:

+- Pausing or scaling operations after receiving a pending maintenance notification while the cycle is initiated.

+- If you are targeting different Service Level Objectives (SLOs) during the maintenance cycle, such as transitioning from any SLO higher than DW400c and then scaling back to an SLO lower or equal to DW400c, or vice versa, a cancellation could occur. This is because maintenance windows are not applicable for DW400c or lower performance levels, and they can undergo maintenance at any time.

+- Internal infrastructure factors, such as actual changes to planned maintenance scheduling by the release team.

+- Maintenance may be canceled or rescheduled if internal monitoring detects that maintenance is taking longer than expected. Maintenance must be completed within the Service Level Agreements (SLAs) defined by customer maintenance window settings.

+### Are there any best practices that I need to consider for our workload during the maintenance window?

+- Yes, if possible, pause all transactional and ETL workloads during the planned maintenance interval to avoid errors or delays in restoring the online service. Long-running transactional operations should be completed prior to an upcoming maintenance period.

+- For workloads to be resilient to interruptions caused by maintenance operations, use retry logic for both the connection and the command (query) levels, applying longer retry intervals and/or more retry attempts to withstand an extended connection loss that can extend up to or greater than 30 minutes in some cases.

+### Distribute target not found in the update request

+#### Error

+```text

+Validation failed: Distribute target with Runoutput name <runoutputname> not found in the update request. Deleting a distribution target is not allowed.

+```

+#### Cause

+This error occurs when an existing distribute target isn't found in the Patch request body.

+#### Solution

+The distribution array should contain all the distribution targets that is, new targets (if any), existing targets with no change and updated targets. If you want to remove an existing distribution target, delete and re-create the image template as deleting a distribution target is currently not supported through the Patch API.

+### Missing required fields

+#### Error

+```text

+Validation failed: 'ImageTemplate.properties.distribute[<index>]': Missing field <fieldname>. Please review http://aka.ms/azvmimagebuildertmplref for details on fields required in the Image Builder Template.

+```

+#### Cause

+This error occurs when a required field is missing from a distribute target.

+#### Solution

+When creating a request, please provide every required field in a distribute target even if there's no change.

+### Dynamic allocation

+Azure assigns the next available unassigned or unreserved IP address in the subnet's address range. While this is normally the next sequentially available address, there's no guarantee that the address will be the next one in the range. For example, if addresses 10.0.0.4-10.0.0.9 are already assigned to other resources, the next IP address assigned is most likely 10.0.0.10. However, it could be any address between 10.0.0.10 and 10.0.0.254. If a specific Private IP address is required for a resource, you should use a static private IP address.

+Dynamic is the default allocation method. Once assigned, dynamic IP addresses are released if a network interface is:

+* Deleted

+* Reassigned to a different subnet within the same virtual network.

+* The allocation method is changed to static, and a different IP address is specified.

+By default, Azure assigns the previous dynamically assigned address as the static address when you change the allocation method from dynamic to static.

+### Static allocation

+With static allocation, you select and assign any unassigned or unreserved IP address in the subnet's address range.

+For example, a subnet's address range is 10.0.0.0/16 and addresses 10.0.0.4-10.0.0.9 are assigned to other resources. You can assign any address between 10.0.0.10 - 10.0.255.254. Static addresses are only released if a network interface is deleted.

+Azure assigns the static IP as the dynamic IP when the allocation method is changed. The reassignment occurs even if the address isn't the next available in the subnet. The address changes when the network interface is assigned to a different subnet.

+To assign the network interface to a different subnet, you change the allocation method from static to dynamic. Assign the network interface to a different subnet, then change the allocation method back to static. Assign an IP address from the new subnet's address range.

+> [!NOTE]

+> When requesting a private IP address, the allocation is not deterministic or sequential. There are no guarantees the next allocated IP address will utilize the next sequential IP address or use previously deallocated addresses. If a specific Private IP address is required for a resource, you should consider using a static private IP address.

+You can add as many [private](#private) and [public](#public) [IPv4](#ipv4) addresses as necessary to a network interface, within the limits listed in the [Azure limits](../../azure-resource-manager/management/azure-subscription-service-limits.md#azure-resource-manager-virtual-networking-limits) article. You can add a private IPv6 address to one [secondary IP configuration](#secondary) (as long as there are no existing secondary IP configurations) for an existing network interface. Each network interface can have one IPv6 private address. You can optionally add a public IPv6 address to an IPv6 network interface configuration. See [IPv6](#ipv6) for details about using IPv6 addresses.

+Situations arise where you need to change the allocation method of an IPv4 address, change the static IPv4 address, or change the public IP address associated with a network interface. Place a virtual machine into the stopped (deallocated) state before changing the private IPv4 address of a secondary IP configuration associated with the secondary network interface. To learn more, see [primary and secondary network interfaces](../../virtual-network/virtual-network-network-interface-vm.md)).

+- Can have a [public](#public) IPv4 address assigned to it. You can't assign a public IPv6 address to a primary (IPv4) IP configuration.

+In addition to a primary IP configuration, a network interface can have zero or more secondary IP configurations assigned to it. A secondary IP configuration:

+- Must have a private IPv4 or IPv6 address assigned to it. If the address is IPv6, the network interface can only have one secondary IP configuration. If the address is IPv4, the network interface can have multiple secondary IP configurations assigned to it. To learn more about how many private and public IPv4 addresses can be assigned to a network interface, see the [Azure limits](../../azure-resource-manager/management/azure-subscription-service-limits.md#azure-resource-manager-virtual-networking-limits).

+- Can have a public IPv4 or IPv6 address assigned to it. Assigning multiple IPv4 addresses to a network interface is helpful in scenarios such as:

+ - The ability to load balance one IPv6 address assigned to a network interface. To learn more about load balancing a private IPv6 address, see [Load balance IPv6 addresses](../../load-balancer/load-balancer-ipv6-overview.md).

+4. [Manually configure](virtual-network-multiple-ip-addresses-portal.md#os-config) the secondary IP addresses within the operating system (and also the primary IP address within Windows) to match what you set within Azure. Don't manually set the primary IP address in the OS network configuration on Linux, or it may not be able to connect to the Internet when the configuration is reloaded.

+5. Reload the network configuration on the guest operating system. This can be done by rebooting the system, or by running 'nmcli con down "System eth0 && nmcli con up "System eth0"' in Linux systems running NetworkManager.

+By following the previous steps, the private IP address assigned to the network interface within Azure, and within a virtual machine's operating system, remain the same. To keep track of virtual machines in your subscription that have manually set IP addresses within an operating system for, consider adding an Azure [tag](../../azure-resource-manager/management/tag-resources.md) to the virtual machines. You might use "IP address allocation: Static", for example. This way, you can easily find the virtual machines within your subscription that you've manually set the IP address for within the operating system.

+- **Private only**: Azure reserves the first four addresses in each subnet address range, and doesn't assign the addresses. Azure assigns the next available unassigned or unreserved IP address in the subnet's address range. While this is normally the next sequentially available address, there's no guarantee that the address will be the next one in the range. For example, if the subnet's address range is 10.0.0.0/16, and addresses 10.0.0.4-10.0.0.14 are already assigned (.0-.3 are reserved), the next IP address assigned is most likely 10.0.0.15. However, it could be any address between 10.0.0.10 and 10.0.0.254. If a specific Private IP address is required for a resource, you should use a static private IP address. Dynamic is the default allocation method. Once assigned, dynamic IP addresses are only released if a network interface is deleted, assigned to a different subnet within the same virtual network, or the allocation method is changed to static, and a different IP address is specified. By default, Azure assigns the previous dynamically assigned address as the static address when you change the allocation method from dynamic to static.

+

+- **Private only**: You select and assign an address from the subnet's address range. The address you assign can be any address within the subnet address range outside one of the first four addresses in the subnet's address range and not currently assigned to an existing resource in the subnet. Static addresses are only released if a network interface is deleted. If you change the allocation method to static, Azure dynamically assigns the previously assigned dynamic IP address as the static address, even if the address isn't the next available address in the subnet's address range. The address also changes if the network interface is assigned to a different subnet within the same virtual network. In order to assign the network interface to a different subnet, you must first change the allocation method from static to dynamic. Once the network interface is assigned to a different subnet, you can change the allocation method back to static, and assign an IP address from the new subnet's address range.

+Azure virtual networks provide DHCP service and DNS to VMs. Client/server DHCP traffic (source port UDP/68, destination port UDP/67) is not supported in a virtual network.