+* Schema discovery is being used on certain gallery applications. Schema discovery is the sole method to add more attributes to the schema of an existing gallery SCIM application. Schema discovery isn't currently supported on custom non-gallery SCIM application.

+|OAuth authorization code grant|Access tokens have a shorter life than passwords, and have an automated refresh mechanism that long-lived bearer tokens don't have. A real user must be present during initial authorization, adding a level of accountability. |Requires a user to be present. If the user leaves the organization, the token is invalid, and authorization will need to be completed again.|Supported for gallery apps, but not non-gallery apps. However, you can provide an access token in the UI as the secret token for short term testing purposes. Support for OAuth code grant on non-gallery is in our backlog, in addition to support for configurable auth / token URLs on the gallery app.|

+|OAuth client credentials grant|Access tokens have a shorter life than passwords, and have an automated refresh mechanism that long-lived bearer tokens don't have. Both the authorization code grant and the client credentials grant create the same type of access token, so moving between these methods is transparent to the API. Provisioning can be automated, and new tokens can be silently requested without user interaction. ||Supported for gallery apps, but not non-gallery apps. However, you can provide an access token in the UI as the secret token for short term testing purposes. Support for OAuth client credentials grant on non-gallery is in our backlog.|

+> Users must have previously registered for multifactor authentication before triggering the user risk policy.

+On Azure AD joined and hybrid Azure AD joined devices, unlocking the device, or signing in interactively will only refresh the Primary Refresh Token (PRT) every 4 hours. The last refresh timestamp recorded for PRT compared with the current timestamp must be within the time allotted in SIF policy for PRT to satisfy SIF and grant access to a PRT that has an existing MFA claim. On [Azure AD registered devices](/azure/active-directory/devices/concept-azure-ad-register), unlock/sign-in would not satisfy the SIF policy because the user is not accessing an Azure AD registered device via an Azure AD account. However, the [Azure AD WAM](../develop/scenario-desktop-acquire-token-wam.md) plugin can refresh a PRT during native application authentication using WAM.

+* If you're ready to configure Conditional Access policies for your environment, see the article [Plan a Conditional Access deployment](plan-conditional-access.md).

+It isn't possible to conduct a dry run using the Backup Authentication Service or simulate the result of a policy with resilience defaults enabled or disabled at this time. Azure AD will conduct monthly exercises using the Backup Authentication Service. The sign-in logs will display if the Backup Authentication Service was used to issue the access token. In **Azure portal** > **Monitoring** > **Sign-in Logs** blade, you can add the filter "Token issuer type == Azure AD Backup Auth" to display the logs processed by Azure AD Backup Authentication service.

+> In directories without appropriate licenses, existing Conditional Access policies for workload identities will continue to function, but can't be modified. For more information see [Microsoft Entra Workload Identities](https://www.microsoft.com/security/business/identity-access/microsoft-entra-workload-identities#office-StandaloneSKU-k3hubfz).  

+> [!NOTE]

+> Manually elevating a user to become a local administrator on the VM by adding the user to a member of the local administrators group or by running `net localgroup administrators /add "AzureAD\UserUpn"` command is not supported. You need to use Azure roles above to authorize VM login.

+Microsoft Azure [national clouds](../develop/authentication-national-cloud.md) are physically isolated instances of Azure. B2B collaboration isn't enabled by default across national cloud boundaries, but you can use Microsoft cloud settings to establish mutual B2B collaboration between the following Microsoft Azure clouds:

+To set up B2B collaboration between tenants in different clouds, both tenants need to configure their Microsoft cloud settings to enable collaboration with the other cloud. Then each tenant must configure inbound and outbound cross-tenant access with the tenant in the other cloud. For details, see [Microsoft cloud settings](cross-cloud-settings.md).

+# Configure Microsoft cloud settings for B2B collaboration

+1. Select **Microsoft cloud settings**.

+> Automatic redemption is currently in preview.

+> [Cross-tenant synchronization](../multi-tenant-organizations/cross-tenant-synchronization-overview.md) is currently in preview.

+For configuration steps, see [Configure Microsoft cloud settings for B2B collaboration](cross-cloud-settings.md).

+### Microsoft cloud settings for B2B collaboration

+If you need to collaborate with an Azure AD organization that's outside of the Azure US Government cloud, you can use [Microsoft cloud settings](cross-cloud-settings.md) to enable B2B collaboration.

+- Use [Microsoft cloud settings](cross-cloud-settings.md) to establish mutual B2B collaboration between the Microsoft Azure global cloud and [Microsoft Azure Government](../../azure-government/index.yml) or [Microsoft Azure China 21Vianet](/azure/china).

+ Title: Convert local guest accounts to Azure AD B2B guest accounts

+description: Learn to convert local guests into Azure AD B2B guest accounts by identifying apps and local guest accounts, migration, and more.

+# Convert local guest accounts to Azure Active Directory B2B guest accounts

+With Azure Active Directory (Azure AD B2B), external users collaborate with their identities. Although organizations can issue local usernames and passwords to external users, this approach isn't recommended. Azure AD B2B has improved security, lower cost, and less complexity, compared to creating local accounts. In addition, if your organization issues local credentials that external users manage, you can use Azure AD B2B instead. Use the guidance in this document to make the transition.

+Learn more: [Plan an Azure AD B2B collaboration deployment](secure-external-access-resources.md)

+Before migrating local accounts to Azure AD B2B, confirm the applications and workloads external users can access. For example, for applications hosted on-premises, validate the application is integrated with Azure AD. On-premises applications are a good reason to create local accounts.

+Learn more: [Grant B2B users in Azure AD access to your on-premises applications](../external-identities/hybrid-cloud-to-on-premises.md)

+We recommend that external-facing applications have single-sign on (SSO) and provisioning integrated with Azure AD for the best end user experience.

+Identify the accounts to be migrated to Azure AD B2B. External identities in Active Directory are identifiable with an attribute-value pair. For example, making ExtensionAttribute15 = `External` for external users. If these users are set up with Azure AD Connect or Cloud Sync, configure synced external users to have the `UserType` attributes set to `Guest`. If the users are set up as cloud-only accounts, you can modify user attributes. Primarily, identify users to convert to B2B.

+Identify user identities or external emails. Confirm that the local account (v-lakshmi@contoso.com) is a user with the home identity and email address: lakshmi@fabrikam.com. To identify home identities:

+- The external user's sponsor provides the information

+- The external user provides the information

+- Refer to an internal database, if the information is known and stored

+After mapping external local accounts to identities, add external identities or email to the user.mail attribute on local accounts.

+Notify external users about migration timing. Communicate expectations, such as when external users must stop using a current password to enable authenticate by home and corporate credentials. Communications can include email campaigns and announcements.

+After local accounts have user.mail attributes populated with the external identity and email, convert local accounts to Azure AD B2B by inviting the local account. You can use PowerShell or the Microsoft Graph API.

+Learn more: [Invite internal users to B2B collaboration](../external-identities/invite-internal-users.md)

+## Post-migration considerations

+If external user local accounts were synced from on-premises, reduce their on-premises footprint and use B2B guest accounts. You can:

+- Transition external user local accounts to Azure AD B2B and stop creating local accounts

+ - Invite external users in Azure AD

+- Randomize external user's local-account passwords to prevent authentication to on-premises resources

+ - This action ensures authentication and user lifecycle is connected to the external user home identity

+1. [Convert local guest accounts to B2B](10-secure-local-guest.md) (YouΓÇÖre here)

+ Title: Manage external access to resources with Conditional Access

+description: Learn to use Conditional Access policies to secure external access to resources.

+# Manage external access to resources with Conditional Access policies

+Conditional Access interprets signals, enforces policies, and determines if a user is granted access to resources. In this article, learn about applying Conditional Access policies to external users. The article assumes you might not have access to entitlement management, a feature you can use with Conditional Access.

+Learn more:

+* [What is Conditional Access?](../conditional-access/overview.md)

+* [Plan a Conditional Access deployment](../conditional-access/plan-conditional-access.md)

+* [What is entitlement management?](../governance/entitlement-management-overview.md)

+The following diagram illustrates signals to Conditional Access that trigger access processes.

+ 

+## Align a security plan with Conditional Access policies

+In the third article, in the set of 10 articles, there's guidance on creating a security plan. Use that plan to help create Conditional Access policies for external access. Part of the security plan includes:

+* Grouped applications and resources for simplified access

+* Sign-in requirements for external users

+> Create internal and external user test accounts to test policies before applying them.

+See article three, [Create a security plan for external access to resources](3-secure-access-plan.md)

+The following sections are best practices for governing external access with Conditional Access policies.

+### Entitlement management or groups

+If you canΓÇÖt use connected organizations in entitlement management, create an Azure AD security group, or Microsoft 365 Group for partner organizations. Assign users from that partner to the group. You can use the groups in Conditional Access policies.

+Learn more:

+* [What is entitlement management?](../governance/entitlement-management-overview.md)

+* [Manage Azure Active Directory groups and group membership](how-to-manage-groups.md)

+* [Overview of Microsoft 365 Groups for administrators](/microsoft-365/admin/create-groups/office-365-groups?view=o365-worldwide&preserve-view=true)

+### Conditional Access policy creation

+Create as few Conditional Access policies as possible. For applications that have the same access requirements, add them to the same policy.

+Conditional Access policies apply to a maximum of 250 applications. If more than 250 applications have the same access requirement, create duplicate policies. For instance, Policy A applies to apps 1-250, Policy B applies to apps 251-500, etc.

+### Naming convention

+Use a naming convention that clarifies policy purpose. External access examples are:

+* ExternalAccess_actiontaken_AppGroup

+* ExternalAccess_Block_FinanceApps

+## Block external users from resources

+You can block external users from accessing resources with Conditional Access policies.

+1. Sign in to the Azure portal as a Conditional Access Administrator, Security Administrator, or Global Administrator.

+2. Browse to **Azure Active Directory** > **Security** > **Conditional Access**.

+3. Select **New policy**.

+4. Enter a policy a name.

+5. Under **Assignments**, select **Users or workload identities**.

+6. Under **Include**, select **All guests and external users**.

+7. Under **Exclude**, select **Users and groups**.

+8. Select emergency access accounts.

+9. Select **Done**.

+10. Under **Cloud apps or actions** > **Include**, select **All cloud apps**.

+11. Under **Exclude**, select applications you want to exclude.

+12. Under **Access controls** > **Grant**, select **Block access**.

+13. Select **Select**.

+14. Select **Enable policy** to **Report-only**.

+15. Select **Create**.

+> [!NOTE]

+> You can confirm settings in **report only** mode. See, Configure a Conditional Access policy in repory-only mode, in [Conditional Access insights and reporting](../conditional-access/howto-conditional-access-insights-reporting.md).

+Learn more: [Manage emergency access accounts in Azure AD](../roles/security-emergency-access.md)

+### Allow external access to specific external users

+There are scenarios when it's necessary to allow access for a small, specific group.

+Before you begin, we recommend you create a security group, which contains external users who access resources. See, [Quickstart: Create a group with members and view all groups and members in Azure AD](active-directory-groups-view-azure-portal.md).

+1. Sign in to the Azure portal as a Conditional Access Administrator, Security Administrator, or Global Administrator.

+2. Browse to **Azure Active Directory** > **Security** > **Conditional Access**.

+3. Select **New policy**.

+4. Enter a policy name.

+5. Under **Assignments**, select **Users or workload identities**.

+6. Under **Include**, select **All guests and external users**.

+7. Under **Exclude**, select **Users and groups**

+8. Select emergency access accounts.

+9. Select the external users security group.

+10. Select **Done**.

+11. Under **Cloud apps or actions** > **Include**, select **All cloud apps**.

+12. Under **Exclude**, select applications you want to exclude.

+13. Under **Access controls** > **Grant**, select **Block access**.

+14. Select **Select**.

+15. Select **Create**.

+> [!NOTE]

+> You can confirm settings in **report only** mode. See, Configure a Conditional Access policy in repory-only mode, in [Conditional Access insights and reporting](../conditional-access/howto-conditional-access-insights-reporting.md).

+Learn more: [Manage emergency access accounts in Azure AD](../roles/security-emergency-access.md)

+### Service provider access

+Conditional Access policies for external users might interfere with service provider access, for example granular delegated administrate privileges.

+Learn more: [Introduction to granular delegated admin privileges (GDAP)](/partner-center/gdap-introduction)

+## Conditional Access templates

+Conditional Access templates are a convenient method to deploy new policies aligned with Microsoft recommendations. These templates provide protection aligned with commonly used policies across various customer types and locations.

+Learn more: [Conditional Access templates (Preview)](../conditional-access/concept-conditional-access-policy-common.md)

+> [!NOTE]

+> Identity Protection detects risk on single tenant, third party SaaS, and multi-tenant apps. Managed Identities are not currently in scope.

+ > [!Note]

+ > **We don't support sending emails to users in group-assigned roles.**

+Organizations can choose to block access when risk is detected. Blocking sometimes stops legitimate users from doing what they need to. A better solution is to allow self-remediation using Azure AD multifactor authentication (MFA) and secure password change.

+> Users must register for Azure AD MFA before they face a situation requiring remediation. For hybrid users that are synced from on-premises to cloud, password writeback must have been enabled on them. Users not registered are blocked and require administrator intervention.

+> Password change (I know my password and want to change it to something new) outside of the risky user policy remediation flow does not meet the requirement for secure password change.

+ - Require a secure password change when user risk level is **High**. Azure AD MFA is required before the user can create a new password with password writeback to remediate their risk.

+Requiring access control when risk level is low will introduce more user interrupts. Choosing to block access rather than allowing self-remediation options, like secure password change and multifactor authentication, will impact your users and administrators. Weigh these choices when configuring your policies.

+Custom logos must be exactly 215x215 pixels in size and be in the PNG format. You should use a solid color background with no transparency in your application logo. The logo file size can't be over 100 KB.

+1. Under **Manage**, select **Admin consent settings**. Under **Admin consent requests**, select **Yes** for **Users can request admin consent to apps they are unable to consent to** .

+https://login.microsoftonline.com/{organization}/adminconsent?client_id={client-id}

+- `{organization}` is the tenant ID or any verified domain name of the tenant you want to consent the application in. You can use the value `common`, which will cause the consent to happen in the home tenant of the user you sign in with.

+$spApplicationPermissions = Get-AzureADServiceAppRoleAssignedTo-ObjectId $sp.ObjectId -All $true | Where-Object { $_.PrincipalType -eq "ServicePrincipal" }

+1. Select **Security > Identity providers**.

+1. Select your Identity provider directory.

+1. Select **Set up user provisioning**.

+1. Copy the values for **SCIM base URL** and **API key**. You'll need them when you configure Azure.

+1. Save your **SCIM configuration**.

+

+1. Log in to the Axis Management Console.

+Add Atmos from the Azure AD application gallery to start managing provisioning to Atmos. If you have previously setup Atmos for SSO, you can use the same application. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+ Title: Azure Active Directory SSO integration with HawkeyeBSB

+description: Learn how to configure single sign-on between Azure Active Directory and HawkeyeBSB.

+# Azure Active Directory SSO integration with HawkeyeBSB

+In this article, you'll learn how to integrate HawkeyeBSB with Azure Active Directory (Azure AD). HawkeyeBSB was developed by Redbridge Debt & Treasury Advisory to help Clients manage their bank fees. When you integrate HawkeyeBSB with Azure AD, you can:

+* Control in Azure AD who has access to HawkeyeBSB.

+* Enable your users to be automatically signed-in to HawkeyeBSB with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+You'll configure and test Azure AD single sign-on for HawkeyeBSB in a test environment. HawkeyeBSB supports both **SP** and **IDP** initiated single sign-on.

+## Prerequisites

+To integrate Azure Active Directory with HawkeyeBSB, you need:

+* An Azure AD user account. If you don't already have one, you can [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+* One of the following roles: Global Administrator, Cloud Application Administrator, Application Administrator, or owner of the service principal.

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* HawkeyeBSB single sign-on (SSO) enabled subscription.

+## Add application and assign a test user

+Before you begin the process of configuring single sign-on, you need to add the HawkeyeBSB application from the Azure AD gallery. You need a test user account to assign to the application and test the single sign-on configuration.

+### Add HawkeyeBSB from the Azure AD gallery

+Add HawkeyeBSB from the Azure AD application gallery to configure single sign-on with HawkeyeBSB. For more information on how to add application from the gallery, see the [Quickstart: Add application from the gallery](../manage-apps/add-application-portal.md).

+### Create and assign Azure AD test user

+Follow the guidelines in the [create and assign a user account](../manage-apps/add-application-portal-assign-users.md) article to create a test user account in the Azure portal called B.Simon.

+Alternatively, you can also use the [Enterprise App Configuration Wizard](https://portal.office.com/AdminPortal/home?Q=Docs#/azureadappintegration). In this wizard, you can add an application to your tenant, add users/groups to the app, and assign roles. The wizard also provides a link to the single sign-on configuration pane in the Azure portal. [Learn more about Microsoft 365 wizards.](/microsoft-365/admin/misc/azure-ad-setup-guides).

+## Configure Azure AD SSO

+Complete the following steps to enable Azure AD single sign-on in the Azure portal.

+1. In the Azure portal, on the **HawkeyeBSB** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, select the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ a. In the **Identifier** textbox, type a URL using the following pattern:

+ `https://hawkeye.redbridgeanalytics.com/sso/saml/metadata/<uniqueSlugPerCustomer>`

+ b. In the **Reply URL** textbox, type a URL using the following pattern:

+ `https://hawkeye.redbridgeanalytics.com/sso/saml/acs/<uniqueSlugPerCustomer>`

+1. If you wish to configure the application in **SP** initiated mode, then perform the following step:

+ In the **Sign on URL** textbox, type a URL using the following pattern:

+ `https://hawkeye.redbridgeanalytics.com/sso/saml/login/<uniqueSlugPerCustomer>`

+ > [!NOTE]

+ > These values are not real. Update these values with the actual Identifier, Reply URL and Sign on URL. Contact [HawkeyeBSB Client support team](mailto:casemanagement@redbridgedta.com) to get these values. You can also refer to the patterns shown in the **Basic SAML Configuration** section in the Azure portal.

+1. On the **Set-up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Federation Metadata XML** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up HawkeyeBSB** section, copy the appropriate URL(s) based on your requirement.

+ 

+## Configure HawkeyeBSB SSO

+To configure single sign-on on **HawkeyeBSB** side, you need to send the downloaded **Federation Metadata XML** and appropriate copied URLs from Azure portal to [HawkeyeBSB support team](mailto:casemanagement@redbridgedta.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create HawkeyeBSB test user

+In this section, you create a user called Britta Simon at HawkeyeBSB. Work with [HawkeyeBSB support team](mailto:casemanagement@redbridgedta.com) to add the users in the HawkeyeBSB platform. Users must be created and activated before you use single sign-on.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+#### SP initiated:

+1. Click on **Test this application** in Azure portal. This will redirect to HawkeyeBSB Sign-on URL where you can initiate the login flow.

+1. Go to HawkeyeBSB Sign-on URL directly and initiate the login flow from there.

+#### IDP initiated:

+1. Click on **Test this application** in Azure portal and you should be automatically signed in to the HawkeyeBSB for which you set up the SSO.

+1. You can also use Microsoft My Apps to test the application in any mode. When you click the HawkeyeBSB tile in the My Apps, if configured in SP mode you would be redirected to the application sign-on page for initiating the login flow and if configured in IDP mode, you should be automatically signed in to the HawkeyeBSB for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Additional resources

+* [What is single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+* [Plan a single sign-on deployment](../manage-apps/plan-sso-deployment.md).

+## Next steps

+Once you configure HawkeyeBSB you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+* An introdus subscription, that includes single sign-on (SSO)

+* A valid introdus API Token.

+Add introDus Pre and Onboarding Platform from the Azure AD application gallery to start managing provisioning to introDus Pre and Onboarding Platform. If you have previously setup introDus Pre and Onboarding Platform for SSO you can use the same application. However it's recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+9. Review the user attributes that are synchronized from Azure AD to introDus Pre and Onboarding Platform in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in introDus Pre and Onboarding Platform for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you'll need to ensure that the introDus Pre and Onboarding Platform API supports filtering users based on that attribute. Select the **Save** button to commit any changes.

+13. When you're ready to provision, click **Save**.

+ Title: Azure Active Directory SSO integration with Parallels Desktop

+description: Learn how to configure single sign-on between Azure Active Directory and Parallels Desktop.

+# Azure Active Directory SSO integration with Parallels Desktop

+In this article, you'll learn how to integrate Parallels Desktop with Azure Active Directory (Azure AD). SSO/SAML authentication for employees to use Parallels Desktop. Enable your employees to sign in and activate Parallels Desktop with a corporate account. When you integrate Parallels Desktop with Azure AD, you can:

+* Control in Azure AD who has access to Parallels Desktop.

+* Enable your users to be automatically signed-in to Parallels Desktop with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+You'll configure and test Azure AD single sign-on for Parallels Desktop in a test environment. Parallels Desktop supports only **SP** initiated single sign-on.

+## Prerequisites

+To integrate Azure Active Directory with Parallels Desktop, you need:

+* An Azure AD user account. If you don't already have one, you can [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+* One of the following roles: Global Administrator, Cloud Application Administrator, Application Administrator, or owner of the service principal.

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* Parallels Desktop single sign-on (SSO) enabled subscription.

+## Add application and assign a test user

+Before you begin the process of configuring single sign-on, you need to add the Parallels Desktop application from the Azure AD gallery. You need a test user account to assign to the application and test the single sign-on configuration.

+### Add Parallels Desktop from the Azure AD gallery

+Add Parallels Desktop from the Azure AD application gallery to configure single sign-on with Parallels Desktop. For more information on how to add application from the gallery, see the [Quickstart: Add application from the gallery](../manage-apps/add-application-portal.md).

+### Create and assign Azure AD test user

+Follow the guidelines in the [create and assign a user account](../manage-apps/add-application-portal-assign-users.md) article to create a test user account in the Azure portal called B.Simon.

+Alternatively, you can also use the [Enterprise App Configuration Wizard](https://portal.office.com/AdminPortal/home?Q=Docs#/azureadappintegration). In this wizard, you can add an application to your tenant, add users/groups to the app, and assign roles. The wizard also provides a link to the single sign-on configuration pane in the Azure portal. [Learn more about Microsoft 365 wizards.](/microsoft-365/admin/misc/azure-ad-setup-guides).

+## Configure Azure AD SSO

+Complete the following steps to enable Azure AD single sign-on in the Azure portal.

+1. In the Azure portal, on the **Parallels Desktop** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, select the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ a. In the **Identifier** textbox, type a URL using the following pattern:

+ `https://account.parallels.com/<ID>`

+ b. In the **Reply URL** textbox, type a URL using the following pattern:

+ `https://account.parallels.com/webapp/sso/acs/<ID>`

+ > [!NOTE]

+ > These values are not real. Update these values with the actual Identifier and Reply URL. Please note the Identifier and Reply URL values are customer specific and should be able to specify it manually by copying it from Parallels My Account to the identity provider Azure. Contact [Parallels Desktop Client support team](mailto:parallels.desktop.sso@alludo.com) for any help. You can also refer to the patterns shown in the **Basic SAML Configuration** section in the Azure portal.

+ c. In the **Sign on URL** textbox, type the URL:-

+ `https://my.parallels.com/login?sso=1`

+1. On the **Set up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Certificate (PEM)** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up Parallels Desktop** section, copy the appropriate URL(s) based on your requirement.

+ 

+## Configure Parallels Desktop SSO

+To configure single sign-on on **Parallels Desktop** side, you need to send the downloaded **Certificate (PEM)** and appropriate copied URLs from Azure portal to [Parallels Desktop support team](mailto:parallels.desktop.sso@alludo.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create Parallels Desktop test user

+In this section, you create a user called Britta Simon at Parallels Desktop. Work with [Parallels Desktop support team](mailto:parallels.desktop.sso@alludo.com) to add the users in the Parallels Desktop platform. Users must be created and activated before you use single sign-on.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+* Click on **Test this application** in Azure portal. This will redirect to Parallels Desktop Sign-on URL where you can initiate the login flow.

+* Go to Parallels Desktop Sign-on URL directly and initiate the login flow from there.

+* You can use Microsoft My Apps. When you click the Parallels Desktop tile in the My Apps, this will redirect to Parallels Desktop Sign-on URL. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Additional resources

+* [What is single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+* [Plan a single sign-on deployment](../manage-apps/plan-sso-deployment.md).

+## Next steps

+Once you configure Parallels Desktop you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+# Automatically upgrade Azure Kubernetes Service cluster node operating system images (preview)

+| `NodeImage`|AKS will update the nodes with a newly patched VHD containing security fixes and bug fixes on a weekly cadence. The update to the new VHD is disruptive, following maintenance windows and surge settings. No extra VHD cost is incurred when choosing this option. If you use this channel, Linux [unattended upgrades][unattended-upgrades] will be disabled by default.|

+> You may need to assign a minimum of *Microsoft.Network/virtualNetworks/subnets/read* and *Microsoft.Network/virtualNetworks/subnets/join/action* permission to AKS MSI on the Azure Virtual Network resources. You can view the cluster identity with [az aks show][az-aks-show], such as `az aks show --resource-group myResourceGroup --name myAKSCluster --query "identity"`. To create a role assignment, use the [az role assignment create][az-role-assignment-create] command.

+### Specify a different subnet

+Node image upgrades can also be performed automatically, and scheduled by using planned maintenance. For more details, see [Automatically upgrade node images][auto-upgrade-node-image].

+[auto-upgrade-node-image]: auto-upgrade-node-image.md

+Node image upgrades can also be performed automatically, and scheduled by using planned maintenance. For more details, see [Automatically upgrade node images][auto-upgrade-node-image].

+[auto-upgrade-node-image]: auto-upgrade-node-image.md

+az ad app credential list --id "$SP_ID" --query "[].endDateTime" -o tsv

+SP_SECRET=$(az ad app credential reset --id "$SP_ID" --query password -o tsv)

+* Image SKUs for SGX and FIPS are not available.

+* It doesn't meet the [Federal Information Processing Standard (FIPS) 140](https://csrc.nist.gov/publications/detail/fips/140/3/final) compliance requirements and [Center for Internet Security (CIS)](https://www.cisecurity.org/) certification.

+### How can I determine the method that was used to deploy my Azure App Service?

+Let us say you take over owning an app and you wish to find out how the Azure App Service was deployed so you can make changes and deploy them. You can determine how an Azure App Service was deployed by checking the application settings. If the app was deployed using an external package URL, you will see the WEBSITE_RUN_FROM_PACKAGE setting in the application settings with a URL value. Or if it was deployed using zip deploy, you will see the WEBSITE_RUN_FROM_PACKAGE setting with a value of 1. If the app was deployed using Azure DevOps, you will see the deployment history in the Azure DevOps portal. If Azure Functions Core Tools was used, you will see the deployment history in the Azure portal.

+| US Gov Virginia | ✅ |✅ | ✅ |

+Since application gateway resources are deployed within a virtual network, Application Gateway performs a check to verify the permission on the provided virtual network resource. This validation is performed during both creation and management operations.

+You should check your [Azure role-based access control](../role-based-access-control/role-assignments-list-portal.md) to verify the users or service principals that operate application gateways have at least **Microsoft.Network/virtualNetworks/subnets/join/action** permission. Use built-in roles, such as [Network contributor](../role-based-access-control/built-in-roles.md#network-contributor), which already support this permission. If a built-in role doesn't provide the right permission, you can [create and assign a custom role](../role-based-access-control/custom-roles-portal.md). Learn more about [managing subnet permissions](../virtual-network/virtual-network-manage-subnet.md#permissions). You may have to allow sufficient time for [Azure Resource Manager cache refresh](../role-based-access-control/troubleshooting.md?tabs=bicep#symptomrole-assignment-changes-are-not-being-detected) after role assignment changes.

+#### Identifying affected users or service principals for your subscription

+By visiting Azure Advisor for your account, you can verify if your subscription has any users or service principals with insufficient permission. The details of that recommendation are as follows:

+**Title**: Update VNet permission of Application Gateway users </br>

+**Category**: Reliability </br>

+**Impact**: High </br>

+#### Using temporary Azure Feature Exposure Control (AFEC) flag

+As a temporary extension, we have introduced a subscription-level [Azure Feature Exposure Control (AFEC)](../azure-resource-manager/management/preview-features.md?tabs=azure-portal) that you can register for, until you fix the permissions for all your users and/or service principals. [Set up this flag](../azure-resource-manager/management/preview-features.md?#required-access) for your Azure subscription.

+**Name**: Microsoft.Network/DisableApplicationGatewaySubnetPermissionCheck </br>

+**Description**: Disable Application Gateway Subnet Permission Check </br>

+**ProviderNamespace**: Microsoft.Network </br>

+**EnrollmentType**: AutoApprove </br>

+> The provision to circumvent the virtual network permission check by using this feature control (AFEC) is available only for a limited period, **until 6th April 2023**. Ensure all the roles and permissions managing Application Gateways are updated by then, as there will be no further extensions. Set up this flag in your Azure subscription.

+> Start/Stop VM during off-hours, version 1 is going to retire soon by CY23 and is unavailable in the marketplace now. We recommend that you start using [version 2](../azure-functions/start-stop-vms/overview.md) which is now generally available. The new version offers all existing capabilities and provides new features, such as multi-subscription support from a single Start/Stop instance. If you have the version 1 solution already deployed, you can still use the feature, and we will provide support until retirement in CY23. The details on announcement will be shared soon.

+> Start/Stop VM during off-hours, version 1 is going to retire soon by CY23 and is unavailable in the marketplace now. We recommend that you start using [version 2](../azure-functions/start-stop-vms/overview.md), which is now generally available. The new version offers all existing capabilities and provides new features, such as multi-subscription support from a single Start/Stop instance. If you have the version 1 solution already deployed, you can still use the feature, and we will provide support until retirement in CY23. The details on announcement will be shared soon.

+- **Compatibility with the unified monitoring agent** - Compatible with the [Azure Monitor Agent (Preview)](../../azure-monitor/agents/agents-overview.md) that enhances security, reliability, and facilitates multi-homing experience to store data.

+- **Multi-homing experience** ΓÇô Provides standardization of management from one central workspace. You can [transition from Log Analytics (LA) to AMA](../../azure-monitor/agents/azure-monitor-agent-migration.md)

+so that all VMs point to a single workspace for data collection and maintenance.

+> Start/Stop VM during off-hours, version 1 is going to retire soon by CY23 and is unavailable in the marketplace now. We recommend that you start using [version 2](../../azure-functions/start-stop-vms/overview.md) which is now generally available. The new version offers all existing capabilities and provides new features, such as multi-subscription support from a single Start/Stop instance. If you have the version 1 solution already deployed, you can still use the feature, and we will provide support until retirement in CY23. The details on announcement will be shared soon.

+You can use the Bicep template to create a new Hybrid Worker group, create a new Azure Windows VM and add it to an existing Hybrid Worker Group. Learn more about [Bicep](../azure-resource-manager/bicep/overview.md).

+KB2267602 is the [Windows Defender definition update](https://www.microsoft.com/en-us/wdsi/defenderupdates). It's updated daily.

+- A Spring Boot application. If you don't have one, create a Maven project with the [Spring Initializr](https://start.spring.io/). Be sure to select **Maven Project** and, under **Dependencies**, add the **Spring Web** dependency, and then select Java version 8 or higher.

+9. Select **Configuration Explorer** > **+ Create** > **Key-value** to add the following key-value pairs:

+ | Key | Value |

+ |||

+ | /application/config.message | Hello |

+ Leave **Label** and **Content Type** empty for now.

+10. Select **Apply**.

+Now that you have an App Configuration store, you can use the Spring Cloud Azure Config starter to have your application communicate with the App Configuration store that you create.

+To install the Spring Cloud Azure Config starter module, add the following dependency to your *pom.xml* file:

+```xml

+<dependency>

+ <groupId>com.azure.spring</groupId>

+ <artifactId>azure-spring-cloud-appconfiguration-config</artifactId>

+ <version>2.11.0</version>

+</dependency>

+```

+> [!NOTE]

+> If you need to support an older version of Spring Boot, see our [old library](https://github.com/Azure/azure-sdk-for-jav).

+### Code the application

+To use the Spring Cloud Azure Config starter to have your application communicate with the App Configuration store that you create, configure the application by using the following steps.

+1. Create a new Java file named *MessageProperties.java*, and add the following lines:

+ ```java

+ import org.springframework.boot.context.properties.ConfigurationProperties;

+ @ConfigurationProperties(prefix = "config")

+ public class MessageProperties {

+ private String message;

+ public String getMessage() {

+ return message;

+ }

+ public void setMessage(String message) {

+ this.message = message;

+ }

+ }

+ ```

+1. Create a new Java file named *HelloController.java*, and add the following lines:

+ ```java

+ import org.springframework.web.bind.annotation.GetMapping;

+ import org.springframework.web.bind.annotation.RestController;

+ @RestController

+ public class HelloController {

+ private final MessageProperties properties;

+ public HelloController(MessageProperties properties) {

+ this.properties = properties;

+ }

+ @GetMapping

+ public String getMessage() {

+ return "Message: " + properties.getMessage();

+ }

+ }

+ ```

+1. In the main application Java file, add `@EnableConfigurationProperties` to enable the *MessageProperties.java* configuration properties class to take effect and register it with the Spring container.

+ ```java

+ import org.springframework.boot.context.properties.EnableConfigurationProperties;

+ @SpringBootApplication

+ @EnableConfigurationProperties(MessageProperties.class)

+ public class DemoApplication {

+ public static void main(String[] args) {

+ SpringApplication.run(DemoApplication.class, args);

+ }

+ }

+ ```

+1. Open the auto-generated unit test and update to disable Azure App Configuration, or it will try to load from the service when running unit tests.

+ ```java

+ import org.junit.jupiter.api.Test;

+ import org.springframework.boot.test.context.SpringBootTest;

+ @SpringBootTest(properties = "spring.cloud.azure.appconfiguration.enabled=false")

+ class DemoApplicationTests {

+ @Test

+ void contextLoads() {

+ }

+ }

+ ```

+1. Create a new file named *bootstrap.properties* under the resources directory of your app, and add the following line to the file.

+ ```properties

+ spring.cloud.azure.appconfiguration.stores[0].connection-string= ${APP_CONFIGURATION_CONNECTION_STRING}

+ ```

+ ```cmd

+ setx APP_CONFIGURATION_CONNECTION_STRING "connection-string-of-your-app-configuration-store"

+ ```

+ If you use Windows PowerShell, run the following command:

+ ```azurepowershell

+ $Env:APP_CONFIGURATION_CONNECTION_STRING = "connection-string-of-your-app-configuration-store"

+ ```

+ If you use macOS or Linux, run the following command:

+ ```cmd

+ export APP_CONFIGURATION_CONNECTION_STRING='connection-string-of-your-app-configuration-store'

+ ```

+### Build and run the app locally

+ ```cmd

+ mvn clean package

+ mvn spring-boot:run

+ ```

+1. After your application is running, use *curl* to test your application, for example:

+ ```cmd

+ curl -X GET http://localhost:8080/

+ ```

+ You see the message that you entered in the App Configuration store.

+ Title: 'Tutorial: Workload management in a multi-cluster environment with GitOps'

+description: This tutorial walks through typical use-cases that Platform and Application teams face on a daily basis working with Kubernetes workloads in a multi-cluster environment.

+keywords: "GitOps, Flux, Kubernetes, K8s, Azure, Arc, AKS, ci/cd, devops"

+# Tutorial: Workload management in a multi-cluster environment with GitOps

+Enterprise organizations, developing cloud native applications, face challenges to deploy, configure and promote a great variety of applications and services across a fleet of Kubernetes clusters at scale. This fleet may include Azure Kubernetes Service (AKS) clusters as well as clusters running on other public cloud providers or in on-premises data centers that are connected to Azure through the Azure Arc.

+This tutorial walks you through typical scenarios of the workload deployment and configuration in a multi-cluster Kubernetes environment. First, you deploy a sample infrastructure with a few GitHub repositories and AKS clusters. Next, you work through a set of use cases where you act as different personas working in the same environment: the Platform Team and the Application Team.

+In this tutorial, you learn how to:

+> [!div class="checklist"]

+> * Onboard a new application

+> * Schedule an application on the cluster types

+> * Promote an application across rollout environemnts

+> * Build and deploy an application

+> * Provide platform configurations

+> * Add a new cluster type to your environment

+If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

+## Prerequisites

+In order to successfully deploy the sample, you need:

+- [Azure CLI](/cli/azure/install-azure-cli).

+- [GitHub CLI](https://cli.github.com)

+- [Helm](https://helm.sh/docs/helm/helm_install/)

+- [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)

+## 1 - Deploy the sample

+To deploy the sample, run the following script:

+```bash

+mkdir kalypso && cd kalypso

+curl -fsSL -o deploy.sh https://raw.githubusercontent.com/microsoft/kalypso/main/deploy/deploy.sh

+chmod 700 deploy.sh

+./deploy.sh -c -p <prefix. e.g. kalypso> -o <github org. e.g. eedorenko> -t <github token> -l <azure-location. e.g. westus2>

+```

+This script may take 10-15 minutes to complete. After it's done, it reports the execution result in the output like this:

+```output

+Deployment is complete!

+Created repositories:

+ - https://github.com/eedorenko/kalypso-control-plane

+ - https://github.com/eedorenko/kalypso-gitops

+ - https://github.com/eedorenko/kalypso-app-src

+ - https://github.com/eedorenko/kalypso-app-gitops

+Created AKS clusters in kalypso-rg resource group:

+ - control-plane

+ - drone (Flux based workload cluster)

+ - large (ArgoCD based workload cluster)

+

+```

+> [!NOTE]

+> If something goes wrong with the deployment, you can delete the created resources with the following command:

+>

+> ```bash

+> ./deploy.sh -d -p <preix. e.g. kalypso> -o <github org. e.g. eedorenko> -t <github token> -l <azure-location. e.g. westus2>

+> ```

+### Sample overview

+This deployment script created an infrastructure, shown in the following diagram:

+There are a few Platform Team repositories:

+- [Control Plane](https://github.com/microsoft/kalypso-control-plane): Contains a platform model defined with high level abstractions such as environments, cluster types, applications and services, mapping rules and configurations, and promotion workflows.

+- [Platform GitOps](https://github.com/microsoft/kalypso-gitops): Contains final manifests that represent the topology of the fleet, such as which cluster types are available in each environment, what workloads are scheduled on them, and what platform configuration values are set.

+- [Services Source](https://github.com/microsoft/kalypso-svc-src): Contains high-level manifest templates of sample dial-tone platform services.

+- [Services GitOps](https://github.com/microsoft/kalypso-svc-gitops): Contains final manifests of sample dial-tone platform services to be deployed across the clusters.

+The infrastructure also includes a couple of the Application Team repositories:

+- [Application Source](https://github.com/microsoft/kalypso-app-src): Contains a sample application source code, including Docker files, manifest templates and CI/CD workflows.

+- [Application GitOps](https://github.com/microsoft/kalypso-app-gitops): Contains final sample application manifests to be deployed to the deployment targets.

+The script created the following Azure Kubernetes Service (AKS) clusters:

+- `control-plane` - This cluster is a management cluster that doesn't run any workloads. The `control-plane` cluster hosts [Kalypso Scheduler](https://github.com/microsoft/kalypso-scheduler) operator that transforms high level abstractions from the [Control Plane](https://github.com/microsoft/kalypso-control-plane) repository to the raw Kubernetes manifests in the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.

+- `drone` - A sample workload cluster. This cluster has the [GitOps extension](conceptual-gitops-flux2.md) installed and it uses `Flux` to reconcile manifests from the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository. For this sample, the `drone` cluster can represent an Azure Arc-enabled cluster or an AKS cluster with the Flux/GitOps extension.

+- `large` - A sample workload cluster. This cluster has `ArgoCD` installed on it to reconcile manifests from the [Platform GitOps](https://github.com/microsoft/kalypso-gitops) repository.

+### Explore Control Plane

+The `control plane` repository contains three branches: `main`, `dev` and `stage`. The `dev` and `stage` branches contain configurations that are specific for `Dev` and `Stage` environments. On the other hand, the `main` branch doesn't represent any specific environment. The content of the `main` branch is common and used by all environments in the fleet. Any change to the `main` branch is a subject to be promoted across environments. For example, a new application or a new template can be promoted to the `Stage` environment only after successful testing on the `Dev` environment.

+The `main` branch:

+|Folder|Description|

+||--|

+|.github/workflows| Contains GitHub workflows that implement the promotional flow.|

+|.environments| Contains a list of environments with pointers to the branches with the environment configurations.|

+|templates| Contains manifest templates for various reconcilers (for example, Flux and ArgoCD) and a template for the workload namespace.|

+|workloads| Contains a list of onboarded applications and services with pointers to the corresponding GitOps repositories.|

+The `dev` and `stage` branches:

+|Item|Description|

+|-|--|

+|cluster-types| Contains a list of available cluster types in the environment. The cluster types are grouped in custom subfolders. Each cluster type is marked with a set of labels. It specifies a reconciler type that it uses to fetch the manifests from GitOps repositories. The subfolders also contain a number of config maps with the platform configuration values available on the cluster types.|

+|configs/dev-config.yaml| Contains config maps with the platform configuration values applicable for all cluster types in the environment.|

+|scheduling| Contains scheduling policies that map workload deployment targets to the cluster types in the environment.|

+|base-repo.yaml| A pointer to the place in the `Control Plane` repository (`main`) from where the scheduler should take templates and workload registrations.|

+|gitops-repo.yaml| A pointer to the place in the `Platform GitOps` repository to where the scheduler should PR generated manifests.|

+> [!TIP]

+> The folder structure in the `Control Plane` repository doesn't really matter. This tutorial provides a sample of how you can organize files in the repository, but feel free to do it in your own preferred way. The scheduler is interested in the content of the files, rather than where the files are located.

+## 2 - Platform Team: Onboard a new application

+The Application Team runs their software development lifecycle. They build their application and promote it across environments. They're not aware of what cluster types are available in the fleet and where their application will be deployed. But they do know that they want to deploy their application in `Dev` environment for functional and performance testing and in `Stage` environment for UAT testing.

+

+The Application Team describes this intention in the [workload](https://github.com/microsoft/kalypso-app-src/blob/main/workload/workload.yaml) file in the [Application Source](https://github.com/microsoft/kalypso-app-src) repository:

+```yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: Workload

+metadata:

+ name: hello-world-app

+ labels:

+ type: application

+ family: force

+spec:

+ deploymentTargets:

+ - name: functional-test

+ labels:

+ purpose: functional-test

+ edge: "true"

+ environment: dev

+ manifests:

+ repo: https://github.com/microsoft/kalypso-app-gitops

+ branch: dev

+ path: ./functional-test

+ - name: performance-test

+ labels:

+ purpose: performance-test

+ edge: "false"

+ environment: dev

+ manifests:

+ repo: https://github.com/microsoft/kalypso-app-gitops

+ branch: dev

+ path: ./performance-test

+ - name: uat-test

+ labels:

+ purpose: uat-test

+ environment: stage

+ manifests:

+ repo: https://github.com/microsoft/kalypso-app-gitops

+ branch: stage

+ path: ./uat-test

+```

+This file contains a list of three deployment targets. These targets are marked with custom labels and point to the folders in [Application GitOps](https://github.com/microsoft/kalypso-app-gitops) repository where the Application Team generates application manifests for each deployment target.

+With this file, Application Team requests Kubernetes compute resources from the Platform Team. In response, the Platform Team must register the application in the [Control Plane](https://github.com/microsoft/kalypso-control-plane) repo.

+

+To register the application, open a terminal and use the following script:

+```bash

+export org=<github org>

+export prefix=<prefix>

+# clone the control-plane repo

+git clone https://github.com/$org/$prefix-control-plane control-plane

+cd control-plane

+# create workload registration file

+cat <<EOF >workloads/hello-world-app.yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: WorkloadRegistration

+metadata:

+ name: hello-world-app

+ labels:

+ type: application

+spec:

+ workload:

+ repo: https://github.com/$org/$prefix-app-src

+ branch: main

+ path: workload/

+ workspace: kaizen-app-team

+EOF

+git add .

+git commit -m 'workload registration'

+git push

+```

+> [!NOTE]

+> For simplicity, this tutorial pushes changes directly to `main`. In practice, you'd create a pull request to submit the changes.

+With that in place, the application is onboarded in the control plane. But the control plane still doesn't know how to map the application deployment targets to the cluster types in the fleet.

+### Define application scheduling policy on Dev

+The Platform Team must define how the application deployment targets will be scheduled on cluster types in the `Dev` environment. To do this, submit scheduling policies for the `functional-test` and `performance-test` deployment targets with the following script:

+```bash

+# Switch to dev branch (representing Dev environemnt) in the control-plane folder

+git checkout dev

+mkdir -p scheduling/kaizen

+# Create a scheduling policy for the functional-test deployment target

+cat <<EOF >scheduling/kaizen/functional-test-policy.yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: SchedulingPolicy

+metadata:

+ name: functional-test-policy

+spec:

+ deploymentTargetSelector:

+ workspace: kaizen-app-team

+ labelSelector:

+ matchLabels:

+ purpose: functional-test

+ edge: "true"

+ clusterTypeSelector:

+ labelSelector:

+ matchLabels:

+ restricted: "true"

+ edge: "true"

+EOF

+# Create a scheduling policy for the performance-test deployment target

+cat <<EOF >scheduling/kaizen/performance-test-policy.yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: SchedulingPolicy

+metadata:

+ name: performance-test-policy

+spec:

+ deploymentTargetSelector:

+ workspace: kaizen-app-team

+ labelSelector:

+ matchLabels:

+ purpose: performance-test

+ edge: "false"

+ clusterTypeSelector:

+ labelSelector:

+ matchLabels:

+ size: large

+EOF

+git add .

+git commit -m 'application scheduling policies'

+git config pull.rebase false

+git pull --no-edit

+git push

+```

+The first policy states that all deployment targets from the `kaizen-app-team` workspace, marked with labels `purpose: functional-test` and `edge: "true"` should be scheduled on all environment cluster types that are marked with label `restricted: "true"`. You can treat a workspace as a group of applications produced by an application team.

+The second policy states that all deployment targets from the `kaizen-app-team` workspace, marked with labels `purpose: performance-test` and `edge: "false"` should be scheduled on all environment cluster types that are marked with label `size: "large"`.

+This push to the `dev` branch triggers the scheduling process and creates a PR to the `dev` branch in the `Platform GitOps` repository:

+Besides `Promoted_Commit_id`, which is just tracking information for the promotion CD flow, the PR contains assignment manifests. The `functional-test` deployment target is assigned to the `drone` cluster type, and the `performance-test` deployment target is assigned to the `large` cluster type. Those manifests will land in `drone` and `large` folders that contain all assignments to these cluster types in the `Dev` environment.

+

+The `Dev` environment also includes `command-center` and `small` cluster types:

+ :::image type="content" source="media/tutorial-workload-management/dev-cluster-types.png" alt-text="Screenshot showing cluster types in the Dev environment.":::

+However, only the `drone` and `large` cluster types were selected by the scheduling policies that you defined.

+### Understand deployment target assignment manifests

+Before you continue, take a closer look at the generated assignment manifests for the `functional-test` deployment target. There are `namespace.yaml`, `config.yaml` and `reconciler.yaml` manifest files.

+`namespace.yaml` defines a namespace that will be created on any `drone` cluster where the `hello-world` application runs.

+

+```yaml

+apiVersion: v1

+kind: Namespace

+metadata:

+ labels:

+ deploymentTarget: hello-world-app-functional-test

+ environment: dev

+ someLabel: some-value

+ workload: hello-world-app

+ workspace: kaizen-app-team

+ name: dev-kaizen-app-team-hello-world-app-functional-test

+```

+`config.yaml` contains all platform configuration values available on any `drone` cluster that the application can use in the `Dev` environment.

+

+```yaml

+apiVersion: v1

+kind: ConfigMap

+metadata:

+ name: platform-config

+ namespace: dev-kaizen-app-team-hello-world-app-functional-test

+data:

+ CLUSTER_NAME: Drone

+ DATABASE_URL: mysql://restricted-host:3306/mysqlrty123

+ ENVIRONMENT: Dev

+ REGION: East US

+ SOME_COMMON_ENVIRONMENT_VARIABLE: "false"

+```

+`reconciler.yaml` contains Flux resources that a `drone` cluster uses to fetch application manifests, prepared by the Application Team for the `functional-test` deployment target.

+

+```yaml

+apiVersion: source.toolkit.fluxcd.io/v1beta2

+kind: GitRepository

+metadata:

+ name: hello-world-app-functional-test

+ namespace: flux-system

+spec:

+ interval: 30s

+ ref:

+ branch: dev

+ secretRef:

+ name: repo-secret

+ url: https://github.com/<github org>/<prefix>-app-gitops

+apiVersion: kustomize.toolkit.fluxcd.io/v1beta2

+kind: Kustomization

+metadata:

+ name: hello-world-app-functional-test

+ namespace: flux-system

+spec:

+ interval: 30s

+ path: ./functional-test

+ prune: true

+ sourceRef:

+ kind: GitRepository

+ name: hello-world-app-functional-test

+ targetNamespace: dev-kaizen-app-team-hello-world-app-functional-test

+```

+> [!NOTE]

+> The `control plane` defines that the `drone` cluster type uses `Flux` to reconcile manifests from the application GitOps repositories. The `large` cluster type, on the other hand, reconciles manifests with `ArgoCD`. Therefore `reconciler.yaml` for the `performance-test` deployment target will look differently and contain `ArgoCD` resources.

+### Promote application to Stage

+Once you approve and merge the PR to the `Platform GitOps` repository, the `drone` and `large` AKS clusters that represent corresponding cluster types start fetching the assignment manifests. The `drone` cluster has [GitOps extension](conceptual-gitops-flux2.md) installed, pointing to the `Platform GitOps` repository. It reports its `compliance` status to Azure Resource Graph:

+The PR merging event starts a GitHub workflow `checkpromote` in the `control plane` repository. This workflow waits until all clusters with the [GitOps extension](conceptual-gitops-flux2.md) installed that are looking at the `dev` branch in the `Platform GitOps` repository are compliant with the PR commit. In this tutorial, the only such cluster is `drone`.

+Once the `checkpromote` is successful, it starts the `cd` workflow that promotes the change (application registration) to the `Stage` environment. For better visibility, it also updates the git commit status in the `control plane` repository:

+

+ :::image type="content" source="media/tutorial-workload-management/dev-git-commit-status.png" alt-text="Screenshot showing git commit status deploying to dev.":::

+> [!NOTE]

+> If the `drone` cluster fails to reconcile the assignment manifests for any reason, the promotion flow will fail. The commit status will be marked as failed, and the application registration will not be promoted to the `Stage` environment.

+Next, configure a scheduling policy for the `uat-test` deployment target in the stage environment:

+```bash

+# Switch to stage branch (representing Stage environemnt) in the control-plane folder

+git checkout stage

+mkdir -p scheduling/kaizen

+# Create a scheduling policy for the uat-test deployment target

+cat <<EOF >scheduling/kaizen/uat-test-policy.yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: SchedulingPolicy

+metadata:

+ name: uat-test-policy

+spec:

+ deploymentTargetSelector:

+ workspace: kaizen-app-team

+ labelSelector:

+ matchLabels:

+ purpose: uat-test

+ clusterTypeSelector:

+ labelSelector: {}

+EOF

+git add .

+git commit -m 'application scheduling policies'

+git config pull.rebase false

+git pull --no-edit

+git push

+```

+The policy states that all deployment targets from the `kaizen-app-team` workspace marked with labels `purpose: uat-test` should be scheduled on all cluster types defined in the environment.

+Pushing this policy to the `stage` branch triggers the scheduling process, which creates a PR with the assignment manifests to the `Platform GitOps` repository, similar to those for the `Dev` environment.

+As in the case with the `Dev` environment, after reviewing and merging the PR to the `Platform GitOps` repository, the `checkpromote` workflow in the `control plane` repository waits until clusters with the [GitOps extension](conceptual-gitops-flux2.md) (`drone`) reconcile the assignment manifests.

+ :::image type="content" source="media/tutorial-workload-management/check-promote-to-stage.png" alt-text="Screenshot showing promotion to stage.":::

+On successful execution, the commit status is updated.

+## 3 - Application Dev Team: Build and deploy application

+The Application Team regularly submits pull requests to the `main` branch in the `Application Source` repository. Once a PR is merged to `main`, it starts a CI/CD workflow. In this tutorial, the workflow will be started manually.

+

+ Go to the `Application Source` repository in GitHub. On the `Actions` tab, select `Run workflow`.

+The workflow performs the following actions:

+- Builds the application Docker image and pushes it to the GitHub repository package.

+- Generates manifests for the `functional-test` and `performance-test` deployment targets. It uses configuration values from the `dev-configs` branch. The generated manifests are added to a pull request and auto-merged in the `dev` branch.

+- Generates manifests for the `uat-test` deployment target. It uses configuration values from the `stage-configs` branch.

+The generated manifests are added to a pull request to the `stage` branch waiting for approval:

+To test the application manually on the `Dev` environment before approving the PR to the `Stage` environment, first verify how the `functional-test` application instance works on the `drone` cluster:

+```bash

+kubectl port-forward svc/hello-world-service -n dev-kaizen-app-team-hello-world-app-functional-test 9090:9090 --context=drone

+# output:

+# Forwarding from 127.0.0.1:9090 -> 9090

+# Forwarding from [::1]:9090 -> 9090

+```

+While this command is running, open `localhost:9090` in your browser. You'll see the following greeting page:

+The next step is to check how the `performance-test` instance works on the `large` cluster:

+```bash

+kubectl port-forward svc/hello-world-service -n dev-kaizen-app-team-hello-world-app-performance-test 8080:8080 --context=large

+# output:

+# Forwarding from 127.0.0.1:8080 -> 8080

+# Forwarding from [::1]:8080 -> 8080

+```

+This time, use `8080` port and open `localhost:8080` in your browser.

+Once you're satisfied with the `Dev` environment, approve and merge the PR to the `Stage` environment. After that, test the `uat-test` application instance in the `Stage` environment on both clusters.

+Run the following command for the `drone` cluster and open `localhost:8001` in your browser:

+

+```bash

+kubectl port-forward svc/hello-world-service -n stage-kaizen-app-team-hello-world-app-uat-test 8001:8000 --context=drone

+```

+Run the following command for the `large` cluster and open `localhost:8002` in your browser:

+ ```bash

+kubectl port-forward svc/hello-world-service -n stage-kaizen-app-team-hello-world-app-uat-test 8002:8000 --context=large

+```

+> [!NOTE]

+> It may take up to three minutes to reconcile the changes from the application GitOps repository on the `large` cluster.

+The application instance on the `large` cluster shows the following greeting page:

+ :::image type="content" source="media/tutorial-workload-management/stage-greeting-page.png" alt-text="Screenshot showing the greeting page on stage.":::

+## 4 - Platform Team: Provide platform configurations

+Applications in the fleet grab the data from the very same database in both `Dev` and `Stage` environments. Let's change it and configure `west-us` clusters to provide a different database url for the applications working in the `Stage` environment:

+```bash

+# Switch to stage branch (representing Stage environemnt) in the control-plane folder

+git checkout stage

+# Update a config map with the configurations for west-us clusters

+cat <<EOF >cluster-types/west-us/west-us-config.yaml

+apiVersion: v1

+kind: ConfigMap

+metadata:

+ name: west-us-config

+ labels:

+ platform-config: "true"

+ region: west-us

+data:

+ REGION: West US

+ DATABASE_URL: mysql://west-stage:8806/mysql2

+EOF

+git add .

+git commit -m 'database url configuration'

+git config pull.rebase false

+git pull --no-edit

+git push

+```

+The scheduler scans all config maps in the environment and collects values for each cluster type based on label matching. Then, it puts a `platform-config` config map in every deployment target folder in the `Platform GitOps` repository. The `platform-config` config map contains all of the platform configuration values that the workload can use on this cluster type in this environment.

+In a few seconds, a new PR to the `stage` branch in the `Platform GitOps` repository appears:

+Approve the PR and merge it.

+The `large` cluster is handled by ArgoCD, which, by default, is configured to reconcile every three minutes. This cluster doesn't report its compliance state to Azure like the clusters such as `drone` that have the [GitOps extension](conceptual-gitops-flux2.md). However, you can still monitor the reconciliation state on the cluster with ArgoCD UI.

+To access the ArgoCD UI on the `large` cluster, run the following command:

+```bash

+# Get ArgoCD username and password

+echo "ArgoCD username: admin, password: $(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" --context large| base64 -d)"

+# output:

+# ArgoCD username: admin, password: eCllTELZdIZfApPL

+kubectl port-forward svc/argocd-server 8080:80 -n argocd --context large

+```

+Next, open `localhost:8080` in your browser and provide the username and password printed by the script. You'll see a web page similar to this one:

+ :::image type="content" source="media/tutorial-workload-management/argocd-ui.png" alt-text="Screenshot showing the Argo CD user interface web page." lightbox="media/tutorial-workload-management/argocd-ui.png":::

+Select the `stage` tile to see more details on the reconciliation state from the `stage` branch to this cluster. You can select the `SYNC` buttons to force the reconciliation and speed up the process.

+Once the new configuration has arrived to the cluster, check the `uat-test` application instance at `localhost:8002` after

+running the following commands:

+```bash

+kubectl rollout restart deployment hello-world-deployment -n stage-kaizen-app-team-hello-world-app-uat-test --context=large

+kubectl port-forward svc/hello-world-service -n stage-kaizen-app-team-hello-world-app-uat-test 8002:8000 --context=large

+```

+You'll see the updated database url:

+## 5 - Platform Team: Add cluster type to environment

+Currently, only `drone` and `large` cluster types are included in the `Stage` environment. Let's include the `small` cluster type to `Stage` as well. Even though there's no physical cluster representing this cluster type, you can see how the scheduler reacts to this change.

+```bash

+# Switch to stage branch (representing Stage environemnt) in the control-plane folder

+git checkout stage

+# Add "small" cluster type in west-us region

+mkdir -p cluster-types/west-us/small

+cat <<EOF >cluster-types/west-us/small/small-cluster-type.yaml

+apiVersion: scheduler.kalypso.io/v1alpha1

+kind: ClusterType

+metadata:

+ name: small

+ labels:

+ region: west-us

+ size: small

+spec:

+ reconciler: argocd

+ namespaceService: default

+EOF

+git add .

+git commit -m 'add new cluster type'

+git config pull.rebase false

+git pull --no-edit

+git push

+```

+In a few seconds, the scheduler submits a PR to the `Platform GitOps` repository. According to the `uat-test-policy` that you created, it assigns the `uat-test` deployment target to the new cluster type, as it's supposed to work on all available cluster types in the environment.

+## Clean up resources

+When no longer needed, delete the resources that you created for this tutorial. To do so, run the following command:

+```bash

+# In kalypso folder

+./deploy.sh -d -p <preix. e.g. kalypso> -o <github org. e.g. eedorenko> -t <github token> -l <azure-location. e.g. westus2>

+```

+## Next steps

+In this tutorial, you have performed tasks for a few of the most common workload management scenarios in a multi-cluster Kubernetes environment. There are many other scenarios you may want to explore. Continue to use the sample and see how you can implement use cases that are most common in your daily activities.

+To understand the underlying concepts and mechanics deeper, refer to the following resources:

+> [!div class="nextstepaction"]

+> - [Workload Management in Multi-cluster environment with GitOps](https://github.com/microsoft/kalypso)

+- The extension service now correctly restarts when the Azure Connected Machine agent is upgraded by Update Management Center

+- Tenant IDs are better validated when connecting the server

+> Version 1.26 is only available for Linux operating systems.

+This topic describes the basic requirements for installing the Connected Machine agent to onboard a physical server or virtual machine to Azure Arc-enabled servers. Some [onboarding methods](deployment-options.md) may have more requirements.

+You shouldn't install Azure Arc on virtual machines hosted in Azure, Azure Stack Hub, or Azure Stack Edge, as they already have similar capabilities. You can, however, [use an Azure VM to simulate an on-premises environment](plan-evaluate-on-azure-virtual-machine.md) for testing purposes, only.

+Take extra care when using Azure Arc on systems that are:

+* Cloned

+* Restored from backup as a second instance of the server

+* Used to create a "golden image" from which other virtual machines are created

+If two agents use the same configuration, you will encounter inconsistent behaviors when both agents try to act as one Azure resource. The best practice for these situations is to use an automation tool or script to onboard the server to Azure Arc after it has been cloned, restored from backup, or created from a golden image.

+> For additional information on using Azure Arc-enabled servers in VMware environments, see the [VMware FAQ](vmware-faq.md).

+Azure Arc supports the following Windows and Linux operating systems. Only x86-64 (64-bit) architectures are supported. Azure Arc does not run on x86 (32-bit) or ARM-based architectures.

+ * Azure Editions are supported on Azure Stack HCI

+* Windows 10, 11 (see [client operating system guidance](#client-operating-system-guidance))

+### Client operating system guidance

+The Azure Arc service and Azure Connected Machine Agent are supported on Windows 10 and 11 client operating systems only when using those computers in a server-like environment. That is, the computer should always be:

+* Connected to the internet

+* Connected to a power source

+* Powered on

+For example, a computer running Windows 11 that's responsible for digital signage, point-of-sale solutions, and general back office management tasks is a good candidate for Azure Arc. End-user productivity machines, such as a laptop, which may go offline for long periods of time, shouldn't use Azure Arc and instead should consider [Microsoft Intune](/mem/intune) or [Microsoft Endpoint Configuration Manager](/mem/configmgr).

+### Short-lived servers and virtual desktop infrastructure

+Microsoft doesn't recommend running Azure Arc on short-lived (ephemeral) servers or virtual desktop infrastructure (VDI) VMs. Azure Arc is designed for long-term management of servers and isn't optimized for scenarios where you are regularly creating and deleting servers. For example, Azure Arc doesn't know if the agent is offline due to planned system maintenance or if the VM was deleted, so it won't automatically clean up server resources that stopped sending heartbeats. As a result, you could encounter a conflict if you re-create the VM with the same name and there's an existing Azure Arc resource with the same name.

+[Azure Virtual Desktop on Azure Stack HCI](../../virtual-desktop/azure-stack-hci-overview.md) doesn't use short-lived VMs and supports running Azure Arc in the desktop VMs.

+* NET Framework 4.6 or later. [Download the .NET Framework](/dotnet/framework/install/guide-for-developers).

+* Windows PowerShell 4.0 or later (already included with Windows Server 2012 R2 and later). For Windows Server 2008 R2 SP1, [Download Windows Management Framework 5.1.](https://www.microsoft.com/download/details.aspx?id=54616).

+You'll need the following Azure built-in roles for different aspects of managing connected machines:

+* To onboard machines, you must have the [Azure Connected Machine Onboarding](../../role-based-access-control/built-in-roles.md#azure-connected-machine-onboarding) or [Contributor](../../role-based-access-control/built-in-roles.md#contributor) role for the resource group where you're managing the servers.

+* To select a resource group from the drop-down list when using the **Generate script** method, you'll also need the [Reader](../../role-based-access-control/built-in-roles.md#reader) role for that resource group (or another role that includes **Reader** access).

+Each Azure Arc-enabled server is associated with an Azure Active Directory object and counts against your directory quota. See [Azure AD service limits and restrictions](../../active-directory/enterprise-users/directory-service-limits-restrictions.md) for information about the maximum number of objects you can have in an Azure AD directory.

+You can register the resource providers using the following commands:

+Register-AzResourceProvider -ProviderNamespace Microsoft.AzureArcData

+az provider register --namespace 'Microsoft.AzureArcData'

+ OrchestrationMetadata orchestration = client.waitForInstanceCompletion(

+ https://us.atlas.microsoft.com/dataRegistries/{udid}?api-version=2022-12-01-preview&subscription-key={Your-Azure-Maps-Subscription-key}

+https://us.atlas.microsoft.com/dataRegistries/{udid}/content?api-version=2022-12-01-preview&subscription-key={Your-Azure-Maps-Subscription-key}

+> - Replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+> - In the URL examples in this article you will need to replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+5. Enter the following URL (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the following URL (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the `status URL` you copied in [Upload pins and path data](#upload-pins-and-path-data). The request should look like the following URL (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the following URL to the [Render Service](/rest/api/maps/render/get-map-image) (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key and `udid` with the `udid` of the uploaded data):

+5. Enter the following URL to the [Render Service](/rest/api/maps/render/get-map-image) (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the following URL to the [Render Service](/rest/api/maps/render/get-map-image) (replace {`Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+ >For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+4. On the **Builder** tab, select the **GET** HTTP method, and then enter the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+6. Now, we'll call the [Post Data for Points API](/rest/api/maps/elevation/postdataforpoints) to get elevation data for the same two points. On the **Builder** tab, select the **POST** HTTP method and then enter the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+4. On the **Builder** tab, select the **GET** HTTP method, and then enter the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+9. Now, we'll call the [Post Data For Polyline API](/rest/api/maps/elevation/postdataforpolyline) to get elevation data for the same three points. On the **Builder** tab, select the **POST** HTTP method, and then enter the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+4. On the **Builder** tab, select the **GET** HTTP method, and then enter the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. In this request, we're searching for a specific address: `400 Braod St, Seattle, WA 98109`. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key. The request should look like the following URL:

+2. Select the **GET** HTTP method in the builder tab and enter the following URL. For this request, and other requests mentioned in this article, replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key. The request should look like the following URL:

+4. Enter the following URL to the [Feature Update States API](/rest/api/maps/v2/feature-state/update-states) (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key and `statesetId` with the `statesetId`):

+3. Add the following initialization code. Make sure to replace `<Your Azure Maps Key>` with your Azure Maps subscription key.

+> * `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+> * In the URL examples in this article you will need to replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+5. Enter the following URL to the [Conversion Service](/rest/api/maps/v2/conversion/convert) (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key and `udid` with the `udid` of the uploaded package):

+> * `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the `status URL` you copied in [Upload Geofencing GeoJSON data](#upload-geofencing-geojson-data). The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the `resource Location URL` you copied in [Check the GeoJSON data upload status](#check-the-geojson-data-upload-status). The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key):

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key, and `{udid}` with the `udid` you saved in the [Upload Geofencing GeoJSON data section](#upload-geofencing-geojson-data)).

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key, and `{udid}` with the `udid` you saved in the [Upload Geofencing GeoJSON data section](#upload-geofencing-geojson-data)).

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key, and `{udid}` with the `udid` you saved in the [Upload Geofencing GeoJSON data section](#upload-geofencing-geojson-data)).

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key, and `{udid}` with the `udid` you saved in the [Upload Geofencing GeoJSON data section](#upload-geofencing-geojson-data)).

+5. Enter the following URL. The request should look like the following URL (replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key, and `{udid}` with the `udid` you saved in the [Upload Geofencing GeoJSON data section](#upload-geofencing-geojson-data)).

+2. Select the **POST** HTTP method in the builder tab, and enter the following URL to upload the geofence to the Data Upload API. Make sure to replace `{Your-Azure-Maps-Subscription-key}` with your Azure Maps subscription key.

+4. Clean up: After you confirm that Azure Monitor Agent is collecting data properly, you may **choose to either disable or uninstall the legacy Log Analytics agents**

+ 1. If you have migrated to Azure Monitor agent for selected features/solutions and you need to continue using the legacy Log Analytics for others, you can

+ 2. If you've migrated to Azure Monitor agent for all your requirements, you may [uninstall the Log Analytics agent](./agent-manage.md#uninstall-agent) from monitored resources. Clean up any configuration files, workspace keys, or certificates that were used previously by the Log Analytics agent.

+ 3. Don't uninstall the legacy agent if you need to use it for uploading data to System Center Operations Manager.

+Annotations show where you deployed a new build or other significant events. Annotations make it easy to see whether your changes had any effect on your application's performance. They can be automatically created by the [Azure Pipelines](/azure/devops/pipelines/tasks/) build system. You can also create annotations to flag any event you want by creating them from PowerShell.

+- The resource to which you're deploying is linked to Application Insights via the `APPINSIGHTS_INSTRUMENTATIONKEY` app setting.

+- The Application Insights resource is in the same subscription as the resource to which you're deploying.

+> If you're still using the Application Insights annotation deployment task, you should delete it.

+If you can't use one of the deployment tasks in the previous section, you need to add an inline script task in your deployment pipeline.

+1. Go to a new or existing pipeline and select a task.

+ :::image type="content" source="./media/annotations/task.png" alt-text="Screenshot that shows a task selected under Stages." lightbox="./media/annotations/task.png":::

+ :::image type="content" source="./media/annotations/add-azure-cli.png" alt-text="Screenshot that shows adding a new task and selecting Azure CLI." lightbox="./media/annotations/add-azure-cli.png":::

+1. Specify the relevant Azure subscription. Change **Script Type** to **PowerShell** and **Script Location** to **Inline**.

+1. Add the [PowerShell script from step 2 in the next section](#create-release-annotations-with-the-azure-cli) to **Inline Script**.

+1. Add the following arguments. Replace the angle-bracketed placeholders with your values to **Script Arguments**. The `-releaseProperties` are optional.

+ The following example shows metadata you can set in the optional `releaseProperties` argument by using [build](/azure/devops/pipelines/build/variables#build-variables-devops-services) and [release](/azure/devops/pipelines/release/variables#default-variablesrelease) variables.

+1. Select **Save**.

+## Create release annotations with the Azure CLI

+You can use the `CreateReleaseAnnotation` PowerShell script to create annotations from any process you want without using Azure DevOps.

+1. Sign in to the [Azure CLI](/cli/azure/authenticate-azure-cli).

+1. Make a local copy of the following script and call it `CreateReleaseAnnotation.ps1`.

+ > [!NOTE]

+ > Your annotations must have **Category** set to **Deployment** to appear in the Azure portal.

+1. Call the PowerShell script with the following code. Replace the angle-bracketed placeholders with your values. The `-releaseProperties` are optional.

+ |Argument | Definition | Note|

+ |--|--|--|

+ |`aiResourceId` | The resource ID to the target Application Insights resource. | Example:<br> /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRGName/providers/microsoft.insights/components/MyResourceName|

+ |`releaseName` | The name to give the created release annotation. | |

+ |`releaseProperties` | Used to attach custom metadata to the annotation. | Optional|

+

+> Release annotations aren't currently available in the **Metrics** pane of Application Insights.

+Whenever you use the release template to deploy a new release, an annotation is sent to Application Insights. You can view annotations in the following locations:

+- **Performance:**

+ :::image type="content" source="./media/annotations/performance.png" alt-text="Screenshot that shows the Performance tab with a release annotation selected to show the Release Properties tab." lightbox="./media/annotations/performance.png":::

+- **Failures:**

+ :::image type="content" source="./media/annotations/failures.png" alt-text="Screenshot that shows the Failures tab with a release annotation selected to show the Release Properties tab." lightbox="./media/annotations/failures.png":::

+- **Usage:**

+ :::image type="content" source="./media/annotations/usage-pane.png" alt-text="Screenshot that shows the Users tab bar with release annotations selected. Release annotations appear as blue arrows above the chart indicating the moment in time that a release occurred." lightbox="./media/annotations/usage-pane.png":::

+- **Workbooks:**

+ In any log-based workbook query where the visualization displays time along the x-axis:

+ :::image type="content" source="./media/annotations/workbooks-annotations.png" alt-text="Screenshot that shows the Workbooks pane with a time series log-based query with annotations displayed." lightbox="./media/annotations/workbooks-annotations.png":::

+To enable annotations in your workbook, go to **Advanced Settings** and select **Show annotations**.

+## Release annotations by using API keys

+> Annotations using API keys is deprecated. We recommend using the [Azure CLI](#create-release-annotations-with-the-azure-cli) instead.

+To create release annotations, install one of the many Azure DevOps extensions available in Visual Studio Marketplace.

+1. On the **Visual Studio Marketplace** [Release Annotations extension](https://marketplace.visualstudio.com/items/ms-appinsights.appinsightsreleaseannotations) page, select your Azure DevOps organization. Select **Install** to add the extension to your Azure DevOps organization.

+ 

+### Configure release annotations by using API keys

+ 

+1. Select **Add task** and then select the **Application Insights Release Annotation** task from the menu.

+ 

+ > The Release Annotation task currently supports only Windows-based agents. It won't run on Linux, macOS, or other types of agents.

+ 

+1. Back in the Application Insights **API Access** window, select **Create API Key**.

+ 

+1. In the **Create API key** window, enter a description, select **Write annotations**, and then select **Generate key**. Copy the new key.

+ 

+1. Under **Name**, enter **ApiKey**. Under **Value**, paste the API key you copied from the **API Access** tab.

+ 

+1. Select **Save** in the main release template window to save the template.

+To use the new release annotations:

+1. Remove the Application Insights Release Annotation task in your Azure Pipelines deployment.

+1. Create new release annotations with [Azure Pipelines](#release-annotations-with-azure-pipelines-build) or the [Azure CLI](#create-release-annotations-with-the-azure-cli).

+3. Once the agent jar file is uploaded, go to App Service configurations. If you

+ need to use **Startup Command** for Linux, please include jvm arguments:

+ :::image type="content"source="./media/azure-web-apps/startup-command.png" alt-text="Screenshot of startup command.":::

+

+ **Startup Command** won't honor `JAVA_OPTS`.

+ If you don't use **Startup Command**, create a new environment variable, `JAVA_OPTS`, with the value

+ `-javaagent:{PATH_TO_THE_AGENT_JAR}/applicationinsights-agent-{VERSION_NUMBER}.jar`.

+4. Restart the app to apply the changes.

+ Title: Application Insights telemetry data model - Event telemetry | Microsoft Docs

+description: Learn about the Application Insights data model for event telemetry.

+You can create event telemetry items (in [Application Insights](./app-insights-overview.md)) to represent an event that occurred in your application. Typically, it's a user interaction such as a button click or order checkout. It can also be an application lifecycle event like initialization or a configuration update.

+Semantically, events may or may not be correlated to requests. However, if used properly, event telemetry is more important than requests or traces. Events represent business telemetry and should be subject to separate, less aggressive [sampling](./api-filtering-sampling.md).

+Event name: To allow proper grouping and useful metrics, restrict your application so that it generates a few separate event names. For example, don't use a separate name for each generated instance of an event.

+**Maximum length:** 512 characters

+- See [Data model](data-model.md) for Application Insights types and data models.

+- [Write custom event telemetry](./api-custom-events-metrics.md#trackevent).

+- Check out [platforms](./app-insights-overview.md#supported-languages) supported by Application Insights.

+Add the JVM arg `-javaagent:"path/to/applicationinsights-agent-3.4.10.jar"` somewhere before `-jar`, for example:

+java -javaagent:"path/to/applicationinsights-agent-3.4.10.jar" -jar <myapp.jar>

+If you're using the *exec* form, add the parameter `-javaagent:"path/to/applicationinsights-agent-3.4.10.jar"` to the parameter list somewhere before the `"-jar"` parameter, for example:

+ENTRYPOINT ["java", "-javaagent:path/to/applicationinsights-agent-3.4.10.jar", "-jar", "<myapp.jar>"]

+If you're using the *shell* form, add the JVM arg `-javaagent:"path/to/applicationinsights-agent-3.4.10.jar"` somewhere before `-jar`, for example:

+ENTRYPOINT java -javaagent:"path/to/applicationinsights-agent-3.4.10.jar" -jar <myapp.jar>

+ <version>3.4.10</version>

+JAVA_OPTS="$JAVA_OPTS -javaagent:path/to/applicationinsights-agent-3.4.10.jar"

+CATALINA_OPTS="$CATALINA_OPTS -javaagent:path/to/applicationinsights-agent-3.4.10.jar"

+If the file `<tomcat>/bin/setenv.sh` already exists, modify that file and add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to `CATALINA_OPTS`.

+set CATALINA_OPTS=%CATALINA_OPTS% -javaagent:path/to/applicationinsights-agent-3.4.10.jar

+set "CATALINA_OPTS=%CATALINA_OPTS% -javaagent:path/to/applicationinsights-agent-3.4.10.jar"

+If the file `<tomcat>/bin/setenv.bat` already exists, modify that file and add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to `CATALINA_OPTS`.

+Locate the file `<tomcat>/bin/tomcat8w.exe`. Run that executable and add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to the `Java Options` under the `Java` tab.

+Add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to the existing `JAVA_OPTS` environment variable in the file `JBOSS_HOME/bin/standalone.conf` (Linux) or `JBOSS_HOME/bin/standalone.conf.bat` (Windows):

+ JAVA_OPTS="-javaagent:path/to/applicationinsights-agent-3.4.10.jar -Xms1303m -Xmx1303m ..."

+Add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to the existing `jvm-options` in `JBOSS_HOME/domain/configuration/host.xml`:

+ <option value="-javaagent:path/to/applicationinsights-agent-3.4.10.jar"/>

+-javaagent:path/to/applicationinsights-agent-3.4.10.jar

+Add `-javaagent:path/to/applicationinsights-agent-3.4.10.jar` to the existing `jvm-options` in `glassfish/domains/domain1/config/domain.xml`:

+ -javaagent:path/to/applicationinsights-agent-3.4.10.jar>

+ -javaagent:path/to/applicationinsights-agent-3.4.10.jar

+-javaagent:path/to/applicationinsights-agent-3.4.10.jar

+By default, Application Insights Java 3.x expects the configuration file to be named `applicationinsights.json`, and to be located in the same directory as `applicationinsights-agent-3.4.10.jar`.

+If you specify a relative path, it will be resolved relative to the directory where `applicationinsights-agent-3.4.10.jar` is located.

+If you specify a relative path, it's resolved relative to the directory where `applicationinsights-agent-3.4.10.jar` is located.

+ <version>3.4.10</version>

+`applicationinsights-agent-3.4.10.jar` is located.

+> [!WARNING]

+> Invoking Profile now will enable the profiler feature, and Application Insights will apply default CPU and memory SLA triggers. When your application breaches those SLAs, Application Insights will gather Java profiles. If you wish to disable profiling later on, you can do so within the trigger menu shown in [Installation](#installation).

+- `APPLICATIONINSIGHTS_PREVIEW_PROFILER_ENABLED`: boolean (default: `true`)

+ Enables/disables the profiling feature. By default the feature is enabled within the agent (since agent 3.4.9). However, even though this feature is enabled within the agent, profiles will not be gathered unless enabled within the Portal as described in [Installation](#installation).

+Java Profiling is a free feature with Application Insights. [Azure Monitor Application Insights pricing](https://azure.microsoft.com/pricing/details/monitor/) is based on ingestion cost.

+Yes, you can profile a JVM running microservices using the JFR.

+-javaagent:path/to/applicationinsights-agent-3.4.10.jar

+Download the [applicationinsights-agent-3.4.10.jar](https://github.com/microsoft/ApplicationInsights-Java/releases/download/3.4.10/applicationinsights-agent-3.4.10.jar) file.

+Point the JVM to the jar file by adding `-javaagent:"path/to/applicationinsights-agent-3.4.10.jar"` to your application's JVM args.

+- Create a configuration file named `applicationinsights.json`, and place it in the same directory as `applicationinsights-agent-3.4.10.jar` with the following content:

+ <version>3.4.10</version>

+description: Start monitoring a new application with a new instrumentation key.

+Application Insights monitors the availability, performance, and usage of your apps. This article shows you how to set it up for a SharePoint site.

+> Because of security concerns, you can't directly add the script that's described in this article to your webpages in the SharePoint modern UX. As an alternative, you can use [SharePoint Framework (SPFx)](/sharepoint/dev/spfx/extensions/overview-extensions) to build a custom extension that you can use to install Application Insights on your SharePoint sites.

+In the [Azure portal](https://portal.azure.com), create a new Application Insights resource. For **Application Type**, select **ASP.NET**.

+

+The window that opens is the place where you see performance and usage data about your app. The next time you sign in to Azure, a tile for it appears on the **Start** screen. Alternatively, select **Browse** to find it.

+## Add the script to your webpages

+The following current snippet is version `"5"`. The version is encoded in the snippet as `sv:"#"`. The [current version is also available on GitHub](https://go.microsoft.com/fwlink/?linkid=2156318).

+> The URL for SharePoint uses a different module format `"...\ai.2.gbl.min.js"` (note the extra `.gbl`.). This alternate module format is required to avoid an issue caused by the order in which scripts are loaded. The issue causes the SDK to fail to initialize and results in the loss of telemetry events.

+> The issue is caused by `requireJS` being loaded and initialized before the SDK.

+Insert the script before the &lt;/head&gt; tag of every page you want to track. If your website has a main page, you can put the script there. For example, in an ASP.NET MVC project, you'd put it in `View\Shared\_Layout.cshtml`.

+You can add the code to your main page or individual pages.

+#### Main page

+If you can edit the site's main page, you can provide monitoring for every page in the site.

+Check out the main page and edit it by using SharePoint Designer or any other editor.

+

+Add the code before the </head> tag.

+#### Individual pages

+To monitor a limited set of pages, add the script separately to each page.

+The first events appear in **Search**.

+Select **Refresh** after a few seconds if you're expecting more data.

+## Capture the user ID

+The standard webpage code snippet doesn't capture the user ID from SharePoint, but you can do that with a small modification.

+1. Copy your app's instrumentation key from the **Essentials** dropdown in Application Insights.

+1. Substitute the instrumentation key for `XXXX` in the following snippet.

+1. Embed the script in your SharePoint app instead of the snippet you get from the portal.

+ ```

+

+

+ <SharePoint:ScriptLink ID="ScriptLink1" name="SP.js" runat="server" localizable="false" loadafterui="true" />

+ <SharePoint:ScriptLink ID="ScriptLink2" name="SP.UserProfiles.js" runat="server" localizable="false" loadafterui="true" />

+

+ <script type="text/javascript">

+ var personProperties;

+

+ // Ensure that the SP.UserProfiles.js file is loaded before the custom code runs.

+ SP.SOD.executeOrDelayUntilScriptLoaded(getUserProperties, 'SP.UserProfiles.js');

+

+ function getUserProperties() {

+ // Get the current client context and PeopleManager instance.

+ var clientContext = new SP.ClientContext.get_current();

+ var peopleManager = new SP.UserProfiles.PeopleManager(clientContext);

+

+ // Get user properties for the target user.

+ // To get the PersonProperties object for the current user, use the

+ // getMyProperties method.

+

+ personProperties = peopleManager.getMyProperties();

+

+ // Load the PersonProperties object and send the request.

+ clientContext.load(personProperties);

+ clientContext.executeQueryAsync(onRequestSuccess, onRequestFail);

+ }

+

+ // This function runs if the executeQueryAsync call succeeds.

+ function onRequestSuccess() {

+ var appInsights=window.appInsights||function(config){

+ function s(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},r=document,f=window,e="script",o=r.createElement(e),i,u;for(o.src=config.url||"//az416426.vo.msecnd.net/scripts/a/ai.0.js",r.getElementsByTagName(e)[0].parentNode.appendChild(o),t.cookie=r.cookie,t.queue=[],i=["Event","Exception","Metric","PageView","Trace"];i.length;)s("track"+i.pop());return config.disableExceptionTracking||(i="onerror",s("_"+i),u=f[i],f[i]=function(config,r,f,e,o){var s=u&&u(config,r,f,e,o);return s!==!0&&t["_"+i](config,r,f,e,o),s}),t

+ }({

+ instrumentationKey:"XXXX"

+ });

+ window.appInsights=appInsights;

+ appInsights.trackPageView(document.title,window.location.href, {User: personProperties.get_displayName()});

+ }

+

+ // This function runs if the executeQueryAsync call fails.

+ function onRequestFail(sender, args) {

+ }

+ </script>

+

+

+ ```

+## Next steps

+* See the [Availability overview](./availability-overview.md) to monitor the availability of your site.

+* See [Application Insights](./app-insights-overview.md) for other types of apps.

+description: Learn how to upload source maps to your Azure Storage account blob container by using Application Insights.

+Application Insights supports the uploading of source maps to your Azure Storage account blob container. You can use source maps to unminify call stacks found on the **End-to-end transaction details** page. You can also use source maps to unminify any exception sent by the [JavaScript SDK][ApplicationInsights-JS] or the [Node.js SDK][ApplicationInsights-Node.js].

+

+## Create a new storage account and blob container

+1. [Create a new storage account][create storage account].

+1. [Create a blob container][create blob container] inside your storage account. Set **Public access level** to **Private** to ensure that your source maps aren't publicly accessible.

+ > [!div class="mx-imgBorder"]

+ >

+## Push your source maps to your blob container

+Integrate your continuous deployment pipeline with your storage account by configuring it to automatically upload your source maps to the configured blob container.

+You can upload source maps to your Azure Blob Storage container with the same folder structure they were compiled and deployed with. A common use case is to prefix a deployment folder with its version, for example, `1.2.3/static/js/main.js`. When you unminify via an Azure blob container called `sourcemaps`, the pipeline tries to fetch a source map located at `sourcemaps/1.2.3/static/js/main.js.map`.

+If you're using Azure Pipelines to continuously build and deploy your application, add an [Azure file copy][azure file copy] task to your pipeline to automatically upload your source maps.

+> 

+## Configure your Application Insights resource with a source map storage account

+You have two options for configuring your Application Insights resource with a source map storage account.

+### End-to-end transaction details tab

+From the **End-to-end transaction details** tab, select **Unminify**. Configure your resource if it's unconfigured.

+1. In the Azure portal, view the details of an exception that's minified.

+1. Select **Unminify**.

+1. If your resource isn't configured, configure it.

+### Properties tab

+To configure or change the storage account or blob container that's linked to your Application Insights resource:

+1. Go to the **Properties** tab of your Application Insights resource.

+1. Select **Change source map Blob Container**.

+1. Select a different blob container as your source map container.

+1. Select **Apply**.

+> 

+This section offers troubleshooting tips for common issues.

+### Required Azure role-based access control settings on your blob container

+Any user on the portal who uses this feature must be assigned at least as a [Storage Blob Data Reader][storage blob data reader] to your blob container. Assign this role to anyone who might use the source maps through this feature.

+> Depending on how the container was created, this role might not have been automatically assigned to you or your team.

+1. Verify that the corresponding source map is uploaded to the correct blob container.

+1. Verify that the source map file is named after the JavaScript file it maps to and uses the suffix `.map`.

+

+ For example, `/static/js/main.4e2ca5fa.chunk.js` searches for the blob named `main.4e2ca5fa.chunk.js.map`.

+1. Check your browser's console to see if any errors were logged. Include this information in any support ticket.

+## Next steps

+[Azure file copy task](/azure/devops/pipelines/tasks/deploy/azure-file-copy)

+description: Analyze different sets or users, sessions, events, or operations that have something in common.

+A cohort is a set of users, sessions, events, or operations that have something in common. In Application Insights, cohorts are defined by an analytics query. In cases where you have to analyze a specific set of users or events repeatedly, cohorts can give you more flexibility to express exactly the set you're interested in.

+## Cohorts vs. basic filters

+You can use cohorts in ways similar to filters. But cohorts' definitions are built from custom analytics queries, so they're much more adaptable and complex. Unlike filters, you can save cohorts so that other members of your team can reuse them.

+> After cohorts are created, they're available from the Users, Sessions, Events, and User Flows tools.

+1. Select **Create a Cohort**.

+1. Select the **Template Gallery** tab to see a collection of templates for various cohorts.

+1. Select **Engaged Users -- by Days Used**.

+ * **Activities**: Where you choose which events and page views count as usage.

+ * **Period**: The definition of a month.

+ * **UsedAtLeastCustom**: The number of times users need to use something within a period to count as engaged.

+1. Change **UsedAtLeastCustom** to **5+ days**. Leave **Period** set as the default of 28 days.

+ Now this cohort represents all user IDs sent with any custom event or page view on 5 separate days in the past 28 days.

+1. Select **Save**.

+ > Give your cohort a name, like *Engaged Users (5+ Days)*. Save it to *My reports* or *Shared reports*, depending on whether you want other people who have access to this Application Insights resource to see this cohort.

+1. Select **Back to Gallery**.

+Open the Users tool. In the **Show** dropdown box, choose the cohort you created under **Users who belong to**.

+Important points to notice:

+* You can further filter this cohort by using the normal filters in the Users tool. Although the cohort is defined on 28-day windows, you can still adjust the time range in the Users tool to be 30, 60, or 90 days.

+You can also make cohorts of events. In this section, you define a cohort of events and page views. Then you see how to use them from the other tools. This cohort might define a set of events that your team considers _active usage_ or a set related to a certain new feature.

+1. Select **Create a Cohort**.

+1. Select the **Template Gallery** tab to see a collection of templates for various cohorts.

+1. Select **Events Picker**.

+1. In the **Activities** dropdown box, select the events you want to be in the cohort.

+1. Save the cohort and give it a name.

+The previous two cohorts were defined by using dropdown boxes. You can also define cohorts by using analytics queries for total flexibility. To see how, create a cohort of users from the United Kingdom.

+ :::image type="content" source="./media/usage-cohorts/cohort.png" alt-text="Screenshot that shows the template gallery for cohorts." lightbox="./media/usage-cohorts/cohort.png":::

+ * **Markdown text**: Where you describe the cohort in more detail for other members on your team.

+ * **Parameters**: Where you make your own parameters, like **Activities**, and other dropdown boxes from the previous two examples.

+ * **Query**: Where you define the cohort by using an analytics query.

+ In the query section, you [write an analytics query](/azure/kusto/query). The query selects the certain set of rows that describe the cohort you want to define. The Cohorts tool then implicitly adds a `| summarize by user_Id` clause to the query. This data appears as a preview underneath the query in a table, so you can make sure your query is returning results.

+ > If you don't see the query, resize the section to make it taller and reveal the query.

+1. Copy and paste the following text into the query editor:

+1. Select **Run Query**. If you don't see user IDs appear in the table, change to a country/region where your application has users.

+1. Save and name the cohort.

+## Frequently asked question

+### I defined a cohort of users from a certain country/region. When I compare this cohort in the Users tool to setting a filter on that country/region, why do I see different results?

+Cohorts and filters are different. Suppose you have a cohort of users from the United Kingdom (defined like the previous example), and you compare its results to setting the filter `Country or region = United Kingdom`:

+* The filters version only shows events from the United Kingdom. If you split by country or region, you see only the United Kingdom.

+> Alternatively or in addition, you can send the guest OS metrics to Azure Monitor Logs by using the same agent. There you can query on those metrics in combination with non-metric data by using Log Analytics. Standard [Log Analytics workspace costs](https://azure.microsoft.com/pricing/details/monitor/) would then apply.

+<!--Gen Date: Wed Feb 01 2023 09:43:49 GMT+0200 (Israel Standard Time)-->

+This Azure Monitor Workspace is in the region specific in [Region mappings](#region-mappings).

+If the Azure Monitor workspace is linked to one or more Grafana workspaces, then the data is available in Grafana.

+The output for each command looks similar to the following:

+- `--ksm-metric-annotations-allow-list` is a comma-separated list of Kubernetes annotations keys that will be used in the resource's labels metric. By default the metric contains only name and namespace labels. To include more annotations provide a list of resource names in their plural form and Kubernetes annotation keys, you would like to allow for them. A single `*` can be provided per resource instead to allow any annotations, but that has severe performance implications.

+- `--ksm-metric-labels-allow-list` is a comma-separated list of more Kubernetes label keys that will be used in the resource's labels metric. By default the metric contains only name and namespace labels. To include more labels provide a list of resource names in their plural form and Kubernetes label keys, you would like to allow for them. A single `*` can be provided per resource instead to allow any labels, but that has severe performance implications.

+The output is similar to the following:

+If you're using an existing Azure Managed Grafana instance that already has been linked to an Azure Monitor workspace, then you need the list of Grafana integrations. Copy the value of the `azureMonitorWorkspaceIntegrations` field. If it doesn't exist, then the instance hasn't been linked with any Azure Monitor workspace.

+ | `metricAnnotationsAllowList` | Comma-separated list of more Kubernetes label keys that will be used in the resource's labels metric. |

+4. Open the template file and update the `grafanaIntegrations` property at the end of the file with the values that you retrieved from the Grafana instance. This is similar to the following:

+If you're using an existing Azure Managed Grafana instance that already has been linked to an Azure Monitor workspace, then you need the list of Grafana integrations. Copy the value of the `azureMonitorWorkspaceIntegrations` field. If it doesn't exist, then the instance hasn't been linked with any Azure Monitor workspace.

+5. The main bicep template creates all the required resources and uses two modules for creating the dcra and monitormetrics profile resources from the other two bicep files.

+ | `metricAnnotationsAllowList` | Comma-separated list of more Kubernetes label keys that will be used in the resource's labels metric. |

+6. Open the template file and update the `grafanaIntegrations` property at the end of the file with the values that you retrieved from the Grafana instance. This is similar to the following:

+## [Azure Policy](#tab/azurepolicy)

+### Prerequisites

+- Register the `AKS-PrometheusAddonPreview` feature flag in the Azure Kubernetes clusters subscription with the following command in Azure CLI: `az feature register --namespace Microsoft.ContainerService --name AKS-PrometheusAddonPreview`.

+- The Azure Monitor workspace and Azure Managed Grafana workspace must already be created.

+### Download Azure policy rules and parameters and deploy

+1. Download the main Azure policy rules template from [here](https://aka.ms/AddonPolicyMetricsProfile) and save it as **AddonPolicyMetricsProfile.rules.json**.

+2. Download the parameter file from [here](https://aka.ms/AddonPolicyMetricsProfile.parameters) and save it as **AddonPolicyMetricsProfile.parameters.json** in the same directory as the rules template.

+3. Create the policy definition using a command like : `az policy definition create --name "(Preview) Prometheus Metrics addon" --display-name "(Preview) Prometheus Metrics addon" --mode Indexed --metadata version=1.0.0 category=Kubernetes --rules .\AddonPolicyMetricsProfile.rules.json --params .\AddonPolicyMetricsProfile.parameters.json`

+4. After creating the policy definition, go to Azure portal -> Policy -> Definitions and select the Policy definition you created.

+5. Click on 'Assign' and then go to the 'Parameters' tab and fill in the details. Then click 'Review + Create'.

+6. Now that the policy is assigned to the subscription, whenever you create a new cluster, which does not have Prometheus enabled, the policy will run and deploy the resources. If you want to apply the policy to existing AKS cluster, create a 'Remediation task' for that AKS cluster resource after going to the 'Policy Assignment'.

+7. Now you should see metrics flowing in the existing linked Grafana resource, which is linked with the corresponding Azure Monitor Workspace.

+In case you create a new Managed Grafana resource from Azure portal, please link it with the corresponding Azure Monitor Workspace from the 'Linked Grafana Workspaces' tab of the relevant Azure Monitor Workspace page. Please assign the role 'Monitoring Data Reader' to the Grafana MSI on the Azure Monitor Workspace resource so that it can read data for displaying the charts, using the instructions below.

+1. From the **Overview** page for the Azure Managed Grafana instance in the Azure portal, select **JSON view**.

+2. Copy the value of the `principalId` field for the `SystemAssigned` identity.

+```json

+"identity": {

+ "principalId": "00000000-0000-0000-0000-000000000000",

+ "tenantId": "00000000-0000-0000-0000-000000000000",

+ "type": "SystemAssigned"

+ },

+```

+3. From the **Access control (IAM)** page for the Azure Managed Grafana instance in the Azure portal, select **Add** and then **Add role assignment**.

+4. Select `Monitoring Data Reader`.

+5. Select **Managed identity** and then **Select members**.

+6. Select the **system-assigned managed identity** with the `principalId` from the Grafana resource.

+7. Click **Select** and then **Review+assign**.

+|[Azure Monitor Metrics](essentials/data-platform-metrics.md)|Metrics are numerical values that describe an aspect of a system at a particular point in time. [Azure Monitor Metrics](./essentials/data-platform-metrics.md) is a time-series database, optimized for analyzing time-stamped data. Azure Monitor collects metrics at regular intervals. Metrics are identified with a timestamp, a name, a value, and one or more defining labels. They can be aggregated using algorithms, compared to other metrics, and analyzed for trends over time. It supports native Azure Monitor metrics and [Prometheus based metrics](essentials/prometheus-metrics-overview.md).|

+For information about configuring VM insights, see [Enable VM insights](vminsights-enable-overview.md).

+To enable the map feature in VM insights, the virtual machine requires one of the following. See [Enable VM insights on unmonitored machine](vminsights-enable-overview.md) for details on each.

+1. Select the **Volumes** blade from the Capacity Pools blade.

+2. Select **+ Add volume** to create a volume.

+3. In the Create a Volume window, select **Create** and provide information for the following fields under the Basics tab:

+ * **Large Volume**

+ If the quota of your volume is less than 100 TiB, select **No**. If the quota of your volume is greater than 100 TiB, select **Yes**.

+ [!INCLUDE [Large volumes warning](includes/large-volumes-notice.md)]

+ If you haven't delegated a subnet, you can select **Create new** on the Create a Volume page. Then in the Create Subnet page, specify the subnet information, and select **Microsoft.NetApp/volumes** to delegate the subnet for Azure NetApp Files. In each VNet, only one subnet can be delegated to Azure NetApp Files.

+ * If you want to apply an existing snapshot policy to the volume, select **Show advanced section** to expand it, specify whether you want to hide the snapshot path, and select a snapshot policy in the pull-down menu.

+4. Select **Protocol** and complete the following information:

+5. Select **Review + Create** to review the volume details. Then select **Create** to create the SMB volume.

+* [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)

+* [Enable Active Directory Domain Services (AD DS) LDAP authentication for NFS volumes](configure-ldap-over-tls.md)

+ * **Large Volume**

+ If the quota of your volume is less than 100 TiB, select **No**. If the quota of your volume is greater than 100 TiB, select **Yes**.

+ [!INCLUDE [Large volumes warning](includes/large-volumes-notice.md)]

+* [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)

+| Number of IPs in a virtual network (including immediately peered VNets) accessing volumes in an Azure NetApp Files hosting VNet | <ul><li>**Basic**: 1000</li><li>**Standard**: [Same standard limits as VMs](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-resource-manager-virtual-networking-limits)</li></ul> | No |

+| Maximum size of a single capacity pool | 500 TiB | Yes |

+| Minimum size of a single regular volume | 100 GiB | No |

+| Maximum size of a single regular volume | 100 TiB | No |

+| Minimum size of a single [large volume](large-volumes-requirements-considerations.md) | 102,401 GiB | No |

+| Maximum size of a single large volume | 500 TiB | No |

+| Maximum number of files [`maxfiles`](#maxfiles) per volume | 106,255,630 | Yes |

+| Maximum number of quota rules per volume | 100 | Yes |

+**For volumes up to 100 TiB in size:**

+| <= 1 TiB | 21,251,126 |

+| > 4 TiB but <= 100 TiB | 106,255,630 |

+**For [large volumes](azure-netapp-files-understand-storage-hierarchy.md#large-volumes):**

+| Volume size (quota) | Automatic readjustment of the `maxfiles` limit |

+| - | - |

+| > 100 TiB | 2,550,135,120 |

+

+You can increase the `maxfiles` limit beyond 2,550,135,120 using a support request. For every 2,550,135,120 files you increase (or a fraction thereof), you need to increase the corresponding volume quota by 120 TiB. For example, if you increase `maxfiles` limit from 2,550,135,120 to 5,100,270,240 files (or any number in between), you need to increase the volume quota to at least 240 TiB.

+

+The maximum `maxfiles` value for a 500 TiB volume is 10,625,563,000 files.

+- [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)

+ For more information, see [QoS types](#qos_types).

+The QoS type is an attribute of a capacity pool. Azure NetApp Files provides two QoS types of capacity pools: *auto (default)* and *manual*.

+When you use a manual QoS capacity pool with, for example, an SAP HANA system, an Oracle database, or other workloads requiring multiple volumes, the capacity pool can be used to create these application volumes. Each volume can provide the individual size and throughput to meet the application requirements. See [Throughput limit examples of volumes in a manual QoS capacity pool](azure-netapp-files-service-levels.md#throughput-limit-examples-of-volumes-in-a-manual-qos-capacity-pool) for details about the benefits.

+- Volumes contain a capacity of between 4 TiB and 100 TiB. You can create a [large volume](#large-volumes) with a size of between 100 TiB and 500 TiB.

+## Large volumes

+Azure NetApp Files allows you to create volumes up to 500 TiB in size, exceeding the previous 100-TiB limit. Large volumes begin at a capacity of 102,401 GiB and scale up to 500 TiB. Regular Azure NetApp Files volumes are offered between 100 GiB and 102,400 GiB.

+For more information, see [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md).

+- [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)

+Azure NetApp Files backup in a region can only protect an Azure NetApp Files volume located in that same region. For example, backups created by the service in West US 2 for a volume located in West US 2 are sent to Azure storage also located in West US 2. Azure NetApp Files doesn't support backups or backup replication to a different region.

+* For volumes larger than 10 TB, it can take multiple hours to transfer all the data from the backup media.

+* Currently, the Azure NetApp Files backup feature supports backing up the daily, weekly, and monthly local snapshots created by the associated snapshot policy to the Azure storage. Hourly backups aren't currently supported.

+* Azure NetApp Files backup uses the [Zone-Redundant storage](../storage/common/storage-redundancy.md#redundancy-in-the-primary-region) (ZRS) account that replicates the data synchronously across three Azure availability zones in the region, except for the regions listed where only [Locally Redundant Storage](../storage/common/storage-redundancy.md#redundancy-in-the-primary-region) (LRS) storage is supported:

+ A configured snapshot policy for snapshots is required for the volume needing backup. The policy will also set the number of backups stored in Azure storage.

+* If an issue occurs (for example, no sufficient space left on the volume) and causes the snapshot policy to stop creating new snapshots, the backup feature won't have any new snapshots to back up.

+* In a cross-region replication setting, Azure NetApp Files backup can be configured on a source volume only. Azure NetApp Files backup isn't supported on a cross-region replication *destination* volume.

+* [Reverting a volume using snapshot revert](snapshots-revert-volume.md) isn't supported on Azure NetApp Files volumes that have backups.

+* See [Restore a backup to a new volume](backup-restore-new-volume.md) for other considerations related to restoring backups.

+* If you need to delete a parent resource group or subscription that contains backups, you should delete any backups first. Deleting the resource group or subscription won't delete the backups. You can remove backups by [disabling backups](backup-disable.md) or [manually deleting the backups](backup-disable.md).

+ Title: Configure AD DS LDAP over TLS for Azure NetApp Files | Microsoft Docs

+description: Describes how to configure AD DS LDAP over TLS for Azure NetApp Files, including root CA certificate management.

+# Configure AD DS LDAP over TLS for Azure NetApp Files

+* PTR records must exist for all domain controllers in the site for AD DS LDAP over TLS to function properly.

+1. Follow [Install the Certification Authority](/windows-server/networking/core-network-guide/cncg/server-certs/install-the-certification-authority) to install and configure AD DS Certificate Authority.

+1. Go to the NetApp account used for the volume, and select **Active Directory connections**. Then, select **Join** to create a new AD connection or **Edit** to edit an existing AD connection.

+2. In the **Join Active Directory** or **Edit Active Directory** window that appears, select the **LDAP over TLS** checkbox to enable LDAP over TLS for the volume. Then select **Server root CA Certificate** and upload the [generated root CA certificate](#generate-and-export-root-ca-certificate) to use for LDAP over TLS.

+1. Go to the NetApp account that is used for the volume and select **Active Directory connections**. Then select **Edit** to edit the existing AD connection.

+2. In the **Edit Active Directory** window that appears, deselect the **LDAP over TLS** checkbox and select **Save** to disable LDAP over TLS for the volume.

+ * **Large Volume**

+ If the quota of your volume is less than 100 TiB, select **No**. If the quota of your volume is greater than 100 TiB, select **Yes**.

+ [!INCLUDE [Large volumes warning](includes/large-volumes-notice.md)]

+* [Requirements and considerations for large volumes](large-volumes-requirements-considerations.md)

+* [Manage default and individual user and group quotas for a volume](manage-default-individual-user-group-quotas.md)

+* [Manage default and individual user and group quotas for a volume](manage-default-individual-user-group-quotas.md)

+ * You can only use cross-zone replication in regions that support the availability zone volume placement. [!INCLUDE [Azure NetApp Files cross-zone-replication supported regions](includes/cross-zone-regions.md)]

+* To establish cross-zone replication, you must create the source volume in an availability zone.

+* You can use cross-zone replication with SMB and NFS volumes. Replication of SMB volumes requires an Active Directory connection in the source and destination NetApp accounts. The destination AD connection must have access to the DNS servers or AD DS Domain Controllers that are reachable from the delegated subnet in the destination zone. For more information, see [Requirements for Active Directory connections](create-active-directory-connections.md#requirements-for-active-directory-connections).

+* The replication destination volume is read-only until you fail over to the destination zone to enable the destination volume for read and write. For more information about the failover process, see [fail over to the destination volume](cross-region-replication-manage-disaster-recovery.md#fail-over-to-destination-volume).

+* Cross-zone replication does not support cascading and fan in/out topologies.

+* At this time, you can't configure volume replication for source volumes created from snapshot with cross-zone replication.

+* After you set up cross-zone replication, the replication process creates *SnapMirror snapshots* to provide references between the source volume and the destination volume. SnapMirror snapshots are cycled automatically when a new one is created for every incremental transfer. You cannot delete SnapMirror snapshots until you delete the replication relationship and volume.

+* You can delete manual snapshots on the source volume of a replication relationship when the replication relationship is active or broken, and also after you've deleted replication relationship. You cannot delete manual snapshots for the destination volume until you break the replication relationship.

+* You can't revert a source or destination volume of cross-zone replication to a snapshot. The snapshot revert functionality is unavailable out for volumes in a replication relationship.

+* You can't currently use cross-zone replication with [large volumes](azure-netapp-files-understand-storage-hierarchy.md#large-volumes) (larger than 100 TiB).

+ Title: Understand default and individual user and group quotas for Azure NetApp Files volumes | Microsoft Docs

+description: Helps you understand the use cases of managing default and individual user and group quotas for Azure NetApp Files volumes.

+documentationcenter: ''

+editor: ''

+ms.assetid:

+ na

+# Understand default and individual user and group quotas

+User and group quotas enable you to restrict the logical space that a user or group can consume in a volume. User and group quotas apply to a specific Azure NetApp Files volume.

+## Introduction

+You can restrict user capacity consumption on Azure NetApp Files volumes by setting user and/or group quotas on volumes. User and group quotas differ from volume quotas in the way that they further restrict volume capacity consumption at the user and group level.

+To set a [volume quota](volume-quota-introduction.md), you can use the Azure portal or the Azure NetApp Files API to specify the maximum storage capacity for a volume. Once you set the volume quota, it defines the size of the volume, and there's no restriction on how much capacity any user can consume.

+To restrict usersΓÇÖ capacity consumption, you can set a user and/or group quota. You can set default and/or individual quotas. Once you set user or group quotas, users can't store more data in the volume than the specified user or group quota limit.

+By combining volume and user quotas, you can ensure that storage capacity is distributed efficiently and prevent any single user, or group of users, from consuming excessive amounts of storage.

+To understand considerations and manage user and group quotas for Azure NetApp Files volumes, see [Manage default and individual user and group quotas for a volume](manage-default-individual-user-group-quotas.md).

+## Behavior of default and individual user and group quotas

+This section describes the behavior of user and group quotas.

+The following concepts and behavioral aspects apply to user and group quotas:

+* The volume capacity that can be consumed can be restricted at the user and/or group level.

+ * User quotas are available for SMB, NFS, and dual-protocol volumes.

+ * Group quotas are **not** supported on SMB and dual-protocol volumes.

+* When a user or group consumption reaches the maximum configured quota, further space consumption is prohibited.

+* Individual user quota takes precedence over default user quota.

+* Individual group quota takes precedence over default group quota.

+* If you set group quota and user quota, the most restrictive quota is the effective quota.

+The following subsections describe and depict the behavior of the various quota types.

+### Default user quota

+A default user quota automatically applies a quota limit to *all users* accessing the volume without creating separate quotas for each target user. Each user can only consume the amount of storage as defined by the default user quota setting. No single user can exhaust the volumeΓÇÖs capacity, as long as the default user quota is less than the volume quota. The following diagram depicts this behavior.

+### Individual user quota

+An individual user quota applies a quota to *individual target user* accessing the volume. You can specify the target user by a UNIX user ID (UID) or a Windows security identifier (SID), depending on volume protocol (NFS or SMB). You can define multiple individual user quota settings on a volume. Each user can only consume the amount of storage as defined by their individual user quota setting. No single user can exhaust the volumeΓÇÖs capacity, as long as the individual user quota is less than the volume quota. Individual user quotas override a default user quota, where applicable. The following diagram depicts this behavior.

+### Combining default and individual user quotas

+You can create quota exceptions for specific users by allowing those users less or more capacity than a default user quota setting by combining default and individual user quota settings. In the following example, individual user quotas are set for `user1`, `user2`, and `user3`. Any other user is subjected to the default user quota setting. The individual quota settings can be smaller or larger than the default user quota setting. The following diagram depicts this behavior.

+### Default group quota

+A default group quota automatically applies a quota limit to *all users within all groups* accessing the volume without creating separate quotas for each target group. The total consumption for all users in any group can't exceed the group quota limit. Group quotas arenΓÇÖt applicable to SMB and dual-protocol volumes. A single user can potentially consume the entire group quota. The following diagram depicts this behavior.

+### Individual group quota

+An individual group quota applies a quota to *all users within an individual target group* accessing the volume. The total consumption for all users *in that group* can't exceed the group quota limit. Group quotas arenΓÇÖt applicable to SMB and dual-protocol volumes. You specify the group by a UNIX group ID (GID). Individual group quotas override default group quotas where applicable. The following diagram depicts this behavior.

+### Combining individual and default group quota

+You can create quota exceptions for specific groups by allowing those groups less or more capacity than a default group quota setting by combining default and individual group quota settings. Group quotas arenΓÇÖt applicable to SMB and dual-protocol volumes. In the following example, individual group quotas are set for `group1` and `group2`. Any other group is subjected to the default group quota setting. The individual group quota settings can be smaller or larger than the default group quota setting. The following diagram depicts this scenario.

+### Combining default and individual user and group quotas

+You can combine the various previously described quota options to achieve very specific quota definitions. You can create very specific quota definitions by (optionally) starting with defining a default group quota, followed by individual group quotas matching your requirements. Then you can further tighten individual user consumption by first (optionally) defining a default user quota, followed by individual user quotas matching individual user requirements. Group quotas arenΓÇÖt applicable to SMB and dual-protocol volumes. In the following example, a default group quota has been set as well as individual group quotas for `group1` and `group2`. Furthermore, a default user quota has been set as well as individual quotas for `user1`, `user2`, `user3`, `user5`, and `userZ`. The following diagram depicts this scenario.

+## Observing user quota settings and consumption

+Users can observe user quota settings and consumption from their client systems connected to the NFS, SMB, or dual-protocol volumes respectively. Azure NetApp Files currently doesn't support reporting of group quota settings and consumption explicitly. The following sections describe how users can view their user quota setting and consumption.

+### Windows client

+Windows users can observe their user quota and consumption in Windows Explorer and by running the dir command. Assume a scenario where a 2-TiB volume with a 100-MiB default or individual user quota has been configured. On the client, this scenario is represented as follows:

+* Administrator view:

+ :::image type="content" source="../media/azure-netapp-files/user-quota-administrator-view.png" alt-text="Screenshot showing administrator view of user quota and consumption.":::

+* User view:

+ :::image type="content" source="../media/azure-netapp-files/user-quota-user-view.png" alt-text="Screenshot showing user view of user quota and consumption.":::

+### Linux client

+Linux users can observe their *user* quota and consumption by using the [`quota(1)`](https://man7.org/linux/man-pages/man1/quota.1.html) command. Assume a scenario where a 2-TiB volume with a 100-MiB default or individual user quota has been configured. On the client, this scenario is represented as follows:

+Azure NetApp Files currently doesn't support group quota reporting. However, you know you've reached your groupΓÇÖs quota limit when you receive a `Disk quota exceeded` error in writing to the volume while you havenΓÇÖt reached your user quota yet.

+In the following scenario, users `user4` and `user5` are members of `group2`. The group `group2` has a 200-MiB default or individual group quota assigned. The volume is already populated with 150 MiB of data owned by user `user4`. User `user5` appears to have a 100-MiB quota available as reported by the `quota(1)` command, but `user5` canΓÇÖt consume more than 50 MiB due to the remaining group quota for `group2`. User `user5` receives a `Disk quota exceeded` error message after writing 50 MiB, despite not reaching the user quota.

+> [!IMPORTANT]

+> For quota reporting to work, the client needs access to port 4049/UDP on the Azure NetApp Files volumesΓÇÖ storage endpoint. When using NSGs with standard network features on the Azure NetApp Files delegated subnet, make sure that access is enabled.

+## Next steps

+* [Manage default and individual user and group quotas for a volume](manage-default-individual-user-group-quotas.md)

+* [Resource limits for Azure NetApp Files](azure-netapp-files-resource-limits.md)

+* [Security identifiers](/windows-server/identity/ad-ds/manage/understand-security-identifiers)

+ Title: Requirements and considerations for large volumes | Microsoft Docs

+description: Describes the requirements and considerations you need to be aware of before using large volumes.

+documentationcenter: ''

+editor: ''

+ms.assetid:

+ na

+# Requirements and considerations for large volumes (preview)

+This article describes the requirements and considerations you need to be aware of before using [large volumes](azure-netapp-files-understand-storage-hierarchy.md#large-volumes) on Azure NetApp Files.

+## Register the feature

+The large volumes feature for Azure NetApp Files is currently in public preview. This preview is offered under the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) and is controlled via Azure Feature Exposure Control (AFEC) settings on a per subscription basis.

+To enroll in the preview for large volumes, use the [large volumes preview sign-up form](https://aka.ms/anflargevolumespreviewsignup).

+## Requirements and considerations

+* Existing regular volumes can't be resized over 100 TiB. You can't convert regular Azure NetApp Files volumes to large volumes.

+* You must create a large volume at a size greater than 100 TiB. A single volume can't exceed 500 TiB.

+* You can't resize a large volume to less than 100 TiB. You can only resize a large volume can up to 30% of lowest provisioned size.

+* Large volumes are currently not supported with Azure NetApp Files backup.

+* Large volumes are not currently supported with cross-region replication.

+* You can't create a large volume with application volume groups.

+* Large volumes aren't currently supported with cross-zone replication.

+* The SDK for large volumes isn't currently available.

+* Throughput ceilings for the three performance tiers (Standard, Premium, and Ultra) of large volumes are based on the existing 100-TiB maximum capacity targets. You'll be able to grow to 500 TiB with the throughput ceiling as per the table below.

+| Capacity tier | Volume size (TiB) | Throughput (MiB/s) |

+| | | |

+| Standard | 100 to 500 | 1,600 |

+| Premium | 100 to 500 | 6,400 |

+| Ultra | 100 to 500 | 10,240 |

+## Supported regions

+Support for Azure NetApp Files large volumes is available in the following regions:

+* Australia East

+* Australia Southeast

+* Brazil South

+* Canada Central

+* Central US

+* East US

+* East US 2

+* Germany West Central

+* Japan East

+* North Central US

+* North Europe

+* South Central US

+* Switzerland North

+* UAE North

+* UK West

+* UK South

+* West Europe

+* West US

+* West US 2

+* West US 3

+## Configure large volumes

+>[!IMPORTANT]

+>Before you can use large volumes, you must first request [an increase in regional capacity quota](azure-netapp-files-resource-limits.md#request-limit-increase).

+Once your [regional capacity quota](regional-capacity-quota.md) has increased, you can create volumes that are up to 500 TiB in size. When creating a volume, after you designate the volume quota, you must select **Yes** for the **Large volume** field. Once created, you can manage your large volumes in the same manner as regular volumes.

+## Next steps

+* [Storage hierarchy of Azure NetApp Files](azure-netapp-files-understand-storage-hierarchy.md)

+* [Resource limits for Azure NetApp Files](azure-netapp-files-resource-limits.md)

+* [Create an NFS volume](azure-netapp-files-create-volumes.md)

+* [Create an SMB volume](azure-netapp-files-create-volumes-smb.md)

+* [Create a dual-protocol volume](create-volumes-dual-protocol.md)

+ Title: Manage default and individual user and group quotas for Azure NetApp Files volumes | Microsoft Docs

+description: Describes the considerations and steps for managing user and group quotas for Azure NetApp Files volumes.

+# Manage default and individual user and group quotas for a volume

+This article explains the considerations and steps for managing user and group quotas on Azure NetApp Files volumes. To understand the use cases for this feature, see [Understand default and individual user and group quotas](default-individual-user-group-quotas-introduction.md).

+## Quotas in cross-region replication relationships

+Quota rules are synced from cross-region replication (CRR) source to destination volumes. Quota rules that you create, delete, or update on a CRR source volume automatically applies to the CRR destination volume.

+Quota rules only come into effect on the CRR destination volume after the replication relationship is deleted because the destination volume is read-only. To learn how to break the replication relationship, see [Delete volume replications](cross-region-replication-delete.md#delete-volume-replications). If source volumes have quota rules and you create the CRR destination volume at the same time as the source volume, all the quota rules are created on destination volume.

+## Considerations

+* A quota rule is specific to a volume and is applied to an existing volume.

+* Deleting a volume results in deleting all the associated quota rules for that volume.

+* You can create a maximum number of 100 quota rules for a volume. You can [request limit increase](azure-netapp-files-resource-limits.md#request-limit-increase) through the portal.

+* Azure NetApp Files doesn't support individual group quota and default group quota for SMB and dual protocol volumes.

+* Group quotas track the consumption of disk space for files owned by a particular group. A file can only be owned by exactly one group.

+* Auxiliary groups only help in permission checks. You can't use auxiliary groups to restrict the quota (disk space) for a file.

+* In a cross-region replication setting:

+ * Currently, Azure NetApp Files doesn't support syncing quota rules to the destination (data protection) volume.

+ * You canΓÇÖt create quota rules on the destination volume until you [delete the replication](cross-region-replication-delete.md).

+ * You need to manually create quota rules on the destination volume if you want them for the volume, and you can do so only after you delete the replication.

+ * If a quota rule is in the error state after you delete the replication relationship, you need to delete and re-create the quota rule on the destination volume.

+ * During sync or reverse resync operations:

+ * If you create, update, or delete a rule on a source volume, you must perform the same operation on the destination volume.

+ * If you create, update, or delete a rule on a destination volume after the deletion of the replication relationship, the rule will be reverted to keep the source and destination volumes in sync.

+* If you're using [large volumes](large-volumes-requirements-considerations.md) (volumes larger than 100 TiB):    

+ * The space and file usage in a large volume might exceed as much as five percent more than the configured hard limit before the quota limit is enforced and rejects traffic.   

+ * To provide optimal performance, the space consumption may exceed configured hard limit before the quota is enforced. The additional space consumption won't exceed either the lower of 1 GB or five percent of the configured hard limit.    

+ * After reaching the quota limit, if a user or administrator deletes files or directories to reduce quota usage under the limit, subsequent quota-consuming file operations may resume with a delay of up to five seconds.

+## Register the feature

+The feature to manage user and group quotas is currently in preview. Before using this feature for the first time, you need to register it.

+1. Register the feature:

+ ```azurepowershell-interactive

+ Register-AzProviderFeature -ProviderNamespace Microsoft.NetApp -FeatureName ANFEnableVolumeUserGroupQuota

+ ```

+2. Check the status of the feature registration:

+ ```azurepowershell-interactive

+ Get-AzProviderFeature -ProviderNamespace Microsoft.NetApp -FeatureName ANFEnableVolumeUserGroupQuota

+ ```

+ > [!NOTE]

+ > The **RegistrationState** may be in the `Registering` state for up to 60 minutes before changing to `Registered`. Wait until the status is **Registered** before continuing.

+You can also use [Azure CLI commands](/cli/azure/feature) `az feature register` and `az feature show` to register the feature and display the registration status.

+## Create new quota rules

+1. From the Azure portal, navigate to the volume for which you want to create a quota rule. Select **User and group quotas** in the navigation pane, then click **Add** to create a quota rule for a volume.

+ 

+2. In the **New quota** window that appears, provide information for the following fields, then click **Create**.

+ * **Quota rule name**:

+ The name must be unique within the volume.

+ * **Quota type**:

+ Select one of the following options. For details, see [Understand default and individual user and group quotas](default-individual-user-group-quotas-introduction.md).

+ * `Default user quota`

+ * `Default group quota`

+ * `Individual user quota`

+ * `Individual group quota`

+ * **Quota target**:

+ * NFS volumes:

+ For individual user quota and individual group quota, specify a value in the range of `0` to `4294967295`.

+ For default quota, specify the value as `""`.

+ * SMB volumes:

+ For individual user quota, specify the range in the `^S-1-[0-59]-\d{2}-\d{8,10}-\d{8,10}-\d{8,10}-[1-9]\d{3}` format.

+ * Dual-protocol volumes:

+ For individual user quota using the SMB protocol, specify the range in the `^S-1-[0-59]-\d{2}-\d{8,10}-\d{8,10}-\d{8,10}-[1-9]\d{3}` format.

+ For individual user quota using the NFS protocol, specify a value in the range of `0` to `4294967295`.

+ * **Quota limit**:

+ Specify the limit in the range of `4` to `1125899906842620`.

+ Select `KiB`, `MiB`, `GiB`, or `TiB` from the pulldown.

+## Edit or delete quota rules

+1. On the Azure portal, navigate to the volume whose quota rule you want to edit or delete. Select `…` at the end of the quota rule row, then select **Edit** or **Delete** as appropriate.

+ 

+ 1. If you're editing a quota rule, update **Quota Limit** in the Edit User Quota Rule window that appears.

+

+ 

+ 1. If you're deleting a quota rule, confirm the deletion by selecting **Yes**.

+

+ 

+## Next steps

+* [Understand default and individual user and group quotas](default-individual-user-group-quotas-introduction.md)

+* [Resource limits for Azure NetApp Files](azure-netapp-files-resource-limits.md)

+From the beginning of the service, Azure NetApp Files has been using a capacity-pool provisioning and automatic growth mechanism. Azure NetApp Files volumes are thinly provisioned on an underlying, customer-provisioned capacity pool of a selected tier and size. Volume sizes (quotas) are used to provide performance and capacity, and the quotas can be adjusted on-the-fly at any time. This behavior means that, currently, the volume quota is a performance lever used to control bandwidth to the volume. Currently, underlaying capacity pools automatically grow when the capacity fills up.

+ Resize every provisioned volume to have appropriate buffer based on change rate and alerting or resize turnaround time (for example, 20% based on typical workload considerations), with a maximum of 100 TiB (which is the regular [volume size limit](azure-netapp-files-resource-limits.md#resource-limits). This new volume size, including buffer capacity, should be based on the following factors:

+* [Volume user and group quotas](default-individual-user-group-quotas-introduction.md) (Preview)

+ Azure NetApp Files volumes provide flexible, large and scalable storage shares for applications and users. Storage capacity and consumption by users is only limited by the size of the volume. In some scenarios, you may want to limit this storage consumption of users and groups within the volume. With Azure NetApp Files volume and group quotas, you can now do so. User and/or group quotas enable you to restrict the storage space that a user or group can use within a specific Azure NetApp Files volume. You can choose to set default (same for all users) or individual user quotas on all NFS, SMB, and dual protocol-enabled volumes. On all NFS-enabled volumes, you can set default (same for all users) or individual group quotas.

+* [Large volumes](large-volumes-requirements-considerations.md) (Preview)

+ Regular Azure NetApp Files volumes are limited to 100 TiB in size. Azure NetApp Files [large volumes](azure-netapp-files-understand-storage-hierarchy.md#large-volumes) break this barrier by enabling volumes of 100 TiB to 500 TiB in size. The large volumes capability enables a variety of use cases and workloads that require large volumes with a single directory namespace.

+

+ Title: "Intent recognition with CLU quickstart - Speech service"

+description: In this quickstart, you recognize intents from audio data with the Speech service and Language service.

+zone_pivot_groups: programming-languages-set-thirteen

+keywords: intent recognition

+# Quickstart: Recognize intents with Conversational Language Understanding

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn more about speech recognition](how-to-recognize-speech.md)

+> [!IMPORTANT]

+> LUIS will be retired on October 1st 2025 and starting April 1st 2023 you will not be able to create new LUIS resources. We recommend [migrating your LUIS applications](/azure/cognitive-services/language-service/conversational-language-understanding/how-to/migrate-from-luis) to [conversational language understanding](/azure/cognitive-services/language-service/conversational-language-understanding/overview) to benefit from continued product support and multilingual capabilities.

+>

+> Conversational Language Understanding (CLU) is available for C# and C++ with the [Speech SDK](speech-sdk.md) version 1.25 or later. See the [quickstart](get-started-intent-recognition-clu.md) to recognize intents with the Speech SDK and CLU.

+## Pronunciation assessment in streaming mode

+Pronunciation assessment supports uninterrupted streaming mode. The recording time can be unlimited through the Speech SDK. As long as you don't stop recording, the evaluation process doesn't finish and you can pause and resume evaluation conveniently. In streaming mode, the `AccuracyScore`, `FluencyScore` , and `CompletenessScore` will vary over time throughout the recording and evaluation process.

+For how to use Pronunciation Assessment in streaming mode in your own application, see [sample code](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/csharp/sharedcontent/console/speech_recognition_samples.cs#:~:text=PronunciationAssessmentWithStream).

+For how to use Pronunciation Assessment in streaming mode in your own application, see [sample code](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/cpp/windows/console/samples/speech_recognition_samples.cpp#:~:text=PronunciationAssessmentWithStream).

+For how to use Pronunciation Assessment in streaming mode in your own application, see [sample code](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/java/android/sdkdemo/app/src/main/java/com/microsoft/cognitiveservices/speech/samples/sdkdemo/MainActivity.java#L548).

+For how to use Pronunciation Assessment in streaming mode in your own application, see [sample code](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/js/node/pronunciationAssessment.js).

+- Learn our quality [benchmark](https://techcommunity.microsoft.com/t5/ai-cognitive-services-blog/speech-service-update-hierarchical-transformer-for-pronunciation/ba-p/3740866)

+- Check out easy-to-deploy Pronunciation Assessment [demo](https://github.com/Azure-Samples/Cognitive-Speech-TTS/tree/master/PronunciationAssessment/BrowserJS) and watch the [video tutorial](https://www.youtube.com/watch?v=zFlwm7N4Awc) of pronunciation assessment.

+Use pattern matching if:

+* You're only interested in matching strictly what the user said. These patterns match more aggressively than [conversational language understanding (CLU)](/azure/cognitive-services/language-service/conversational-language-understanding/overview).

+* You don't have access to a CLU model, but still want intents.

+Use pattern matching if:

+* You're only interested in matching strictly what the user said. These patterns match more aggressively than [conversational language understanding (CLU)](/azure/cognitive-services/language-service/conversational-language-understanding/overview).

+* You don't have access to a CLU model, but still want intents.

+The Speech SDK provides an embedded pattern matcher that you can use to recognize intents in a very strict way. This is useful for when you need a quick offline solution. This works especially well when the user is going to be trained in some way or can be expected to use specific phrases to trigger intents. For example: "Go to floor seven", or "Turn on the lamp" etc. It is recommended to start here and if it no longer meets your needs, switch to using LUIS or a combination of the two.

+Use pattern matching if:

+* You're only interested in matching strictly what the user said. These patterns match more aggressively than [conversational language understanding (CLU)](/azure/cognitive-services/language-service/conversational-language-understanding/overview).

+* You don't have access to a CLU model, but still want intents.

+For more information, see the [pattern matching concepts](./pattern-matching-overview.md) and then:

+* Start with [simple pattern matching](how-to-use-simple-language-pattern-matching.md).

+* Improve your pattern matching by using [custom entities](how-to-use-custom-entity-pattern-matching.md).

+## Conversational Language Understanding

+Conversational language understanding (CLU) enables users to build custom natural language understanding models to predict the overall intention of an incoming utterance and extract important information from it.

+Both a Speech resource and Language resource are required to use CLU with the Speech SDK. The Speech resource is used to transcribe the user's speech into text, and the Language resource is used to recognize the intent of the utterance. To get started, see the [quickstart](get-started-intent-recognition-clu.md).

+> [!IMPORTANT]

+> When you use conversational language understanding with the Speech SDK, you are charged both for the Speech-to-text recognition request and the Language service request for CLU. For more information about pricing for conversational language understanding, see [Language service pricing](https://azure.microsoft.com/pricing/details/cognitive-services/language-service/).

+For information about how to use conversational language understanding without the Speech SDK and without speech recognition, see the [Language service documentation](/azure/cognitive-services/language-service/conversational-language-understanding/overview).

+> [!IMPORTANT]

+> LUIS will be retired on October 1st 2025 and starting April 1st 2023 you will not be able to create new LUIS resources. We recommend [migrating your LUIS applications](/azure/cognitive-services/language-service/conversational-language-understanding/how-to/migrate-from-luis) to [conversational language understanding](/azure/cognitive-services/language-service/conversational-language-understanding/overview) to benefit from continued product support and multilingual capabilities.

+>

+> Conversational Language Understanding (CLU) is available for C# and C++ with the [Speech SDK](speech-sdk.md) version 1.25 or later. See the [quickstart](get-started-intent-recognition-clu.md) to recognize intents with the Speech SDK and CLU.

+* [Intent recognition with simple pattern matching](how-to-use-simple-language-pattern-matching.md)

+* [Intent recognition with CLU quickstart](get-started-intent-recognition-clu.md)

+- Syllable-level accuracy scores are currently available via the [JSON file](?tabs=json#pronunciation-assessment-results) or [Speech SDK](how-to-pronunciation-assessment.md).

+### Assessment scores in streaming mode

+Pronunciation Assessment supports uninterrupted streaming mode. The demo on the Speech Studio supports up to 60 minutes of recording in streaming mode for evaluation. The Speech Studio demo allows for up to 60 minutes of recording in streaming mode for evaluation. As long as you do not press the stop recording button, the evaluation process does not finish and you can pause and resume evaluation conveniently.

+Pronunciation Assessment evaluates three aspects of pronunciation: accuracy, fluency, and completeness. At the bottom of **Assessment result**, you can see **Pronunciation score** as aggregated overall score which includes 3 sub aspects: **Accuracy score**, **Fluency score**, and **Completeness score**. In streaming mode, since the **Accuracy score**, **Fluency score and Completeness score** will vary over time throughout the recording process, we demonstrate an approach on Speech Studio to display approximate overall score incrementally before the end of the evaluation, which weighted only with Accuracy score and Fluency score. The **Completeness score** is only calculated at the end of the evaluation after you press the stop button, so the final overall score is aggregated from **Accuracy score**, **Fluency score**, and **Completeness score** with weight.

+Refer to the demo examples below for the whole process of evaluating pronunciation in streaming mode.

+**Start recording**

+As you start recording, the scores at the bottom begin to alter from 0.

+**During recording**

+During recording a long paragraph, you can pause recording at any time. You can continue to evaluate your recording as long as you don't press the stop button.

+**Finish recording**

+After you press the stop button, you can see **Pronunciation score**, **Accuracy score**, **Fluency score**, and **Completeness score** at the bottom.

+Regarding the size of the training data, in most cases you can build a reasonable custom neural voice with 500 utterances. According to our tests, adding more training data in most languages does not necessarily improve naturalness of the voice itself (tested using the MOS score), however, with more training data that covers more word instances, you have higher possibility to reduce the ratio of dissatisfactory parts of speech for the voice, such as the glitches. To hear what dissatisfactory parts of speech sound like, refer to [the GitHub examples](https://github.com/Azure-Samples/Cognitive-Speech-TTS/blob/master/CustomVoice/DSAT-examples.md).

+For the free (F0) pricing tier, see also the monthly allowances at the [pricing page](https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/).

+The following sections provide you with a quick guide to the quotas and limits that apply to the Speech service.

+For information about adjustable quotas for Standard (S0) Speech resources, see [additional explanations](#detailed-description-quota-adjustment-and-best-practices), [best practices](#general-best-practices-to-mitigate-throttling-during-autoscaling), and [adjustment instructions](#speech-to-text-increase-online-transcription-concurrent-request-limit). The quotas and limits for Free (F0) Speech resources aren't adjustable.

+This section describes speech-to-text quotas and limits per Speech resource. Unless otherwise specified, the limits aren't adjustable.

+#### Online transcription and speech translation

+> [!IMPORTANT]

+> These limits apply to concurrent speech-to-text online transcription requests and speech translation requests combined. For example, if you have 60 concurrent speech-to-text requests and 40 concurrent speech translation requests, you'll reach the limit of 100 concurrent requests.

+| Quota | Free (F0) | Standard (S0) |

+| Concurrent request limit - base model endpoint | 1 <br/><br/>This limit isn't adjustable. | 100 (default value)<br/><br/>The rate is adjustable for Standard (S0) resources. See [additional explanations](#detailed-description-quota-adjustment-and-best-practices), [best practices](#general-best-practices-to-mitigate-throttling-during-autoscaling), and [adjustment instructions](#speech-to-text-increase-online-transcription-concurrent-request-limit). |

+| Concurrent request limit - custom endpoint | 1 <br/><br/>This limit isn't adjustable. | 100 (default value)<br/><br/>The rate is adjustable for Standard (S0) resources. See [additional explanations](#detailed-description-quota-adjustment-and-best-practices), [best practices](#general-best-practices-to-mitigate-throttling-during-autoscaling), and [adjustment instructions](#speech-to-text-increase-online-transcription-concurrent-request-limit). |

+| Quota | Free (F0) | Standard (S0) |

+The limits in this table apply per Speech resource when you create a Custom Speech model.

+| Quota | Free (F0) | Standard (S0) |

+### Text-to-speech quotas and limits per resource

+This section describes text-to-speech quotas and limits per Speech resource. Unless otherwise specified, the limits aren't adjustable.

+#### Common text-to-speech quotas and limits

+| Quota | Free (F0) | Standard (S0) |

+| Maximum number of transactions per time period for prebuilt neural voices and custom neural voices. | 20 transactions per 60 seconds<br/><br/>This limit isn't adjustable. | 200 transactions per second (TPS) (default value)<br/><br/>The rate is adjustable up to 1000 TPS for Standard (S0) resources. See [additional explanations](#detailed-description-quota-adjustment-and-best-practices), [best practices](#general-best-practices-to-mitigate-throttling-during-autoscaling), and [adjustment instructions](#text-to-speech-increase-concurrent-request-limit). |

+| Max SSML message size per turn for websocket | 64 KB | 64 KB |

+| Quota | Free (F0)| Standard (S0) |

+| Max number of transactions per second (TPS) | Not available for F0 | 200 transactions per second (TPS) (default value) |

+### Speaker recognition quotas and limits per resource

+Speaker recognition is limited to 20 transactions per second (TPS).

+Some of the Speech service quotas are adjustable. This section provides additional explanations, best practices, and adjustment instructions.

+The following quotas are adjustable for Standard (S0) resources. The Free (F0) request limits aren't adjustable.

+- Speech-to-text [concurrent request limit](#online-transcription-and-speech-translation) for base model endpoint and custom endpoint

+- Text-to-speech [maximum number of transactions per time period](#text-to-speech-quotas-and-limits-per-resource) for prebuilt neural voices and custom neural voices

+- Speech translation [concurrent request limit](#online-transcription-and-speech-translation)

+By default, the number of concurrent speech-to-text [online transcription requests and speech translation requests](#online-transcription-and-speech-translation) combined is limited to 100 per resource in the base model, and 100 per custom endpoint in the custom model. For the standard pricing tier, you can increase this amount. Before submitting the request, ensure that you're familiar with the material discussed earlier in this article, such as the best practices to mitigate throttling.

+> Concurrent request limits for base and custom models need to be adjusted separately. You can have a Speech service resource that's associated with many custom endpoints hosting many custom model deployments. As needed, the limit adjustments per custom endpoint must be requested separately.

+Increasing the limit of concurrent requests doesn't directly affect your costs. The Speech service uses a payment model that requires that you pay only for what you use. The limit defines how high the service can scale before it starts throttle your requests.

+Azure OpenAI Service includes a content management system that works alongside core models to filter content. This system works by running both the input prompt and generated content through an ensemble of classification models aimed at detecting misuse. If the system identifies harmful content, you'll receive either an error on the API call if the prompt was deemed inappropriate or the finish_reason on the response will be `content_filter` to signify that some of the generation was filtered. You can generate content with the completions API using many different configurations that will alter the filtering behavior you should expect. The following section aims to enumerate all of these scenarios for you to appropriately design your solution.

+description: Learn about the different model capabilities that are available with Azure OpenAI.

+Azure OpenAI provides access to many different models, grouped by family and capability. A model family typically associates models by their intended task. The following table describes model families currently available in Azure OpenAI. Not all models are available in all regions currently. Refer to the [model capability table](#model-capabilities) in this article for a full breakdown.

+Each model family has a series of models that are further distinguished by capability. These capabilities are typically identified by names, and the alphabetical order of these names generally signifies the relative capability and cost of that model within a given model family. For example, GPT-3 models use names such as Ada, Babbage, Curie, and Davinci to indicate relative capability and cost. Davinci is more capable and more expensive than Curie, which in turn is more capable and more expensive than Babbage, and so on.

+Azure OpenAI model names typically correspond to the following standard naming convention:

+> The older versions of GPT-3 models named `ada`, `babbage`, `curie`, and `davinci` that don't follow the standard naming convention are primarily intended for fine tuning. For more information, see [Learn how to customize a model for your application](../how-to/fine-tuning.md).

+You can get a list of models that are available for both inference and fine-tuning by your Azure OpenAI resource by using the [Models List API](/rest/api/cognitiveservices/azureopenaistable/models/list).

+We recommend starting with the most capable model in a model family to confirm whether the model capabilities meet your requirements. Then you can stay with that model or move to a model with lower capability and cost, optimizing around that model's capabilities.

+The GPT-3 models can understand and generate natural language. The service offers four model capabilities, each with different levels of power and speed suitable for different tasks. Davinci is the most capable model, while Ada is the fastest. In the order of greater to lesser capability, the models are:

+- `text-curie-001`

+- `text-babbage-001`

+- `text-ada-001`

+While Davinci is the most capable, the other models provide significant speed advantages. Our recommendation is for users to start with Davinci while experimenting, because it produces the best results and validate the value that Azure OpenAI can provide. Once you have a prototype working, you can then optimize your model choice with the best latency/performance balance for your application.

+TheyΓÇÖre most capable in Python and proficient in over a dozen languages, including C#, JavaScript, Go, Perl, PHP, Ruby, Swift, TypeScript, SQL, and Shell. In the order of greater to lesser capability, the Codex models are:

+- `code-cushman-001`

+Similar to GPT-3, Davinci is the most capable Codex model and can perform any task the other models can perform, often with less instruction. For applications requiring deep understanding of the content, Davinci produces the best results. Greater capabilities require more compute resources, so Davinci costs more and isn't as fast as other models.

+When using our embeddings models, keep in mind their limitations and risks.

+| Model ID | Supports Completions | Supports Embeddings | Base model Regions | Fine-Tuning Regions |

+| ada | Yes | No | N/A | East US, South Central US, West Europe |

+| text-ada-001 | Yes | No | East US, South Central US, West Europe | N/A |

+| babbage | Yes | No | N/A | East US, South Central US, West Europe |

+| text-babbage-001 | Yes | No | East US, South Central US, West Europe | N/A |

+| curie | Yes | No | N/A | East US, South Central US, West Europe |

+| text-curie-001 | Yes | No | East US, South Central US, West Europe | N/A |

+| davinci¹ | Yes | No | N/A | East US, South Central US, West Europe |

+| text-davinci-001 | Yes | No | South Central US, West Europe | N/A |

+| text-davinci-002 | Yes | No | East US, South Central US, West Europe | N/A |

+| text-davinci-003 | Yes | No | East US | N/A |

+| text-davinci-fine-tune-002¹ | Yes | No | N/A | East US, West Europe |

+¹ The model is available by request only. Currently we aren't accepting new requests to use the model.

+| Model ID | Supports Completions | Supports Embeddings | Base model Regions | Fine-Tuning Regions |

+| code-cushman-001² | Yes | No | South Central US, West Europe | East US, South Central US, West Europe |

+| code-davinci-002 | Yes | No | East US, West Europe | N/A |

+| code-davinci-fine-tune-002² | Yes | No | N/A | East US, West Europe |

+² The model is available for fine-tuning by request only. Currently we aren't accepting new requests to fine-tune the model.

+| Model ID | Supports Completions | Supports Embeddings | Base model Regions | Fine-Tuning Regions |

+[Learn more about Azure OpenAI](../overview.md)

+Azure Communications Gateway also offers metrics for monitoring your deployment.

+The Azure Communications Gateway service includes continuous monitoring for potential faults in your deployment. The metrics we monitor cover all metrics that operators must monitor as part of the Operator Connect program and include:

+##### Representation challenges workarounds

+It is possible that an old document, with an incorrect schema, was used to create your container's analytical store base schema. Based on all the rules presented above, you may be receiving `NULL` for certain properties when querying your analytical store using Azure Synapse Link. To delete or update the problematic documents won't help because base schema reset isn't currently supported. The possible solutions are:

+ * To abandon the property with the wrong schema and add a new one, with another name, that has the correct schema in all documents. Example: You have billions of documents in the **Orders** container where the **status** property is a string. But the first document in that container has **status** defined with integer. So, one document will have **status** correctly represented and all other documents will have `NULL`. You can add the **status2** property to all documents and start to use it, instead of the original property.

+#### Best practices for multi-region writes

+Here are some best practices to consider when writing to multiple regions.

+#### Keep local traffic local

+When you use multi-region writes, the application should issue read and write traffic originating in the local region, strictly to the local Cosmos DB region. You must avoid cross-region calls for optimal performance.

+It's important for the application to minimize conflicts by avoiding the following anti-patterns:

+* Sending the same write operation to all regions to hedge bets on response times from the fastest region.

+* Randomly determining the target region for a read or write operation on a per request basis.

+* Using a Round Robin policy to determine the target region for a read or write operation on a per request basis.

+#### Avoid dependency on replication lag

+Multi-region write accounts can't be configured for Strong Consistency. Thus, the region being written to responds immediately after replicating the data locally while asynchronously replicating the data globally.

+While infrequent, a replication lag may occur on one or a few partitions when geo-replicating data. Replication lag can occur due to rare blips in network traffic or higher than usual rates of conflict resolution.

+For instance, an architecture in which the application writes to Region A but reads from Region B introduces a dependency on replication lag between the two regions. However, if the application reads and writes to the same region, performance remains constant even in the presence of replication lag.

+#### Session Consistency Usage for Write operations

+In Session Consistency, the session token is used for both read and write operations.

+For read operations, the cached session token is sent to the server with a guarantee of receiving data corresponding to the specified (or a more recent) session token.

+For write operations, the session token is sent to the database with a guarantee of persisting the data only if the server has caught up to the session token provided. In single-region write accounts, the write region is always guaranteed to have caught up to the session token. However, in multi-region write accounts, the region you write to may not have caught up to writes issued to another region. If the client writes to Region A with a session token from Region B, Region A won't be able to persist the data until it has caught up to changes made in Region B.

+It's best to use session tokens only for read operations and not for write operations when passing session tokens between client instances.

+#### Rapid updates to the same document

+The server's updates to resolve or confirm the absence of conflicts can collide with writes triggered by the application when the same document is repeatedly updated. Repeated updates in rapid succession to the same document experience higher latencies during conflict resolution. While occasional bursts in repeated updates to the same document are inevitable, it would be worth exploring an architecture where new documents are created instead if steady state traffic sees rapid updates to the same document over an extended period.

+Multi-region accounts experience different behaviors depending on the following table.

+| Single write region | Read region outage | All clients will automatically redirect reads to other regions. No read or write availability loss for all configurations, except 2 regions with strong consistency, which loses write availability until restoration of the service or, if you enable **service-managed failover**, the service marks the region as failed and a failover occurs. | No data loss. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support read traffic. <br/> When the outage is over, readjust provisioned RUs as appropriate. |

+| Single write region | Write region outage | Clients will redirect reads to other regions. <br/> **Without service-managed failover**, clients experience write availability loss, until restoration of write availability occurs automatically when the outage ends. <br/> **With service-managed failover** clients experience write availability loss until the services manages a failover to a new write region selected according to your preferences. | If you haven't selected the strong consistency level, the service may not replicate some data to the remaining active regions. This replication depends on the consistency level selected as described in [this section](consistency-levels.md#rto). If the affected region suffers permanent data loss, you could lose unreplicated data. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support read traffic. <br/> *Don't* trigger a manual failover during the outage, as it can't succeed. <br/> When the outage is over, readjust provisioned RUs as appropriate. Accounts using API for NoSQLs may also recover the non-replicated data in the failed region from your [conflicts feed](how-to-manage-conflicts.md#read-from-conflict-feed). |

+| Multiple write regions | Any regional outage | Possibility of temporary write availability loss, analogously to single write region with service-managed failover. The failover of the [conflict-resolution region](#conflict-resolution-region) may also cause a loss of write availability if a high number of conflicting writes happen at the time of the outage. | Recently updated data in the failed region may be unavailable in the remaining active regions, depending on the selected [consistency level](consistency-levels.md). If the affected region suffers permanent data loss, you may lose unreplicated data. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support more traffic. <br/> When the outage is over, you may readjust provisioned RUs as appropriate. If possible, Azure Cosmos DB automatically recovers non-replicated data in the failed region. This automatic recovery uses the configured conflict resolution method for API for NoSQL accounts. For accounts other APIs, this automatic recovery uses *Last Write Wins*. |

+* No changes are required in your application code to handle read region outage. When the impacted read region is back online, it will automatically sync with the current write region and will be available again to serve read requests.

+* During a write region outage, the Azure Cosmos DB account will automatically promote a secondary region to be the new primary write region when **automatic (service-managed) failover** is configured on the Azure Cosmos DB account. The failover occurs to another region in the order of region priority you've specified.

+* Manual failover shouldn't be triggered and will not succeed in presence of an outage of the source or destination region. This is because of a consistency check required by the failover procedure, which requires connectivity between the regions.

+|Regional outage ΓÇô data loss | Data loss | Data loss | Dependent on consistency level. For more information, see [Consistency, availability, and performance tradeoffs](./consistency-levels.md). | Dependent on consistency level. For more information, see [Consistency, availability, and performance tradeoffs](./consistency-levels.md). | Dependent on consistency level. For more information, see [Consistency, availability, and performance tradeoffs](./consistency-levels.md).

+| Single write region | Not enabled | If there was an outage in a read region when not using strong consistency, all clients will redirect to other regions. No read or write availability loss. No data loss. When using strong consistency, read region outage can affect write availability if fewer than two read regions remaining.<br/> If there was an outage in the write region, clients experience write availability loss. If you haven't selected the strong consistency level, the service may not replicate some data to the remaining active regions. This replication depends on the consistency level selected as described in [this section](consistency-levels.md#rto). If the affected region suffers permanent data loss, you may lose unreplicated data. <br/> Azure Cosmos DB restores write availability automatically when the outage ends. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support read traffic. <br/> *Don't* trigger a manual failover during the outage, as it can't succeed. <br/> When the outage is over, readjust provisioned RUs as appropriate. |

+| Single write region | Enabled | If there was an outage in a read region when not using strong consistency, all clients will redirect to other regions. No read or write availability loss. No data loss. When using strong consistency, read region outage can affect write availability if fewer than two read regions remaining.<br/> If there was an outage in the write region, clients experience write availability loss until Azure Cosmos DB automatically elects a new region as the new write region according to your preferences. If you haven't selected the strong consistency level, the service may not replicate some data to the remaining active regions. This replication depends on the consistency level selected as described in [this section](consistency-levels.md#rto). If the affected region suffers permanent data loss, you may lose unreplicated data. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support read traffic. <br/> *Don't* trigger a manual failover during the outage, as it can't succeed. <br/> When the outage is over, you may move the write region back to the original region, and readjust provisioned RUs as appropriate. Accounts using API for NoSQLs may also recover the non-replicated data in the failed region from your [conflicts feed](how-to-manage-conflicts.md#read-from-conflict-feed). |

+| Multiple write regions | Not applicable | Recently updated data in the failed region may be unavailable in the remaining active regions. Eventual, consistent prefix, and session consistency levels guarantee a staleness of <15 mins. Bounded staleness guarantees less than K updates or T seconds, depending on the configuration. If the affected region suffers permanent data loss, you may lose unreplicated data. | During the outage, ensure that there are enough provisioned RUs in the remaining regions to support more traffic. <br/> When the outage is over, you may readjust provisioned RUs as appropriate. If possible, Azure Cosmos DB automatically recovers non-replicated data in the failed region. This automatic recovery uses the configured conflict resolution method for API for NoSQL accounts. For accounts other APIs, this automatic recovery uses *Last Write Wins*. |

+The query plan, for a query scoped to a single partition, is cached on the client. This eliminates the need to make a call to the gateway to retrieve the query plan after the first call. The key for the cached query plan is the SQL query string. You need to **make sure the query is [parametrized](query/parameterized-queries.md)**. If not, the query plan cache lookup will often be a cache miss as the query string is unlikely to be identical across calls. Query plan caching is **enabled by default for Java SDK version 4.20.0 and above** and **for Spring Data Azure Cosmos DB SDK version 3.13.0 and above**.

+ Title: Quickstart - Use Spring Data Azure Cosmos DB v3 to create a document database using Azure Cosmos DB

+description: This quickstart presents a Spring Data Azure Cosmos DB v3 code sample you can use to connect to and query the Azure Cosmos DB for NoSQL

+# Quickstart: Build a Spring Data Azure Cosmos DB v3 app to manage Azure Cosmos DB for NoSQL data

+In this quickstart, you create and manage an Azure Cosmos DB for NoSQL account from the Azure portal, and by using a Spring Data Azure Cosmos DB v3 app cloned from GitHub.

+First, you create an Azure Cosmos DB for NoSQL account using the Azure portal. Alternately, without a credit card or an Azure subscription, you can set up a free [Try Azure Cosmos DB account](https://aka.ms/trycosmosdb). You can then create a Spring Boot app using the Spring Data Azure Cosmos DB v3 connector, and then add resources to your Azure Cosmos DB account by using the Spring Boot application.

+Azure Cosmos DB is a multi-model database service that lets you quickly create and query document, table, key-value, and graph databases with global distribution and horizontal scale capabilities.

+> [!IMPORTANT]

+> These release notes are for version 3 of Spring Data Azure Cosmos DB. You can find release notes for version 2 at [Spring Data Azure Cosmos DB v2 for API for NoSQL (legacy): Release notes and resources](sdk-java-spring-data-v2.md).

+> Spring Data Azure Cosmos DB supports only the API for NoSQL.

+>

+> See the following articles for information about Spring Data on other Azure Cosmos DB APIs:

+* An Azure account with an active subscription.

+ * No Azure subscription? You can [try Azure Cosmos DB free](../try-free.md) with no credit card required.

+* [Java Development Kit (JDK) 8](https://www.azul.com/downloads/azure-only/zulu/?&version=java-8-lts&architecture=x86-64-bit&package=jdk). Set the `JAVA_HOME` environment variable to the JDK install folder.

+* A [Maven binary archive](https://maven.apache.org/download.cgi). On Ubuntu, run `apt-get install maven` to install Maven.

+* [Git](https://www.git-scm.com/downloads). On Ubuntu, run `sudo apt-get install git` to install Git.

+*The structure of an Azure Cosmos DB account.* Irrespective of API or programming language, an Azure Cosmos DB *account* contains zero or more *databases*, a *database* (DB) contains zero or more *containers*, and a *container* contains zero or more items, as shown in the following diagram:

+For more information about databases, containers, and items, see [Azure Cosmos DB resource model](../account-databases-containers-items.md). A few important properties are defined at the level of the container, among them *provisioned throughput* and *partition key*.

+The provisioned throughput is measured in Request Units (*RUs*) which have a monetary price and are a substantial determining factor in the operating cost of the account. You can select provisioned throughput at per-container granularity or per-database granularity. However, you should prefer container-level throughput specification. For more information, see [Introduction to provisioned throughput in Azure Cosmos DB](../set-throughput.md).

+As items are inserted into an Azure Cosmos DB container, the database grows horizontally by adding more storage and compute to handle requests. Storage and compute capacity are added in discrete units known as *partitions*. You must choose one field in your documents to be the partition key, which maps each document to a partition.

+The way partitions are managed is that each partition is assigned a roughly equal slice out of the range of partition key values. For this reason, you should choose a partition key that's relatively random or evenly distributed. Otherwise, you get *hot partitions* and *cold partitions*, which see substantially more or fewer requests. For information on avoiding this condition, see [Partitioning and horizontal scaling in Azure Cosmos DB](../partitioning-overview.md).

+Before you can create a document database, you need to create an API for NoSQL account with Azure Cosmos DB.

+Now let's switch to working with code. Let's clone an API for NoSQL app from GitHub, set the connection string, and run it.

+git clone https://github.com/Azure-Samples/azure-spring-boot-samples.git

+This step is optional. If you're interested in learning how the database resources are created in the code, you can review the following snippets. Otherwise, you can skip ahead to [Run the app](#run-the-app).

+### [Passwordless (Recommended)](#tab/passwordless)

+In this section, the configurations and the code don't have any authentication operations. However, connecting to Azure service requires authentication. To complete the authentication, you need to use Azure Identity. Spring Cloud Azure uses `DefaultAzureCredential`, which Azure Identity provides to help you get credentials without any code changes.

+`DefaultAzureCredential` supports multiple authentication methods and determines which method to use at runtime. This approach enables your app to use different authentication methods in different environments (local vs. production) without implementing environment-specific code. For more information, see the [Default Azure credential](/azure/developer/java/sdk/identity-azure-hosted-auth#default-azure-credential) section of [Authenticate Azure-hosted Java applications](/azure/developer/java/sdk/identity-azure-hosted-auth).

+### Authenticate using DefaultAzureCredential

+You can authenticate to Cosmos DB for NoSQL using `DefaultAzureCredential` by adding the `azure-identity` [dependency](https://mvnrepository.com/artifact/com.azure/azure-identity) to your application. `DefaultAzureCredential` automatically discovers and uses the account you signed in with in the previous step.

+Configure the Azure Database for MySQL credentials in the *application.yml* configuration file in the *cosmos/spring-cloud-azure-starter-data-cosmos/spring-cloud-azure-data-cosmos-sample* directory. Replace the values of `${AZURE_COSMOS_ENDPOINT}` and `${COSMOS_DATABASE}`.

+```yaml

+spring:

+ cloud:

+ azure:

+ cosmos:

+ endpoint: ${AZURE_COSMOS_ENDPOINT}

+ database: ${COSMOS_DATABASE}

+```

+After Spring Boot and Spring Data create the Azure Cosmos DB account, database, and container, they connect to the database and container for `delete`, `add`, and `find` operations.

+### [Password](#tab/password)

+### Application configuration file

+The following section shows how Spring Boot and Spring Data use configuration instead of code to establish an Azure Cosmos DB client and connect to Azure Cosmos DB resources. At application startup Spring Boot handles all of this boilerplate using the following settings in *application.yml*:

+```yaml

+spring:

+ cloud:

+ azure:

+ cosmos:

+ key: ${AZURE_COSMOS_KEY}

+ endpoint: ${AZURE_COSMOS_ENDPOINT}

+ database: ${COSMOS_DATABASE}

+Once you create an Azure Cosmos DB account, database, and container, just fill-in-the-blanks in the config file and Spring Boot/Spring Data does the following: (1) creates an underlying Java SDK `CosmosClient` instance with the URI and key, and (2) connects to the database and container. You're all set - no more resource management code!

+Spring Data provides a simple, clean, standardized, and platform-independent interface for operating on datastores, as shown in the following examples. These CRUD and query examples enable you to manipulate Azure Cosmos DB documents by using Spring Data Azure Cosmos DB. These examples build on the Spring Data GitHub sample linked to earlier in this article.

+ ```java

+ // Save the User class to Azure Cosmos DB database.

+ final Mono<User> saveUserMono = repository.save(testUser);

+ ```

+* Point-reads using the derived query method defined in the repository. The `findById` performs point-reads for `repository`. The fields mentioned in the method name cause Spring Data to execute a point-read defined by the `id` field:

+ ```java

+ // Nothing happens until we subscribe to these Monos.

+ // findById will not return the user as user is not present.

+ final Mono<User> findByIdMono = repository.findById(testUser.getId());

+ final User findByIdUser = findByIdMono.block();

+ Assert.isNull(findByIdUser, "User must be null");

+ ```

+ ```java

+ repository.deleteAll().block();

+ LOGGER.info("Deleted all data in container.");

+ ```

+* Derived query based on repository method name. Spring Data implements the `repository` `findByFirstName` method as a Java SDK SQL query on the `firstName` field. You can't implement this query as a point-read.

+ ```java

+ final Flux<User> firstNameUserFlux = repository.findByFirstName("testFirstName");

+ ```

+Now go back to the Azure portal to get your connection string information. Then, use the following steps to launch the app with your endpoint information so your app can communicate with your hosted database.

+1. In the Git terminal window, `cd` to the sample code folder.

+ ```bash

+ cd azure-spring-boot-samples/cosmos/spring-cloud-azure-starter-data-cosmos/spring-cloud-azure-data-cosmos-sample

+ ```

+1. In the Git terminal window, use the following command to install the required Spring Data Azure Cosmos DB packages.

+ ```bash

+ mvn clean package

+ ```

+1. In the Git terminal window, use the following command to start the Spring Data Azure Cosmos DB application:

+ ```bash

+ mvn spring-boot:run

+ ```

+1. The app loads *application.yml* and connects the resources in your Azure Cosmos DB account.

+1. The app performs point CRUD operations described previously.

+1. The app performs a derived query.

+1. The app doesn't delete your resources. Switch back to the portal to [clean up the resources](#clean-up-resources) from your account if you want to avoid incurring charges.

+In this quickstart, you learned how to create an Azure Cosmos DB for NoSQL account and create a document database and container using the Data Explorer. You then ran a Spring Data app to do the same thing programmatically. You can now import more data into your Azure Cosmos DB account.

+* If all you know is the number of vCores and servers in your existing database cluster, read about [estimating request units using vCores or vCPUs](../convert-vcore-to-request-unit.md)

+# Azure Cosmos DB for NoSQL: Spring Data Azure Cosmos DB v3 examples

+> These release notes are for version 3 of Spring Data Azure Cosmos DB. You can find [release notes for version 2 here](sdk-java-spring-data-v2.md).

+> Spring Data Azure Cosmos DB supports only the API for NoSQL.

+* Links to the tasks in each of the example Spring Data Azure Cosmos DB project files.

+* Spring Data Azure Cosmos DB v3

+You can optionally use Maven to get the latest Spring Data Azure Cosmos DB v3 binaries for use in your project. Maven automatically adds any necessary dependencies. Otherwise, you can directly download the dependencies listed in the **pom.xml** file and add them to your build path.

+ Title: 'Spring Data Azure Cosmos DB v2 for API for NoSQL release notes and resources'

+description: Learn about the Spring Data Azure Cosmos DB v2 for API for NoSQL, including release dates, retirement dates, and changes made between each version of the Azure Cosmos DB SQL Async Java SDK.

+# Spring Data Azure Cosmos DB v2 for API for NoSQL (legacy): Release notes and resources

+ Spring Data Azure Cosmos DB version 2 for NoSQL allows developers to use Azure Cosmos DB in Spring applications. Spring Data Azure Cosmos DB exposes the Spring Data interface for manipulating databases and collections, working with documents, and issuing queries. Both Sync and Async (Reactive) APIs are supported in the same Maven artifact.

+> This version of Spring Data Azure Cosmos DB SDK depends on a retired version of Azure Cosmos DB Java SDK. This Spring Data Azure Cosmos DB SDK will be announced as retiring in the near future! This is *not* the latest Azure Spring Data Azure Cosmos DB SDK for Azure Cosmos DB and is outdated. Because of performance issues and instability in Azure Spring Data Azure Cosmos DB SDK V2, we highly recommend to use [Azure Spring Data Azure Cosmos DB v3](sdk-java-spring-data-v3.md) for your project. To upgrade, follow the instructions in the [Migrate to Azure Cosmos DB Java SDK v4](migrate-java-v4-sdk.md) guide to understand the difference in the underlying Java SDK V4.

+You can use Spring Data Azure Cosmos DB in your applications hosted in [Azure Spring Apps](https://azure.microsoft.com/services/spring-apps/).

+> These release notes are for version 2 of Spring Data Azure Cosmos DB. You can find [release notes for version 3 here](sdk-java-spring-data-v3.md).

+> Spring Data Azure Cosmos DB supports only the API for NoSQL.

+> 2. Create a Spring Data Azure Cosmos DB app by using the [starter](/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-cosmos-db). It's easy!

+> 3. Work through the [Spring Data Azure Cosmos DB developer's guide](/azure/developer/java/spring-framework/how-to-guides-spring-data-cosmosdb), which walks through basic Azure Cosmos DB requests.

+|**API documentation** | [Spring Data Azure Cosmos DB reference documentation]() |

+|**Contribute to the SDK** | [Spring Data Azure Cosmos DB repo on GitHub](https://github.com/microsoft/spring-data-cosmosdb) |

+|**Developer's guide** | [Spring Data Azure Cosmos DB developer's guide](/azure/developer/java/spring-framework/how-to-guides-spring-data-cosmosdb) |

+ Title: 'Spring Data Azure Cosmos DB v3 for API for NoSQL release notes and resources'

+description: Learn about the Spring Data Azure Cosmos DB v3 for API for NoSQL, including release dates, retirement dates, and changes made between each version of the Azure Cosmos DB SQL Async Java SDK.

+# Spring Data Azure Cosmos DB v3 for API for NoSQL: Release notes and resources

+The Spring Data Azure Cosmos DB version 3 for NoSQL allows developers to use Azure Cosmos DB in Spring applications. Spring Data Azure Cosmos DB exposes the Spring Data interface for manipulating databases and collections, working with documents, and issuing queries. Both Sync and Async (Reactive) APIs are supported in the same Maven artifact.

+You can use Spring Data Azure Cosmos DB in your applications hosted in [Azure Spring Apps](https://azure.microsoft.com/services/spring-apps/).

+### Which Version of Azure Spring Data Azure Cosmos DB Should I Use

+Azure Spring Data Azure Cosmos DB library supports multiple versions of Spring Boot / Spring Cloud. Refer to [azure Spring Data Azure Cosmos DB version mapping](https://github.com/Azure/azure-sdk-for-jav#which-version-of-azure-spring-data-cosmos-should-i-use) for detailed information on which version of Azure Spring Data Azure Cosmos DB to use with Spring Boot / Spring Cloud version.

+> These release notes are for version 3 of Spring Data Azure Cosmos DB.

+> Azure Spring Data Azure Cosmos DB SDK has dependency on the Spring Data framework, and supports only the API for NoSQL.

+ Get up and running with Spring Data Azure Cosmos DB by following our [Spring Boot Starter guide](/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-cosmos-db). The Spring Boot Starter approach is the recommended way to get started using the Spring Data Azure Cosmos DB connector.

+ Alternatively, you can add the Spring Data Azure Cosmos DB dependency to your `pom.xml` file as shown below:

+| **Release notes** | [Release notes for Spring Data Azure Cosmos DB SDK v3](https://github.com/Azure/azure-sdk-for-jav) |

+| **SDK Documentation** | [Azure Spring Data Azure Cosmos DB SDK v3 documentation](https://github.com/Azure/azure-sdk-for-jav) |

+| **Get started** | [Quickstart: Build a Spring Data Azure Cosmos DB app to manage Azure Cosmos DB for NoSQL data](./quickstart-java-spring-data.md) <br> [GitHub repo with quickstart code](https://github.com/Azure-Samples/azure-spring-data-cosmos-java-sql-api-getting-started) |

+| **Basic code samples** | [Azure Cosmos DB: Spring Data Azure Cosmos DB examples for the API for NoSQL](samples-java-spring-data.md) <br> [GitHub repo with sample code](https://github.com/Azure-Samples/azure-spring-data-cosmos-java-sql-api-samples)|

+* Spring Data Azure Cosmos DB supports Java JDK 8 and Java JDK 11.

+> [Spring Data Azure Cosmos DB v3 for API for NoSQL](sdk-java-spring-data-v3.md)

+### Fully Qualified Domain Name ( FQDN ) of Azure HDInsight

+If you created a custom private link service, FQDN should end with **azurehdinsight.net** without leading *privatelink* in domain name when you create a private end point. If you use privatelink in domain name, make sure it is valid and you are able to resolve it.

+- [I'm not able to configure Pull Request Annotations](#im-not-able-to-configure-pull-request-annotations)

+- [What programming languages are supported by Defender for DevOps?](#what-programming-languages-are-supported-by-defender-for-devops)

+- [I'm getting an error that informs me that there's no CLI tool](#im-getting-an-error-that-informs-me-that-theres-no-cli-tool)

+When you select the *Authorize* button, the account that you're logged in with is used. That account can have the same email but may have a different tenant. Make sure you have the right account/tenant combination selected in the popup consent screen and Visual Studio.

+You can [check which account is signed in](https://app.vssps.visualstudio.com/profile/view).

+The Azure DevOps service only supports `TfsGit`.

+Ensure that you've [onboarded your repositories](/azure/defender-for-cloud/quickstart-onboard-devops?branch=main) to Microsoft Defender for Cloud. If you still can't see your repository, ensure that you're signed in with the correct Azure DevOps organization user account. Your Azure subscription and Azure DevOps Organization need to be in the same tenant. If the user for the connector is wrong, you need to delete the previously created connector, sign in with the correct user account and re-create the connector.

+If secret scan isn't enabled (meaning MSDO isn't configured for your pipeline) or a scan isn't performed for at least 14 days, the resource shows as `N/A` in Defender for Cloud.

+If no scan is performed for 14 days, the scan results revert to `N/A`.

+Exemptions aren't available for Defender for DevOps within Microsoft Defender for Cloud.

+The ability to block developers from committing code with exposed secrets isn't currently available.

+### I'm not able to configure Pull Request Annotations

+### What programming languages are supported by Defender for DevOps?

+- JavaScript

+- TypeScript

+### I'm getting an error that informs me that there's no CLI tool

+When you run the pipeline in Azure DevOps, you receive the following error:

+`no such file or directory, scandir 'D:\a\_msdo\versions\microsoft.security.devops.cli'`.

+This error can be seen in the extensions job as well.

+This error occurs if you're missing the dependency of `dotnet6` in the pipeline's YAML file. DotNet6 is required to allow the Microsoft Security DevOps extension to run. Include this as a task in your YAML file to eliminate the error.

+Users working in hybrid environments may be managing OT alerts in [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, the OT sensor, and an on-premises management console.

+The following image shows how you can connect your sensors to the Defender for IoT portal in Azure directly over the internet from remote sites, without traversing the enterprise network.

+Defender for IoT provides IoT security functionality across both the Microsoft 365 Defender and [Azure portals](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) using the following methods:

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Sensor settings (Preview)**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Sensor settings (Preview)**

+- **In the Azure portal**: Defender for IoT customers can view their discovered IoT devices in the **Device inventory** page in [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal. Register an Enterprise IoT network sensor, currently in **Public preview** to gain visibility to additional devices that aren't covered by Defender for Endpoint.

+To cancel the plan and remove all Defender for IoT services from the associated subscription, cancel the plan in [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal. For more information, see [Cancel your Enterprise IoT plan](manage-subscriptions-enterprise.md#cancel-your-enterprise-iot-plan).

+1. In the Azure portal, go to [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) and select **Plans and pricing** > **Add plan**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select the **Alerts** page on the left. By default, the following details are shown in the grid:

+We recommend that you update alert severity In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal as soon as you've triaged an alert so that you can prioritize the riskiest alerts as soon as possible. Make sure to update your alert status once you've taken remediation steps so that the progress is recorded.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select the **Alerts** page on the left.

+Use the **Device inventory** page in [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal to manage all network devices detected by cloud-connected sensors, including OT, IoT, and IT. Identify new devices detected, devices that might need troubleshooting, and more.

+Verify that your sensor is successfully connected to the Azure portal directly from the sensor's **Overview** page.

+In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal, use one of the following options:

+Make sure you can reach the SMTP server from the [sensor's management port](./best-practices/understand-network-architecture.md).

+In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal, use one of the following options:

+- For a new installation or standalone update, select **Getting started** > **On-premises management console**.

+ - For a new installation, select a version in the **Purchase an appliance and install software** area, and then select **Download**.

+ - **To download threat intelligence packages from the Azure portal**, you need access to the Azure portal as a [Security Reader](../../role-based-access-control/built-in-roles.md#security-reader), [Security Admin](../../role-based-access-control/built-in-roles.md#security-admin), [Contributor](../../role-based-access-control/built-in-roles.md#contributor), or [Owner](../../role-based-access-control/built-in-roles.md#owner) role.

+ - **To push threat intelligence updates to cloud-connected OT sensors from the Azure portal**, you need access to Azure portal as a [Security Admin](../../role-based-access-control/built-in-roles.md#security-admin), [Contributor](../../role-based-access-control/built-in-roles.md#contributor), or [Owner](../../role-based-access-control/built-in-roles.md#owner) role.

+ - **To manually upload threat intelligence packages to OT sensors or on-premises management consoles**, you need access to the OT sensor or on-premises management console as an **Admin** user.

+Threat intelligence packages can be automatically updated to *cloud connected* sensors as they're released by Defender for IoT.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors**, and then locate the sensor you want to change.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors**, and locate the OT sensor you want to update.

+1. Select the options (**...**) menu for the selected sensor and then select **Push Threat Intelligence update**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors**. Locate and select the OT sensors you want to update.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Threat intelligence update (Preview)** > **Local update**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Site and sensors**.

+After you've onboarded your plan, you'll see it listed in [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal. Go to the Defender for IoT **Plans and pricing** page and find your subscription with the new **Enterprise IoT** plan listed. For example:

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal, go to the **Alerts** page. By default, alerts are sorted by the **Last detection** column, from most recent to oldest alert, so that you can first see the latest alerts in your network.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) in the Azure portal, select **Sites and sensors** and then locate the OT sensors with legacy, but [supported versions](#prerequisites) installed.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Sensor update (Preview)**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Sensor update (Preview)**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** > **Sensor update (Preview)**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Getting started** > **On-premises management console**.

+1. In [Defender for IoT](https://ms.portal.azure.com/#view/Microsoft_Azure_IoT_Defender/IoTDefenderDashboard/~/Getting_started) on the Azure portal, select **Sites and sensors** and then select the legacy OT sensor you want to update.

+ Title: Azure Data Box as Event Grid source

+description: Describes the properties that are provided for Data Box events with Azure Event Grid.

+# Azure Data Box as an Event Grid source

+This article provides the properties and schema for Azure Data Box events. For an introduction to event schemas, see [Azure Event Grid event schema](event-schema.md).

+## Data Box events

+ |Event name |Description|

+ |-|--|

+ | Microsoft.DataBox.CopyStarted |Triggered when the copy has started from the device and the first byte of data copy is copied. |

+ |Microsoft.DataBox.CopyCompleted |Triggered when the copy has completed from device.|

+ | Microsoft.DataBox.OrderCompleted |Triggered when the order has completed copying and copy logs are available. |

+### Example events

+# [Event Grid event schema](#tab/event-grid-event-schema)

+### Microsoft.DataBox.CopyStarted event

+```json

+[{

+ "topic": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "eventType": "Microsoft.DataBox.CopyStarted",

+ "id": "049ec3f6-5b7d-4052-858e-6f4ce6a46570",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "CopyStarted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "dataVersion": "1",

+ "metadataVersion": "1",

+ "eventTime": "2022-10-16T02:51:26.4248221Z"

+}]

+```

+### Microsoft.DataBox.CopyCompleted event

+```json

+[{

+ "topic": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "eventType": "Microsoft.DataBox.CopyCompleted",

+ "id": "759c892a-a628-4e48-a116-2e1d54c555ce",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "CopyCompleted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "dataVersion": "1",

+ "metadataVersion": "1",

+ "eventTime": "2022-10-16T02:58:18.503829Z"

+}]

+```

+### Microsoft.DataBox.OrderCompleted event

+```json

+{

+ "topic": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "eventType": "Microsoft.DataBox.OrderCompleted",

+ "id": "5eb07c79-39a8-439c-bb4b-bde1f6267c37",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "OrderCompleted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "dataVersion": "1",

+ "metadataVersion": "1",

+ "eventTime": "2022-10-16T02:51:26.4248221Z"

+}

+```

+# [Cloud event schema](#tab/cloud-event-schema)

+### Microsoft.DataBox.CopyStarted event

+```json

+[{

+ "source": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "type": "Microsoft.DataBox.CopyStarted",

+ "time": "2022-10-16T02:51:26.4248221Z",

+ "id": "049ec3f6-5b7d-4052-858e-6f4ce6a46570",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "CopyStarted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "specVersion": "1.0"

+}]

+```

+### Microsoft.DataBox.CopyCompleted event

+```json

+{

+ "source": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "type": "Microsoft.DataBox.CopyCompleted",

+ "time": "2022-10-16T02:51:26.4248221Z",

+ "id": "759c892a-a628-4e48-a116-2e1d54c555ce",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "CopyCompleted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "specVersion": "1.0"

+}

+```

+### Microsoft.DataBox.OrderCompleted event

+```json

+[{

+ "source": "/subscriptions/{subscription-id}/resourceGroups/{your-rg}/providers/Microsoft.DataBox/jobs/{your-resource}",

+ "subject": "/jobs/{your-resource}",

+ "type": "Microsoft.DataBox.OrderCompleted",

+ "time": "2022-10-16T02:51:26.4248221Z",

+ "id": "5eb07c79-39a8-439c-bb4b-bde1f6267c37",

+ "data": {

+ "serialNumber": "SampleSerialNumber",

+ "stageName": "OrderCompleted",

+ "stageTime": "2022-10-12T19:38:08.0218897Z"

+ },

+ "specVersion": "1.0"

+}]

+```

+## Next steps

+* For an introduction to Azure Event Grid, see [What is Event Grid?](overview.md)

+* For more information about creating an Azure Event Grid subscription, see [Event Grid subscription schema](subscription-creation-schema.md).

+### Configure logs

+Front Door tracks extensive telemetry about every request. When you enable caching, your origin servers might not receive every request, so it's important that you use the Front Door logs to understand how your solution is running and responding to your clients. For more information about the metrics and logs that Azure Front Door records, see [Monitor metrics and logs in Azure Front Door](front-door-diagnostics.md) and [WAF logs](../web-application-firewall/afds/waf-front-door-monitor.md#waf-logs).

+To configure logging for your own application, see [Configure Azure Front Door logs](./standard-premium/how-to-logs.md)

+When Front Door manages your TLS certificates, it reduces your operational costs, and helps you to avoid costly outages caused by forgetting to renew a certificate. Front Door automatically issues and rotates the managed TLS certificates.

+Front Door can rewrite the `Host` header of incoming requests. This feature can be helpful when you manage a set of customer-facing custom domain names that route to a single origin. This feature can also help when you want to avoid configuring custom domain names in Front Door and at your origin. However, when you rewrite the `Host` header, request cookies and URL redirections might break. In particular, when you use platforms like Azure App Service, features like [session affinity](../app-service/configure-common.md#configure-general-settings) and [authentication and authorization](../app-service/overview-authentication-authorization.md) might not work correctly.

+For internet-facing applications, we recommend you enable the Front Door web application firewall (WAF) and configure it to use managed rules. When you use a WAF and Microsoft-managed rules, your application is protected from a wide range of attacks.

+* **Ignore Query String**: In this mode, Azure Front Door passes the query strings from the client to the origin on the first request and caches the asset. Future requests for the asset that are served from the Front Door environment ignore the query strings until the cached asset expires.

+* **Use Query String**: In this mode, each request with a unique URL, including the query string, is treated as a unique asset with its own cache. For example, the response from the origin for a request for `www.example.ashx?q=test1` is cached at the Front Door environment and returned for ensuing caches with the same query string. A request for `www.example.ashx?q=test2` is cached as a separate asset with its own time-to-live setting.

+ The order of the query string parameters doesn't matter. For example, if the Azure Front Door environment includes a cached response for the URL `www.example.ashx?q=test1&r=test2`, then a request for `www.example.ashx?r=test2&q=test1` is also served from the cache.

+* **Ignore Specified Query Strings** and **Include Specified Query Strings**: In this mode, you can configure Azure Front Door to include or exclude specified parameters when the cache key is generated.

+ For example, suppose that the default cache key is `/foo/image/asset.html`, and a request is made to the URL `https://contoso.com/foo/image/asset.html?language=EN&userid=100&sessionid=200`. If there's a rules engine rule to exclude the `userid` query string parameter, then the query string cache key would be `/foo/image/asset.html?language=EN&sessionid=200`.

+## Logs and reports

+The [access log](front-door-diagnostics.md#access-log) includes the cache status for each request. Also, [reports](standard-premium/how-to-reports.md#caching-report) include information about how Azure Front Door's cache is used in your application.

+The [access log](front-door-diagnostics.md#access-log) includes the cache status for each request.

+* **When caching is enabled**, the cache behavior is different depending on the cache behavior value applied by the Rules Engine:

+ Title: Monitor metrics and logs - Azure Front Door

+description: This article describes the different metrics and logs that Azure Front Door records.

+zone_pivot_groups: front-door-tiers

+# Monitor metrics and logs in Azure Front Door

+Azure Front Door provides several features to help you monitor your application, track requests, and debug your Front Door configuration.

+Logs and metrics are stored and managed by [Azure Monitor](../azure-monitor/overview.md).

+[Reports](standard-premium/how-to-reports.md) provide insight into how your traffic is flowing through Azure Front Door, the web application firewall (WAF), and to your application.

+## Metrics

+Azure Front Door measures and sends its metrics in 60-second intervals. The metrics can take up to 3 minutes to be processed by Azure Monitor, and they might not appear until processing is completed. Metrics can also be displayed in charts or grids, and are accessible through the Azure portal, Azure PowerShell, the Azure CLI, and the Azure Monitor APIs. For more information, seeΓÇ»[Azure Monitor metrics](../azure-monitor/essentials/data-platform-metrics.md).

+The metrics listed in the following table are recorded and stored free of charge for a limited period of time. For an extra cost, you can store for a longer period of time.

+| Metrics | Description | Dimensions |

+| - | - | - |

+| Byte Hit Ratio | The percentage of traffic that was served from the Azure Front Door cache, computed against the total egress traffic. The byte hit ratio is low if most of the traffic is forwarded to the origin rather than served from the cache. <br/><br/> **Byte Hit Ratio** = (egress from edge - egress from origin)/egress from edge. <br/><br/> Scenarios excluded from bytes hit ratio calculations:<ul><li>You explicitly disable caching, either through the Rules Engine or query string caching behavior.</li><li>You explicitly configure a `Cache-Control` directive with the `no-store` or `private` cache directives.</li></ul> | Endpoint |

+| Origin Health Percentage | The percentage of successful health probes sent from Azure Front Door to origins.| Origin, Origin Group |

+| Origin Latency | The time calculated from when the request was sent by the Azure Front Door edge to the origin until Azure Front Door received the last response byte from the origin. | Endpoint, Origin |

+| Origin Request Count | The number of requests sent from Azure Front Door to origins. | Endpoint, Origin, HTTP Status, HTTP Status Group |

+| Percentage of 4XX | The percentage of all the client requests for which the response status code is 4XX. | Endpoint, Client Country, Client Region |

+| Percentage of 5XX | The percentage of all the client requests for which the response status code is 5XX. | Endpoint, Client Country, Client Region |

+| Request Count | The number of client requests served through Azure Front Door, including requests served entirely from the cache. | Endpoint, Client Country, Client Region, HTTP Status, HTTP Status Group |

+| Request Size | The number of bytes sent in requests from clients to Azure Front Door. | Endpoint, Client Country, client Region, HTTP Status, HTTP Status Group |

+| Response Size | The number of bytes sent as responses from Front Door to clients. |Endpoint, client Country, client Region, HTTP Status, HTTP Status Group |

+| Total Latency | The total time taken from when the client request was received by Azure Front Door until the last response byte was sent from Azure Front Door to the client. |Endpoint, Client Country, Client Region, HTTP Status, HTTP Status Group |

+| Web Application Firewall Request Count | The number of requests processed by the Azure Front Door web application firewall. | Action, Policy Name, Rule Name |

+> [!NOTE]

+> If a request to the origin times out, the value of the *Http Status* dimension is **0**.

+## Logs

+Logs track all requests that pass through Azure Front Door. It can take a few minutes for logs to be processed and stored.

+There are multiple Front Door logs, which you can use for different purposes:

+- [Access logs](#access-log) can be used to identify slow requests, determine error rates, and understand how Front Door's caching behavior is working for your solution.

+- Web application firewall (WAF) logs can be used to detect potential attacks, and false positive detections that might indicate legitimate requests that the WAF blocked. For more information on the WAF logs, see [Azure Web Application Firewall monitoring and logging](../web-application-firewall/afds/waf-front-door-monitor.md).

+- [Health probe logs](#health-probe-log) can be used to identify origins that are unhealthy or that don't respond to requests from some of Front Door's geographically distributed PoPs.

+- [Activity logs](#activity-logs) provide visibility into the operations performed on your Azure resources, such as configuration changes to your Azure Front Door profile.

+The activity log and web application firewall log includes a *tracking reference*, which is also propagated in requests to origins and to client responses by using the `X-Azure-Ref` header. You can use the tracking reference to gain an end-to-end view of your application request processing.

+Access logs, health probe logs, and WAF logs aren't enabled by default. To enable and store your diagnostic logs, see [Configure Azure Front Door logs](./standard-premium/how-to-logs.md). Activity log entries are collected by default, and you can view them in the Azure portal.

+## <a name="access-log"></a>Access log

+Information about every request is logged into the access log. Each access log entry contains the information listed in the following table.

+| Property | Description |

+|-|-|

+| TrackingReference | The unique reference string that identifies a request served by Azure Front Door. The tracking reference is sent to the client and to the origin by using the `X-Azure-Ref` headers. Use the tracking reference when searching for a specific request in the access or WAF logs. |

+| Time | The date and time when the Azure Front Door edge delivered requested contents to client (in UTC). |

+| HttpMethod | HTTP method used by the request: DELETE, GET, HEAD, OPTIONS, PATCH, POST, or PUT. |

+| HttpVersion | The HTTP version that the client specified in the request. |

+| RequestUri | The URI of the received request. This field contains the full scheme, port, domain, path, and query string. |

+| HostName | The host name in the request from client. If you enable custom domains and have wildcard domain (`*.contoso.com`), the HostName log field's value is `subdomain-from-client-request.contoso.com`. If you use the Azure Front Door domain (`contoso-123.z01.azurefd.net`), the HostName log field's value is `contoso-123.z01.azurefd.net`. |

+| RequestBytes | The size of the HTTP request message in bytes, including the request headers and the request body. |

+| ResponseBytes | The size of the HTTP response message in bytes. |

+| UserAgent | The user agent that the client used. Typically, the user agent identifies the browser type. |

+| ClientIp | The IP address of the client that made the original request. If there was an `X-Forwarded-For` header in the request, then the client IP address is taken from the header. |

+| SocketIp | The IP address of the direct connection to the Azure Front Door edge. If the client used an HTTP proxy or a load balancer to send the request, the value of SocketIp is the IP address of the proxy or load balancer. |

+| timeTaken | The length of time from when the Azure Front Door edge received the client's request to the time that Azure Front Door sent the last byte of the response to the client, in seconds. This field doesn't take into account network latency and TCP buffering. |

+| RequestProtocol | The protocol that the client specified in the request. Possible values include: **HTTP**, **HTTPS**. |

+| SecurityProtocol | The TLS/SSL protocol version used by the request, or null if the request didn't use encryption. Possible values include: **SSLv3**, **TLSv1**, **TLSv1.1**, **TLSv1.2**. |

+| SecurityCipher | When the value for the request protocol is HTTPS, this field indicates the TLS/SSL cipher negotiated by the client and Azure Front Door. |

+| Endpoint | The domain name of the Azure Front Door endpoint, such as `contoso-123.z01.azurefd.net`. |

+| HttpStatusCode | The HTTP status code returned from Azure Front Door. If the request to the origin timed out, the value for the HttpStatusCode field is **0**. If the client closed the connection, the value for the HttpStatusCode field is **499**. |

+| Pop | The Azure Front Door edge point of presence (PoP) that responded to the user request. |

+| Cache Status | How the request was handled by the Azure Front Door cache. Possible values are: <ul><li>**HIT** and **REMOTE_HIT**: The HTTP request was served from the Azure Front Door cache.</li><li>**MISS**: The HTTP request was served from origin. </li><li> **PARTIAL_HIT**: Some of the bytes were served from the Front Door edge PoP cache, and other bytes were served from the origin. This status indicates an [object chunking](./front-door-caching.md#delivery-of-large-files) scenario. </li><li> **CACHE_NOCONFIG**: The request was forwarded without caching settings, including bypass scenarios. </li><li> **PRIVATE_NOSTORE**: There was no cache configured in the caching settings by the customer. </li><li> **N/A**: The request was denied by a signed URL or the Rules Engine.</li></ul> |

+| MatchedRulesSetName | The names of the Rules Engine rules that were processed. |

+| RouteNameΓÇ»| The name of the route that the request matched. |

+| ClientPort | The IP port of the client that made the request. |

+| Referrer | The URL of the site that originated the request. |

+| TimetoFirstByte | The length of time, in seconds, from when the Azure Front Door edge received the request to the time the first byte was sent to client, as measured by Azure Front Door. This property doesn't measure the client data. |

+| ErrorInfo | If an error occurred during the processing of the request, this field provides detailed information about the error. Possible values are: <ul><li> **NoError**: Indicates no error was found. </li><li> **CertificateError**: Generic SSL certificate error. </li><li> **CertificateNameCheckFailed**: The host name in the SSL certificate is invalid or doesn't match the requested URL. </li><li> **ClientDisconnected**: The request failed because of a client network connection issue. </li><li> **ClientGeoBlocked**: The client was blocked due to the geographical location of the IP address. </li><li> **UnspecifiedClientError**: Generic client error. </li><li> **InvalidRequest**: Invalid request. This response indicates a malformed header, body, or URL. </li><li> **DNSFailure**: A failured occurred during DNS resolution. </li><li> **DNSTimeout**: The DNS query to resolve the origin IP address timed out. </li><li> **DNSNameNotResolved**: The server name or address couldn't be resolved. </li><li> **OriginConnectionAborted**: The connection with the origin was disconnected abnormally. </li><li> **OriginConnectionError**: Generic origin connection error. </li><li> **OriginConnectionRefused**: The connection with the origin wasn't established. </li><li> **OriginError**: Generic origin error. </li><li> **OriginInvalidRequest**: An invalid request was sent to the origin. </li><li> **ResponseHeaderTooBig**: The origin returned a too large of a response header. </li><li> **OriginInvalidResponse**: The origin returned an invalid or unrecognized response. </li><li> **OriginTimeout**: The timeout period for the origin request expired. </li><li> **ResponseHeaderTooBig**: The origin returned a too large of a response header. </li><li> **RestrictedIP**: The request was blocked because of restricted IP address. </li><li> **SSLHandshakeError**: Azure Front Door was unable to establish a connection with the origin because of an SSL handshake failure. </li><li> **SSLInvalidRootCA**: The root certification authority's certificate was invalid. </li><li> **SSLInvalidCipher**: The HTTPS connection was established using an invalid cipher. </li><li> **OriginConnectionAborted**: The connection with the origin was disconnected abnormally. </li><li> **OriginConnectionRefused**: The connection with the origin wasn't established. </li><li> **UnspecifiedError**: An error occurred that didnΓÇÖt fit in any of the errors in the table. </li></ul> |

+| OriginURL | The full URL of the origin where the request was sent. The URL is composed of the scheme, host header, port, path, and query string. <br> **URL rewrite**: If the request URL was rewritten by the Rules Engine, the path refers to the rewritten path. <br> **Cache on edge PoP**: If the request was served from the Azure Front Door cache, the origin is **N/A**. <br> **Large request**: If the requested content is large and there are multiple chunked requests going back to the origin, this field corresponds to the first request to the origin. For more information, see [Object Chunking](./front-door-caching.md#delivery-of-large-files). |

+| OriginIP | The IP address of the origin that served the request. <br> **Cache on edge PoP**: If the request was served from the Azure Front Door cache, the origin is **N/A**. <br> **Large request**: If the requested content is large and there are multiple chunked requests going back to the origin, this field corresponds to the first request to the origin. For more information, see [Object Chunking](./front-door-caching.md#delivery-of-large-files). |

+| OriginName| The full hostname (DNS name) of the origin. <br> **Cache on edge PoP**: If the request was served from the Azure Front Door cache, the origin is **N/A**. <br> **Large request**: If the requested content is large and there are multiple chunked requests going back to the origin, this field corresponds to the first request to the origin. For more information, see [Object Chunking](./front-door-caching.md#delivery-of-large-files). |

+## Health probe log

+Azure Front Door logs every failed health probe request. These logs can help you to diagnose problems with an origin. The logs provide you with information that you can use to investigate the failure reason and then bring the origin back to a healthy status.

+Some scenarios this log can be useful for are:

+- You noticed Azure Front Door traffic was sent to a subset of the origins. For example, you might have noticed that only three out of four origins receive traffic. You want to know if the origins are receiving and responding to health probes so you know whether the origins are healthy.

+- You noticed the origin health percentage metric is lower than you expected. You want to know which origins are recorded as unhealthy and the reason for the health probe failures.

+Each health probe log entry has the following schema:

+| Property | Description |

+| | |

+| HealthProbeId | A unique ID to identify the health probe request. |

+| Time | The date and time when the health probe was sent (in UTC). |

+| HttpMethod | The HTTP method used by the health probe request. Values include **GET** and **HEAD**, based on the health probe's configuration. |

+| Result | The status of health probe. The value is either **success** or a description of the error the probe received. |

+| HttpStatusCode | The HTTP status code returned by the origin. |

+| ProbeURL | The full target URL to where the probe request was sent. The URL is composed of the scheme, host header, path, and query string. |

+| OriginName | The name of the origin that the health probe was sent to. This field helps you to locate origins of interest if origin is configured to use an FDQN. |

+| POP | The edge PoP that sent the probe request. |

+| Origin IP | The IP address of the origin that the health probe was sent to. |

+| TotalLatency | The time from when the Azure Front Door edge sent the health probe request to the origin to when the origin sent the last response to Azure Front Door. |

+| ConnectionLatency| The time spent setting up the TCP connection to send the HTTP probe request to the origin. |

+| DNSResolution Latency | The time spent on DNS resolution. This field only has a value if the origin is configured to be an FDQN instead of an IP address. If the origin is configured to use an IP address, the value is **N/A**. |

+The following example JSON snippet shows a health probe log entry for a failed health probe request.

+```json

+{

+ "records": [

+ {

+ "time": "2021-02-02T07:15:37.3640748Z",

+ "resourceId": "/SUBSCRIPTIONS/27CAFCA8-B9A4-4264-B399-45D0C9CCA1AB/RESOURCEGROUPS/AFDXPRIVATEPREVIEW/PROVIDERS/MICROSOFT.CDN/PROFILES/AFDXPRIVATEPREVIEW-JESSIE",

+ "category": "FrontDoorHealthProbeLog",

+ "operationName": "Microsoft.Cdn/Profiles/FrontDoorHealthProbeLog/Write",

+ "properties": {

+ "healthProbeId": "9642AEA07BA64675A0A7AD214ACF746E",

+ "POP": "MAA",

+ "httpVerb": "HEAD",

+ "result": "OriginError",

+ "httpStatusCode": "400",

+ "probeURL": "http://afdxprivatepreview.blob.core.windows.net:80/",

+ "originName": "afdxprivatepreview.blob.core.windows.net",

+ "originIP": "52.239.224.228:80",

+ "totalLatencyMilliseconds": "141",

+ "connectionLatencyMilliseconds": "68",

+ "DNSLatencyMicroseconds": "1814"

+ }

+ }

+ ]

+}

+```

+## Web application firewall log

+For more information on the Front Door web application firewall (WAF) logs, see [Azure Web Application Firewall monitoring and logging](../web-application-firewall/afds/waf-front-door-monitor.md).

+## Activity logs

+Activity logs provide information about the management operations on your Azure Front Door resources. The logs include details about each write operation that was performed on an Azure Front Door resource, including when the operation occurred, who performed it, and what the operation was.

+> [!NOTE]

+> Activity logs don't include read operations. They also might not include all operations that you perform by using either the Azure portal or classic management APIs.

+For more information, see [View your activity logs](./standard-premium/how-to-logs.md#view-your-activity-logs).

+## Next steps

+To enable and store your diagnostic logs, see [Configure Azure Front Door logs](./standard-premium/how-to-logs.md).

+>If a request to the the origin times out, the value for HttpStatusCode is set to **0**.

+1. Select **Activity log**.

+1. Choose a filtering scope, and then select **Apply**.

+> [!NOTE]

+> Activity log doesn't include any GET operations or operations that you perform by using either the Azure portal or the original Management API.

+>

+1. Choose **Diagnostic settings**.

+1. Select **Turn on diagnostics**. Archive diagnostic logs along with metrics to a storage account, stream them to an event hub, or send them to Azure Monitor logs.

+| HttpStatusCode | The HTTP status code returned from the proxy. If a request to the origin times out, the value for HttpStatusCode is set to **0**.|

+| ErrorInfo | This field contains the specific type of error for further troubleshooting. </br> Possible values include: </br> **NoError**: Indicates no error was found. </br> **CertificateError**: Generic SSL certificate error.</br> **CertificateNameCheckFailed**: The host name in the SSL certificate is invalid or doesn't match. </br> **ClientDisconnected**: Request failure because of client network connection. </br> **UnspecifiedClientError**: Generic client error. </br> **InvalidRequest**: Invalid request. It might occur because of malformed header, body, and URL. </br> **DNSFailure**: DNS Failure. </br> **DNSNameNotResolved**: The server name or address couldn't be resolved. </br> **OriginConnectionAborted**: The connection with the origin was stopped abruptly. </br> **OriginConnectionError**: Generic origin connection error. </br> **OriginConnectionRefused**: The connection with the origin wasn't able to established. </br> **OriginError**: Generic origin error. </br> **OriginInvalidResponse**: Origin returned an invalid or unrecognized response. </br> **OriginTimeout**: The timeout period for origin request expired. </br> **ResponseHeaderTooBig**: The origin returned too large of a response header. </br> **RestrictedIP**: The request was blocked because of restricted IP. </br> **SSLHandshakeError**: Unable to establish connection with origin because of SSL hand shake failure. </br> **UnspecifiedError**: An error occurred that didnΓÇÖt fit in any of the errors in the table. </br> **SSLMismatchedSNI**:The request was invalid because the HTTP message header didn't match the value presented in the TLS SNI extension during SSL/TLS connection setup.|

+For every request that goes to an origin shield, there are two log entries:

+> For various routing configurations and traffic behaviors, some of the fields like backendHostname, cacheStatus, isReceivedFromClient, and POP field may respond with different values. The following table explains the different values these fields will have for various scenarios:

+| Ignore Query String | Once the asset is cached, all ensuing requests ignore the query strings until the cached asset expires. |

+| Use Query String | Each request with a unique URL, including the query string, is treated as a unique asset with its own cache. |

+| Ignore Specified Query Strings | Request URL query strings listed in "Query parameters" setting are ignored for caching. |

+| Include Specified Query Strings | Request URL query strings listed in "Query parameters" setting are used for caching. |

+Front Door is designed to be internet-facing, and this scenario is optimized for publicly available blobs. If you need to authenticate access to blobs, consider using [shared access signatures](../storage/common/storage-sas-overview.md), and ensure that you enable the [*Use Query String* query string behavior](front-door-caching.md#query-string-behavior) to avoid Front Door from serving requests to unauthenticated clients. However, this approach might not make effective use of the Front Door cache, because each request with a different shared access signature must be sent to the origin separately.

+description: This article explains how to configure Azure Front Door logs.

+# Configure Azure Front Door logs

+Azure Front Door captures several types of logs. Logs can help you monitor your application, track requests, and debug your Front Door configuration. For more information about Azure Front Door's logs, see [Monitor metrics and logs in Azure Front Door](../front-door-diagnostics.md).

+Access logs, health probe logs, and WAF logs aren't enabled by default. In this article, you'll learn how to enable diagnostic logs for your Azure Front Door profile.

+ * Azure Log Analytics in Azure Monitor is best used for general real-time monitoring and analysis of Azure Front Door performance.

+ * Select the *Subscription* and *Log Analytics workspace*.

+ * Storage accounts are best used for scenarios when logs are stored for a longer duration and are reviewed when needed.

+ * Select the *Subscription* and the *Storage Account*. and set **Retention (days)**.

+ * Event hubs are a great option for integrating with other security information and event management (SIEM) tools or external data stores, such as Splunk, DataDog, or Sumo.

+ * Select the *Subscription, Event hub namespace, Event hub name (optional)*, and *Event hub policy name*.

+ > [!TIP]

+ > Most Azure customers use Log Analytics.

+## View your activity logs

+# Real-time monitoring in Azure Front Door

+Azure Front Door is integrated with Azure Monitor. You can use metrics in real time to measure traffic to your application, and to track, troubleshoot, and debug issues.

+You can also configure alerts for each metric such as a threshold for 4XXErrorRate or 5XXErrorRate. When the error rate exceeds the threshold, it will trigger an alert as configured. For more information, see [Create, view, and manage metric alerts using Azure Monitor](../../azure-monitor/alerts/alerts-metric.md).

+## Access metrics in the Azure portal

+1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your Azure Front Door Standard/Premium profile.

+1. Under **Monitoring**, select **Metrics**.

+1. In **Metrics**, select the metric to add:

+1. Select **Add filter** to add a filter:

+1. Select **Apply splitting** to split data by different dimensions:

+1. Select **New chart** to add a new chart:

+## Configure alerts in the Azure portal

+1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your Azure Front Door Standard/Premium profile.

+1. Under **Monitoring**, select **Alerts**.

+Azure Front Door analytics reports provide a built-in, all-around view of how your Azure Front Door profile behaves, along with associated web application firewall (WAF) metrics. You can also take advantage of [Azure Front Door's logs](../front-door-diagnostics.md?pivot=front-door-standard-premium) to do further troubleshooting and debugging.

+The built-in reports include information about your traffic and your application's security. Azure Front Door provides traffic reports and security reports.

+| Traffic report | Details |

+| [Key metrics in all reports](#key-metrics-included-in-all-reports) | Shows overall data that were sent from Azure Front Door edge points of presence (PoPs) to clients, including:<ul><li>Peak bandwidth</li><li>Requests</li><li>Cache hit ratio</li><li>Total latency</li><li>5XX error rate</li></ul> |

+| [Traffic by domain](#traffic-by-domain-report) | Provides an overview of all the domains within your Azure Front Door profile:<ul><li>Breakdown of data transferred out from the Azure Front Door edge to the client</li><li>Total requests</li><li>3XX/4XX/5XX response code by domains</li></ul> |

+| [Traffic by location](#traffic-by-location-report) | <ul><li>Shows a map view of request and usage by top countries/regions<br/></li><li>Trend view of top countries/regions</li></ul> |

+| [Usage](#usage-report) | <ul><li>Data transfer out from Azure Front Door edge to clients<br/></li><li>Data transfer out from origin to Azure Front Door edge<br/></li><li>Bandwidth from Azure Front Door edge to clients<br/></li><li>Bandwidth from origin to Azure Front Door edge<br/></li><li>Requests<br/></li><li>Total latency<br/></li><li>Request count trend by HTTP status code</li></ul> |

+| [Caching](#caching-report) | <ul><li>Shows cache hit ratio by request count<br/></li><li>Trend view of hit and miss requests</li></ul> |

+| [Top URL](#top-url-report) | <ul><li>Shows request count <br/></li><li>Data transferred <br/></li><li>Cache hit ratio <br/></li><li>Response status code distribution for the most requested 50 assets</li></ul> |

+| [Top referrer](#top-referrer-report) | <ul><li>Shows request count <br/></li><li>Data transferred <br/></li><li>Cache hit ratio <br/></li><li>Response status code distribution for the top 50 referrers that generate traffic</li></ul> |

+| [Top user agent](#top-user-agent-report) | <ul><li>Shows request count <br/></li><li>Data transferred <br/></li><li>Cache hit ratio <br/></li><li>Response status code distribution for the top 50 user agents that were used to request content</li></ul> |

+| Security report | Details |

+| Overview of key metrics | <ul><li>Shows matched WAF rules<br/></li><li>Matched OWASP rules<br/></li><li>Matched bot protection rules<br/></li><li>Matched custom rules</li></ul> |

+| Metrics by dimensions | <ul><li>Breakdown of matched WAF rules trend by action<br/></li><li>Doughnut chart of events by Rule Set Type and event by rule group<br/></li><li>Break down list of top events by rule ID, countries/regions, IP address, URL, and user agent</li></ul> |

+> Security reports are only available when you use the Azure Front Door premium tier.

+Reports are free of charge. Most reports are based on access log data, but you don't need to enable access logs or make any configuration changes to use the reports.

+## How to access reports

+Reports are accessible through the Azure portal and through the Azure Resource Manager API. You can also [download reports as comma-separated values (CSV) files](#export-reports-in-csv-format).

+### Access reports by using the Azure portal

+1. In the navigation pane, select **Reports** or **Security** under *Analytics*.

+1. Select the report you want to view.

+1. After choosing the report, you can select different filters.

+ - **Show data for:** Select the date range for which you want to view traffic by domain. Available ranges are:

+ By default, data is shown for the last seven days. For reports with line charts, the data granularity goes with the date ranges you selected as the default behavior.

+ * 5 minutes - one data point every 5 minutes for date ranges less than or equal to 24 hours. This granularity level can be used for date ranges that are 14 days or shorter.

+ * By hour ΓÇô one data point every hour for date ranges between 24 hours and 30 days.

+ * By day ΓÇô one data point per day for date ranges longer than 30 days.

+ Select **Aggregation** to change the default aggregation granularity.

+ - **Location:** Select one or more countries/regions to filter by the client locations. Countries/regions are grouped into six regions: North America, Asia, Europe, Africa, Oceania, and South America. Refer to [countries/regions mapping](https://en.wikipedia.org/wiki/Subregion). By default, all countries are selected.

+ - **Protocol:** Select either HTTP or HTTPS to view traffic data for the selected protocol.

+ - **Domains** - Select one or more endpoints or custom domains. By default, all endpoints and custom domains are selected.

+ * If you delete an endpoint or a custom domain in one profile and then recreate the same endpoint or domain in another profile, the report counts the new endpoint as a second endpoint.

+ * If you delete a custom domain and bind it to a different endpoint, the behavior depends on how you view the report. If you view the report by custom domain then they'll be treated as one custom domain. If you view the report by endpoint, they'll be treated as separate items.

+### Export reports in CSV format

+You can download any of the Azure Front Door reports as a CSV file. Every CSV report includes some general information and the information is available in all CSV files:

+| Value | Description |

+| Report | The name of the report. |

+| Domains | The list of the endpoints or custom domains for the report. |

+| StartDateUTC | The start of the date range for which you generated the report, in Coordinated Universal Time (UTC). |

+| EndDateUTC | The end of the date range for which you generated the report, in Coordinated Universal Time (UTC). |

+| GeneratedTimeUTC | The date and time when you generated the report, in Coordinated Universal Time (UTC). |

+| Location | The list of the countries/regions where the client requests originated. The value is **All** by default. Not applicable to the *Security* report. |

+| Protocol | The protocol of the request, which is either HTTP or HTTPS. Not applicable to *Top URL*, *Traffic by user agent*, and *Security* reports. |

+| Aggregation | The granularity of data aggregation in each row, every 5 minutes, every hour, and every day. Not applicable to *Traffic by domain*, *Top URL*, *Traffic by user agent* reports, and *Security* reports. |

+Each report also includes its own variables. Select a report to view the variables that the report includes.

+# [Traffic by domain](#tab/traffic-by-domain)

+The *Traffic by domain* report includes these fields:

+* Domain

+* Total Request

+* Cache Hit Ratio

+* 3XX Requests

+* 4XX Requests

+* 5XX Requests

+* ByteTransferredFromEdgeToClient

+# [Traffic by location](#tab/traffic-by-location)

+The *Traffic by location* report includes these fields:

+* Location

+* TotalRequests

+* Request%

+* BytesTransferredFromEdgeToClient

+# [Usage](#tab/usage)

+There are three reports in the usage report's CSV file: one for HTTP protocol, one for HTTPS protocol, and one for HTTP status codes.

+The *Usage* report's HTTP and HTTPS data sets include these fields:

+* Time

+* Protocol

+* DataTransferred(bytes)

+* TotalRequest

+* bpsFromEdgeToClient

+* 2XXRequest

+* 3XXRequest

+* 4XXRequest

+* 5XXRequest

+The *Usage* report's HTTP status codes data set include these fields:

+* Time

+* DataTransferred(bytes)

+* TotalRequest

+* bpsFromEdgeToClient

+* 2XXRequest

+* 3XXRequest

+* 4XXRequest

+* 5XXRequest

+# [Caching](#tab/caching)

+The *Caching* report includes these fields:

+* Time

+* CacheHitRatio

+* HitRequests

+* MissRequests

+# [Top URL](#tab/top-url)

+The *Top URL* report includes these fields:

+* URL

+* TotalRequests

+* Request%

+* DataTransferred(bytes)

+* DataTransferred%

+# [Top user agent](#tab/topuser-agent)

+The *Top user agent* report includes these fields:

+* UserAgent

+* TotalRequests

+* Request%

+* DataTransferred(bytes)

+* DataTransferred%

+# [Security](#tab/security)

+The *Security* report includes seven tables:

+* Time

+* Rule ID

+* Countries/regions

+* IP address

+* URL

+* Hostname

+* User agent

+All of the tables in the *Security* report include the following fields:

+* BlockedRequests

+* AllowedRequests

+* LoggedRequests

+* RedirectedRequests

+* OWASPRuleRequests

+* CustomRuleRequests

+* BotRequests

+## Key metrics included in all reports

+The following metrics are used within the reports.

+| Metric | Description |

+|||

+| Data Transferred | Shows data transferred from Azure Front Door edge PoPs to client for the selected time frame, client locations, domains, and protocols. |

+| Peak Bandwidth | Peak bandwidth usage in bits per seconds from Azure Front Door edge PoPs to clients for the selected time frame, client locations, domains, and protocols. |

+| Total Requests | The number of requests that Azure Front Door edge PoPs responded to clients for the selected time frame, client locations, domains, and protocols. |

+| Cache Hit Ratio | The percentage of all the cacheable requests for which Azure Front Door served the contents from its edge caches for the selected time frame, client locations, domains, and protocols. |

+| 5XX Error Rate | The percentage of requests for which the HTTP status code to client was a 5XX for the selected time frame, client locations, domains, and protocols. |

+| Total Latency | Average latency of all the requests for the selected time frame, client locations, domains, and protocols. The latency for each request is measured as the total time of when the client request gets received by Azure Front Door until the last response byte sent from Azure Front Door to client. |

+## Traffic by domain report

+The **traffic by domain** report provides a grid view of all the domains under this Azure Front Door profile.

+In this report you can view:

+* Request counts

+* Data transferred out from Azure Front Door to client

+* Requests with status code (3XX, 4XX and 5XX) of each domain

+Domains include endpoint domains and custom domains.

+You can go to other tabs to investigate further or view access log for more information if you find the metrics below your expectation.

+## Usage report

+The **usage report** shows the trends of traffic and response status code by various dimensions.

+The dimensions included in the usage report are:

+* Data transferred from edge to client and from origin to edge, in a line chart.

+* Data transferred from edge to client by protocol, in a line chart.

+* Number of requests from edge to clients, in a line chart.

+* Number of requests from edge to clients by protocol (HTTP and HTTPS), in a line chart.

+* Bandwidth from edge to client, in a line chart.

+* Total latency, which measures the total time from the client request received by Azure Front Door until the last response byte sent from Azure Front Door to the client, in a line chart.

+* Number of requests from edge to clients by HTTP status code, in a line chart. Every request generates an HTTP status code. HTTP status code appears as the HTTPStatusCode in the raw access log. The status code describes how the Azure Front Door edge PoP handled the request. For example, a 2XX status code indicates that the request was successfully served to a client. While a 4XX status code indicates that an error occurred.

+* Number of requests from the edge to clients by HTTP status code, in a line chart. The percentage of requests by HTTP status code is shown in a grid.

+## Traffic by location report

+The **traffic by location** report displays:

+* The top 50 countries/regions of visitors that access your assets the most.

+* A breakdown of metrics by countries/regions and gives you an overall view of countries/regions where the most traffic gets generated.

+* The countries/regions that have higher cache hit ratios, and higher 4XX/5XX error code rates.

+The following items are included in the reports:

+* A world map view of the top 50 countries/regions by data transferred out or requests of your choice.

+* Two line charts showing a trend view of the top five countries/regions by data transferred out and requests of your choice.

+* A grid of the top countries/regions with corresponding data transferred out from Azure Front Door to clients, the percentage of data transferred out, the number of requests, the percentage of requests by the country/region, cache hit ratio, 4XX response code counts, and 5XX response code counts.

+## Caching report

+The **caching report** provides a chart view of cache hits and misses, and the cache hit ratio, based on requests. Understanding how Azure Front Door caches your content helps you to improve your application's performance because cache hits give you the fastest performance. You can optimize data delivery speeds by minimizing cache misses.

+The caching report includes:

+* Cache hit and miss count trend, in a line chart.

+* Cache hit ratio, in a line chart.

+Cache hits/misses describe the request number cache hits and cache misses for client requests.

+* Hits: the client requests that are served directly from Azure Front Door edge PoPs. Refers to those requests whose values for CacheStatus in the raw access logs are *HIT*, *PARTIAL_HIT*, or *REMOTE_HIT*.

+* Miss: the client requests that are served by Azure Front Door edge POPs fetching contents from origin. Refers to those requests whose values for the field CacheStatus in the raw access raw logs are *MISS*.

+**Cache hit ratio** describes the percentage of cached requests that are served from edge directly. The formula of the cache hit ratio is: `(PARTIAL_HIT +REMOTE_HIT+HIT/ (HIT + MISS + PARTIAL_HIT + REMOTE_HIT)*100%`.

+Requests that meet the following requirements are included in the calculation:

+* The requested content was cached on an Azure Front Door PoP.

+* Partial cached contents for [object chunking](../front-door-caching.md#delivery-of-large-files).

+It excludes all of the following cases:

+* Requests that are denied because of a Rule Set.

+* Requests that contain matching Rules Set, which has been set to disable the cache.

+* Requests that are blocked by the Azure Front Door WAF.

+* Requests when the origin response headers indicate that they shouldn't be cached. For example, requests with `Cache-Control: private`, `Cache-Control: no-cache`, or `Pragma: no-cache` headers prevent the response from being cached.

+## Top URL report

+The **top URL report** allow you to view the amount of traffic incurred through a particular endpoint or custom domain. You'll see data for the most requested 50 assets during any period in the past 90 days.

+Popular URLs will be displayed with the following values:

+* URL, which refers to the full path of the requested asset in the format of `http(s)://contoso.com/https://docsupdatetracker.net/index.html/images/example.jpg`. URL refers to the value of the RequestUri field in the raw access log.

+* Request counts.

+* Request counts as a percentage of the total requests served by Azure Front Door.

+* Data transferred.

+* Data transferred percentage.

+* Cache hit ratio percentage.

+* Requests with response codes of 4XX.

+* Requests with response codes of 5XX.

+User can sort URLs by request count, request count percentage, data transferred, and data transferred percentage. All the metrics are aggregated by hour and might vary based on the timeframe selected.

+> [!NOTE]

+> Top URLs might change over time. To get an accurate list of the top 50 URLs, Azure Front Door counts all your URL requests by hour and keep the running total over the course of a day. The URLs at the bottom of the 50 URLs may rise onto or drop off the list over the day, so the total number of these URLs are approximations.

+>

+> The top 50 URLs may rise and fall in the list, but they rarely disappear from the list, so the numbers for top URLs are usually reliable. When a URL drops off the list and rise up again over a day, the number of request during the period when they are missing from the list is estimated based on the request number of the URL that appear in that period.

+## Top referrer report

+The **top referrer** report shows you the top 50 referrers to a particular Azure Front Door endpoint or custom domain. You can view data for any period in the past 90 days. A referrer indicates the URL from which a request was generated. Referrer may come from a search engine or other websites. If a user types a URL (for example, `https://contoso.com/https://docsupdatetracker.net/index.html`) directly into the address bar of a browser, the referrer for the requested is *Empty*.

+The top referrer report includes the following values.

+* Referrer, which is the value of the Referrer field in the raw access log.

+* Request counts.

+* Request count as a percentage of total requests served by Azure Front Door in the selected time period.

+* Data transferred.

+* Data transferred percentage.

+* Cache hit ratio percentage.

+* Requests with response code as 4XX.

+* Requests with response code as 5XX.

+You can sort by request count, request %, data transferred and data transferred %. All the metrics are aggregated by hour and may vary per the time frame selected.

+## Top user agent report

+The **top user agent** report shows graphical and statistics views of the top 50 user agents that were used to request content. The following list shows example user agents:

+* Mozilla/5.0 (Windows NT 10.0; WOW64)

+* AppleWebKit/537.36 (KHTML, like Gecko)

+* Chrome/86.0.4240.75

+* Safari/537.36.

+A grid displays the request counts, request %, data transferred and data transferred, cache Hit Ratio %, requests with response code as 4XX and requests with response code as 5XX. User Agent refers to the value of UserAgent in access logs.

+> [!NOTE]

+> Top user agents might change over time. To get an accurate list of the top 50 user agents, Azure Front Door counts all your user agent requests by hour and keep the running total over the course of a day. The user agents at the bottom of the 50 user agents may rise onto or drop off the list over the day, so the total number of these user agents are approximations.

+>

+> The top 50 user agents may rise and fall in the list, but they rarely disappear from the list, so the numbers for top user agents are usually reliable. When a user agent drops off the list and rise up again over a day, the number of request during the period when they are missing from the list is estimated based on the request number of the user agents that appear in that period.

+## Security report

+The **security report** provides graphical and statistics views of WAF activity.

+| Dimensions | Description |

+|||

+| Overview metrics - Matched WAF rules | Requests that match custom WAF rules, managed WAF rules and bot protection rules. |

+| Overview metrics - Blocked Requests | The percentage of requests that are blocked by WAF rules among all the requests that matched WAF rules. |

+| Overview metrics - Matched Managed Rules | Requests that match managed WAF rules. |

+| Overview metrics - Matched Custom Rule | Requests that match custom WAF rules. |

+| Overview metrics - Matched Bot Rule | Requests that match bot protection rules. |

+| WAF request trend by action | Four line-charts trend for requests by action. Actions are *Block*, *Log*, *Allow*, and *Redirect*. |

+| Events by Rule Type | Doughnut chart of the WAF requests distribution by rule type. Rule types include bot protection rules, custom rules, and managed rules. |

+| Events by Rule Group | Doughnut chart of the WAF requests distribution by rule group. |

+| Requests by actions | A table of requests by actions, in descending order. |

+| Requests by top Rule IDs | A table of requests by top 50 rule IDs, in descending order. |

+| Requests by top countries/regions | A table of requests by top 50 countries/regions, in descending order. |

+| Requests by top client IPs | A table of requests by top 50 IPs, in descending order. |

+| Requests by top Request URL | A table of requests by top 50 URLs, in descending order. |

+| Request by top Hostnames | A table of requests by top 50 hostname, in descending order. |

+| Requests by top user agents | A table of requests by top 50 user agents, in descending order. |

+A Web Application Firewall (WAF) policy allows you to control access to your web applications by using a set of custom and managed rules. You can change the state of the policy or configure a specific mode type for the policy. Depending on policy level settings you can choose to either actively inspect incoming requests, monitor only, or to monitor and take actions against requests that match a rule. You can also configure the WAF to only detect threats without blocking them, which is useful when you first enable the WAF. After evaluating how the WAF works with your application, you can reconfigure the WAF settings and enable the WAF in prevention mode. For more information, see [WAF policy settings](../web-application-firewall/afds/waf-front-door-policy-settings.md).

+West US region in the management group called "Corp". This policy will inherit onto all the Enterprise

+ The diagram focuses on the root management group with child Landing zones and Sandbox management groups. The Landing zones management group has two child management groups named Corp and Online while the Sandbox management group has two child subscriptions.

+Let's say there's a custom role defined on the Sandbox management group. That custom role is then

+assigned on the two Sandbox subscriptions.

+If we try to move one of those subscriptions to be a child of the Corp management group, this

+move would break the path from subscription role assignment to the Sandbox management group role

+ assignable scopes from Sandbox to the root management group so that the definition can be reached by

+Access the roles specified in the policy definition using the [az policy definition show](/cli/azure/policy/definition?view=azure-cli-latest#az-policy-definition-show&preserve-view=true) command, then iterate over each **roleDefinitionId** to create the role assignment using the [az role assignment create](/cli/azure/role/assignment?view=azure-cli-latest#az-role-assignment-create&preserve-view=true) command.

+Ingest is the first stage where device messages are received from an [Azure Event Hubs](../../event-hubs/index.yml) event hub and immediately pulled into the MedTech service. The Event Hubs service supports high scale and throughput with the ability to receive and process millions of device messages per second. It also enables the MedTech service to consume messages asynchronously, removing the need for devices to wait while device messages are processed.

+> The `Resolution Type` can also be adjusted post deployment of the MedTech service if a different `Resolution Type` is later required.

+The MedTech service provides near real-time processing and will also attempt to reduce the number of requests made to the FHIR service by grouping requests into batches of 300 [normalized messages](#normalize). If there's a low volume of data, and 300 normalized messages haven't been added to the group, then the corresponding FHIR Observations in that group are persisted to the FHIR service after ~five minutes. This means that when there's fewer than 300 normalized messages to be processed, there may be a delay of ~five minutes before FHIR Observations are created or updated in the FHIR service.

+> [!NOTE]

+> When multiple device messages contain data for the same FHIR Observation, have the same timestamp, and are sent within the same device message batch (for example, within the ~five minute window or in groups of 300 normalized messages), only the data corresponding to the latest device message for that FHIR Observation is persisted.

+>

+> For example:

+>

+> Device message 1:

+> ```json

+> {   

+> "patientid": "testpatient1",   

+> "deviceid": "testdevice1",

+> "systolic": "129",   

+> "diastolic": "65",   

+> "measurementdatetime": "2022-02-15T04:00:00.000Z"

+> } 

+> ```

+>

+> Device message 2:

+> ```json

+> {   

+> "patientid": "testpatient1",   

+> "deviceid": "testdevice1",   

+> "systolic": "113",   

+> "diastolic": "58",   

+> "measurementdatetime": "2022-02-15T04:00:00.000Z"

+> }

+> ```

+>

+> Assuming these device messages were ingested within the same ~five minute window or in the same group of 300 normalized messages, and since the `measurementdatetime` is the same for both device messages (indicating these contain data for the same FHIR Observation), only device message 2 is persisted to represent the latest/most recent data.

+# [Portal](#tab/azure-portal-preview)

+Perform the following steps to order an import job in Azure Import/Export. The Azure Import/Export service creates a job of the type "Data Box."

+ [](./media/storage-import-export-data-from-blobs/import-export-order-preview-03-export-job.png#lightbox)

+ 

+ :::image type="complex" source="./media/storage-import-export-data-from-blobs/import-export-order-preview-06-b-export-job.png" alt-text="Screenshot showing selected containers and blobs for a new Azure Import/Export export job in the portal.":::

+ - Choose **Export from blob list file (XML format)**, and select an XML file that contains a list of paths and prefixes for the blobs to be exported from the storage account. You must construct the XML file and store it in a container for the storage account. The file can't be empty.

+ 1. Enter a **Carrier account number**. The account number for a valid carrier account is required.

+ 

+ 

+ 

+ [](./media/storage-import-export-data-from-blobs/import-export-order-preview-12-export-job.png#lightbox)

+If you don't know the number of drives you need, see [Determine how many drives you need](storage-import-export-determine-drives-for-export.md#determine-how-many-drives-you-need). If you know the number of drives, proceed to ship the drives.

+1. After you receive the drives with exported data, you need to get the BitLocker keys to unlock the drives. Go to the export job in the Azure portal. Select **Import/Export** tab.

+2. Select your export job from the list. Go to **Encryption** and copy the keys.

+Here's an example of the sample input.

+2. Create a single NTFS volume on each drive. Assign a drive letter to the volume. Don't use mountpoints.

+ * If you have added data to a pre-encrypted drive (WAImportExport tool wasn't used for encryption), use the BitLocker key (a numerical password that you specify) in the popup to unlock the drive.

+ |/blobtype: |This option specifies the type of blobs you want to import the data to. For block blobs, the blob type is `BlockBlob` and for page blobs, it's `PageBlob`. |

+ |/skipwrite: | Specifies that there's no new data required to be copied and existing data on the disk is to be prepared. |

+ |/enablecontentmd5: |The option when enabled, ensures that MD5 is computed and set as `Content-md5` property on each blob. Use this option only if you want to use the `Content-md5` field after the data is uploaded to Azure. <br> This option doesn't affect the data integrity check (that occurs by default). The setting does increase the time taken to upload data to cloud. |

+ Together with the journal file, a `<Journal file name>_DriveInfo_<Drive serial ID>.xml` file is also created in the same folder where the tool resides. The .xml file is used in place of the journal file when creating a job if the journal file is too large.

+# [Portal](#tab/azure-portal-preview)

+The Import/Export service supports only import of Azure Files into Azure Storage. Exporting Azure Files isn't supported.

+ - **To import a file**: In the following example, the data to copy is on the F: drive. Your file *MyFile1.txt* is copied to the root of the *MyAzureFileshare1*. If the *MyAzureFileshare1* doesn't exist, it's created in the Azure Storage account. Folder structure is maintained.

+ > The /Disposition parameter, which let you choose what to do when you import a file that already exists in earlier versions of the tool, isn't supported in Azure Import/Export version 2.2.0.300. In the earlier tool versions, an imported file with the same name as an existing file was renamed by default.

+ - **For a disk that isn't encrypted**: Specify *Encrypt* to enable BitLocker encryption on the disk.

+### [Portal](#tab/azure-portal-preview)

+> For Blob storage, ensure that your devices are sending messages that have `contentType: application/JSON` and `contentEncoding:utf-8` (or `utf-16`, `utf-32`). See the [IoT Hub documentation](../../iot-hub/iot-hub-devguide-routing-query-syntax.md#query-based-on-message-body) for an example.

+ dotnet run -- -s <id-scope> -c <your-certificate-folder>\certs\device-01-full-chain.cert.pfx -p 1234

+ dotnet run -- -s <id-scope> -c <your-certificate-folder>\certs\device-02-full-chain.cert.pfx -p 1234

+During the EFLOW VM deployment, the VM had a switch assigned for all communications between the Windows host OS and the virtual machine. You always use the switch for VM lifecycle management communications, and it's not possible to delete it.

+The following steps in this section show how to assign a network interface to the EFLOW virtual machine. Ensure that the virtual switch and the networking configuration align with your networking environment. For more information about networking concepts like type of switches, DHCP and DNS, see [Azure IoT Edge for Linux on Windows networking](./iot-edge-for-linux-on-windows-networking.md).

+1. Check that the virtual switch you assign to the EFLOW VM is available.

+1. Check that you correctly assigned the virtual switch to the EFLOW VM.

+Once you successfully assign the virtual switch to the EFLOW VM, create a networking endpoint assigned to virtual switch to finalize the network interface creation. If you're using Static IP, ensure to use the appropriate parameters: _ip4Address_, _ip4GatewayAddress_ and _ip4PrefixLength_.

+ - If you're using DHCP, you don't need Static IP parameters.

+1. Check that you correctly created the network endpoint and assigned it to the EFLOW VM. You should see two network interfaces assigned to the virtual machine.

+The final step is to make sure the networking configurations applied correctly and the EFLOW VM has the new network interface configured. The new interface shows up as _"eth1"_ if it's the first extra interface added to the VM.

+ The default interface **eth0** is the one used for all the VM management. You should see another interface, like **eth1**, which is the new interface you assigned to the VM. Following the examples, if you previously assigned a new endpoint with the static IP 192.168.0.103 you should see the interface **eth1** with the _inet addr: 192.168.0.103_.

+ :::image type="content" source="./medilet-eflow-ifconfig.png" alt-text="Screenshot of EFLOW virtual machine network interfaces.":::

+Follow the steps in [How to configure networking for Azure IoT Edge for Linux on Windows](./how-to-configure-iot-edge-for-linux-on-windows-networking.md) to make sure you applied all the networking configurations correctly.

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/initial-project.png" alt-text="Screenshot that shows how to open your DevOps project.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/add-new-pipeline.png" alt-text="Screenshot that shows how to create a new build pipeline.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/create-without-yaml.png" alt-text="Screenshot that shows how to use the classic editor.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/pipeline-source.png" alt-text="Screenshot showing how to select your pipeline source.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/start-with-empty-build-job.png" alt-text="Screenshot showing how to start with an empty job for your build pipeline.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment-classic/configure-trigger.png" alt-text="Screenshot showing how to turn on continuous integration trigger.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/initial-project.png" alt-text="Screenshot showing how to open your DevOps project.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/add-new-pipeline.png" alt-text="Screenshot showing how to create a new build pipeline using the New pipeline button .":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/select-repository.png" alt-text="Screenshot showing how to select the repository for your build pipeline.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/show-assistant.png" alt-text="Screenshot that shows how to select Show assistant to open Tasks palette.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/add-build-task.png" alt-text="Screenshot of the Use Tasks palette and how to add tasks to your pipeline.":::

+ :::image type="content" source="./media/how-to-continuous-integration-continuous-deployment/check-trigger-settings.png" alt-text="Screenshot showing how to review your pipeline's trigger settings from the Triggers menu under More actions.":::

+ :::image type="content" source="./media/how-to-deploy-monitor/metric-list.png" alt-text="Screenshot showing how to edit custom metrics in a deployment.":::

+ :::image type="content" source="./media/how-to-deploy-blob/addmodule-tab1.png" alt-text="Screenshot showing the Module Settings tab of the Add I o T Edge Module page. .":::

+ :::image type="content" source="./media/how-to-deploy-blob/addmodule-tab3.png" alt-text="Screenshot showing the Container Create Options tab of the Add I o T Edge Module page..":::

+ :::image type="content" source="./media/how-to-deploy-blob/addmodule-tab4.png" alt-text="Screenshot showing the Module Twin Settings tab of the Add I o T Edge Module page.":::

+ :::image type="content" source="./media/how-to-develop-csharp-module/new-solution.png" alt-text="Screenshot showing how to run the New IoT Edge Solution.":::

+ :::image type="content" source="./media/how-to-deploy-blob/create-options.png" alt-text="Screenshot showing how to update module createOptions - Visual Studio Code .":::

+ :::image type="content" source="./media/how-to-deploy-blob/devicetocloud-deviceautodelete.png" alt-text="Screenshot showing how to set desired properties for azureblobstorageoniotedge in Visual Studio Code .":::

+ :::image type="content" source="./media/how-to-deploy-blob/https-proxy-config.png" alt-text="Screenshot showing the Update I o T Edge Module pane where you can enter the specified values.":::

+ :::image type="content" source="./media/how-to-deploy-blob/verify-proxy-config.png" alt-text="Screenshot showing the Environment Variables tab.":::

+Once you create Azure IoT Edge modules with your business logic, you want to deploy them to your devices to operate at the edge. If you have multiple modules that work together to collect and process data, you can deploy them all at once. You can also declare the routing rules that connect them.

+[Azure CLI](/cli/azure) is an open-source cross platform, command-line tool for managing Azure resources such as IoT Edge. It enables you to manage Azure IoT Hub resources, device provisioning service instances, and linked-hubs out of the box. The new IoT extension enriches Azure CLI with features such as device management and full IoT Edge capability.

+* [Azure CLI](/cli/azure/install-azure-cli) in your environment. At a minimum, your Azure CLI version must be 2.0.70 or higher. Use `az --version` to validate. This version supports az extension commands and introduces the Knack command framework.

+To deploy modules using the Azure CLI, save the deployment manifest locally as a .json file. You use the file path in the next section when you run the command to apply the configuration to your device.

+Change directories into the folder where you saved your deployment manifest. If you used one of the Visual Studio Code IoT Edge templates, use the `deployment.json` file in the **config** folder of your solution directory and not the `deployment.template.json` file.

+ :::image type="content" source="./media/how-to-deploy-modules-vscode/azure-iot-hub-devices.png" alt-text="Screenshot showing the expanded Azure I o T Hub section.":::

+ :::image type="content" source="./media/how-to-deploy-modules-vscode/select-deployment-manifest.png" alt-text="Screenshot showing where to select the I o T Edge Deployment Manifest.":::

+ :::image type="content" source="./media/how-to-deploy-monitor-vscode/create-deployment-at-scale.png" alt-text="Screenshot showing how to specify a deployment ID.":::

+ :::image type="content" source="./media/how-to-install-iot-edge-ubuntuvm/iotedge-vm-dns-name.png" alt-text="Screenshot showing the DNS name of the I o T Edge virtual machine." lightbox="./media/how-to-install-iot-edge-ubuntuvm/iotedge-vm-dns-name.png":::

+ sudo iotedge config apply

+Installing certificates on an IoT Edge device is a necessary step before deploying your solution in production. Learn more about how to [Prepare to deploy your IoT Edge solution in production](production-checklist.md).

+ :::image type="content" source="./media/how-to-monitor-iot-edge-deployments/target-devices.png" alt-text="Screenshot showing targeted devices for a deployment.":::

+ :::image type="content" source="./media/how-to-monitor-iot-edge-deployments/deployment-metrics-tab.png" alt-text="Screenshot showing the metrics for a deployment.":::

+ :::image type="content" source="./media/how-to-monitor-module-twins/select-module-twin.png" alt-text="Screenshot showing how to select a module twin to view in the Azure portal .":::

+ :::image type="content" source="./media/how-to-monitor-module-twins/edit-module-twin-vscode.png" alt-text="Screenshot showing how to get a module twin to edit in Visual Studio Code .":::

+ :::image type="content" source="./media/how-to-monitor-module-twins/update-module-twin-vscode.png" alt-text="Screenshot showing how to update a module twin in Visual Studio Code.":::

+ For example, create a *root folder* under _C:\Shared_ named **EFLOW-Shared**.

+ :::image type="content" source="media/how-to-share-windows-folder-to-vm/root-folder.png" alt-text="Screenshot of the Windows root folder.":::

+ For example, create two folders one named **Read-Access** and one named **Read-Write-Access**.

+ :::image type="content" source="media/how-to-share-windows-folder-to-vm/shared-folders.png" alt-text="Screenshot of Windows shared folders.":::

+ :::image type="content" source="./media/how-to-visual-studio-develop-csharp-module/add-module.png" alt-text="Screenshot of how to add Application and Module.":::

+The module project folder contains a file for your module code named either `Program.cs` or `main.c` depending on the language you chose. This folder also contains a file named `module.json` that describes the metadata of your module. Various Docker files included here provide the information needed to build your module as a Windows or Linux container.

+ * If developing in C#, set a breakpoint in the `PipeMessage()` function in **ModuleBackgroundService.cs**.

+ * If developing in C#, set a breakpoint in the `PipeMessage()` function in **ModuleBackgroundService.cs**.

+ :::image type="content" source="./media/how-to-develop-csharp-module/new-solution.png" alt-text="Screenshot of how to run a new IoT Edge solution.":::

+1. Provide the name of the module's image repository. Visual Studio Code autopopulates the module name with **localhost:5000/<your module name\>**. Replace it with your own registry information. Use **localhost** if you use a local Docker registry for testing. If you use Azure Container Registry, then use sign in server from your registry's settings. The sign-in server looks like **_\<registry name\>_.azurecr.io**. Only replace the **localhost:5000** part of the string so that the final result looks like **\<*registry name*\>.azurecr.io/_\<your module name\>_**.

+ :::image type="content" source="./media/how-to-develop-csharp-module/repository.png" alt-text="Screenshot of how to provide a Docker image repository.":::

+- A **modules** folder has subfolders for each module. Within the folder for each module, there's a file called **module.json** that controls how modules are built and deployed. You need to modify this file to change the module deployment container registry from a localhost to a remote registry. At this point, you only have one module. But you can add more if needed

+The sample modules allow you to build the solution, push to your container registry, and deploy to a device. This process lets you start testing without modifying any code. The sample module takes input from a source (in this case, the *SimulatedTemperatureSensor* module that simulates data) and pipes it to IoT Hub.

+Your default solution contains two modules, one is a simulated temperature sensor module and the other is the pipe module. The simulated temperature sensor sends messages to the pipe module and then the messages go to the IoT Hub. In the module folder you created, there are several Docker files for different container types. Use any of the files that end with the extension **.debug** to build your module for testing.

+1. You can see the successful set up of the IoT Edge Simulator by reading the progress detail in the integrated terminal.

+ :::image type="content" source="media/how-to-vs-code-develop-module/view-log.png" alt-text="Screenshot of the Watch Variables.":::

+- Update the `launch.json` so that Visual Studio Code can attach to the process in the container on the remote machine. You can find this file in the `.vscode` folder in your workspace and updates each time you add a new module that supports debugging.

+In the Visual Studio Code Debug view, select the debug configuration file for your module. By default, the **.debug** Dockerfile, module's container `createOptions` settings, and the `launch.json` file use *localhost*.

+1. Connecting to Docker requires root-level privileges. Follow the steps in [Manage docker as a non-root user](https://docs.docker.com/engine/install/linux-postinstall) to allow connection to the Docker daemon on the remote device. When you finish debugging, you may want to remove your user from the Docker group.

+* [Message routing query based on message properties](../iot-hub/iot-hub-devguide-routing-query-syntax.md#query-based-on-message-properties)

+* [Message routing query based on message body](../iot-hub/iot-hub-devguide-routing-query-syntax.md#query-based-on-message-body)

+* [Message routing query based on device twin](../iot-hub/iot-hub-devguide-routing-query-syntax.md#query-based-on-device-or-module-twin)

+IoT Edge does not currently support network proxies that use NTLM authentication. Users may consider bypassing the proxy by adding the required endpoints to the firewall allowlist.

+1. In the Visual Studio Code explorer, open **modules** > **CSharpModule** > **ModuleBackgroundService.cs**.

+1. Save the ModuleBackgroundService.cs file.

+The module folder contains a file for your module code, named either `Program.cs` or `main.c` depending on the language you chose. This folder also contains a file named `module.json` that describes the metadata of your module. Various Docker files provide the information needed to build your module as a Windows or Linux container.

+ * If developing in C#, set a breakpoint in the `PipeMessage()` function in **ModuleBackgroundService.cs**.

+ * **Generate a synthetic partition key for messages**: Select **Enable** if needed.

+ To effectively support high-scale scenarios, you can enable [synthetic partition keys](../cosmos-db/nosql/synthetic-partition-keys.md) for the Cosmos DB endpoint. You can specify the partition key property name in **Partition key name**. The partition key property name is defined at the container level and can't be changed once it has been set.

+ You can configure the synthetic partition key value by specifying a template in **Partition key template** based on your estimated data volume. The generated partition key value is automatically added to the partition key property for each new Cosmos DB record.

+ For more information about partitioning, see [Partitioning and horizontal scaling in Azure Cosmos DB](../cosmos-db/partitioning-overview.md).

+ > [!CAUTION]

+ > If you're using the system assigned managed identity for authenticating to Cosmos DB, you must use Azure CLI or Azure PowerShell to assign the Cosmos DB Built-in Data Contributor built-in role definition to the identity. Role assignment for Cosmos DB isn't currently supported from the Azure portal. For more information about the various roles, see [Configure role-based access for Azure Cosmos DB](../cosmos-db/how-to-setup-rbac.md). To understand assigning roles via CLI, see [Manage Azure Cosmos DB SQL role resources.](/cli/azure/cosmosdb/sql/role)

+You can use any X.509 certificate to authenticate a device with IoT Hub by uploading either a certificate thumbprint or a certificate authority (CA) to Azure IoT Hub. To learn more, see [Device Authentication using X.509 CA Certificates](iot-hub-x509ca-overview.md). For information about how to upload and verify a certificate authority with your IoT hub, see [Set up X.509 security in your Azure IoT hub](./tutorial-x509-prove-possession.md).

+ Title: Understand Azure IoT Hub message routing

+Message routing enables you to send messages from your devices to cloud services in an automated, scalable, and reliable manner. Message routing can be used for:

+* **Sending device telemetry messages and events** to the built-in endpoint and custom endpoints. Events that can be routed include device lifecycle events, device twin change events, digital twin change events, and device connection state events.

+* **Filtering data before routing it to various endpoints** by applying rich queries. Message routing allows you to query on the message properties and message body as well as device twin tags and device twin properties. For more information, see [queries in message routing](iot-hub-devguide-routing-query-syntax.md).

+The IoT Hub defines a [common format](iot-hub-devguide-messages-construct.md) for all device-to-cloud messaging for interoperability across protocols.

+Each IoT hub has a default built-in endpoint (**messages/events**) that is compatible with Event Hubs. You also can create [custom endpoints](iot-hub-devguide-endpoints.md#custom-endpoints) that point to other services in your Azure subscription.

+Each message is routed to all endpoints whose routing queries it matches. In other words, a message can be routed to multiple endpoints. If a message matches multiple routes that point to the same endpoint, IoT Hub delivers the message to that endpoint only once.

+* Built-in endpoint

+* Storage containers

+* Service Bus queues

+* Service Bus topics

+* Event Hubs

+* Cosmos DB (preview)

+IoT Hub needs write access to these service endpoints for message routing to work. If you configure your endpoints through the Azure portal, the necessary permissions are added for you. If you configure your endpoints using PowerShell or the Azure CLI, you need to provide the write access permission.

+To learn how to create endpoints, see the following articles:

+* [Manage routes and endpoints using the Azure portal](how-to-routing-portal.md)

+* [Manage routes and endpoints using the Azure CLI](how-to-routing-azure-cli.md)

+* [Manage routes and endpoints using PowerShell](how-to-routing-powershell.md)

+* [Manage routes and endpoints using Azure Resource Manager](how-to-routing-arm.md)

+Make sure you configure your services to support the expected throughput. For example, if you're using Event Hubs as a custom endpoint, you must configure the **throughput units** for that event hub so it can handle the ingress of events you plan to send via IoT Hub message routing. Similarly, when using a Service Bus queue as an endpoint, you must configure the **maximum size** to ensure the queue can hold all the data ingressed, until it's egressed by consumers. When you first configure your IoT solution, you may need to monitor your other endpoints and make any necessary adjustments for the actual load.

+If your custom endpoint has firewall configurations, consider using the [Microsoft trusted first party exception.](./virtual-network-support.md#egress-connectivity-from-iot-hub-to-other-azure-resources)

+### Built-in endpoint

+You can use standard [Event Hubs integration and SDKs](iot-hub-devguide-messages-read-builtin.md) to receive device-to-cloud messages from the built-in endpoint (**messages/events**). Once a route is created, data stops flowing to the built-in endpoint unless a route is created to that endpoint. Even if no routes are created, a fallback route must be enabled to route messages to the built-in endpoint. The fallback is enabled by default if you create your hub using the portal or the CLI.

+### Azure Storage as a routing endpoint

+There are two storage services IoT Hub can route messages to: [Azure Blob Storage](../storage/blobs/storage-blobs-introduction.md) and [Azure Data Lake Storage Gen2](../storage/blobs/data-lake-storage-introduction.md) (ADLS Gen2) accounts. Azure Data Lake Storage accounts are [hierarchical namespace-enabled](../storage/blobs/data-lake-storage-namespace.md) storage accounts built on top of blob storage. Both of these use blobs for their storage.

+IoT Hub supports writing data to Azure Storage in the [Apache Avro](https://avro.apache.org/) format and the JSON format. The default is AVRO. When using JSON encoding, you must set the contentType property to **application/json** and contentEncoding property to **UTF-8** in the message [system properties](iot-hub-devguide-routing-query-syntax.md#system-properties). Both of these values are case-insensitive. If the content encoding isn't set, then IoT Hub writes the messages in base 64 encoded format.

+The encoding format can be set only when the blob storage endpoint is configured; it can't be edited for an existing endpoint

+IoT Hub batches messages and writes data to storage whenever the batch reaches a certain size or a certain amount of time has elapsed. IoT Hub defaults to the following file naming convention: `{iothub}/{partition}/{YYYY}/{MM}/{DD}/{HH}/{mm}`.

+You may use any file naming convention, however you must use all listed tokens. IoT Hub writes to an empty blob if there's no data to write.

+We recommend listing the blobs or files and then iterating over them, to ensure all blobs or files are read without making any assumptions of partition. The partition range could potentially change during a [Microsoft-initiated failover](iot-hub-ha-dr.md#microsoft-initiated-failover) or IoT Hub [manual failover](iot-hub-ha-dr.md#manual-failover). You can use the [List Blobs API](/rest/api/storageservices/list-blobs) to enumerate the list of blobs or [List ADLS Gen2 API](/rest/api/storageservices/datalakestoragegen2/path) for the list of files. For example:

+### Service Bus queues and Service Bus topics as a routing endpoint

+### Event Hubs as a routing endpoint

+Apart from the built-in-Event Hubs compatible endpoint, you can also route data to custom endpoints of type Event Hubs.

+### Azure Cosmos DB as a routing endpoint (preview)

+IoT Hub supports writing to Cosmos DB in JSON (if specified in the message content-type) or as Base64 encoded binary.

+To effectively support high-scale scenarios, you can enable [synthetic partition keys](../cosmos-db/nosql/synthetic-partition-keys.md) for the Cosmos DB endpoint. As Cosmos DB is a hyperscale data store, all data/documents written to it must contain a field that represents a logical partition. Each logical partition has a maximum size of 20 GB. You can specify the partition key property name in **Partition key name**. The partition key property name is defined at the container level and can't be changed once it has been set.

+You can configure the synthetic partition key value by specifying a template in **Partition key template** based on your estimated data volume. For example, in manufacturing scenarios, your logical partition might be expected to approach its maximum limit of 20 GB within a month. In that case, you can define a synthetic partition key as a combination of the device ID and the month. The generated partition key value is automatically added to the partition key property for each new Cosmos DB record, ensuring logical partitions are created each month for each device.

+> [!CAUTION]

+> If you're using the system assigned managed identity for authenticating to Cosmos DB, you must use Azure CLI or Azure PowerShell to assign the Cosmos DB Built-in Data Contributor built-in role definition to the identity. Role assignment for Cosmos DB isn't currently supported from the Azure portal. For more information about the various roles, see [Configure role-based access for Azure Cosmos DB](../cosmos-db/how-to-setup-rbac.md). To understand assigning roles via CLI, see [Manage Azure Cosmos DB SQL role resources.](/cli/azure/cosmosdb/sql/role)

+## Routing queries

+IoT Hub message routing provides a querying capability to filter the data before routing it to the endpoints. Each routing query you configure has the following properties:

+| Property | Description |

+| - | -- |

+| **Name** | The unique name that identifies the query. |

+| **Source** | The origin of the data stream to be acted upon. For example, device telemetry. |

+| **Condition** | The query expression for the routing query that is run against the message application properties, system properties, message body, device twin tags, and device twin properties to determine if it's a match for the endpoint. For more information about constructing a query, see the see [message routing query syntax](iot-hub-devguide-routing-query-syntax.md) |

+| **Endpoint** | The name of the endpoint where IoT Hub sends messages that match the query. We recommend that you choose an endpoint in the same region as your IoT hub. |

+A single message may match the condition on multiple routing queries, in which case IoT Hub delivers the message to the endpoint associated with each matched query. IoT Hub also automatically deduplicates message delivery, so if a message matches multiple queries that have the same destination, it's only written once to that destination.

+For more information, see [IoT Hub message routing query syntax](./iot-hub-devguide-routing-query-syntax.md).

+## Read data that has been routed

+Use the following articles to learn how to read messages from an endpoint.

+* Read from a [built-in endpoint](../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-nodejs)

+* Read from [Blob storage](../storage/blobs/storage-blob-event-quickstart.md)

+* Read from [Event Hubs](../event-hubs/event-hubs-dotnet-standard-getstarted-send.md)

+* Read from [Service Bus queues](../service-bus-messaging/service-bus-dotnet-get-started-with-queues.md)

+* Read from [Service Bus topics](../service-bus-messaging/service-bus-dotnet-how-to-use-topics-subscriptions.md)

+The fallback route sends all the messages that don't satisfy query conditions on any of the existing routes to the built-in endpoint (**messages/events**), which is compatible with [Event Hubs](../event-hubs/index.yml). If message routing is enabled, you can enable the fallback route capability. Once a route is created, data stops flowing to the built-in endpoint, unless a route is created to that endpoint. If there are no routes to the built-in endpoint and a fallback route is enabled, only messages that don't match any query conditions on routes will be sent to the built-in endpoint. Also, if all existing routes are deleted, the fallback route capability must be enabled to receive all data at the built-in endpoint.

+### Limitations for device connection state events

+## Test routes

+## Monitor and troubleshoot

+To learn how to create message routes, see:

+* [Create and delete routes and endpoints by using the Azure portal](./how-to-routing-portal.md)

+* [Create and delete routes and endpoints by using the Azure CLI](./how-to-routing-azure-cli.md)

+* If you want to route your device-to-cloud messages to custom endpoints, see [Use message routes and custom endpoints for device-to-cloud messages](iot-hub-devguide-messages-d2c.md).

+ Title: Query on Azure IoT Hub message routing

+Message routing allows you to query on the message properties and message body as well as device twin tags and device twin properties. If the message body isn't in JSON format, message routing can still route the message, but queries can't be applied to the message body. Queries are described as Boolean expressions where, if true, the query succeeds and routes all the incoming data; otherwise, the query fails and the incoming data isn't routed. If the expression evaluates to a null or undefined value, it's treated as a Boolean false value, and generates an error in the IoT Hub [routes resource logs](monitor-iot-hub-reference.md#routes). The query syntax must be correct for the route to be saved and evaluated.

+## Query based on message properties

+System properties help identify the contents and source of the messages, some of which are described in the following table:

+| contentEncoding | string | The user specifies the encoding type of the message. If the contentType property is set to `application/JSON`, then allowed values are `UTF-8`, `UTF-16`, and `UTF-32`. |

+For more information about the other available system properties, see [Create and read IoT Hub messages](iot-hub-devguide-messages-construct.md).

+### Message properties query expressions

+A full list of supported operators and functions is provided in the [expression and conditions](iot-hub-devguide-query-language.md#expressions-and-conditions) section of [IoT Hub query language for device and module twins, jobs, and message routing](iot-hub-devguide-query-language.md).

+## Query based on message body

+To enable querying on a message body, the message should be in a JSON format and encoded in either UTF-8, UTF-16 or UTF-32. The `contentType` system property must be `application/JSON`. The `contentEncoding` system property must be one of the UTF encoding values supported by that system property. If these system properties aren't specified, IoT Hub won't evaluate the query expression on the message body.

+The following JavaScript example shows how to create a message with a properly formed and encoded JSON body:

+For a message encoding sample in C#, see the [HubRoutingSample](https://github.com/Azure/azure-iot-sdk-csharp/tree/main/iothub/device/samples/how%20to%20guides/HubRoutingSample) provided in the Microsoft Azure IoT SDK for .NET. This sample is the same one used in the [Message Routing tutorial](tutorial-routing.md). The Program.cs file also has a method named `ReadOneRowFromFile`, which reads one of the encoded files, decodes it, and writes it back out as ASCII so you can read it.

+### Message body query expressions

+You can run queries and functions only on properties in the body reference. You can't run queries or functions on the entire body reference. For example, the following query is *not* supported and returns `undefined`:

+```sql

+$body[0] = 'Feb'

+```

+To filter a twin notification payload based on what changed, run your query on the message body. For example, to filter when there's a desired property change on `sendFrequency` and the value is greater than 10:

+```sql

+$body.properties.desired.telemetryConfig.sendFrequency > 10

+```

+To filter messages that contains a property change, no matter the value of the property, you can use the `is_defined()` function (when the value is a primitive type):

+```sql

+is_defined($body.properties.desired.telemetryConfig.sendFrequency)

+```

+## Query based on device or module twin

+Message routing enables you to query on [device twin](iot-hub-devguide-device-twins.md) or [module twin](iot-hub-devguide-module-twins.md) tags and properties, which are JSON objects. The following sample illustrates a device twin with tags and properties:

+### Twin query expressions

+ Title: Compare Event Grid, routing for IoT Hub

+**[IoT Hub message routing](iot-hub-devguide-messages-d2c.md)**: This IoT Hub feature enables users to route device-to-cloud messages to service endpoints like Azure Storage containers, Event Hubs, Service Bus queues, and Service Bus topics. Routing also provides a querying capability to filter the data before routing it to the endpoints. In addition to device telemetry data, you can also [route non-telemetry events](iot-hub-devguide-messages-d2c.md#non-telemetry-events) and use them to trigger actions.

+| **Device messages and events** | Yes, message routing supports telemetry data, device twin changes, device lifecycle events, digital twin change events, and device connection state events. | Yes, Event Grid supports telemetry data and device events like device created/deleted/connected/disconnected. But Event Grid doesn't support device twin change events and digital twin change events. |

+| **Ordering** | Yes, message routing maintains the order of events. | No, Event Grid doesn't guarantee the order of events. |

+| **Filtering** | Rich filtering on message application properties, message system properties, message body, device twin tags, and device twin properties. Filtering isn't applied to digital twin change events. For examples, see [Message Routing Query Syntax](iot-hub-devguide-routing-query-syntax.md). | Filtering based on event type, subject type and attributes in each event. For examples, see [Understand filtering events in Event Grid Subscriptions](../event-grid/event-filtering.md). When subscribing to telemetry events, you can apply filters on the data to filter on message properties, message body and device twin in your IoT Hub, before publishing to Event Grid. See [how to filter events](../iot-hub/iot-hub-event-grid.md#filter-events). |

+| **Endpoints** | <ul><li>Event Hubs</li> <li>Azure Blob Storage</li> <li>Service Bus queue</li> <li>Service Bus topics</li><li>Cosmos DB (preview)</li></ul><br>Paid IoT Hub SKUs (S1, S2, and S3) can have 10 custom endpoints and 100 routes per IoT Hub. | <ul><li>Azure Functions</li> <li>Azure Automation</li> <li>Event Hubs</li> <li>Logic Apps</li> <li>Storage Blob</li> <li>Custom Topics</li> <li>Queue Storage</li> <li>Power Automate</li> <li>Third-party services through WebHooks</li></ul><br>Event Grid supports 500 endpoints per IoT Hub. For the most up-to-date list of endpoints, see [Event Grid event handlers](../event-grid/overview.md#event-handlers). |

+| **Cost** | There is no separate charge for message routing. Only ingress of telemetry into IoT Hub is charged. For example, if you have a message routed to three different endpoints, you're billed for only one message. | There is no charge from IoT Hub. Event Grid offers the first 100,000 operations per month for free, and then $0.60 per million operations afterwards. |

+| **Reliability** | High: Delivers each message to the endpoint at least once for each route. Expires all messages that aren't delivered within one hour. | High: Delivers each message to the webhook at least once for each subscription. Expires all events that aren't delivered within 24 hours. |

+| **Send to multiple endpoints** | Yes, send a single message to multiple endpoints. | Yes, send a single message to multiple endpoints. |

+ Event Grid does not guarantee that endpoints receive events in the same order that they occurred. For those cases in which absolute order of messages is significant and/or in which a consumer needs a trustworthy unique identifier for messages, we recommend using message routing.

+* Learn more about [IoT Hub message routing](../iot-hub/iot-hub-devguide-messages-d2c.md) and the [IoT Hub endpoints](../iot-hub/iot-hub-devguide-endpoints.md).

+If your hub uses [message routing](iot-hub-devguide-messages-d2c.md), exporting the template for the hub includes the routing configuration, but it doesn't include the resources themselves. You must choose whether to move the routing resources to the new location or to leave them in place and continue to use them "as is".

+You have to make some changes before you can use the template to create the new hub in the new region. Use [Visual Studio Code](https://code.visualstudio.com) or a text editor to edit the template.

+ The reason for this is because the connection strings are long and ungainly, and unlikely to change, but you might want to change the options and run the application more than once. To change the value of an environment variable, you have to close the command window and Visual Studio or Visual Studio Code, whichever you are using.

+Add a custom endpoint for the Service Bus queue to your IoT hub and create a message routing rule to direct messages that contain a temperature alert to that endpoint, where they will be picked up by your logic app. The routing rule uses a routing query, `temperatureAlert = "true"`, to forward messages based on the value of the `temperatureAlert` application property set by the client code running on the device. To learn more, see [Message routing query based on message properties](./iot-hub-devguide-routing-query-syntax.md#query-based-on-message-properties).

+ > If you use X.509 certificate authentication, SAS token passwords are not required. For more information, see [Set up X.509 security in your Azure IoT Hub](./tutorial-x509-prove-possession.md) and follow code instructions in the [TLS/SSL configuration section](#tlsssl-configuration).

+| [Message routing](iot-hub-devguide-messages-d2c.md), [message enrichments](iot-hub-message-enrichments-overview.md), and [Event Grid integration](iot-hub-event-grid.md) | Yes | Yes |

+* If you want to use self-signed certificates for testing, see the [Create a self-signed certificate](reference-x509-certificates.md#create-a-self-signed-certificate) section of [X.509 certificates](reference-x509-certificates.md).

+ >[!IMPORTANT]

+ >We recommend that you use certificates signed by an issuing Certificate Authority (CA), even for testing purposes. Never use self-signed certificates in production.

+Learn how to [register your CA certificate](./tutorial-x509-prove-possession.md)

+Learn how to [manually create a device in IoT Hub](./iot-hub-create-through-portal.md#register-a-new-device-in-the-iot-hub).

+Learn how to [complete this device connection step](./tutorial-x509-prove-possession.md).

+X.509 certificates are digital documents that represent a user, computer, service, or device. A certificate authority (CA), subordinate CA, or registration authority issues X.509 certificates. The certificates contain the public key of the certificate subject. They don't contain the subject's private key, which must be stored securely. [RFC 5280](https://tools.ietf.org/html/rfc5280) documents public key certificates, including their fields and extensions. Public key certificates are digitally signed and typically contain the following information:

+| [Validity](https://www.rfc-editor.org/rfc/rfc5280#section-4.1.2.5) | The inclusive time period for which the certificate is valid. |

+The X.509 standard defines the extensions included in this section, for use in the Internet public key infrastructure (PKI).

+| PKCS #7 certificate | A format designed for the transport of signed or encrypted data. It can include the entire certificate chain. [RFC 2315](https://tools.ietf.org/html/rfc2315) defines this format. |

+| PKCS #8 key | The format for a private key store. [RFC 5208](https://tools.ietf.org/html/rfc5208) defines this format. |

+| PKCS #12 key and certificate | A complex format that can store and protect a key and the entire certificate chain. It's commonly used with a .p12 or .pfx extension. PKCS #12 is synonymous with the PFX format. [RFC 7292](https://tools.ietf.org/html/rfc7292) defines this format. |

+## Self-signed certificates

+You can authenticate a device to your IoT hub for testing purposes by using two self-signed certificates. This type of authentication is sometimes called *thumbprint authentication* because the certificates are identified by calculated hash values called *fingerprints* or *thumbprints*. These calculated hash values are used by IoT Hub to authenticate your devices.

+>[!IMPORTANT]

+>We recommend that you use certificates signed by an issuing Certificate Authority (CA), even for testing purposes. Never use self-signed certificates in production.

+### Create a self-signed certificate

+You can use [OpenSSL](https://www.openssl.org/) to create self-signed certificates. The following steps show you how to run OpenSSL commands in a bash shell to create a self-signed certificate and retrieve a certificate fingerprint that can be used for authenticating your device in IoT Hub.

+>[!NOTE]

+>If you want to use self-signed certificates for testing, you must create two certificates for each device.

+1. Run the following command to generate a private key and create a PEM-encoded private key (.key) file, replacing the following placeholders with their corresponding values. The private key generated by the following command uses the RSA algorithm with 2048-bit encryption.

+ *{KeyFile}*. The name of your private key file.

+ ```bash

+ openssl genpkey -out {KeyFile} -algorithm RSA -pkeyopt rsa_keygen_bits:2048

+ ```

+1. Run the following command to generate a PKCS #10 certificate signing request (CSR) and create a CSR (.csr) file, replacing the following placeholders with their corresponding values. Make sure that you specify the device ID of the IoT device for your self-signed certificate when prompted.

+ *{KeyFile}*. The name of your private key file.

+ *{CsrFile}*. The name of your CSR file.

+ *{DeviceID}*. The name of your IoT device.

+ ```bash

+ openssl req -new -key {KeyFile} -out {CsrFile}

+

+ Country Name (2 letter code) [XX]:.

+ State or Province Name (full name) []:.

+ Locality Name (eg, city) [Default City]:.

+ Organization Name (eg, company) [Default Company Ltd]:.

+ Organizational Unit Name (eg, section) []:.

+ Common Name (eg, your name or your server hostname) []:{DeviceID}

+ Email Address []:.

+

+ Please enter the following 'extra' attributes

+ to be sent with your certificate request

+ A challenge password []:.

+ An optional company name []:.

+ ```

+1. Run the following command to examine and verify your CSR, replacing the following placeholders with their corresponding values.

+ *{CsrFile}*. The name of your certificate file.

+ ```bash

+ openssl req -text -in {CsrFile} -verify -noout

+ ```

+

+1. Run the following command to generate a self-signed certificate and create a PEM-encoded certificate (.crt) file, replacing the following placeholders with their corresponding values. The command converts and signs your CSR with your private key, generating a self-signed certificate that expires in 365 days.

+ *{KeyFile}*. The name of your private key file.

+ *{CsrFile}*. The name of your CSR file.

+ *{CrtFile}*. The name of your certificate file.

+ ```bash

+ openssl x509 -req -days 365 -in {CsrFile} -signkey {KeyFile} -out {CrtFile}

+ ```

+1. Run the following command to retrieve the fingerprint of the certificate, replacing the following placeholders with their corresponding values. The fingerprint of a certificate is a calculated hash value that is unique to that certificate. You need the fingerprint to configure your IoT device in IoT Hub for testing.

+ *{CrtFile}*. The name of your certificate file.

+ ```bash

+ openssl x509 -in {CrtFile} -noout -fingerprint

+ ```

+* For X.509 certificate authentication, the device certificate or the CA certificate associated with the device isn't expired. To learn how to register X.509 CA certificates with IoT Hub, see [Set up X.509 security in your Azure IoT hub](tutorial-x509-prove-possession.md).

+* If you want to use self-signed certificates for testing, see the [Create a self-signed certificate](reference-x509-certificates.md#create-a-self-signed-certificate) section of [X.509 certificates](reference-x509-certificates.md).

+ >[!IMPORTANT]

+ >We recommend that you use certificates signed by an issuing Certificate Authority (CA), even for testing purposes. Never use self-signed certificates in production.

+For production environments, we recommend that you purchase an X.509 CA certificate from a public root certificate authority (CA). However, creating your own test certificate hierarchy is adequate for testing IoT Hub device authentication. For more information about getting an X.509 CA certificate from a public root CA, see the [Get an X.509 CA certificate](iot-hub-x509ca-overview.md#get-an-x509-ca-certificate) section of [Authenticate devices using X.509 CA certificates](iot-hub-x509ca-overview.md).

+The following example uses [OpenSSL](https://www.openssl.org/) and the [OpenSSL Cookbook](https://www.feistyduck.com/library/openssl-cookbook/online/ch-openssl.html) to create a certificate authority (CA), a subordinate CA, and a device certificate. The example then signs the subordinate CA and the device certificate into a certificate hierarchy. This example is presented for demonstration purposes only.

+>[!NOTE]

+>Microsoft provides PowerShell and Bash scripts to help you understand how to create your own X.509 certificates and authenticate them to an IoT hub. The scripts are included with the [Azure IoT Hub Device SDK for C](https://github.com/Azure/azure-iot-sdk-c). The scripts are provided for demonstration purposes only. Certificates created by them must not be used for production. The certificates contain hard-coded passwords (ΓÇ£1234ΓÇ¥) and expire after 30 days. You must use your own best practices for certificate creation and lifetime management in a production environment. For more information, see [Managing test CA certificates for samples and tutorials](https://github.com/Azure/azure-iot-sdk-c/blob/main/tools/CACertificates/CACertificateOverview.md) in the GitHub repository for the [Azure IoT Hub Device SDK for C](https://github.com/Azure/azure-iot-sdk-c).

+Create a directory structure for the certificate authority.

+* The *db* directory stores the certificate database.

+Next, create a self-signed CA certificate. Self-signing is suitable for testing purposes. Specify the `ca_ext` configuration file extensions on the command line. These extensions indicate that the certificate is for a root CA and can be used to sign certificates and certificate revocation lists (CRLs). Sign the certificate, and commit it to the database.

+This example shows you how to create a subordinate or registration CA. Because you can use the root CA to sign certificates, creating a subordinate CA isnΓÇÖt strictly necessary. Having a subordinate CA does, however, mimic real world certificate hierarchies in which the root CA is kept offline and a subordinate CA issues client certificates.

+ * If you're using the PowerShell script supplied by Microsoft, run `New-CACertsVerificationCert "<verification code>"` to create a certificate named `VerifyCert4.cer`, replacing `<verification code>` with the previously generated verification code. For more information, see [Tutorial: Use OpenSSL to create test certificates](tutorial-x509-openssl.md).

+ * If you're using the Bash script supplied by Microsoft, run `./certGen.sh create_verification_certificate "<verification code>"` to create a certificate named `verification-code.cert.pem`, replacing `<verification code>` with the previously generated verification code. For more information, see [Tutorial: Use OpenSSL to create test certificates](tutorial-x509-openssl.md).

+### Health Probes

+Azure cross-region Load Balancer utilizes the health of the backend regional load balancers when deciding where to distribute traffic to. These health checks by the cross-region load balancer are done automatically every 20 seconds, given that a user has set up health probes on their regional load balancer.  

+Use the `init()` method for any costly or common preparation. For example, use it to load the model into a global object. This function will be called once at the beginning of the process. You model's files will be available in an environment variable called `AZUREML_MODEL_DIR`. Use this variable to locate the files associated with the model. Notice that some models may be contained in a folder (in the following example, the model has several files in a folder named `model`). See [how you can find out what's the folder used by your model](#using-models-that-are-folders).

+def run(mini_batch: List[str]) -> Union[List[Any], pandas.DataFrame]:

+### Relationship between the degree of parallelism and the scoring script

+Your deployment configuration controls the size of each mini-batch and the number of workers on each node. Take into account them when deciding if you want to read the entire mini-batch to perform inference, or if you want to run inference file by file, or row by row (for tabular). See [Running inference at the mini-batch, file or the row level](#running-inference-at-the-mini-batch-file-or-the-row-level) to see the different approaches.

+When running multiple workers on the same instance, take into account that memory will be shared across all the workers. Usually, increasing the number of workers per node should be accompanied by a decrease in the mini-batch size or by a change in the scoring strategy (if data size and compute SKU remains the same).

+For an example about how to achieve it see [High throughput deployments](how-to-image-processing-batch.md#high-throughput-deployments). This example processes an entire batch of files at a time.

+For an example about how to achieve it see [Image processing with batch deployments](how-to-image-processing-batch.md). This example processes a file at a time.

+For an example about how to achieve it see [Text processing with batch deployments](how-to-nlp-processing-batch.md). This example processes a row at a time.

+### Using models that are folders

+When authoring scoring scripts, the environment variable `AZUREML_MODEL_DIR` is typically used in the `init()` function to load the model. However, some models may contain its files inside of a folder. When reading the files in this variable, you may need to account for that. You can identify the folder where your MLflow model is placed as follows:

+1. Go to [Azure Machine Learning portal](https://ml.azure.com).

+1. Go to the section __Models__.

+1. Select the model you are trying to deploy and click on the tab __Artifacts__.

+1. Take note of the folder that is displayed. This folder was indicated when the model was registered.

+ :::image type="content" source="media/how-to-deploy-mlflow-models-online-endpoints/mlflow-model-folder-name.png" lightbox="media/how-to-deploy-mlflow-models-online-endpoints/mlflow-model-folder-name.png" alt-text="Screenshot showing the folder where the model artifacts are placed.":::

+Then you can use this path to load the model:

+```python

+def init():

+ global model

+ # AZUREML_MODEL_DIR is an environment variable created during deployment

+ # The path "model" is the name of the registered model's folder

+ model_path = os.path.join(os.environ["AZUREML_MODEL_DIR"], "model")

+ model = load_model(model_path)

+```

+## Securing batch endpoints

+Batch endpoints inherent the networking configuration from the workspace where they are deployed. All the batch endpoints created inside of secure workspace are deployed as private batch endpoints by default. In order to have fully operational batch endpoints working with private networking, follow the following steps:

+1. You have configured your Azure Machine Learning workspace for private networking. For more details about how to achieve it read [Create a secure workspace](tutorial-create-secure-workspace.md).

+2. For Azure Container Registry in private networks, there are [some prerequisites about their configuration](how-to-secure-workspace-vnet.md#prerequisites).

+ > [!WARNING]

+ > Azure Container Registries with Quarantine feature enabled are not supported by the moment.

+3. Ensure blob, file, queue, and table private endpoints are configured for the storage accounts as explained at [Secure Azure storage accounts](how-to-secure-workspace-vnet.md#secure-azure-storage-accounts). Batch deployments require all the 4 to properly work.

+4. Create the batch endpoint as regularly done.

+* Put the second set of private endpoints in a different resource group and hence in different private DNS zones. It prevents a name resolution conflict between the set of IPs used for the workspace and the ones used by the client VNets. Azure Private DNS provides a reliable, secure DNS service to manage and resolve domain names in a virtual network without the need to add a custom DNS solution. By using private DNS zones, you can use your own custom domain names rather than the Azure-provided names available today. Note that the DNS resolution against a private DNS zone works only from virtual networks that are linked to it. For more details, see [recommended zone names for Azure services](../private-link/private-endpoint-dns.md#azure-services-dns-zone-configuration).

+## Limitations

+Consider the following limitations when working on batch endpoints deployed regarding networking:

+- If you change the networking configuration of the workspace from public to private, or from private to public, such doesn't affect existing batch endpoints networking configuration. Batch endpoints rely on the configuration of the workspace at the time of creation. You can recreate your endpoints if you want them to reflect changes you made in the workspace.

+- When working on a private link-enabled workspace, batch endpoints can be created and managed using Azure Machine Learning studio. However, they can't be invoked from the UI in studio. Use the Azure ML CLI v2 instead for job creation. For more details about how to use it see [Run batch endpoint to start a batch scoring job](how-to-use-batch-endpoint.md#run-endpoint-and-configure-inputs-and-outputs).

+> - Outbound communication from managed online endpoint deployment is to the _workspace API_. When the endpoint is configured to use __public outbound__, then the workspace must be able to accept that public communication (allow public access).

+> - When `egress_public_network_access` is disabled, the deployment can only access the resources secured in the VNET. When `egress_public_network_access` is enabled, the deployment can only access the resources with public access, which means it cannot access the resources secured in the VNET.

+from azure.ai.ml.entities import AmlCompute, NetworkSettings

+network_settings = NetworkSettings(vnet_name="<vnet-name>", subnet="<subnet-name>")

+compute = AmlCompute(

+ name=cpu_compute_target,

+ size="STANDARD_D2_V2",

+ min_instances=0,

+ max_instances=4,

+ enable_node_public_ip=False,

+ network_settings=network_settings

+)

+ml_client.begin_create_or_update(entity=compute)

+from azure.ai.ml.entities import AmlCompute, NetworkSettings

+network_settings = NetworkSettings(vnet_name="<vnet-name>", subnet="<subnet-name>")

+compute = AmlCompute(

+ name=cpu_compute_target,

+ size="STANDARD_D2_V2",

+ min_instances=0,

+ max_instances=4,

+ network_settings=network_settings

+)

+ml_client.begin_create_or_update(entity=compute)

+* [Use a firewall](how-to-access-azureml-behind-firewall.md)

+## *Make issues*

+### No targets specified and no makefile found

+<!--issueDescription-->

+This issue can happen when no targets are specified and no makefile is found when running `make`.

+**Potential causes:**

+* Makefile doesn't exist in the current directory

+* No targets are specified

+**Affected areas (symptoms):**

+* Failure in building environments from UI, SDK, and CLI.

+* Failure in running jobs because it will implicitly build the environment in the first step.

+**Troubleshooting steps**

+* Ensure that the makefile is spelled correctly

+* Ensure that the makefile exists in the current directory

+* If you have a custom makefile, specify it using ```make -f custommakefile```

+* Specify targets in the makefile or in the command line

+* Configure your build and generate a makefile

+* Ensure your makefile is formatted correctly and tabs are used for indentation

+**Resources**

+* [GNU Make](https://www.gnu.org/software/make/manual/make.html)

+<!--/issueDescription-->

+This article uses a compute created here named `batch-cluster`. Adjust as needed and reference your compute using `azureml:<your-compute-name>` or create one as shown.

+1. Batch deployments require a scoring script that indicates how a given model should be executed and how input data must be processed. Batch Endpoints support scripts created in Python. In this case, we're deploying a model that reads image files representing digits and outputs the corresponding digit. The scoring script is as follows:

+ > [!NOTE]

+ > For MLflow models, Azure Machine Learning automatically generates the scoring script, so you're not required to provide one. If your model is an MLflow model, you can skip this step. For more information about how batch endpoints work with MLflow models, see the dedicated tutorial [Using MLflow models in batch deployments](how-to-mlflow-batch.md).

+ > [!WARNING]

+ > If you're deploying an Automated ML model under a batch endpoint, notice that the scoring script that Automated ML provides only works for online endpoints and is not designed for batch execution. Please see [Author scoring scripts for batch deployments](how-to-batch-scoring-script.md) to learn how to create one depending on what your model does.

+ __mnist/code/batch_driver.py__

+ :::code language="python" source="~/azureml-examples-main/sdk/python/endpoints/batch/mnist/code/batch_driver.py" :::

+## Run endpoint and configure inputs and outputs

+You need the following prerequisites to follow this tutorial:

+To connect MLflow to an Azure Machine Learning workspace, you need the tracking URI for the workspace. Each workspace has its own tracking URI and it has the protocol `azureml://`.

+1. __Environment__: it reads account information specified via environment variables and use it to authenticate.

+1. __Managed Identity__: If the application is deployed to an Azure host with Managed Identity enabled, it authenticates with it.

+1. __Azure CLI__: if a user has signed in via the Azure CLI `az login` command, it authenticates as that user.

+1. __Azure PowerShell__: if a user has signed in via Azure PowerShell's `Connect-AzAccount` command, it authenticates as that user.

+1. __Interactive browser__: it interactively authenticates a user via the default browser.

+## Non-public Azure Clouds support

+The Azure Machine Learning plugin for MLflow is configured by default to work with the global Azure cloud. However, you can configure the Azure cloud you are using by setting the environment variable `AZUREML_CURRENT_CLOUD`.

+# [MLflow SDK](#tab/mlflow)

+```Python

+import os

+os.environ["AZUREML_CURRENT_CLOUD"] = "AzureChinaCloud"

+```

+# [Using environment variables](#tab/environ)

+```bash

+export AZUREML_CURRENT_CLOUD="AzureChinaCloud"

+```

+You can identify the cloud you are using with the following Azure CLI command:

+```bash

+az cloud list

+```

+The current cloud has the value `IsActive` set to `True`.

+Create an **azuredeploy.json** file with the following content to create a server using public access connectivity method and also create a database on the server. Update the **firewallRules** default value if needed.

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "resourceNamePrefix": {

+ "type": "string",

+ "metadata": {

+ "description": "Provide a prefix for creating resource names."

+ }

+ },

+ "location": {

+ "type": "string",

+ "defaultValue": "[resourceGroup().location]"

+ },

+ "firewallRules": {

+ "type": "array",

+ "defaultValue": [

+ {

+ "name": "rule1",

+ "startIPAddress": "192.168.0.1",

+ "endIPAddress": "192.168.0.255"

+ },

+ {

+ "name": "rule2",

+ "startIPAddress": "192.168.1.1",

+ "endIPAddress": "192.168.1.255"

+ }

+ ]

+ "allowedValues": [

+ "Burstable",

+ "Generalpurpose",

+ "MemoryOptimized"

+ ],

+ "description": "The tier of the particular SKU. High Availability is available only for GeneralPurpose and MemoryOptimized sku."

+ "version": {

+ "defaultValue": "8.0.21",

+ "allowedValues": [

+ "5.7",

+ "8.0.21"

+ ],

+ "description": "Server version"

+ "defaultValue": "1",

+ "allowedValues": [

+ "Disabled",

+ "SameZone",

+ "ZoneRedundant"

+ ],

+ "defaultValue": "2",

+ "storageSizeGB": {

+ "type": "int",

+ "defaultValue": 20

+ },

+ "storageIops": {

+ "type": "int",

+ "defaultValue": 360

+ },

+ "storageAutogrow": {

+ "type": "string",

+ "defaultValue": "Enabled",

+ "allowedValues": [

+ "Enabled",

+ "Disabled"

+ ]

+ },

+ "skuName": {

+ "type": "string",

+ "defaultValue": "Standard_B1ms",

+ "metadata": {

+ "description": "The name of the sku, e.g. Standard_D32ds_v4."

+ }

+ "type": "int",

+ "defaultValue": 7

+ "type": "string",

+ "defaultValue": "Disabled",

+ "allowedValues": [

+ "Disabled",

+ "Enabled"

+ ]

+ },

+ "serverName": {

+ "type": "string",

+ "defaultValue": "[format('{0}mysqlserver', parameters('resourceNamePrefix'))]"

+ "type": "string",

+ "defaultValue": "[format('{0}mysqldb', parameters('resourceNamePrefix'))]"

+ "apiVersion": "2021-12-01-preview",

+ "location": "[parameters('location')]",

+ "storage": {

+ "autoGrow": "[parameters('storageAutogrow')]"

+ "backup": {

+ }

+ },

+ {

+ "type": "Microsoft.DBforMySQL/flexibleServers/databases",

+ "apiVersion": "2021-12-01-preview",

+ "name": "[format('{0}/{1}', parameters('serverName'), parameters('databaseName'))]",

+ "properties": {

+ "charset": "utf8",

+ "collation": "utf8_general_ci"

+ "dependsOn": [

+ "[resourceId('Microsoft.DBforMySQL/flexibleServers', parameters('serverName'))]"

+ ]

+ "name": "createFirewallRules",

+ "count": "[length(range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1)))]",

+ "mode": "serial",

+ "batchSize": 1

+ "type": "Microsoft.Resources/deployments",

+ "apiVersion": "2020-10-01",

+ "name": "[format('firewallRules-{0}', range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1))[copyIndex()])]",

+ "expressionEvaluationOptions": {

+ "scope": "inner"

+ },

+ "parameters": {

+ "ip": {

+ "value": "[parameters('firewallRules')[range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1))[copyIndex()]]]"

+ },

+ "serverName": {

+ "value": "[parameters('serverName')]"

+ }

+ },

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "parameters": {

+ "serverName": {

+ "type": "string"

+ },

+ "ip": {

+ "type": "object"

+ }

+ },

+ "apiVersion": "2021-12-01-preview",

+ "name": "[format('{0}/{1}', parameters('serverName'), parameters('ip').name)]",

+ "startIpAddress": "[parameters('ip').startIPAddress]",

+ "endIpAddress": "[parameters('ip').endIPAddress]"

+ },

+ "[resourceId('Microsoft.DBforMySQL/flexibleServers', parameters('serverName'))]"

+ ]

+Create an **azuredeploy.json** file with the following content to create a server using private access connectivity method inside a virtual network.

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "resourceNamePrefix": {

+ "type": "string",

+ "metadata": {

+ "description": "Provide a prefix for creating resource names."

+ }

+ },

+ "location": {

+ "type": "string",

+ "defaultValue": "[resourceGroup().location]"

+ },

+ "firewallRules": {

+ "type": "array",

+ "defaultValue": [

+ {

+ "name": "rule1",

+ "startIPAddress": "192.168.0.1",

+ "endIPAddress": "192.168.0.255"

+ },

+ {

+ "name": "rule2",

+ "startIPAddress": "192.168.1.1",

+ "endIPAddress": "192.168.1.255"

+ }

+ ]

+ "allowedValues": [

+ "Burstable",

+ "Generalpurpose",

+ "MemoryOptimized"

+ ],

+ "description": "The tier of the particular SKU. High Availability is available only for GeneralPurpose and MemoryOptimized sku."

+ "version": {

+ "defaultValue": "8.0.21",

+ "allowedValues": [

+ "5.7",

+ "8.0.21"

+ ],

+ "description": "Server version"

+ "defaultValue": "1",

+ "allowedValues": [

+ "Disabled",

+ "SameZone",

+ "ZoneRedundant"

+ ],

+ "defaultValue": "2",

+ "storageSizeGB": {

+ "type": "int",

+ "defaultValue": 20

+ "storageIops": {

+ "type": "int",

+ "defaultValue": 360

+ "storageAutogrow": {

+ "defaultValue": "Enabled",

+ "allowedValues": [

+ "Enabled",

+ "Disabled"

+ ]

+ "skuName": {

+ "defaultValue": "Standard_B1ms",

+ "metadata": {

+ "description": "The name of the sku, e.g. Standard_D32ds_v4."

+ }

+ "type": "int",

+ "defaultValue": 7

+ "type": "string",

+ "defaultValue": "Disabled",

+ "allowedValues": [

+ "Disabled",

+ "Enabled"

+ ]

+ },

+ "serverName": {

+ "type": "string",

+ "defaultValue": "[format('{0}mysqlserver', parameters('resourceNamePrefix'))]"

+ "type": "string",

+ "defaultValue": "[format('{0}mysqldb', parameters('resourceNamePrefix'))]"

+ "apiVersion": "2021-12-01-preview",

+ "location": "[parameters('location')]",

+ "storage": {

+ "autoGrow": "[parameters('storageAutogrow')]"

+ "backup": {

+ }

+ "apiVersion": "2021-12-01-preview",

+ "name": "[format('{0}/{1}', parameters('serverName'), parameters('databaseName'))]",

+ },

+ "dependsOn": [

+ "[resourceId('Microsoft.DBforMySQL/flexibleServers', parameters('serverName'))]"

+ ]

+ },

+ {

+ "copy": {

+ "name": "createFirewallRules",

+ "count": "[length(range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1)))]",

+ "mode": "serial",

+ "batchSize": 1

+ },

+ "type": "Microsoft.Resources/deployments",

+ "apiVersion": "2020-10-01",

+ "name": "[format('firewallRules-{0}', range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1))[copyIndex()])]",

+ "properties": {

+ "expressionEvaluationOptions": {

+ "scope": "inner"

+ },

+ "mode": "Incremental",

+ "parameters": {

+ "ip": {

+ "value": "[parameters('firewallRules')[range(0, if(greater(length(parameters('firewallRules')), 0), length(parameters('firewallRules')), 1))[copyIndex()]]]"

+ },

+ "serverName": {

+ "value": "[parameters('serverName')]"

+ }

+ },

+ "template": {

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "contentVersion": "1.0.0.0",

+ "parameters": {

+ "serverName": {

+ "type": "string"

+ },

+ "ip": {

+ "type": "object"

+ }

+ },

+ "resources": [

+ {

+ "type": "Microsoft.DBforMySQL/flexibleServers/firewallRules",

+ "apiVersion": "2021-12-01-preview",

+ "name": "[format('{0}/{1}', parameters('serverName'), parameters('ip').name)]",

+ "properties": {

+ "startIpAddress": "[parameters('ip').startIPAddress]",

+ "endIpAddress": "[parameters('ip').endIPAddress]"

+ }

+ }

+ ]

+ }

+ },

+ "dependsOn": [

+ "[resourceId('Microsoft.DBforMySQL/flexibleServers', parameters('serverName'))]"

+ ]

+Deploy the Bicep file using either Azure CLI or Azure PowerShell.

+# [CLI](#tab/CLI)

+```azurecli

+az group create --name exampleRG --location eastus

+az deployment group create --resource-group exampleRG --template-file azuredeploy.json

+```

+# [PowerShell](#tab/PowerShell)

+```azurepowershell

+New-AzResourceGroup -Name exampleRG -Location eastus

+New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile azuredeploy.json

+Follow the instructions to enter the parameter values. When the deployment finishes, you should see a message indicating the deployment succeeded.

+## Review deployed resources

+Follow these steps to verify if your server was created in the resource group.

+# [CLI](#tab/CLI)

+```azurecli

+az resource list --resource-group exampleRG

+# [PowerShell](#tab/PowerShell)

+```azurepowershell

+Get-AzResource -ResourceGroupName exampleRG

+To delete the resource group and the resources contained in the resource group:

+# [CLI](#tab/CLI)

+```azurecli

+az group delete --name exampleRG

+# [PowerShell](#tab/PowerShell)

+```azurepowershell

+Remove-AzResourceGroup -Name exampleRG

+ Title: 'Quickstart: Create an Azure DB for MySQL - Flexible Server - Bicep'

+description: In this Quickstart, learn how to create an Azure Database for MySQL - Flexible Server by using Bicep.

+# Quickstart: Use a Bicep file to create an Azure Database for MySQL - Flexible Server

+## Prerequisites

+- An Azure account with an active subscription.

+## Create server with public access

+Create a **main.bicep** file and a **crateFirewallRules.bicep** file with the following content to create a server using public access connectivity method and also create a database on the server. Update the **firewallRules** default value if needed.

+**main.bicep**

+```bicep

+@description('Provide a prefix for creating resource names.')

+param resourceNamePrefix string

+@description('Provide the location for all the resources.')

+param location string = resourceGroup().location

+@description('Provide the administrator login name for the MySQL server.')

+param administratorLogin string

+@description('Provide the administrator login password for the MySQL server.')

+@secure()

+param administratorLoginPassword string

+@description('Provide an array of firewall rules to be applied to the MySQL server.')

+param firewallRules array = [

+ {

+ name: 'rule1'

+ startIPAddress: '192.168.0.1'

+ endIPAddress: '192.168.0.255'

+ }

+ {

+ name: 'rule2'

+ startIPAddress: '192.168.1.1'

+ endIPAddress: '192.168.1.255'

+ }

+]

+@description('The tier of the particular SKU. High Availability is available only for GeneralPurpose and MemoryOptimized sku.')

+@allowed([

+ 'Burstable'

+ 'Generalpurpose'

+ 'MemoryOptimized'

+])

+param serverEdition string = 'Burstable'

+@description('Server version')

+@allowed([

+ '5.7'

+ '8.0.21'

+])

+param version string = '8.0.21'

+@description('Availability Zone information of the server. (Leave blank for No Preference).')

+param availabilityZone string = '1'

+@description('High availability mode for a server : Disabled, SameZone, or ZoneRedundant')

+@allowed([

+ 'Disabled'

+ 'SameZone'

+ 'ZoneRedundant'

+])

+param haEnabled string = 'Disabled'

+@description('Availability zone of the standby server.')

+param standbyAvailabilityZone string = '2'

+param storageSizeGB int = 20

+param storageIops int = 360

+@allowed([

+ 'Enabled'

+ 'Disabled'

+])

+param storageAutogrow string = 'Enabled'

+@description('The name of the sku, e.g. Standard_D32ds_v4.')