+## Common deprovisioning scenarios to test

+To learn more about deprovisioning scenarios, see [How Application Provisioning Works](how-provisioning-works.md#deprovisioning).

+The Azure AD provisioning service uses the [SCIM 2.0 protocol](https://techcommunity.microsoft.com/t5/Identity-Standards-Blog/bg-p/IdentityStandards) for automatic provisioning. The service connects to the SCIM endpoint for the application, and uses SCIM user object schema and REST APIs to automate the provisioning and deprovisioning of users and groups. A SCIM-based provisioning connector is provided for most applications in the Azure AD gallery. Developers use the SCIM 2.0 user management API in Azure AD to build endpoints for their apps that integrate with the provisioning service. For details, see [Build a SCIM endpoint and configure user provisioning](../app-provisioning/use-scim-to-provision-users-and-groups.md).

+There's a preconfigured set of attributes and attribute mappings between Azure AD user objects and each SaaS appΓÇÖs user objects. Some apps manage other types of objects along with Users, such as Groups.

+* **Groups.** With an Azure AD Premium license plan, you can use groups to assign access to a SaaS application. Then, when the provisioning scope is set to **Sync only assigned users and groups**, the Azure AD provisioning service provisions or deprovisions users based on whether they're members of a group that's assigned to the application. The group object itself isn't provisioned unless the application supports group objects. Ensure that groups assigned to your application have the property "SecurityEnabled" set to "True".

+ * How fast a user in a dynamic group is provisioned or deprovisioned in a SaaS application depends on how fast the dynamic group can evaluate membership changes. For information about how to check the processing status of a dynamic group, see [Check processing status for a membership rule](../enterprise-users/groups-create-rule.md).

+ * When a user loses membership in the dynamic group, it's considered a deprovisioning event. Consider this scenario when creating rules for dynamic groups.

+## Deprovisioning

+The Azure AD provisioning service keeps source and target systems in sync by deprovisioning accounts when user access is removed.

+* A user is soft-deleted in Azure AD (sent to the recycle bin / AccountEnabled property set to false). Thirty days after a user is deleted in Azure AD, they're permanently deleted from the tenant. At this point, the provisioning service sends a DELETE request to permanently delete the user in the application. At any time during the 30-day window, you can [manually delete a user permanently](../fundamentals/active-directory-users-restore.md), which sends a delete request to the application.

+ A `regedit` file you can use to set these values follows:

+

+ ```

+ Windows Registry Editor Version 5.00

+ [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2]

+ [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client]

+ "DisabledByDefault"=dword:00000000

+ "Enabled"=dword:00000001

+ [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server]

+ "DisabledByDefault"=dword:00000000

+ "Enabled"=dword:00000001

+ [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]

+ "SchUseStrongCrypto"=dword:00000001

+ ```

+

+- Registration of FIDO2 security keys may fail for some users if the FIDO2 Authentication method policy is targeted for a group and the overall Authentication methods policy has more than 20 groups configured. We're working on increasing the policy size limit and in the mean time recommend limiting the number of group targets to no more than 20.

+This is because TOTP will be preferred over the **Approve**/**Deny** push notification and Remote Desktop Gateway doesn't provide the option to enter a verification code with Azure AD Multi-Factor Authentication. For more information, see [Configure accounts for two-step verification](howto-mfa-nps-extension-rdg.md#configure-accounts-for-two-step-verification).

+Organizations can also integrate NPS with Azure AD MFA to enhance security and provide a high level of compliance. This helps ensure that users establish two-step verification to sign in to the Remote Desktop Gateway. For users to be granted access, they must provide their username/password combination along with information that the user has in their control. This information must be trusted and not easily duplicated, such as a cell phone number, landline number, application on a mobile device, and so on. RDG currently supports phone call and **Approve**/**Deny** push notifications from Microsoft authenticator app methods for 2FA. For more information about supported authentication methods see the section [Determine which authentication methods your users can use](howto-mfa-nps-extension.md#determine-which-authentication-methods-your-users-can-use).

+> The sign-in behavior for Remote Desktop Gateway doesn't provide the option to enter a verification code with Azure AD Multi-Factor Authentication. A user account must be configured for phone verification or the Microsoft Authenticator App with **Approve**/**Deny** push notifications.

+> If neither phone verification or the Microsoft Authenticator App with **Approve**/**Deny** push notifications is configured for a user, the user won't be able to complete the Azure AD Multi-Factor Authentication challenge and sign in to Remote Desktop Gateway.

+[Integrate your on-premises directories with Azure Active Directory](../hybrid/whatis-hybrid-identity.md)

+> [!VIDEO https://www.youtube.com/embed/1tPA7B9ztz0]

+> [!VIDEO https://www.youtube.com/embed/fxQGVIwX8_4]

+- Learn how to [troubleshoot your custom extensions API](custom-extension-troubleshoot.md).

+ Title: 'Tutorial: Configure ALVAO for automatic user provisioning with Azure Active Directory'

+description: Learn how to automatically provision and de-provision user accounts from Azure AD to ALVAO.

+writer: twimmers

+ms.assetid: a72aa8af-28e0-4378-9d74-59b128c9cf16

+# Tutorial: Configure ALVAO for automatic user provisioning

+This tutorial describes the steps you need to perform in both ALVAO and Azure Active Directory (Azure AD) to configure automatic user provisioning. When configured, Azure AD automatically provisions and de-provisions users and groups to [ALVAO](https://www.alvao.com) using the Azure AD Provisioning service. For important details on what this service does, how it works, and frequently asked questions, see [Automate user provisioning and deprovisioning to SaaS applications with Azure Active Directory](../app-provisioning/user-provisioning.md).

+## Supported capabilities

+> [!div class="checklist"]

+> * Create users in ALVAO.

+> * Remove users in ALVAO when they do not require access anymore.

+> * Keep user attributes synchronized between Azure AD and ALVAO.

+> * Provision groups and group memberships in ALVAO.

+## Prerequisites

+The scenario outlined in this tutorial assumes that you already have the following prerequisites:

+* [An Azure AD tenant](../develop/quickstart-create-new-tenant.md)

+* A user account in Azure AD with [permission](../roles/permissions-reference.md) to configure provisioning (for example, Application Administrator, Cloud Application administrator, Application Owner, or Global Administrator).

+* A user account in ALVAO with Admin permissions.

+## Step 1. Plan your provisioning deployment

+1. Learn about [how the provisioning service works](../app-provisioning/user-provisioning.md).

+1. Determine who will be in [scope for provisioning](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. Determine what data to [map between Azure AD and ALVAO](../app-provisioning/customize-application-attributes.md).

+## Step 2. Configure ALVAO to support provisioning with Azure AD

+Contact ALVAO support to configure ALVAO to support provisioning with Azure AD.

+## Step 3. Add ALVAO from the Azure AD application gallery

+Add ALVAO from the Azure AD application gallery to start managing provisioning to ALVAO. If you have previously setup ALVAO for SSO you can use the same application. However it's recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+## Step 4. Define who will be in scope for provisioning

+The Azure AD provisioning service allows you to scope who will be provisioned based on assignment to the application and or based on attributes of the user / group. If you choose to scope who will be provisioned to your app based on assignment, you can use the following [steps](../manage-apps/assign-user-or-group-access-portal.md) to assign users and groups to the application. If you choose to scope who will be provisioned based solely on attributes of the user or group, you can use a scoping filter as described [here](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* Start small. Test with a small set of users and groups before rolling out to everyone. When scope for provisioning is set to assigned users and groups, you can control this by assigning one or two users or groups to the app. When scope is set to all users and groups, you can specify an [attribute based scoping filter](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* If you need more roles, you can [update the application manifest](../develop/howto-add-app-roles-in-azure-ad-apps.md) to add new roles.

+## Step 5. Configure automatic user provisioning to ALVAO

+This section guides you through the steps to configure the Azure AD provisioning service to create, update, and disable users and/or groups in TestApp based on user and/or group assignments in Azure AD.

+### To configure automatic user provisioning for ALVAO in Azure AD:

+1. Sign in to the [Azure portal](https://portal.azure.com). Select **Enterprise Applications**, then select **All applications**.

+ 

+1. In the applications list, select **ALVAO**.

+ 

+1. Select the **Provisioning** tab.

+ 

+1. Set the **Provisioning Mode** to **Automatic**.

+ 

+1. Under the **Admin Credentials** section, input your ALVAO Tenant URL and Secret Token. Click **Test Connection** to ensure Azure AD can connect to ALVAO. If the connection fails, ensure your ALVAO account has Admin permissions and try again.

+ 

+1. In the **Notification Email** field, enter the email address of a person or group who should receive the provisioning error notifications and select the **Send an email notification when a failure occurs** check box.

+ 

+1. Select **Save**.

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Users to ALVAO**.

+1. Review the user attributes that are synchronized from Azure AD to ALVAO in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in ALVAO for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you'll need to ensure that the ALVAO API supports filtering users based on that attribute. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by ALVAO|

+ |||||

+ |userName|String|✓|✓

+ |externalId|String|✓|✓

+ |active|Boolean||✓

+ |displayName|String||✓

+ |title|String||

+ |emails[type eq "work"].value|String||

+ |name.givenName|String||

+ |name.familyName|String||

+ |name.formatted|String||

+ |addresses[type eq "work"].formatted|String||

+ |addresses[type eq "work"].locality|String||

+ |addresses[type eq "work"].region|String||

+ |addresses[type eq "work"].country|String||

+ |addresses[type eq "work"].postalCode|String||

+ |addresses[type eq "work"].streetAddress|String||

+ |phoneNumbers[type eq "work"].value|String||

+ |phoneNumbers[type eq "mobile"].value|String||

+ |phoneNumbers[type eq "fax"].value|String||

+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:employeeNumber|String||

+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:organization|String||

+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:department|String||

+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:manager|String||

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Groups to ALVAO**.

+1. Review the group attributes that are synchronized from Azure AD to ALVAO in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the groups in ALVAO for update operations. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by ALVAO|

+ |||||

+ |displayName|String|✓|✓

+ |externalId|String||

+ |members|Reference||

+

+1. To configure scoping filters, refer to the following instructions provided in the [Scoping filter tutorial](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. To enable the Azure AD provisioning service for ALVAO, change the **Provisioning Status** to **On** in the **Settings** section.

+ 

+1. Define the users and/or groups that you would like to provision to ALVAO by choosing the desired values in **Scope** in the **Settings** section.

+ 

+1. When you're ready to provision, click **Save**.

+ 

+This operation starts the initial synchronization cycle of all users and groups defined in **Scope** in the **Settings** section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running.

+## Step 6. Monitor your deployment

+Once you've configured provisioning, use the following resources to monitor your deployment:

+* Use the [provisioning logs](../reports-monitoring/concept-provisioning-logs.md) to determine which users have been provisioned successfully or unsuccessfully

+* Check the [progress bar](../app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user.md) to see the status of the provisioning cycle and how close it's to completion

+* If the provisioning configuration seems to be in an unhealthy state, the application goes into quarantine. Learn more about quarantine states [here](../app-provisioning/application-provisioning-quarantine-status.md).

+## More resources

+* [Managing user account provisioning for Enterprise Apps](../app-provisioning/configure-automatic-user-provisioning-portal.md)

+* [What is application access and single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+## Next steps

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)

+ Title: 'Tutorial: Configure Better Stack for automatic user provisioning with Azure Active Directory'

+description: Learn how to automatically provision and de-provision user accounts from Azure AD to Better Stack.

+writer: twimmers

+ms.assetid: ceb66a35-ca28-4a43-b9be-c8074cd406ff

+# Tutorial: Configure Better Stack for automatic user provisioning

+This tutorial describes the steps you need to perform in both Better Stack and Azure Active Directory (Azure AD) to configure automatic user provisioning. When configured, Azure AD automatically provisions and de-provisions users and groups to [Better Stack](https://betterstack.com) using the Azure AD Provisioning service. For important details on what this service does, how it works, and frequently asked questions, see [Automate user provisioning and deprovisioning to SaaS applications with Azure Active Directory](../app-provisioning/user-provisioning.md).

+## Supported capabilities

+> [!div class="checklist"]

+> * Create users in Better Stack.

+> * Remove users in Better Stack when they do not require access anymore.

+> * Keep user attributes synchronized between Azure AD and Better Stack.

+> * Provision groups and group memberships in Better Stack.

+> * [Single sign-on](../manage-apps/add-application-portal-setup-oidc-sso.md) to Better Stack (recommended).

+## Prerequisites

+The scenario outlined in this tutorial assumes that you already have the following prerequisites:

+* [An Azure AD tenant](../develop/quickstart-create-new-tenant.md)

+* A user account in Azure AD with [permission](../roles/permissions-reference.md) to configure provisioning (for example, Application Administrator, Cloud Application administrator, Application Owner, or Global Administrator).

+* A user account in Better Stack with Admin permissions.

+## Step 1. Plan your provisioning deployment

+1. Learn about [how the provisioning service works](../app-provisioning/user-provisioning.md).

+1. Determine who will be in [scope for provisioning](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. Determine what data to [map between Azure AD and Better Stack](../app-provisioning/customize-application-attributes.md).

+## Step 2. Configure Better Stack to support provisioning with Azure AD

+Contact Better Stack support to configure Better Stack to support provisioning with Azure AD.

+## Step 3. Add Better Stack from the Azure AD application gallery

+Add Better Stack from the Azure AD application gallery to start managing provisioning to Better Stack. If you have previously setup Better Stack for SSO you can use the same application. However it's recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+## Step 4. Define who will be in scope for provisioning

+The Azure AD provisioning service allows you to scope who will be provisioned based on assignment to the application and or based on attributes of the user / group. If you choose to scope who will be provisioned to your app based on assignment, you can use the following [steps](../manage-apps/assign-user-or-group-access-portal.md) to assign users and groups to the application. If you choose to scope who will be provisioned based solely on attributes of the user or group, you can use a scoping filter as described [here](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* Start small. Test with a small set of users and groups before rolling out to everyone. When scope for provisioning is set to assigned users and groups, you can control this by assigning one or two users or groups to the app. When scope is set to all users and groups, you can specify an [attribute based scoping filter](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* If you need more roles, you can [update the application manifest](../develop/howto-add-app-roles-in-azure-ad-apps.md) to add new roles.

+## Step 5. Configure automatic user provisioning to Better Stack

+This section guides you through the steps to configure the Azure AD provisioning service to create, update, and disable users and/or groups in TestApp based on user and/or group assignments in Azure AD.

+### To configure automatic user provisioning for Better Stack in Azure AD:

+1. Sign in to the [Azure portal](https://portal.azure.com). Select **Enterprise Applications**, then select **All applications**.

+ 

+1. In the applications list, select **Better Stack**.

+ 

+1. Select the **Provisioning** tab.

+ 

+1. Set the **Provisioning Mode** to **Automatic**.

+ 

+1. Under the **Admin Credentials** section, input your Better Stack Tenant URL and Secret Token. Click **Test Connection** to ensure Azure AD can connect to Better Stack. If the connection fails, ensure your Better Stack account has Admin permissions and try again.

+ 

+1. In the **Notification Email** field, enter the email address of a person or group who should receive the provisioning error notifications and select the **Send an email notification when a failure occurs** check box.

+ 

+1. Select **Save**.

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Users to Better Stack**.

+1. Review the user attributes that are synchronized from Azure AD to Better Stack in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in Better Stack for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you'll need to ensure that the Better Stack API supports filtering users based on that attribute. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by Better Stack|

+ |||||

+ |userName|String|✓|✓

+ |active|Boolean||

+ |emails[type eq "work"].value|String||

+ |name.givenName|String||

+ |name.familyName|String||

+ |phoneNumbers[type eq "work"].value|String||

+ |phoneNumbers[type eq "mobile"].value|String||

+ |externalId|String||

+ |timezone|String||

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Groups to Better Stack**.

+1. Review the group attributes that are synchronized from Azure AD to Better Stack in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the groups in Better Stack for update operations. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by Better Stack|

+ |||||

+ |displayName|String|✓|✓

+ |externalId|String||

+ |members|Reference||

+

+1. To configure scoping filters, refer to the following instructions provided in the [Scoping filter tutorial](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. To enable the Azure AD provisioning service for Better Stack, change the **Provisioning Status** to **On** in the **Settings** section.

+ 

+1. Define the users and/or groups that you would like to provision to Better Stack by choosing the desired values in **Scope** in the **Settings** section.

+ 

+1. When you're ready to provision, click **Save**.

+ 

+This operation starts the initial synchronization cycle of all users and groups defined in **Scope** in the **Settings** section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running.

+## Step 6. Monitor your deployment

+Once you've configured provisioning, use the following resources to monitor your deployment:

+* Use the [provisioning logs](../reports-monitoring/concept-provisioning-logs.md) to determine which users have been provisioned successfully or unsuccessfully

+* Check the [progress bar](../app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user.md) to see the status of the provisioning cycle and how close it's to completion

+* If the provisioning configuration seems to be in an unhealthy state, the application goes into quarantine. Learn more about quarantine states [here](../app-provisioning/application-provisioning-quarantine-status.md).

+## More resources

+* [Managing user account provisioning for Enterprise Apps](../app-provisioning/configure-automatic-user-provisioning-portal.md)

+* [What is application access and single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+## Next steps

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)

+ Title: 'Tutorial: Configure Kno2fy for automatic user provisioning with Azure Active Directory'

+description: Learn how to automatically provision and de-provision user accounts from Azure AD to Kno2fy.

+writer: twimmers

+ms.assetid: 68cc23f3-000b-421b-9d23-41f2eb5db521

+# Tutorial: Configure Kno2fy for automatic user provisioning

+This tutorial describes the steps you need to perform in both Kno2fy and Azure Active Directory (Azure AD) to configure automatic user provisioning. When configured, Azure AD automatically provisions and de-provisions users and groups to [Kno2fy](https://www.kno2.com) using the Azure AD Provisioning service. For important details on what this service does, how it works, and frequently asked questions, see [Automate user provisioning and deprovisioning to SaaS applications with Azure Active Directory](../app-provisioning/user-provisioning.md).

+## Supported capabilities

+> [!div class="checklist"]

+> * Create users in Kno2fy.

+> * Remove users in Kno2fy when they do not require access anymore.

+> * Keep user attributes synchronized between Azure AD and Kno2fy.

+> * Provision groups and group memberships in Kno2fy

+> * [Single sign-on](kno2fy-tutorial.md) to Kno2fy (recommended).

+## Prerequisites

+The scenario outlined in this tutorial assumes that you already have the following prerequisites:

+* [An Azure AD tenant](../develop/quickstart-create-new-tenant.md)

+* A user account in Azure AD with [permission](../roles/permissions-reference.md) to configure provisioning (for example, Application Administrator, Cloud Application administrator, Application Owner, or Global Administrator).

+* One or more Kno2 organizations that have the provisioning service enabled.

+* A Kno2 administrator account with permission to manage the organizations that should have their users managed through Azure AD.

+## Step 1. Plan your provisioning deployment

+1. Learn about [how the provisioning service works](../app-provisioning/user-provisioning.md).

+1. Determine who will be in [scope for provisioning](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. Determine what data to [map between Azure AD and Kno2fy](../app-provisioning/customize-application-attributes.md).

+## Step 2. Configure Kno2fy to support provisioning with Azure AD

+1. Provisioning with Azure AD is intended for use with Single Sign on using Azure AD as the Identity Provider. Enable Single Sign on for the Kno2fy Application in Azure AD and add the Azure AD Identity Provider by adding the appropriate issuer value in the Kno2 settings for your organization(s).

+1. A Kno2 team member will assist in acquiring a provisioning token and the URL for use with the service. Save these values for use in Step 5.

+## Step 3. Add Kno2fy from the Azure AD application gallery

+Add Kno2fy from the Azure AD application gallery to start managing provisioning to Kno2fy. If you have previously setup Kno2fy for SSO you can use the same application. However it's recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+## Step 4. Define who will be in scope for provisioning

+The Azure AD provisioning service allows you to scope who will be provisioned based on assignment to the application and or based on attributes of the user / group. If you choose to scope who will be provisioned to your app based on assignment, you can use the following [steps](../manage-apps/assign-user-or-group-access-portal.md) to assign users and groups to the application. If you choose to scope who will be provisioned based solely on attributes of the user or group, you can use a scoping filter as described [here](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* Start small. Test with a small set of users and groups before rolling out to everyone. When scope for provisioning is set to assigned users and groups, you can control this by assigning one or two users or groups to the app. When scope is set to all users and groups, you can specify an [attribute based scoping filter](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* If you need more roles, you can [update the application manifest](../develop/howto-add-app-roles-in-azure-ad-apps.md) to add new roles.

+## Step 5. Configure automatic user provisioning to Kno2fy

+This section guides you through the steps to configure the Azure AD provisioning service to create, update, and disable users and/or groups in TestApp based on user and/or group assignments in Azure AD.

+### To configure automatic user provisioning for Kno2fy in Azure AD:

+1. Sign in to the [Azure portal](https://portal.azure.com). Select **Enterprise Applications**, then select **All applications**.

+ 

+1. In the applications list, select **Kno2fy**.

+ 

+1. Select the **Provisioning** tab.

+ 

+1. Set the **Provisioning Mode** to **Automatic**.

+ 

+1. Under the **Admin Credentials** section, input your Kno2fy Tenant URL and Secret Token. Click **Test Connection** to ensure Azure AD can connect to Kno2fy. If the connection fails, ensure your Kno2fy account has Admin permissions and try again.

+ 

+1. In the **Notification Email** field, enter the email address of a person or group who should receive the provisioning error notifications and select the **Send an email notification when a failure occurs** check box.

+ 

+1. Select **Save**.

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Users to Kno2fy**.

+1. Review the user attributes that are synchronized from Azure AD to Kno2fy in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in Kno2fy for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you'll need to ensure that the Kno2fy API supports filtering users based on that attribute. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by Kno2fy|

+ |||||

+ |userName|String|✓|✓

+ |active|Boolean||✓

+ |displayName|String||✓

+ |emails[type eq "work"].value|String||✓

+ |name.givenName|String||✓

+ |name.familyName|String||✓

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Groups to Kno2fy**.

+1. Review the group attributes that are synchronized from Azure AD to Kno2fy in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the groups in Kno2fy for update operations. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by Kno2fy|

+ |||||

+ |displayName|String|✓|✓

+ |members|Reference||

+

+1. To configure scoping filters, refer to the following instructions provided in the [Scoping filter tutorial](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. To enable the Azure AD provisioning service for Kno2fy, change the **Provisioning Status** to **On** in the **Settings** section.

+ 

+1. Define the users and/or groups that you would like to provision to Kno2fy by choosing the desired values in **Scope** in the **Settings** section.

+ 

+1. When you're ready to provision, click **Save**.

+ 

+This operation starts the initial synchronization cycle of all users and groups defined in **Scope** in the **Settings** section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running.

+## Step 6. Monitor your deployment

+Once you've configured provisioning, use the following resources to monitor your deployment:

+* Use the [provisioning logs](../reports-monitoring/concept-provisioning-logs.md) to determine which users have been provisioned successfully or unsuccessfully

+* Check the [progress bar](../app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user.md) to see the status of the provisioning cycle and how close it's to completion

+* If the provisioning configuration seems to be in an unhealthy state, the application goes into quarantine. Learn more about quarantine states [here](../app-provisioning/application-provisioning-quarantine-status.md).

+## More resources

+* [Managing user account provisioning for Enterprise Apps](../app-provisioning/configure-automatic-user-provisioning-portal.md)

+* [What is application access and single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+## Next steps

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)

+- The last 7 days of utilization data are analyzed. Note that you can change your lookback period in the configurations.

+- The last 7 days of utilization data are analyzed. Note that you can change your lookback period in the configurations.

+ - If we determine that the Burstable SKU credits are sufficient to support the average CPU utilization over 7 days. Note that you can change your lookback period in the configurations.

+ NAMESPACE=$1

+ for PVC in $(kubectl get pvc -n $namespace | awk '{ print $1}'); do

+ PV="$(kubectl get pvc $PVC -n $NAMESPACE -o jsonpath='{.spec.volumeName}')"

+ RECLAIMPOLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ echo "Reclaim Policy for Persistent Volume $PV is $RECLAIMPOLICY"

+ if [[ $RECLAIMPOLICY == "Delete" ]]; then

+ kubectl patch pv $PV -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

+ NAMESPACE=$1

+ FILENAME=$(date +%Y%m%d%H%M)-$NAMESPACE

+ EXISTING_STORAGE_CLASS=$2

+ STORAGE_CLASS_NEW=$3

+ STARTTIMESTAMP=$4

+ ENDTIMESTAMP=$5

+ for PVC in $(kubectl get pvc -n $NAMESPACE | awk '{ print $1}'); do

+ PVC_CREATION_TIME=$(kubectl get pvc $PVC -n $NAMESPACE -o jsonpath='{.metadata.creationTimestamp}')

+ if [[ $PVC_CREATION_TIME > $STARTTIMESTAMP ]]; then

+ if [[ $ENDTIMESTAMP > $PVC_CREATION_TIME ]]; then

+ PV="$(kubectl get pvc $PVC -n $NAMESPACE -o jsonpath='{.spec.volumeName}')"

+ RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ STORAGECLASS="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.storageClassName}')"

+ echo $PVC

+ RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ if [[ $RECLAIM_POLICY == "Retain" ]]; then

+ if [[ $STORAGECLASS == $EXISTING_STORAGE_CLASS ]]; then

+ STORAGE_SIZE="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.capacity.storage}')"

+ SKU_NAME="$(kubectl get storageClass $STORAGECLASS -o jsonpath='{.reclaimPolicy}')"

+ DISK_URI="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.azureDisk.diskURI}')"

+ PERSISTENT_VOLUME_RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ cat >$PVC-csi.yaml <<EOF

+ name: $PV-csi

+ storage: $STORAGE_SIZE

+ name: $PVC-csi

+ namespace: $NAMESPACE

+ csi.storage.k8s.io/pv/name: $PV-csi

+ csi.storage.k8s.io/pvc/name: $PVC-csi

+ csi.storage.k8s.io/pvc/namespace: $NAMESPACE

+ requestedsizegib: "$STORAGE_SIZE"

+ skuname: $SKU_NAME

+ volumeHandle: $DISK_URI

+ persistentVolumeReclaimPolicy: $PERSISTENT_VOLUME_RECLAIM_POLICY

+ storageClassName: $STORAGE_CLASS_NEW

+ name: $PVC-csi

+ namespace: $NAMESPACE

+ storageClassName: $STORAGE_CLASS_NEW

+ storage: $STORAGE_SIZE

+ volumeName: $PV-csi

+ kubectl apply -f $PVC-csi.yaml

+ LINE="PVC:$PVC,PV:$PV,StorageClassTarget:$STORAGE_CLASS_NEW"

+ printf '%s\n' "$LINE" >>$FILENAME

+ NAMESPACE=$1

+ FILENAME=$NAMESPACE-$(date +%Y%m%d%H%M)

+ EXISTING_STORAGE_CLASS=$2

+ STORAGE_CLASS_NEW=$3

+ VOLUME_STORAGE_CLASS=$4

+ START_TIME_STAMP=$5

+ END_TIME_STAMP=$6

+ for PVC in $(kubectl get pvc -n $NAMESPACE | awk '{ print $1}'); do

+ PVC_CREATION_TIME=$(kubectl get pvc $PVC -n $NAMESPACE -o jsonpath='{.metadata.creationTimestamp}')

+ if [[ $PVC_CREATION_TIME > $START_TIME_STAMP ]]; then

+ if [[ $END_TIME_STAMP > $PVC_CREATION_TIME ]]; then

+ PV="$(kubectl get pvc $PVC -n $NAMESPACE -o jsonpath='{.spec.volumeName}')"

+ RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ STORAGE_CLASS="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.storageClassName}')"

+ echo $PVC

+ RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ if [[ $STORAGE_CLASS == $EXISTING_STORAGE_CLASS ]]; then

+ STORAGE_SIZE="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.capacity.storage}')"

+ SKU_NAME="$(kubectl get storageClass $STORAGE_CLASS -o jsonpath='{.reclaimPolicy}')"

+ DISK_URI="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.azureDisk.diskURI}')"

+ TARGET_RESOURCE_GROUP="$(cut -d'/' -f5 <<<"$DISK_URI")"

+ echo $DISK_URI

+ echo $TARGET_RESOURCE_GROUP

+ PERSISTENT_VOLUME_RECLAIM_POLICY="$(kubectl get pv $PV -n $NAMESPACE -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')"

+ az snapshot create --resource-group $TARGET_RESOURCE_GROUP --name $PVC-$FILENAME --source "$DISK_URI"

+ SNAPSHOT_PATH=$(az snapshot list --resource-group $TARGET_RESOURCE_GROUP --query "[?name == '$PVC-$FILENAME'].id | [0]")

+ SNAPSHOT_HANDLE=$(echo "$SNAPSHOT_PATH" | tr -d '"')

+ echo $SNAPSHOT_HANDLE

+ cat <<EOF >$PVC-csi.yml

+ name: $PVC-$FILENAME

+ volumeSnapshotClassName: $VOLUME_STORAGE_CLASS

+ snapshotHandle: $SNAPSHOT_HANDLE

+ name: $PVC-$FILENAME

+ name: $PVC-$FILENAME

+ volumeSnapshotClassName: $VOLUME_STORAGE_CLASS

+ volumeSnapshotContentName: $PVC-$FILENAME

+ name: csi-$PVC

+ storageClassName: $STORAGE_CLASS_NEW

+ storage: $STORAGE_SIZE

+ name: $PVC-$FILENAME

+ kubectl create -f $PVC-csi.yml

+ LINE="OLDPVC:$PVC,OLDPV:$PV,VolumeSnapshotContent:volumeSnapshotContent-$FILENAME,VolumeSnapshot:volumesnapshot$FILENAME,OLDdisk:$DISK_URI"

+ printf '%s\n' "$LINE" >>$FILENAME

+ export SUBSCRIPTION_ID=<subscription id>

+ export RESOURCE_GROUP=<resource group name>

+ export CLUSTER_NAME=<aks cluster name>

+ az account set --subscription $SUBSCRIPTION_ID

+ az identity create --name $UAMI --resource-group $RESOURCE_GROUP

+ export USER_ASSIGNED_CLIENT_ID="$(az identity show -g $RESOURCE_GROUP --name $UAMI --query 'clientId' -o tsv)"

+ export IDENTITY_TENANT=$(az aks show --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --query identity.tenantId -o tsv)

+ export AKS_OIDC_ISSUER="$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv)"

+ export SERVICE_ACCOUNT_NAME="workload-identity-sa" # sample name; can be changed

+ export SERVICE_ACCOUNT_NAMESPACE="default" # can be changed to namespace of your workload

+ name: ${SERVICE_ACCOUNT_NAME}

+ namespace: ${SERVICE_ACCOUNT_NAMESPACE}

+ export FEDERATED_IDENTITY_NAME="aksfederatedidentity" # can be changed as needed

+ az identity federated-credential create --name $FEDERATED_IDENTITY_NAME --identity-name $UAMI --resource-group $RESOURCE_GROUP --issuer ${AKS_OIDC_ISSUER} --subject system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}

+ serviceAccountName: ${SERVICE_ACCOUNT_NAME}

+| `brazilsouth` | :heavy_check_mark: | :x: |

+| `francesouth` | :heavy_check_mark: | :x: |

+| `japanwest` | :heavy_check_mark: | :x: |

+| `koreasouth` | :heavy_check_mark: | :x: |

+| `southindia` | :heavy_check_mark: | :x: |

+| `uaenorth` | :heavy_check_mark: | :x: |

+| `ukwest` | :heavy_check_mark: | :x: |

+Use the following steps to enable Application Insights logging for an API. You can also enable Application Insights logging for all APIs.

+ > [!TIP]

+ > To enable logging for all APIs, select **All APIs**.

+ > [!WARNING]

+ > Overriding the default **Number of payload bytes to log** value **0** may significantly decrease the performance of your APIs.

+- By default, the single API logger (more granular level) overrides the one for all APIs.

+> [!NOTE]

+> See [Application Insights limits](../azure-monitor/service-limits.md#application-insights) for information about the maximum size and number of metrics and events per Application Insights instance.

+## Emit custom metrics

+You can emit [custom metrics](../azure-monitor/essentials/metrics-custom-overview.md) to Application Insights from your API Management instance. API Management emits custom metrics using the [emit-metric](emit-metric-policy.md) policy.

+> Custom metrics are a preview feature of Azure Monitor and subject to limitations.

+To emit custom metrics, perform the following configuration steps.

+1. Enable **Custom metrics (Preview)** with custom dimensions in your Application Insights instance.

+ 1. Navigate to your Application Insights instance in the portal.

+ 1. In the left menu, select **Usage and estimated costs**.

+ 1. Select **Custom metrics (Preview)** > **With dimensions**.

+ 1. Select **OK**.

+1. Add the `"metrics": true` property to the `applicationInsights` diagnostic entity that's configured in API Management. Currently you must add this property using the API Management [Diagnostic - Create or Update](/rest/api/apimanagement/current-ga/diagnostic/create-or-update) REST API. For example:

+ ```http

+ PUT https://management.azure.com/subscriptions/{SubscriptionId}/resourceGroups/{ResourceGroupName}/providers/Microsoft.ApiManagement/service/{APIManagementServiceName}/diagnostics/applicationinsights

+ {

+ [...]

+ {

+ "properties": {

+ "loggerId": "/subscriptions/{SubscriptionId}/resourceGroups/{ResourceGroupName}/providers/Microsoft.ApiManagement/service/{APIManagementServiceName}/loggers/{ApplicationInsightsLoggerName}",

+ "metrics": true

+ [...]

+ }

+ }

+ ```

+1. Ensure that the Application Insights logger is configured at the scope you intend to emit custom metrics (either all APIs, or a single API). For more information, see [Enable Application Insights logging for your API](#enable-application-insights-logging-for-your-api), earlier in this article.

+1. Configure the `emit-metric` policy at a scope where Application Insights logging is configured (either all APIs, or a single API) and is enabled for custom metrics. For policy details, see the [`emit-metric`](emit-metric-policy.md) policy reference.

+| &#9679; Windows Server 2022 (including Server Core) <br> &#9679; Windows Server 2019 (including Server Core) <br> &#9679; Windows Server 2016, version 1709, and 1803 (excluding Server Core) <br> &#9679; Windows Server 2012, 2012 R2 (excluding Server Core) <br> &#9679; Windows 10 Enterprise (including multi-session) and Pro | &#9679; Debian GNU/Linux 8, 9, 10, and 11 <br> &#9679; Ubuntu 18.04 LTS, 20.04 LTS, and 22.04 LTS <br> &#9679; SUSE Linux Enterprise Server 15.2, and 15.3 <br> &#9679; Red Hat Enterprise Linux Server 7, and 8ΓÇ»</br> *Hybrid Worker extension would follow support timelines of the OS vendor.|

+| &#9679; Windows Server 2022 (including Server Core) <br> &#9679; Windows Server 2019 (including Server Core) <br> &#9679; Windows Server 2016, version 1709 and 1803 (excluding Server Core) <br> &#9679; Windows Server 2012, 2012 R2 (excluding Server Core) <br> &#9679; Windows 10 Enterprise (including multi-session) and Pro| &#9679; Debian GNU/Linux 8,9,10, and 11 <br> &#9679; Ubuntu 18.04 LTS, 20.04 LTS, and 22.04 LTS <br> &#9679; SUSE Linux Enterprise Server 15.2, and 15.3 <br> &#9679; Red Hat Enterprise Linux Server 7, and 8 </br> *Hybrid Worker extension would follow support timelines of the OS vendor.ΓÇ»|

+**Connect with Connection String**

+```properties

+spring.cloud.azure.appconfiguration.stores[0].connection-strings[0]="${FIRST_REPLICA_CONNECTION_STRING}"

+spring.cloud.azure.appconfiguration.stores[0].connection-strings[1]="${SECOND_REPLICA_CONNECTION_STRING}"

+```

+> The failover support is available if you use version of **4.7.0** or later of any of the following packages.

+Select the storage account that your cache is using for persistence. Select **Data Protection** from the Resource menu. In the working pane, check the state of *Enable soft delete for blobs*. For more information on soft delete in Azure storage accounts, see [Enable soft delete for blobs](/azure/storage/blobs/soft-delete-blob-enable?tabs=azure-portal).

+description: This article demonstrates how to visualize warnings and errors returned by the Creator Conversion API.

+The *Drawing Error Visualizer* is a stand-alone web application that displays [Drawing package warnings and errors] detected during the conversion process. The Error Visualizer web application consists of a static page that you can use without connecting to the internet. You can use the Error Visualizer to fix errors and warnings in accordance with [Drawing package requirements]. The [Azure Maps Conversion API] returns a response with a link to the Error Visualizer only when an error is detected.

+This tutorial uses the [Postman] application, but you may choose a different API development environment.

+1. Upload your drawing package to the Azure Maps Creator service to obtain a `udid` for the uploaded package. For steps on how to upload a package, see [Upload a drawing package].

+2. Now that the drawing package is uploaded, use `udid` for the uploaded package to convert the package into map data. For steps on how to convert a package, see [Convert a drawing package].

+The downloaded zipped package from the `diagnosticPackageLocation` link contains the following two files.

+Open the _https://docsupdatetracker.net/index.html_ file using any of the following browsers, with the respective version number. You may use a different version, if the version offers equally compatible behavior as the listed version.

+The _ConversionWarningsAndErrors.json_ file has been placed at the root of the downloaded directory. To load the _ConversionWarningsAndErrors.json_, drag & drop the file onto the box. Or, select on the box, find the file in the `File Explorer dialogue`, and upload the file.

+The _ConversionWarningsAndErrors.json_ contains a list of your drawing package errors and warnings. To view detailed information about an error or warning, select the **Details** link. An intractable section appears below the list. You may now navigate to each error to learn more details on how to resolve the error.

+> [Creator for indoor maps]

+[Azure Maps Conversion API]: /rest/api/maps/v2/conversion

+[Convert a drawing package]: tutorial-creator-indoor-maps.md#convert-a-drawing-package

+[Creator for indoor maps]: creator-indoor-maps.md

+[Creator resource]: how-to-manage-creator.md

+[Drawing package requirements]: drawing-requirements.md

+[Drawing package warnings and errors]: drawing-conversion-error-codes.md

+[Postman]: https://www.postman.com/

+[Upload a drawing package]: tutorial-creator-indoor-maps.md#upload-a-drawing-package

+- `click` - Coordinates are added when the mouse or touch is clicked.

+- `freehand ` - Coordinates are added when the mouse or touch is dragged on the map.

+- `hybrid` - Coordinates are added when the mouse or touch is clicked or dragged.

+ Before any shape can be drawn, set the `drawingMode` option of the drawing manager to a supported drawing setting. This setting can be programmed, or invoked by pressing one of the drawing buttons on the toolbar. The drawing mode stays enabled, even after a shape has been drawn, making it easy to draw more shapes of the same type. Programmatically set the drawing mode to an idle state. Or, switch to an idle state by clicking the current drawing modes button on the toolbar.

+- Select the left mouse button, or touch the map to add a point to the map.

+- If the mouse is over the map, press the `F` key, and a point is added at the coordinate of the mouse pointer. This method provides higher accuracy for adding a point to the map. There's less movement on the mouse due to the pressing motion of the left mouse button.

+- Keep clicking, touching, or pressing `F` to add more points to the map.

+- Select any button in the drawing toolbar.

+- Programmatically set the drawing mode.

+- Press the `C` key.

+- Press the `Escape` key.

+- Click mode

+ - Select left mouse button, or touch the map to add each point of a line on the map. A coordinate is added to the line for each click or touch.

+ - If the mouse is over the map, press the `F` key, and a point is added at the coordinate of the mouse pointer. This method provides higher accuracy for adding a point to the map. There's less movement on the mouse due to the pressing motion of the left mouse button.

+ - Keep clicking until all the desired points have been added to the line.

+- Freehand mode

+ - Press down the left mouse button, or touch-down on the map and drag the mouse, or touch point around. Coordinates are added to the line as the mouse or touch point moves around the map. As soon as the mouse or touch-up event is triggered, the drawing is completed. The drawing managers `freehandInterval` option defines the frequency at which coordinates are added.

+- Hybrid mode

+ - Alternate between click and freehand methods, as desired, while drawing a single line. For example, click a few points, then hold and drag the mouse to add a bunch of points, then click a few more.

+- Hybrid/Click mode

+ - Double-click the map at the last point.

+ - Click on any button in the drawing toolbar.

+ - Programmatically set the drawing mode.

+- Freehand mode

+ - Release the mouse button or touch point.

+- Press the `C` key.

+- Press the `Escape` key.

+- Click mode

+ - Select the left mouse button, or touch the map to add each point of a polygon on the map. A coordinate is added to the polygon for each click or touch.

+ - If the mouse is over the map, press the `F` key, and a point is added at the coordinate of the mouse pointer. This method provides higher accuracy for adding a point to the map. There's less movement on the mouse due to the pressing motion of the left mouse button.

+ - Keep clicking until all the desired points have been added to the polygon.

+- Freehand mode

+ - Press down the left mouse button, or touch-down on the map and drag the mouse, or touch point around. Coordinates are added to the polygon as the mouse or touch point moves around the map. As soon as the mouse or touch-up event is triggered, the drawing is completed. The drawing managers `freehandInterval` option defines the frequency at which coordinates are added.

+- Hybrid mode

+ - Alternate between click and freehand methods, as desired, while drawing a single polygon. For example, click a few points, then hold and drag the mouse to add a bunch of points, then click a few more.

+- Hybrid/Click mode

+ - Double-click the map at the last point.

+ - Click on the first point in the polygon.

+ - Click on any button in the drawing toolbar.

+ - Programmatically set the drawing mode.

+- Freehand mode

+ - Release the mouse button or touch point.

+- Press the `C` key.

+- Press the `Escape` key.

+When the drawing manager is in `draw-rectangle` mode, the following actions can be done to draw points on the map, depending on the interaction mode. The generated shape follows the [extended GeoJSON specification for rectangles].

+- Press down the left mouse button, or touch-down on the map to add the first corner of the rectangle and drag to create the rectangle.

+- Release the mouse button or touch point.

+- Programmatically set the drawing mode.

+- Press the `C` key.

+- Press the `Escape` key.

+When the drawing manager is in `draw-circle` mode, the following actions can be done to draw points on the map, depending on the interaction mode. The generated shape follows the [extended GeoJSON specification for circles].

+- Press down the left mouse button, or touch-down on the map to add the center of the circle and drag give the circles a radius.

+- Release the mouse button or touch point.

+- Programmatically set the drawing mode.

+- Press the `C` key.

+- Press the `Escape` key.

+| `C` | Completes any drawing that is in progress and sets the drawing mode to idle. Focus moves to top-level map element. |

+| `Escape` | Cancels any drawing that is in progress and sets the drawing mode to idle. Focus moves to top-level map element. |

+| `Delete` or `Backspace` | If shapes are selected while the edit mode, delete them. |

+> [Drawing manager]

+> [Drawing toolbar]

+[extended GeoJSON specification for rectangles]: extend-geojson.md#rectangle

+[extended GeoJSON specification for circles]: extend-geojson.md#circle

+[Drawing manager]: /javascript/api/azure-maps-drawing-tools/atlas.drawing.drawingmanager

+[Drawing toolbar]: /javascript/api/azure-maps-drawing-tools/atlas.control.drawingtoolbar

+The data for geofence or set of geofences, represented by the `Feature` Object and `FeatureCollection` Object in `GeoJSON` format, is defined in [rfc7946]. In Addition to it:

+* Feature with `polygon` and `multipolygon` geometry type doesn't have a radius property.

+* The `expiredTime` is the expiration date and time of geofencing data. If the value of `userTime` in the request is later than this value, the corresponding geofence data is considered as expired data and isn't queried. Upon which, the geometryId of this geofence data is included in `expiredGeofenceGeometryId` array within the geofence response.

+* The `validityPeriod` is a list of validity time period of the geofence. If the value of `userTime` in the request falls outside of the validity period, the corresponding geofence data is considered as invalid and isn't queried. The geometryId of this geofence data is included in `invalidPeriodGeofenceGeometryId` array within geofence response. The following table shows the properties of validityPeriod element.

+* For each Feature, which contains `MultiPoint`, `MultiLineString`, `MultiPolygon` , or `GeometryCollection`, the properties are applied to all the elements. for example: All the points in `MultiPoint` use the same radius to form a multiple circle geofence.

+Following is a sample request body for a geofence represented as a circle geofence geometry in `GeoJSON` using a center point and a radius. The valid period of the geofence data starts from `2018-10-22`, 9AM to 5PM, repeated every day except for the weekend. `expiredTime` indicates this geofence data is considered expired, if `userTime` in the request is later than `2019-01-01`.

+For example, if the measurement is **Table rows**, the alert logic may be **Greater than 0** indicating that at least one record was returned. If the measurement is a columns value, then the logic may need to be greater than or less than a particular threshold value. In the example below, the log query is looking for anonymous requests to a storage account. If an anonymous request has been made, then we should trigger an alert. In this case, a single row returned would trigger the alert, so the alert logic should be **Greater than 0**.

+> [Azure Monitor Workbooks](../visualize/workbooks-overview.md)

+- The aks-preview version 0.5.136 or higher is required for this feature. Check the aks-preview version by using the `az version` command.

+Application volume groups (AVG) enable you to deploy all volumes for a single HANA host in one atomic step. The Azure portal and the Azure Resource Manager template have implemented prechecks and recommendations for deployment in areas including throughputs and volume naming conventions. As a REST API user, those checks and recommendations are not available.

+* Kerberos and LDAP enablement are not supported.

+1. **Networking:** You need to decide on the networking architecture. To use Azure NetApp Files, you need to create create a VNet that will host a delegated subnet for the Azure NetApp Files storage endpoints (IPs). To ensure that the size of this subnet is large enough, see [Considerations about delegating a subnet to Azure NetApp Files](azure-netapp-files-delegate-subnet.md#considerations).

+ 2. Create a virtual machine (VM) subnet and delegated subnet for Azure NetApp Files.

+1. **Create AvSet and proximity placement group (PPG):** For production landscapes, you should create an AvSet that is manually pinned to a data center where Azure NetApp Files resources are available in proximity. The AvSet pinning ensures that VMs won't be moved on restart. The proximity placement group (PPG) needs to be assigned to the AvSet. With the help of application volume groups, the PPG can find the closest Azure NetApp Files hardware. For more information, see [Best practices about proximity placement groups](application-volume-group-considerations.md#best-practices-about-proximity-placement-groups).

+1. **Manual Steps - Request AvSet pinning**: AvSet pinning is required for long term SAP HANA systems. The Microsoft capacity planning team ensures that the required VMs for SAP HANA and Azure NetApp Files resources are in proximity to available VMs. VMs will not move on restart.

+| `volumes` | Array of volumes to be created (see the next table for volume-granular details) | Volume count depends upon host configuration: <ul><li>Single-host (3-5 volumes) <br /> **Required**: _data_, _log_ and _shared_ <br /> **Optional**: _data-backup_, _log-backup_ </li><li> Multiple-host (two volumes) <br /> **Required**: _data_ and _log_ </li></ul> |

+| `name` | Volume name | None. Examples or recommended volume names: <ul><li> `SH9-data-mnt00001` data for Single-Host.</li><li> `SH9-log-backup` log-backup for Single-Host.</li><li> `HSR-SH9-shared` shared for HSR Secondary.</li><li> `DR-SH9-data-backup` data-backup for cross-region replication destination </li><li> `DR2-SH9-data-backup` data-backup for cross-region replication destination of HSR Secondary.</li></ul> |

+| `creationToken` | Export path name, typically same as the volume name. | None. Example: `SH9-data-mnt00001` |

+| `usageThreshhold` | Size of the volume in bytes. This must be in the 100 GiB to 100-TiB range. For instance, 100 GiB = 107374182400 bytes. | None. You should set volume size depending on the volume type. |

+| `subnetId` | Delegated subnet ID for Azure NetApp Files. | In a normal case where there are sufficient resources available, the number of IP addresses required in the subnet depends on the order of the application volume group created in the subscription: <ol><li> First application volume group created: the creation usually requires to 3-4 IP addresses but can require up to 5</li><li> Second application volume group created: Normally requires two IP addresses</li><li></li>Third and subsequent application volume group created: Normally, more IP addresses are not required</ol> |

+In the following examples, selected placeholders are specified. You should replace them with the values specific to your configuration. These values include:

+1. Extract the subscription ID. This automates the extraction of the subscription ID and generate the authorization token:

+#### Example single-host SAP HANA application volume group creation request

+This example pertains to data, log, shared, data-backup, and log-backup volumes demonstrating best practices for naming, sizing, and throughputs. This example serves as the primary volume if you're configuring an HSR pair.

+### Example 4: Deploy volumes for a disaster recovery HANA system using cross-region replication

+Cross-region replication is one way to set up a disaster recovery configuration for HANA, where the volumes of the HANA database in the DR-region are replicated on the storage side using cross-region replication in contrast to HSR, which replicates at the application level where it requires to have the HANA VMs deployed and running. Refer to [Add volumes for an SAP HANA system as a DR system using cross-region replication](application-volume-group-disaster-recovery.md) to understand for which volumes in cross-region replication relations are required (data, shared, log-backup), not allowed (log), or optional (data-backup).

+3. `<SrcVolumeId_data>`, `<SrcVolumeId_shared>`, `<SrcVolumeId_data-backup>`, `<SrcVolumeId_log-backup>`: cross-region replication source volume IDs for the data, shared, and log-backup cross-region replication destination volumes.

+The following example is extracted from a quickstart template, [Virtual Network with diagnostic logs settings](https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.network/vnet-create-with-diagnostic-logs/main.bicep):

+# [Python](#tab/azure-python)

+```python

+import os

+from azure.identity import AzureCliCredential

+from azure.mgmt.resource import ResourceManagementClient

+credential = AzureCliCredential()

+subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]

+resource_client = ResourceManagementClient(credential, subscription_id)

+rg_result = resource_client.resource_groups.begin_delete("exampleGroup")

+```

+# [Python](#tab/azure-python)

+```python

+import os

+from azure.identity import AzureCliCredential

+from azure.mgmt.resource import ResourceManagementClient

+credential = AzureCliCredential()

+subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]

+resource_client = ResourceManagementClient(credential, subscription_id)

+resource_client.resources.begin_delete_by_id(

+ "/subscriptions/{}/resourceGroups/{}/providers/{}/{}".format(

+ subscription_id,

+ "exampleGroup",

+ "Microsoft.Compute",

+ "virtualMachines/exampleVM"

+ ),

+ "2022-11-01"

+)

+```

+**HCX Enterprise Edition - Default**

+VMware HCX Enterprise is now available and supported on Azure VMware Solution at no extra cost. VMware HCX Enterprise brings valuable [services](https://docs.vmware.com/en/VMware-HCX/4.5/hcx-user-guide/GUID-32AF32BD-DE0B-4441-95B3-DF6A27733EED.html), like Replicated Assisted vMotion (RAV) and Mobility Optimized Networking (MON). VMware HCX Enterprise is now automatically installed for all new VMware HCX add-on requests, and existing VMware HCX Advanced customers can upgrade to VMware HCX Enterprise using the Azure portal. Learn more on how to [Install and activate VMware HCX in Azure VMware Solution](install-vmware-hcx.md).

+### Install Update Rollup 2 for Microsoft Azure Backup Server (MABS) version 3

+Installing the Update Rollup 2 for Microsoft Azure Backup Server (MABS) version 3 is mandatory for protecting the workloads. You can find the bug fixes and installation instructions in the [knowledge base article](https://support.microsoft.com/help/5004579/).

+<a name="ultra-disk-backup">Ultra SSD disks</a> | Supported with [Enhanced policy](backup-azure-vms-enhanced-policy.md). The support is currently in preview. <br><br> Supported region(s) - Sweden Central <br><br> To enroll your subscription for this feature, [fill this form](https://forms.office.com/r/1GLRnNCntU). <br><br> - Configuration of Ultra disk protection is supported via Recovery Services vault only. This configuration is currently not supported via virtual machine blade. <br><br> - Cross-region restore is currently not supported for machines using Ultra disks.

+|On-premises connectivity to Delegated Subnet via Global and Local Expressroute |Yes|

+|On-premises connectivity to Delegated Subnet via VPN GW| Yes |

+|On-premises connectivity via Secured HUB(Az Firewall NVA) | No|

+> On June 02, 2023, Microsoft will retire the Custom Translator v1.0 model platform. Existing v1.0 models must migrate to the new platform for continued processing and support.

+Following measured and consistent high-quality results using models trained on the Custom Translator new platform, the v1.0 platform is retiring. The new Custom Translator platform delivers significant improvements in many domains compared to both standard and Custom v1.0 platform translations. Migrate your v1.0 models to the new platform by June 02, 2023.

+* **May 01, 2023 through June 02, 2023** → Customers voluntarily migrate to new platform models.

+## Upgrade to new platform

+> * Select any or all of your v1.0 models then select **Train** to start new platform model training.

+* **Use the upgrade wizard**. Follow the steps listed in **Upgrade to the latest version** wizard. Depending on your training data size, it may take from a few hours to a full day to upgrade your models to the new platform.

+| Chinese-Simplified | `zh-hans` | `zh` also accepted |

+| French | `fr` | |

+| German | `de` | |

+| Italian | `it` | |

+| Japanese | `ja` | |

+| Korean | `ko` | |

+| Spanish | `es` | |

+| Portuguese | `pt` | |

+ Title: Skip setup screen of the UI Library

+description: Use Azure Communication Services UI Library for Mobile native to skip the setup screen

+zone_pivot_groups: acs-plat-ios-android

+#Customer intent: As a developer, I want to skip setup screen of the library in my application

+# Skip setup screen

+The feature enables the option to join a call without passing through the setup screen. It empowers developers to build their communication application using UI Library in a way that users can join a call directly without any user interaction. The feature also provides capability to configure camera and microphone default state. We're providing APIs to turn the camera and microphone on or off so that developers have the capability to configure the default state of the camera and microphone before joining a call.

+Learn how to set up the skip setup screen feature correctly in your application.

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- A deployed Communication Services resource. [Create a Communication Services resource](../../quickstarts/create-communication-resource.md).

+- A `User Access Token` to enable the call client. For more information on [how to get a `User Access Token`](../../quickstarts/access-tokens.md)

+- Optional: Complete the quickstart for [getting started with the UI Library composites](../../quickstarts/ui-library/get-started-composites.md)

+## Next steps

+- [Learn more about UI Library](../../concepts/ui-library/ui-library-overview.md)

+To perform a custom container application that extends the capability of Azure Key Vault (AKV) - Secure Key Release and Microsoft Azure Attestation (MAA), use the below as a high level reference flow. An easy approach is to review the current side-car implementation code in this [side-car GitHub project](https://github.com/microsoft/confidential-sidecar-containers/tree/d933d0f4e3d5498f7ed9137189ab6a23ade15466/pkg/common).

+ systemctl stop striim-node

+ systemctl stop striim-dbms

+ systemctl start striim-dbms

+ systemctl start striim-node

+ systemctl stop striim-node

+ systemctl stop striim-dbms

+ systemctl start striim-dbms

+ systemctl start striim-node

+ Title: Concepts of customer managed keys in Azure Cosmos DB for PostgreSQL.

+description: Concepts of customer managed keys.

+# Customer-managed keys in Azure Cosmos DB for PostgreSQL

+Data stored in your Azure Cosmos DB for PostgreSQL cluster is automatically and seamlessly encrypted with keys managed by Microsoft. These keys are referred to as **service-managed keys**. Azure Cosmos DB for PostgreSQL uses [Azure Storage encryption](../../storage/common/storage-service-encryption.md) to encrypt data at-rest by default using service-managed keys. You can optionally choose to add an extra layer of security by enabling encryption with **customer-managed keys**.

+## Service-managed keys

+The Azure Cosmos DB for PostgreSQL service uses the FIPS 140-2 validated cryptographic module for storage encryption of data at-rest. All Data including backups and temporary files created while running queries are encrypted on disk. The service uses the AES 256-bit cipher included in Azure storage encryption, and the keys are system-managed. Storage encryption is always on and cannot be disabled.

+## Customer-managed keys

+Many organizations require full control of access to data using a customer-managed key. Data encryption with customer-managed keys for Azure Cosmos DB for PostgreSQL enables you to bring your own key for protecting data at rest. It also allows organizations to implement separation of duties in the management of keys and data. With customer-managed encryption, you're responsible for, and in full control of, a key's lifecycle, usage permissions, and auditing of operations.

+Data encryption with customer-managed keys for Azure Cosmos DB for PostgreSQL is set at the server level. Data, including backups, are encrypted on disk. This encryption includes the temporary files created while running queries. For a given cluster, a customer-managed key, called the key encryption key (**KEK**), is used to encrypt the service's data encryption key (**DEK**). The KEK is an asymmetric key stored in a customer-owned and customer-managed [Azure Key Vault](../../key-vault/index.yml) instance.

+| | Description |

+| | |

+| **Data encryption key (DEK)** | A data encryption key is a symmetric AES256 key used to encrypt a partition or block of data. Encrypting each block of data with a different key makes crypto analysis attacks more difficult. The resource provider or application instance that is encrypting and decrypting a specific block requires access to DEKs. When you replace a DEK with a new key, only the data in its associated block must be re-encrypted with the new key. |

+| **Key encryption key (KEK)** | A key encryption key is an encryption key used to encrypt the DEKs. A KEK that never leaves a key vault allows the DEKs themselves to be encrypted and controlled. The entity that has access to the KEK might be different than the entity that requires the DEK. Since the KEK is required to decrypt the DEKs, the KEK is effectively a single point and deletion of the KEK effectively deletes the DEKs. |

+> [!NOTE]

+> Azure Key Vault is a cloud-based key management system. It's highly available and provides scalable, secure storage for RSA cryptographic keys, optionally backed by FIPS 140-2 Level 2 validated hardware security modules (**HSM**s). A key vault doesn't allow direct access to a stored key but provides encryption and decryption services to authorized entities. A key vault can generate the key, import it, or have it transferred from an on-premises HSM device.

+The DEKs, encrypted with the KEKs, are stored separately. Only an entity with access to the KEK can decrypt these DEKs. For more information, see [Security in encryption at rest.](../../security/fundamentals/encryption-atrest.md).

+## How data encryption with a customer-managed key works

+For a cluster to use customer-managed keys stored in Key Vault for encryption of the DEK, a Key Vault administrator gives the following access rights to the server:

+| | Description |

+| | |

+| **get** | Enables retrieving the public part and properties of the key in the key vault. |

+| **wrapKey** | Enables encryption of the DEK. The encrypted DEK is stored in Azure Cosmos DB for PostgreSQL. |

+| **unwrapKey** | Enables decryption of the DEK. Azure Cosmos DB for PostgreSQL requires the decrypted DEK to encrypt/decrypt data. |

+The key vault administrator can also enable logging of Key Vault audit events, so they can be audited later.

+When the Azure Cosmos DB for PostgreSQL cluster is configured to use the customer-managed key stored in the key vault, the cluster sends the DEK to the key vault for encryptions. Key Vault returns the encrypted DEK, which is stored in the user database. Similarly, when needed, the server sends the protected DEK to the key vault for decryption. Auditors can use [Azure Monitor](../../azure-monitor/index.yml) to review Key Vault audit event logs, if logging is enabled.

+[ ](media/concepts-customer-managed-keys/architecture-customer-managed-keys.png#lightbox)

+## Benefits

+Data encryption with customer-managed keys for Azure Cosmos DB for PostgreSQL provides the following benefits:

+- You fully control data access with the ability to remove the key and make the database inaccessible.

+- Full control over the key lifecycle, including rotation of the key to align with specific corporate policies.

+- Central management and organization of keys in Azure Key Vault.

+- Ability to implement separation of duties between security officers, database administrators, and system administrators.

+- Enabling encryption doesn't have any extra performance effect with or without customer-managed keys. Azure Cosmos DB for PostgreSQL relies on Azure Storage for data encryption in both customer-managed and service-managed key scenarios.

+## Next steps

+>[!div class="nextstepaction"]

+>[Enable encryption with customer managed keys](how-to-customer-managed-keys.md)

+ Title: How to enable encryption with customer managed keys in Azure Cosmos DB for PostgreSQL.

+description: How to enable data encryption with customer managed keys.

+# Enable data encryption with customer-managed keys in Azure Cosmos DB for PostgreSQL

+## Prerequisites

+- An existing Azure Cosmos DB for PostgreSQL account.

+ - If you have an Azure subscription, [create a new account](../nosql/how-to-create-account.md?tabs=azure-portal).

+ - If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

+ - Alternatively, you can [try Azure Cosmos DB free](../try-free.md) before you commit.

+## Enable data encryption with customer-managed keys

+> [!IMPORTANT]

+> Create all the following resources in the same region where your Azure Cosmos DB for PostgreSQL cluster will be deployed.

+1. Create a User-Assigned Managed Identity. Currently, Azure Cosmos DB for PostgreSQL only supports user-assigned managed identities.

+1. Create an Azure Key Vault and add an access policy to the created User-Assigned Managed Identity with the following key permissions: Get, Unwrap Key, and Wrap Key.

+1. Generate a Key in the Key Vault (supported key types: RSA 2048, 3071, 4096).

+1. Select the Customer-Managed Key encryption option during the creation of the Azure Cosmos DB for PostgreSQL cluster and select the appropriate User-Assigned Managed Identity, Key Vault, and Key created in Steps 1, 2, and 3.

+## Detailed steps

+### User Assigned Managed Identity

+1. Search for Managed Identities in the global search bar.

+ 

+1. Create a new User assigned managed Identity in the same region as your Azure Cosmos DB for PostgreSQL cluster.

+ 

+Learn more about [User Assigned Managed Identity.](../../active-directory/managed-identities-azure-resources/how-manage-user-assigned-managed-identities.md?pivots=identity-mi-methods-azp#create-a-user-assigned-managed-identity).

+### Key Vault

+Using customer-managed keys with Azure Cosmos DB for PostgreSQL requires you to set two properties on the Azure Key Vault instance that you plan to use to host your encryption keys: Soft Delete and Purge Protection.

+1. If you create a new Azure Key Vault instance, enable these properties during creation:

+ [  ](media/how-to-customer-managed-keys/key-vault-soft-delete.png#lightbox)

+

+1. If you're using an existing Azure Key Vault instance, you can verify that these properties are enabled by looking at the Properties section on the Azure portal. If any of these properties arenΓÇÖt enabled, see the "Enabling soft delete" and "Enabling Purge Protection" sections in one of the following articles.

+ * How to use [soft-delete with PowerShell.](../../key-vault/general/key-vault-recovery.md)

+ * How to use [soft-delete with Azure CLI.](../../key-vault/general/key-vault-recovery.md)

+1. The key Vault must be set with 90 days for 'Days to retain deleted vaults'. If the existing key Vault has been configured with a lower number, you'll need to create a new key vault as it can't be modified after creation.

+ > [!IMPORTANT]

+ > Your Azure Key Vault instance must be allow public access from all the networks.

+### Add an Access Policy to the Key Vault

+1. From the Azure portal, go to the Azure Key Vault instance that you plan to use to host your encryption keys. Select Access configuration from the left menu and then select Go to access policies.

+ [  ](media/how-to-customer-managed-keys/access-policy.png#lightbox)

+1. Select + Create.

+1. In the Permissions Tab under the Key permissions drop-down menu, select Get, Unwrap Key, and Wrap Key permissions.

+ [  ](media/how-to-customer-managed-keys/access-policy-permissions.png#lightbox)

+1. In the Principal Tab, select the User Assigned Managed Identity you had created in prerequisite step.

+1. Navigate to Review + create select Create.

+### Create / Import Key

+1. From the Azure portal, go to the Azure Key Vault instance that you plan to use to host your encryption keys.

+1. Select Keys from the left menu and then select +Generate/Import.

+ [  ](media/how-to-customer-managed-keys/create-key.png#lightbox)

+1. The customer-managed key to be used for encrypting the DEK can only be asymmetric RSA Key type. All RSA Key sizes 2048, 3072 and 4096 are supported.

+1. The key activation date (if set) must be a date and time in the past. The expiration date (if set) must be a future date and time.

+1. The key must be in the Enabled state.

+1. If you're importing an existing key into the key vault, make sure to provide it in the supported file formats (`.pfx`, `.byok`, `.backup`).

+1. If you're manually rotating the key, the old key version shouldn't be deleted for at least 24 hours.

+### Enable CMK encryption during the provisioning for a new cluster

+ # [Portal](#tab/portal)

+ 1. During the provisioning of a new Cosmos DB for PostgreSQL cluster, after providing the necessary information under Basics and Networking Tab, Navigate to the Encryption (Preview) Tab.

+ [ ](media/how-to-customer-managed-keys/encryption-tab.png#lightbox)

+ 1. Select Customer Managed Key under Data encryption key option.

+ 1. Select the User Assigned Managed Identity created in the previous section.

+

+ 1. Select the Key Vault created in the previous step, which has the access policy to the user managed identity selected in the previous step.

+ 1. Select the Key created in the previous step, and then select Review+create.

+ 1. Verify that CMK is encryption is enabled by Navigating to the Data Encryption(preview) blade of the Cosmos DB for PostgreSQL cluster in the Azure portal.

+ 

+ > [!NOTE]

+ > Data encryption can only be configured during the creation of a new cluster and can't be updated on an existing cluster. A workaround for updating the encryption configuration on an existing cluster is to restore an existing PITR backup to a new cluster and configure the data encryption during the creation of the newly restored cluster.

+ # [ARM Template](#tab/arm)

+ ```json

+ {

+ "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",

+ "contentVersion": "1.0.0.0",

+ "parameters": {

+ "serverGroupName": {

+ "type": "string"

+ },

+ "location": {

+ "type": "string"

+ },

+ "administratorLoginPassword": {

+ "type": "secureString"

+ },

+ "previewFeatures": {

+ "type": "bool"

+ },

+ "postgresqlVersion": {

+ "type": "string"

+ },

+ "coordinatorVcores": {

+ "type": "int"

+ },

+ "coordinatorStorageSizeMB": {

+ "type": "int"

+ },

+ "numWorkers": {

+ "type": "int"

+ },

+ "workerVcores": {

+ "type": "int"

+ },

+ "workerStorageSizeMB": {

+ "type": "int"

+ },

+ "enableHa": {

+ "type": "bool"

+ },

+ "enablePublicIpAccess": {

+ "type": "bool"

+ },

+ "serverGroupTags": {

+ "type": "object"

+ },

+ "userAssignedIdentityUrl": {

+ "type": "string"

+ },

+ "encryptionKeyUrl": {

+ "type": "string"

+ }

+ },

+ "variables": {},

+ "resources": [

+ {

+ "name": "[parameters('serverGroupName')]",

+ "type": "Microsoft.DBforPostgreSQL/serverGroupsv2",

+ "kind": "CosmosDBForPostgreSQL",

+ "apiVersion": "2020-10-05-privatepreview",

+ "identity":

+ {

+ "type": "UserAssigned",

+ "userAssignedIdentities":

+ {

+ "/subscriptions/04b0358b-392b-41d6-899e-b75cb292321e/resourcegroups/yogkuleuapcmkmarlintest/providers/Microsoft.ManagedIdentity/userAssignedIdentities/marlincmktesteus2euapuai1": {}

+ }

+ },

+ "location": "[parameters('location')]",

+ "tags": "[parameters('serverGroupTags')]",

+ "properties": {

+ "createMode": "Default",

+ "administratorLogin": "citus",

+ "administratorLoginPassword": "[parameters('administratorLoginPassword')]",

+ "backupRetentionDays": 35,

+ "enableMx": false,

+ "enableZfs": false,

+ "previewFeatures": "[parameters('previewFeatures')]",

+ "postgresqlVersion": "[parameters('postgresqlVersion')]",

+ "dataencryption":

+ {

+ "primaryKeyUri": "[parameters('encryptionKeyUrl')]",

+ "primaryUserAssignedIdentityId": "[parameters('userAssignedIdentityUrl')]",

+ "type": "AzureKeyVault"

+ },

+ "serverRoleGroups": [

+ {

+ "name": "",

+ "role": "Coordinator",

+ "serverCount": 1,

+ "serverEdition": "GeneralPurpose",

+ "vCores": "[parameters('coordinatorVcores')]",

+ "storageQuotaInMb": "[parameters('coordinatorStorageSizeMB')]",

+ "enableHa": "[parameters('enableHa')]"

+ },

+ {

+ "name": "",

+ "role": "Worker",

+ "serverCount": "[parameters('numWorkers')]",

+ "serverEdition": "MemoryOptimized",

+ "vCores": "[parameters('workerVcores')]",

+ "storageQuotaInMb": "[parameters('workerStorageSizeMB')]",

+ "enableHa": "[parameters('enableHa')]",

+ "enablePublicIpAccess": "[parameters('enablePublicIpAccess')]"

+ }

+ ]

+ },

+ "dependsOn": []

+ }

+ ],

+ "outputs": {}

+ }

+ ```

+### High availability

+ When CMK encryption is enabled on the primary cluster, all standby HA replicas are automatically encrypted by the primary clusterΓÇÖs CMK

+### Restrictions

+* CMK encryption can't be enabled on cross region read replicas.

+* CMK encryption can only be enabled during the creation of a new Azure Cosmos DB for PostgreSQL cluster.

+* CMK encryption isn't supported with Private access (including VNET).

+### Changing encryption configuration by performing a PITR

+Encryption configuration can be changed from service managed encryption to CMK encryption or vice versa while performing a Point in restore operation to a new cluster.

+# [Portal](#tab/portal)

+ 1. Navigate to the Data Encryption blade, and select Initiate restore operation. Alternatively, you can perform PITR by selecting the Restore option in the overview blade.

+ [ ](media/how-to-customer-managed-keys/point-in-time-restore.png#lightbox)

+ 1. You can change/configure the Data Encryption from the Encryption(preview) Tab.

+# [ARM Template](#tab/arm)

+ ```json

+ {

+ "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",

+ "contentVersion": "1.0.0.0",

+ "parameters": {

+ "location": {

+ "type": "string"

+ },

+ "sourceLocation": {

+ "type": "string"

+ },

+ "subscriptionId": {

+ "type": "string"

+ },

+ "resourceGroupName": {

+ "type": "string"

+ },

+ "serverGroupName": {

+ "type": "string"

+ },

+ "sourceServerGroupName": {

+ "type": "string"

+ },

+ "restorePointInTime": {

+ "type": "string"

+ },

+ "encryptionKeyUrl": {

+ "type": "string"

+ },

+ "userAssignedIdentityUrl": {

+ "type": "string"

+ }

+ },

+ "variables": {

+ "api": "2020-02-14-privatepreview"

+ },

+ "resources": [

+ {

+ "apiVersion": "2020-10-05-privatepreview",

+ "location": "[parameters('location')]",

+ "name": "[parameters('serverGroupName')]",

+ "identity":

+ {

+ "type": "UserAssigned",

+ "userAssignedIdentities":

+ {

+ "/subscriptions/04b0358b-392b-41d6-899e-b75cb292321e/resourcegroups/yogkuleuapcmkmarlintest/providers/Microsoft.ManagedIdentity/userAssignedIdentities/marlincmktesteus2euapuai1": {}

+ }

+ },

+ "properties": {

+ "createMode": "PointInTimeRestore",

+ "sourceServerGroupName": "[parameters('sourceServerGroupName')]",

+ "pointInTimeUTC": "[parameters('restorePointInTime')]",

+ "sourceLocation": "[parameters('location')]",

+ "sourceSubscriptionId": "[parameters('subscriptionId')]",

+ "sourceResourceGroupName": "[parameters('resourceGroupName')]",

+ "enableMx": false,

+ "enableZfs": false,

+ "dataencryption":

+ {

+ "primaryKeyUri": "[parameters('encryptionKeyUrl')]",

+ "primaryUserAssignedIdentityId": "[parameters('userAssignedIdentityUrl')]",

+ "type": "AzureKeyVault"

+ },

+ }

+ "type": "Microsoft.DBforPostgreSQL/serverGroupsv2"

+ }

+ ]

+ }

+ ```

+### Monitor the customer-managed key in Key Vault

+To monitor the database state, and to enable alerting for the loss of transparent data encryption protector access, configure the following Azure features:

+* [Azure Resource Health](../../service-health/resource-health-overview.md): An inaccessible database that has lost access to the Customer Key shows as "Inaccessible" after the first connection to the database has been denied.

+* [Activity log](../../service-health/alerts-activity-log-service-notifications-portal.md): When access to the Customer Key in the customer-managed Key Vault fails, entries are added to the activity log. You can reinstate access as soon as possible, if you create alerts for these events.

+* [Action groups](../../azure-monitor/alerts/action-groups.md): Define these groups to send you notifications and alerts based on your preference.

+### April 2023

+* Public preview: Data Encryption at rest using [Customer Managed Keys](./concepts-customer-managed-keys.md) is now supported for all available regions.

+ * See [this guide](./how-to-customer-managed-keys.md) for the steps to enable data encryption using customer managed keys.

+* Data encryption at rest using customer managed keys.

+Microsoft recommends that you manage access to resources using Azure RBAC. However, if you are still using the classic deployment model and managing the classic resources by using [Azure Service Management PowerShell Module](/powershell/azure/servicemanagement/install-azure-ps), you'll need to use a classic administrator.

+To activate your enrollment, the initial enterprise administrator signs in to the [Azure Enterprise portal](https://ea.azure.com) using their work, school, or Microsoft account.

+You can add multiple activities for error handling.

+ Title: Enable logging for Azure Data Manager for Agriculture

+description: Learn how enable logging and debugging in Azure Data Manager for Agriculture

+# Azure Data Manager for Agriculture logging

+After you create a Data Manager for Agriculture resource instance, you can monitor how and when your resources are accessed, and by whom. You can also debug reasons for failure for data-plane requests. To do this, you need to enable logging for Azure Data Manager for Agriculture. You can then save log information at a destination such as a storage account, event hub or a log analytics workspace, that you provide.

+This article provides you with the steps to setup logging for Azure Data Manager for Agriculture.

+## Enable collection of logs

+After creating a Data Manager for Agriculture service resource, navigate to diagnostics settings and then select `add diagnostics settings`. Follow these steps to start collecting and storing logs:

+1. Provide a name for the diagnostic setting.

+2. Select the categories that you want to start collecting logs for.

+3. Choose the destination for collection from storage account, event hub or a log analytics workspace.

+Now you can navigate to the destination you specified in the diagnostic setting to access logs. You can access your logging information 10 minutes (at most) after the Data Manager for Agriculture operation. In most cases, it's quicker.

+## Interpret your logs

+Each log follows the schema listed in the table. The table contains the field names and descriptions:

+| Field name | Description |

+| | |

+| **time** |Date and time in UTC. |

+| **resourceId** |Azure Resource Manager resource ID. For logs, this is the Data Manager for Agriculture resource ID. |

+| **operationName** |Name of the operation, as documented. |

+| **operationVersion** |REST API version requested by the client. |

+| **category** |Type of result. |

+| **resultType** |Result of the REST API request (success or failure). |

+| **resultSignature** |HTTP status. |

+| **resultDescription** |Extra description about the result, when available. |

+| **durationMs** |Time it took to service the REST API request, in milliseconds.|

+| **callerIpAddress** |IP address of the client that made the request. |

+|**level**|The severity level of the event (Informational, Warning, Error, or Critical).

+| **correlationId** |An optional GUID that can be used to correlate logs. |

+| **identity** |Identity from the token that was presented in the REST API request. This is usually an object ID and an application ID or either of the two.|

+|**location**|The region of the resource emitting the event such as "East US" |

+| **properties** |For each `operationName` this contains: `requestUri` (URI of the API request), `partyId`(partyId associated with the request, wherever applicable),`dataPlaneResourceId` (ID that uniquely identifies the data-plane resource in the request) and `requestBody` (contains the request body for the API call associated with the `operationName` for all categories other than ApplicationAuditLogs). </br> Other than the common one's mentioned before `jobProcessesLogs` category has: </br> 1. This list is of fields across operationNames: </br> `jobRunType` (can be oneTime or periodic), `jobId` (ID of the job), `initiatedBy` (indicates whether a job was triggered by a user or by the service). </br> 2. This list is of fields for failed farmOperation related jobs: </br> `farmOperationEntityId` (ID of the entity that failed to be created by the farmOperation job), `farmOperationEntityType`(type of the entity that failed to be created), `errorCode`(code for job failure), `errorMessage`(description of failure), `internalErrorCode`(failure code provide by the provider), `internalErrorMessage`(description of the failure provided by the provider), `providerId`(ID of the provider).

+The `categories` field for Data Manager for Agriculture can have values that are listed in the following table:

+### Categories table

+| category| Description |

+| | |

+|FarmManagementLogs| Logs for CRUD operations for party, Farm, Field, Boundary, Seasonal Field, Crop, CropVariety, Season, Attachment, prescription maps, prescriptions, management zones, zones, plant tissue analysis and nutrient analyses.

+|FarmOperationsLogs|Logs for CRUD operations for FarmOperations data ingestion job, ApplicationData, PlantingData, HarvestingData, TillageData

+|SatelliteLogs| Logs for create and get operations for Satellite data ingestion job

+|WeatherLogs|Logs for create, delete and get operations for weather data ingestion job

+|ProviderAuthLogs| Logs for create, update, delete, cascade delete, get and get all for Oauth providers. It also has logs for get, get all, cascade delete for oauth tokens.

+|JobProcessedLogs| Logs for indicating success or failure and reason of failure for jobs. In addition to logs for resource cascade delete jobs, data-ingestion jobs, it also contains logs for farm operations and event handling jobs.

+|ModelInferenceLogs| Logs for create and get operations for biomass model job.

+|InsightLogs| Logs for get and get all operations for insights.

+|ApplicationAuditLogs| Logs for privileged actions such as data-plane resource create, update, delete and subscription management operations. Complete list is in the operation name table below.

+The `operationName` field values are in *Microsoft.AgFoodPlatform/resource-name/read or write or delete or action* format.

+* `/write` suffix in the operation name corresponds to a create or update the resource-name

+* `/read`suffix in the operation name corresponds to a GET/ LIST /GET ALL API calls or GET status for a cascade delete job for the resource-name

+* `/delete` suffix corresponds to the deletion of the resource-name

+* `/action` suffix corresponds to POST method calls for a resource name

+* `/processed` suffix corresponds to completion of a job (a PUT method call). This indicates status of the job (success or failure).

+* `/failures` suffix corresponds to failure of a farm operation job (a PUT method call) and contains description about the reason of failure.

+The nomenclature for Jobs is as following:

+* For data-ingestion jobs: *Microsoft.AgFoodPlatform/ingestionJobs/<'resource-name'>DataingestionJobs/write*

+* For deletion jobs: *Microsoft.AgFoodPlatform/deletionJobs/<'resource-name'>cascadeDeleteJobs/write*

+The following table lists the **operationName** values and corresponding REST API commands for a category as a tab:

+### FarmManagementLogs

+| operationName |

+| |

+|Microsoft.AgFoodPlatform/farmers/write|

+|Microsoft.AgFoodPlatform/farmers/read|

+|Microsoft.AgFoodPlatform/deletionJobs/farmersCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/farms/write|

+|Microsoft.AgFoodPlatform/farms/read|

+|Microsoft.AgFoodPlatform/farms/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/farmsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/field/write|

+|Microsoft.AgFoodPlatform/field/read|

+|Microsoft.AgFoodPlatform/field/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/fieldsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/seasonalField/write|

+|Microsoft.AgFoodPlatform/seasonalField/read|

+|Microsoft.AgFoodPlatform/seasonalField/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/seasonalFieldsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/boundaries/write|

+|Microsoft.AgFoodPlatform/boundaries/read|

+|Microsoft.AgFoodPlatform/boundaries/delete|

+|Microsoft.AgFoodPlatform/boundaries/action|

+|Microsoft.AgFoodPlatform/deletionJobs/fieldsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/crops/write|

+|Microsoft.AgFoodPlatform/crops/read|

+|Microsoft.AgFoodPlatform/crops/delete|

+|Microsoft.AgFoodPlatform/cropVarieties/write|

+|Microsoft.AgFoodPlatform/cropVarieties/read|

+|Microsoft.AgFoodPlatform/cropVarieties/delete|

+|Microsoft.AgFoodPlatform/seasons/write|

+|Microsoft.AgFoodPlatform/seasons/read|

+|Microsoft.AgFoodPlatform/seasons/delete|

+|Microsoft.AgFoodPlatform/attachments/write|

+|Microsoft.AgFoodPlatform/attachments/read|

+|Microsoft.AgFoodPlatform/attachments/delete|

+|Microsoft.AgFoodPlatform/prescriptions/write|

+|Microsoft.AgFoodPlatform/prescriptions/read|

+|Microsoft.AgFoodPlatform/prescriptions/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/prescriptionsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/prescriptionMaps/write|

+|Microsoft.AgFoodPlatform/prescriptionMaps/read|

+|Microsoft.AgFoodPlatform/prescriptionMaps/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/prescriptionMapsCascadeDeleteJobs/write|

+|Microsoft.AgFoodPlatform/managementZones/write|

+|Microsoft.AgFoodPlatform/managementZones/read|

+|Microsoft.AgFoodPlatform/managementZones/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/managementZonescascadeDeletejobs/write|

+|Microsoft.AgFoodPlatform/zones/write|

+|Microsoft.AgFoodPlatform/zones/read|

+|Microsoft.AgFoodPlatform/zones/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/zonesCascadedeleteJobs/write|

+|Microsoft.AgFoodPlatform/plantTissueanalyses/write|

+|Microsoft.AgFoodPlatform/plantTissueanalyses/read|

+|Microsoft.AgFoodPlatform/plantTissueanalyses/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/plantTissueanalysesCascadedeleteJobs/write|

+|Microsoft.AgFoodPlatform/nutrientAnalyses/write|

+|Microsoft.AgFoodPlatform/nutrientAnalyses/read|

+|Microsoft.AgFoodPlatform/nutrientAnalyses/delete|

+|Microsoft.AgFoodPlatform//deletionJobs/nutrientAnalysescascadeDeletejobs/delete|

+### FarmOperationLogs

+| operationName |

+| |

+|Microsoft.AgFoodPlatform/ingetsionJobs/farmOperationsdataIngestionjobs/write|

+|Microsoft.AgFoodPlatform/applicationData/read|

+|Microsoft.AgFoodPlatform/applicationData/write|

+|Microsoft.AgFoodPlatform/applicationData/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/applicationDatacascadeDeletejob/write|

+|Microsoft.AgFoodPlatform/plantingData/write|

+|Microsoft.AgFoodPlatform/plantingData/read|

+|Microsoft.AgFoodPlatform/plantingData/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/plantingDatacascadeDeletejob/write|

+|Microsoft.AgFoodPlatform/harvestingData/write|

+|Microsoft.AgFoodPlatform/harvestingData/read|

+|Microsoft.AgFoodPlatform/harvestingData/delete|

+|Microsoft.AgFoodPlatform/deletionJobs/harvestingDatacascadeDeletejob/write|

+|Microsoft.AgFoodPlatform/tillageData/Write|

+|Microsoft.AgFoodPlatform/tillageData/Read|

+|Microsoft.AgFoodPlatform/tillageData/Delete|

+|Microsoft.AgFoodPlatform/deletionJobs/tillageDatacascadeDeletejob/write|

+### SatelliteLogs

+| operationName |

+| |

+|Microsoft.AgFoodPlatform/ingestionJobs/satelliteDataingestionJob/write|

+|Microsoft.AgFoodPlatform/scenes/read|

+### WeatherLogs

+| operationName |

+| |

+|Microsoft.AgFoodPlatform/ingestionJobs/weatherDataingestionJob/write|

+|Microsoft.AgFoodPlatform/weather/read|

+|Microsoft.AgFoodPlatform/deletionJobs/weatherDeletejob/delete|

+### ProviderAuthLogs

+| operationName|

+| |

+|Microsoft.AgFoodPlatform/oauthProviders/write|

+|Microsoft.AgFoodPlatform/oauthProviders/read|

+|Microsoft.AgFoodPlatform/oauthProviders/delete|

+|Microsoft.AgFoodPlatform/oauthTokens/read|

+|Microsoft.AgFoodPlatform/oauthTokens/delete|

+

+### JobProcessesLogs

+ |operationName|

+ | |

+ |Microsoft.AgFoodPlatform/ingestionJobs/satelliteDataIngestionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/satelliteDataDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/weatherDataIngestionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/weatherDataDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/oauthProvidersCascadeDeleteJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/oauthTokensRemoveJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/biomassModelJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/ImageProcessingRasterizeJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationDataIngestionJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationDataIngestionJobs/processed/failures

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationPeriodicJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationPeriodicJobs/processed/failures

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationEventHandlingJobs/processed

+ |Microsoft.AgFoodPlatform/ingestionJobs/farmOperationEventHandlingJobs/processed/failures

+ |Microsoft.AgFoodPlatform/deletionJobs/applicationDataCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/tillageDataCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/plantingDataCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/harvestDataCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/managementZonesCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/zonesCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/plantTissueAnalysesCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/prescriptionsCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/prescriptionMapsCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/insightsCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/farmersCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/farmsCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/fieldsCascadeDeletionJobs/processed

+ |Microsoft.AgFoodPlatform/deletionJobs/seasonalFieldsCascadeDeletionJobs/processed

+

+### ApplicationAuditLogs

+The operation names corresponding to write operations in other categories are present in this category. These common logs don't contain the request body. These common logs can be correlated using the correlationId field. Some of the control plane operations that aren't part of the rest of the categories are listed below.

+|operationName|

+| |

+|Create Data Manager for Agriculture Resource|

+|Update Data Manager for Agriculture Resource|

+|Delete Data Manager for Agriculture Resource|

+|Create Subscription|

+|Update Subscription|

+|Data Plane Authentication|

+## Query resource logs in a log analytics workspace

+All the `categories` of resource logs are mapped as a table in log analytics. To access logs for each category, you need to create a diagnostic setting to send data to a log analytics workspace. In this workspace, you can query any of the tables listed to obtain the relevant logs.

+### List of tables in log analytics and their mapping to categories in resource logs

+| Table name in log analytics| Categories in resource logs |Description

+| | | |

+|AgriFoodFarmManagementLogs|FarmManagementLogs| Logs for CRUD operations for party, Farm, Field, Boundary, Seasonal Field, Crop, CropVariety, Season, Attachment, prescription maps, prescriptions, management zones, zones, plant tissue analysis and nutrient analyses.

+|AgriFoodFarmOperationsLogs|FarmOperationsLogs| Logs for CRUD operations for FarmOperations data ingestion job, ApplicationData, PlantingData, HarvestingData, TillageData.

+|AgriFoodSatelliteLogs|SatelliteLogs| Logs for create and get operations for satellite data ingestion job.

+|AgriFoodWeatherLogs|WeatherLogs|Logs for create, delete and get operations for weather data ingestion job.

+|AgriFoodProviderAuthLogs|ProviderAuthLogs| Logs for create, update, delete, cascade delete, get and get all for oauth providers. It also has logs for get, get all, cascade delete for oauth tokens.

+|AgriFoodInsightLogs|InsightLogs| Logs for get and get all operations for insights.

+|AgriFoodModelInferenceLogs|ModelInferenceLogs| Logs for create and get operations for biomass model job.

+|AgriFoodJobProcessedLogs|JobProcessedLogs| Logs for indicating success or failure and reason of failure for jobs. In addition to logs for resource cascade delete jobs, data-ingestion jobs. It also contains logs for farm operations and event handling jobs.

+|AgriFoodApplicationAuditLogs|ApplicationAuditLogs| Logs for privileged actions such as data-plane resource create, update, delete and subscription management operations.

+### List of columns in log analytics tables

+| Field name | Description |

+| | |

+|**Time** |Date and time in UTC. |

+|**ResourceId** |Azure Resource Manager resource ID for Data Manager for Agriculture logs.|

+|**OperationName** |Name of the operation, as documented in the earlier table. |

+|**OperationVersion** |REST API version requested by the client. |

+|**Category** |Category details in the Data Manager for Agriculture logs, this can be any value as listed in the category table. |

+|**ResultType** |Result of the REST API request (success or failure). |

+|**ResultSignature** |HTTP status. |

+|**ResultDescription** |More description about the result, when available. |

+|**DurationMs** |Time it took to service the REST API request, in milliseconds.|

+|**CallerIpAddress** |IP address of the client that made the request. |

+|**Level**|The severity level of the event (informational, warning, error, or critical).|

+|**CorrelationId** |An optional GUID that can be used to correlate logs. |

+|**ApplicationId**| Application ID indicating identity of the caller.|

+|**ObjectId**| Object ID indicating identity of the caller.|

+|**ClientTenantId**| ID of the tenant of the caller.|

+|**SubscriptionId**| ID of the subscription used by the caller.

+|**Location**|The region of the resource emitting the event such as "East US" |

+|**JobRunType**| Available only in `AgriFoodJobProcessesLogs` table, indicates type of the job run. Value can be either of periodic or one time. |

+|**JobId**| Available in`AgriFoodJobProcessesLogs`, `AgriFoodSatelliteLogs`, `AgriFoodWeatherLogs`, and `AgriFoodModelInferenceLogs`, indicates ID of the job. |

+|**InitiatedBy**| Available only in `AgriFoodJobProcessesLogs` table. Indicates whether a job was initiated by a user or by the service. |

+|**partyId**| ID of the party associated with the operation. |

+|**Properties** | Available only in`AgriFoodJobProcessesLogs` table, it contains: `farmOperationEntityId` (ID of the entity that failed to be created by the farmOperation job), `farmOperationEntityType`(Type of the entity that failed to be created, can be ApplicationData, PeriodicJob, etc.), `errorCode`(Code for failure of the job at Data Manager for Agriculture end),`errorMessage`(Description of failure at the Data Manager for Agriculture end),`internalErrorCode`(Code of failure of the job provide by the provider),`internalErrorMessage`(Description of the failure provided by the provider),`providerId`(ID of the provider such as JOHN-DEERE). |

+Each of these tables can be queried by creating a log analytics workspace. Reference for query language is [here](https://learn.microsoft.com/azure/data-explorer/kql-quick-reference).

+### List of sample queries in the log analytics workspace

+| Query name | Description |

+| | |

+|**Status of farm management operations for a party** |Fetches a count of successes and failures of operations within the `FarmManagementLogs` category for each party.

+|**Job execution statistics for a party**| Provides a count of successes and failures of for all operations in the `JobProcessedLogs` category for each party.

+|**Failed Authorization**|Identifies a list of users who failed to access your resource and the reason for this failure.

+|**Status of all operations for a party**|Aggregates failures and successes across categories for a party.

+|**Usage trends for top 100 parties based on the operations performed**|Retrieves a list of top 100 parties based on the number of hits received across categories. This query can be edited to track trend of usage for a particular party.|

+All the queries listed above can be used as base queries to form custom queries in a log analytics workspace. This list of queries can also be accessed in the `Logs` tab in your Azure Data Manager for Agriculture resource on Azure portal.

+## Next steps

+Learn how to [setup private links](./how-to-set-up-private-links.md).

+- You must have [added a resource group to your IoT solution](quickstart-configure-your-solution.md).

+- You must have [installed the Defender for IoT micro agent](quickstart-standalone-agent-binary-installation.md).

+- You must have [configured the Microsoft Defender for IoT agent-based solution](how-to-configure-agent-based-solution.md).

+- Learned how to [investigate security recommendations](quickstart-investigate-security-recommendations.md).

+Opening each aggregated alert displays the detailed alert description, remediation steps, and device ID for each device that triggered an alert. The alert severity and direct investigation is accessible using Log Analytics.

+1. Review the alert **description**, **severity**, **source of the detection**, and **device details** of all devices that issued this alert in the aggregation period.

+1. After reviewing the alert specifics, use the **manual remediation step** instructions to help remediate and resolve the issue that caused the alert.

+## Investigate alerts in your Log Analytics workspace

+> - Investigate recommendations in a Log Analytics workspace

+- You must have [added a resource group to your IoT solution](quickstart-configure-your-solution.md).

+- You must have [installed the Defender for IoT micro agent](quickstart-standalone-agent-binary-installation.md).

+- You must have [configured the Microsoft Defender for IoT agent-based solution](how-to-configure-agent-based-solution.md).

+Open each aggregated recommendation to display the detailed recommendation description, remediation steps, and device ID for each device that triggered a recommendation. It also displays recommendation severity and direct-investigation access using Log Analytics.

+## Investigate recommendations in a Log Analytics workspace

+**To access your recommendations in a Log Analytics workspace**:

+ - For Ubuntu 18.04:

+ - For Ubuntu 20.04:

+ - For Debian 9 (both AMD64 and ARM64):

+This procedure describes how you can connect the Defender for IoT micro agent to the IoT Hub via a proxy.

+1. On your micro agent machine, create a `/etc/defender_iot_micro_agent/conf.json` file with the following content:

+1. Restart the micro agent. Run:

+1. On your micro agent machine, open the `/etc/defender_iot_micro_agent/conf.json` file and add the following content:

+1. Restart the micro agent. Run:

+1. On your micro agent machine, open the `/etc/defender_iot_micro_agent/conf.json` file and add the following content:

+1. Restart the micro agent. Run:

+- [Authenticate using a module identity connection string](#authenticate-using-a-module-identity-connection-string).

+1. Navigate to the **IoT Hub** > **`Your hub`** > **Device management** > **Devices**.

+ > [!NOTE]

+ > The connection string includes a key that enables direct access to the module itself, therefore includes sensitive information that should only be used and readable by root users.

+# Manage your deployment environment

+To help you manage [Azure resources](../azure-resource-manager/management/overview.md#terminology) more easily, you can create automated management tasks for a specific resource or resource group. These tasks vary in number and availability, based on the resource type. For example:

+- For an [Azure storage account](../storage/common/storage-account-overview.md), you can set up an automation task that sends the monthly cost for that storage account.

+- For an [Azure virtual machine](../virtual-machines/overview.md), you can create an automation task that turns on or turns off that virtual machine on a predefined schedule. Specifically, you can create a task that automatically starts or stops the virtual machine a specific number of times every day, week, or month. On the task's **Configure** tab, set the **Interval** value to the number of times and the **Frequency** value to **Day**, **Week**, or **Month**. The automation task continues to work until you delete or disable the task.

+

+ For example, you can create a task that automatically starts a virtual machine once every day. On the task's **Configure** tab, set **Interval** to **1** and **Frequency** to **Day**.

+ |`sslSecret`| The name of the Kubernetes secret in the `azureml` namespace. This config is used to store `cert.pem` (PEM-encoded TLS/SSL cert) and `key.pem` (PEM-encoded TLS/SSL key), which are required for inference HTTPS endpoint support when ``allowInsecureConnections`` is set to `False`. For a sample YAML definition of `sslSecret`, see [Configure sslSecret](./how-to-secure-kubernetes-online-endpoint.md). Use this config or a combination of `sslCertPemFile` and `sslKeyPemFile` protected config settings. |N/A| Optional | Optional |

+> When __loggiging large files__ with `log_artifact` or `log_model`, you may encounter time out errors before the upload of the file is completed. Consider increasing the timeout value by adjusting the environment variable `AZUREML_ARTIFACTS_DEFAULT_TIMEOUT`. It's default value is `300` (seconds).

+Batch Endpoints can be used for processing tabular data that contain text. Those deployments are supported in both MLflow and custom models. In this tutorial we will learn how to deploy a model that can perform text summarization of long sequences of text using a model from HuggingFace.

+The information in this article is based on code samples contained in the [azureml-examples](https://github.com/azure/azureml-examples) repository. To run the commands locally without having to copy/paste YAML and other files, clone the repo and then change directories to the [`cli/endpoints/batch/deploy-models/huggingface-text-summarization`](https://github.com/azure/azureml-examples/tree/main/cli/endpoints/batch/deploy-models/huggingface-text-summarization) if you are using the Azure CLI or [`sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization`](https://github.com/azure/azureml-examples/tree/main/sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization) if you are using our SDK for Python.

+# [Azure CLI](#tab/cli)

+# [Python](#tab/python)

+In a Jupyter notebook:

+```python

+!git clone https://github.com/Azure/azureml-examples --depth 1

+!cd azureml-examples/sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization

+```

+# [Azure CLI](#tab/cli)

+Due to the size of the model, it hasn't been included in this repository. Instead, you can download a copy from the HuggingFace model's hub. You need the packages `transformers` and `torch` installed in the environment you are using.

+```python

+%pip install transformers torch

+```

+Use the following code to download the model to a folder `model`:

+# [Python](#tab/python)

+ # [Azure CLI](#tab/cli)

+ # [Azure CLI](#tab/cli)

+ # [Python](#tab/python)

+ # [Python](#tab/python)

+ # [Azure CLI](#tab/cli)

+ # [Python](#tab/python)

+ # [Python](#tab/python)

+ # [Python](#tab/python)

+ # [Python](#tab/python)

+ # [Python](#tab/python)

+In this article, you'll learn how to troubleshoot common workload (including training jobs and endpoints) errors on the [Kubernetes compute](./how-to-attach-kubernetes-to-workspace.md).

+The common Kubernetes endpoint errors on Kubernetes compute are categorized into two scopes: **compute scope** and **cluster scope**. The compute scope errors are related to the compute target, such as the compute target is not found, or the compute target is not accessible. The cluster scope errors are related to the underlying Kubernetes cluster, such as the cluster itself is not reachable, or the cluster is not found.

+### How to check sslCertPemFile and sslKeyPemFile is correct?

+Use the commands below to run a baseline check for your cert and key. This is to allow for any known errors to be surfaced. Expect the second command to return "RSA key ok" without prompting you for password.

+```bash

+openssl x509 -in cert.pem -noout -text

+openssl rsa -in key.pem -noout -check

+```

+Run the commands below to verify whether sslCertPemFile and sslKeyPemFile are matched:

+```bash

+openssl x509 -in cert.pem -noout -modulus | md5sum

+openssl rsa -in key.pem -noout -modulus | md5sum

+```

+When the training job is running, you can check the job status in the workspace portal. When you encounter some abnormal job status, such as the job retried multiple times, or the job has been stuck in initializing state, or even the job has eventually failed, you can follow the guide below to troubleshoot the issue.

+### Job retry debugging

+### Job pod get stuck in Init state

+If the job runs longer than you expected and if you find that your job pods are getting stuck in an Init state with this warning `Unable to attach or mount volumes: *** failed to get plugin from volumeSpec for volume ***-blobfuse-*** err=no volume plugin matched`, the issue might be occurring because Azure Machine Learning extension doesn't support download mode for input data.

+To resolve this issue, change to mount mode for your input data.

+### Common job failure errors

+Below is a list of common error types that you might encounter when using Kubernetes compute to create and execute a training job, which you can trouble shoot by following the guideline:

+* [Job failed. 137](#job-failed-137)

+* [Job failed. E45004](#job-failed-e45004)

+* [Job failed. 400](#job-failed-400)

+* [Give either an account key or SAS token](#give-either-an-account-key-or-sas-token)

+* [AzureBlob authorization failed](#azureblob-authorization-failed)

+#### Job failed. 137

+Azure Machine Learning Kubernetes job failed. 137:PodPattern matched: {"containers":[{"name":"training-identity-sidecar","message":"Updating certificates in /etc/ssl/certs...\n1 added, 0 removed; done.\nRunning hooks in /etc/ca-certificates/update.d...\ndone.\n * Serving Flask app 'msi-endpoint-server' (lazy loading)\n * Environment: production\n WARNING: This is a development server. Do not use it in a production deployment.\n Use a production WSGI server instead.\n * Debug mode: off\n * Running on http://127.0.0.1:12342/ (Press CTRL+C to quit)\n","code":137}]}

+Check your proxy setting and check whether 127.0.0.1 was added to proxy-skip-range when using `az connectedk8s connect` by following this [network configuring](how-to-access-azureml-behind-firewall.md#scenario-use-kubernetes-compute).

+#### Job failed. E45004

+If the error message is:

+```bash

+Azure Machine Learning Kubernetes job failed. E45004:"Training feature is not enabled, please enable it when install the extension."

+```

+Please check whether you have `enableTraining=True` set when doing the Azure Machine Learning extension installation. More details could be found at [Deploy Azure Machine Learning extension on AKS or Arc Kubernetes cluster](how-to-deploy-kubernetes-extension.md)

+### Job failed. 400

+#### Give either an account key or SAS token

+If you need to access Azure Container Registry (ACR) for Docker image, and to access the Storage Account for training data, this issue should occur when the compute is not specified with a managed identity.

+To access Azure Container Registry (ACR) from a Kubernetes compute cluster for Docker images, or access a storage account for training data, you need to attach the Kubernetes compute with a system-assigned or user-assigned managed identity enabled.

+In the above training scenario, this **computing identity** is necessary for Kubernetes compute to be used as a credential to communicate between the ARM resource bound to the workspace and the Kubernetes computing cluster. So without this identity, the training job will fail and report missing account key or sas token. Take accessing storage account for example, if you don't specify a managed identity to your Kubernetes compute, the job fails with the following error message:

+```bash

+Unable to mount data store workspaceblobstore. Give either an account key or SAS token

+```

+This is because machine learning workspace default storage account without any credentials is not accessible for training jobs in Kubernetes compute.

+To mitigate this issue, you can assign Managed Identity to the compute in compute attach step, or you can assign Managed Identity to the compute after it has been attached. More details could be found at [Assign Managed Identity to the compute target](how-to-attach-kubernetes-to-workspace.md#assign-managed-identity-to-the-compute-target).

+#### AzureBlob authorization failed

+If you need to access the AzureBlob for data upload or download in your training jobs on Kubernetes compute, then the job fails with the following error message:

+Unable to upload project files to working directory in AzureBlob because the authorization failed.

+This is because the authorization failed when the job tries to upload the project files to the AzureBlob. You can check the following items to troubleshoot the issue:

+* Make sure the storage account has enabled the exceptions of ΓÇ£Allow Azure services on the trusted service list to access this storage accountΓÇ¥ and the workspace is in the resource instances list.

+* Make sure the workspace has a system assigned managed identity.

+Errors regarding to identity and authentication:

+* [TokenRefreshFailed](#error-tokenrefreshfailed)

+* [GetAADTokenFailed](#error-getaadtokenfailed)

+* [ACRAuthenticationChallengeFailed](#error-acrauthenticationchallengefailed)

+* [ACRTokenExchangeFailed](#error-acrtokenexchangefailed)

+Errors regarding to crashloopbackoff:

+Errors regarding to scoring script:

+Others:

+* [NamespaceNotFound](#error-namespacenotfound)

+### ERROR: TokenRefreshFailed

+This is because extension cannot get principal credential from Azure because the Kubernetes cluster identity is not set properly, please re-install the [Azure Machine Learning extension](../machine-learning/how-to-deploy-kubernetes-extension.md) and try again.

+### ERROR: GetAADTokenFailed

+This is because the Kubernetes cluster request AAD token failed or timeout, please check your network accessibility then try again.

+* You can follow the [Configure required network traffic](../machine-learning/how-to-access-azureml-behind-firewall.md#scenario-use-kubernetes-compute ) to check the outbound proxy, make sure the cluster can connect to workspace.

+* The workspace endpoint url can be found in online endpoint CRD in cluster.

+If your workspace is a private workspace which disabled public network access, the Kubernetes cluster should only communicate with that private workspace through the private link.

+* You can check if the workspace access allows public access, no matter if an AKS cluster itself is public or private, it cannot access the private workspace.

+* More information you can refer to [Secure Azure Kubernetes Service inferencing environment](../machine-learning/how-to-secure-kubernetes-inferencing-environment.md#what-is-a-secure-aks-inferencing-environment)

+### ERROR: ACRAuthenticationChallengeFailed

+This is because the Kubernetes cluster cannot reach ACR service of the workspace to do authentication challenge. Please check your network, especially the ACR public network access, then try again.

+You can follow the troubleshooting steps in [GetAADTokenFailed](#error-getaadtokenfailed) to check the network.

+### ERROR: ACRTokenExchangeFailed

+This is because the Kubernetes cluster exchange ACR token failed because AAD token is unauthorized yet, since the role assignment takes some time, so you can wait a moment then try again.

+This failure may also be due to too many requests to the ACR service at that time, it should be a transient error, you can try again later.

+| CRM Appetency Labels Shared |Labels from the KDD Cup 2009 customer relationship prediction challenge ([orange_small_train_appetency.labels](https://kdd.org/cupfiles/KDDCupData/2009/orange_small_train_appetency.labels)).|

+|CRM Churn Labels Shared|Labels from the KDD Cup 2009 customer relationship prediction challenge ([orange_small_train_churn.labels](https://kdd.org/cupfiles/KDDCupData/2009/files/orange_small_train_churn.labels)).|

+|CRM Dataset Shared | This data comes from the KDD Cup 2009 customer relationship prediction challenge ([orange_small_train.data.zip](https://kdd.org/cupfiles/KDDCupData/2009/orange_small_train.data.zip)). <br/>The dataset contains 50K customers from the French Telecom company Orange. Each customer has 230 anonymized features, 190 of which are numeric and 40 are categorical. The features are very sparse. |

+|CRM Upselling Labels Shared|Labels from the KDD Cup 2009 customer relationship prediction challenge ([orange_large_train_upselling.labels](https://kdd.org/cupfiles/KDDCupData/2009/orange_small_train_upselling.labels)|

+ Title: 'Quickstart: Create an Azure Database for MariaDB - ARM template'

+Download the certificate needed to communicate over SSL with your Azure Database for MySQL server from [https://cacerts.digicert.com/BaltimoreCyberTrustRoot.crt.pem](https://cacerts.digicert.com/BaltimoreCyberTrustRoot.crt.pem) and save the certificate file to your local drive (this tutorial uses c:\ssl for example).

+See the following links for certificates for servers in sovereign clouds: [Azure Government](https://cacerts.digicert.com/BaltimoreCyberTrustRoot.crt.pem), [Azure China](https://dl.cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem), and [Azure Germany](https://www.d-trust.net/cgi-bin/D-TRUST_Root_Class_3_CA_2_2009.crt).

+RESOURCE_ID=$(az identity show --resource-group myResourceGroup --name myManagedIdentity --query id --output tsv)

+CLIENT_ID=$(az identity show --resource-group myResourceGroup --name myManagedIdentity --query clientId --output tsv)

+az vm identity assign --resource-group myResourceGroup --name myVM --identities $RESOURCE_ID

+echo $CLIENT_ID

+ACCESS_TOKEN=$(curl -s 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fossrdbms-aad.database.windows.net&client_id=CLIENT_ID' -H Metadata:true | jq -r .access_token)

+This article presents the different options available for organizations to establish remote access for their users. It also covers ways to supplement their existing solutions with extra capacity during periods of peak utilization.

+For example, real-time telemedicine traffic of doctor/patient interaction has a high importance and is sensitive to delay or jitter. Replication of traffic between storage solutions isn't delay sensitive. Telemedicine traffic must be routed via the most optimal network path with a high quality of service, whereas it's acceptable to use a suboptimal route for traffic between storage solutions.

+The Microsoft network is also designed to handle various types of network traffic. This traffic can include delay-sensitive multimedia traffic for Skype and Teams, content delivery networks, real-time big data analysis, Azure Storage, Bing, and Xbox. For optimal performance, Microsoft's network directs traffic intended for its resources or passing through them to be routed as close as possible to the traffic's point of origin.

+Another way to support a remote workforce is to deploy a virtual desktop infrastructure (VDI) hosted in your Azure virtual network, secured with Azure Firewall. For example, Azure Virtual Desktop is a desktop and app virtualization service that runs in Azure. With Virtual Desktop, you can set up a scalable and flexible environment in your Azure subscription without the need to run any extra gateway servers. You're responsible only for the Virtual Desktop virtual machines in your virtual network. For more information, see [Azure Firewall remote work support](../firewall/remote-work-support.md).

+- **Azure virtual network peering**: You can connect virtual networks together by using virtual network peering. Virtual network peering is useful if your resources are in more than one Azure region or if you need to connect multiple virtual networks to support remote workers. For more information, see [Virtual network peering][VNet-peer].

+- **Azure VPN-based solution**: For remote employees connected to Azure, you can provide them with access to your on-premises networks by establishing a S2S VPN connection. This connection is between your on-premises networks and Azure VPN Gateway. For more information, see [Create a site-to-site connection][S2S].

+- **Azure ExpressRoute**: By using ExpressRoute private peering, you can enable private connectivity between your Azure deployments and on-premises infrastructure or your infrastructure in a colocation facility. ExpressRoute, via Microsoft peering, also permits accessing public endpoints at Microsoft from your on-premises network.

+| [Remote work using Azure VPN Gateway point-to-site](../vpn-gateway/work-remotely-support.md) | Review available options to set up remote access for users or to supplement their existing solutions with extra capacity for your organization.|

+| Parameter|Description|Example|Required|

+|||||

+|resource-group |Use an appropriate resource group name specifically for ISD of your choice|ResourceGroupName|True

+|resource-name |Resource Name of the l2isolationDomain|example-l2domain| True

+|location|AODS Azure Region used during NFC Creation|eastus| True

+|nf-Id |network fabric ID|/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFresourcegroupname/providers/Microsoft.ManagedNetworkFabric/NetworkFabrics/NFname"| True

+|Vlan-id | VLAN identifier value. VLANs 1-500 are reserved and can't be used. The VLAN identifier value can't be changed once specified. The isolation-domain must be deleted and recreated if the VLAN identifier value needs to be modified. The range is between 501-4095|501| True

+|mtu | maximum transmission unit is 1500 by default, if not specified|1500|

+|administrativeState| Enable/Disable indicate the administrative state of the isolationDomain|Enable|

+| provisioningState | Indicates provisioning state |

+## L2 Isolation-Domain

+--resource-group "ResourceGroupName" \

+--nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/NetworkFabrics/NFname" \

+--vlan-id 750\

+--mtu 1501

+ "annotation": null,user

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l2IsolationDomains/example-l2domain",

+ "location": "eastus",

+ "mtu": 1501,

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT14:57:59.167177+00:00",

+ "lastModifiedAt": "2023-XX-XXT14:57:59.167177+00:00",

+ "vlanId": 750

+az nf l2domain show --resource-group "ResourceGroupName" --resource-name "example-l2domain"

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l2IsolationDomains/example-l2domain",

+ "location": "eastus",

+ "mtu": 1501,

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT14:57:59.167177+00:00",

+ "lastModifiedAt": "2023-XX-XXT14:57:59.167177+00:00",

+ "vlanId": 750

+az nf l2domain list --resource-group "ResourceGroupName"

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l2IsolationDomains/example-l2domain",

+ "mtu": 1501,

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxxxxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2022-XX-XXT22:26:33.065672+00:00",

+ "lastModifiedAt": "2022-XX-XXT14:46:45.753165+00:00",

+ "vlanId": 750

+ }

+Only after the Isolation-Domain is Enabled, that the layer 2 Isolation-Domain configuration is pushed to the Network Fabric devices.

+az nf l2domain update-admin-state --resource-group "ResourceGroupName" --resource-name "example-l2domain" --state Enable/Disable

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l2IsolationDomains/example-l2domain",

+ "location": "eastus",

+ "mtu": 1501,

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT14:57:59.167177+00:00",

+ "lastModifiedAt": "2023-XX-XXT14:57:59.167177+00:00",

+az nf l2domain delete --resource-group "ResourceGroupName" --resource-name "example-l2domain"

+Procedure to show, enable/disable and delete IPv6 based isolation-domains is same as used for IPv4.

+Vlan range for creation Isolation Domain 501-4095

+| Parameter|Description|Example|Required|

+|||||

+|resource-group |Use an appropriate resource group name specifically for ISD of your choice|ResourceGroupName|True|

+|resource-name |Resource Name of the l3isolationDomain|example-l3domain|True|

+|location|AODS Azure Region used during NFC Creation|eastus|True|

+|nf-Id |azure subscriptionId used during NFC Creation|/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/NetworkFabrics/NFName"| True|

+az nf l3domain create

+--resource-group "ResourceGroupName"

+--location "eastus"

+--nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/NetworkFabrics/NFName"

+ "aggregateRouteConfiguration": null,

+ "connectedSubnetRoutePolicy": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain",

+ "location": "eastus",

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/NFresourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "redistributeConnectedSubnets": "True",

+ "redistributeStaticRoutes": "False",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2022-XX-XXT06:23:43.372461+00:00",

+ "createdBy": "email@example.com",

+ "lastModifiedAt": "2023-XX-XXT09:40:38.815959+00:00",

+ "lastModifiedBy": "email@example.com",

+az nf l3domain show --resource-group "ResourceGroupName" --resource-name "example-l3domain"

+ "aggregateRouteConfiguration": null,

+ "connectedSubnetRoutePolicy": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain",

+ "location": "eastus",

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/NFresourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "provisioningState": "Succeeded",

+ "redistributeConnectedSubnets": "True",

+ "redistributeStaticRoutes": "False",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT09:40:38.815959+00:00",

+ "createdBy": "email@example.com",

+ "lastModifiedAt": "2023-XX-XXT09:40:46.923037+00:00",

+ "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "lastModifiedByType": "Application"

+az nf l3domain list --resource-group "ResourceGroupName"

+ "aggregateRouteConfiguration": null,

+ "connectedSubnetRoutePolicy": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain",

+ "location": "eastus",

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/NFresourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "redistributeConnectedSubnets": "True",

+ "redistributeStaticRoutes": "False",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT09:40:38.815959+00:00",

+ "createdBy": "email@example.com",

+ "lastModifiedAt": "2023-XX-XXT09:40:46.923037+00:00",

+ "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "lastModifiedByType": "Application"

+## Optional parameters for Isolation Domain

+| Parameter|Description|Example|Required|

+|||||

+| redistributeConnectedSubnet | Advertise connected subnets default value is True |True | |

+| redistributeStaticRoutes |Advertise Static Routes can have value of true/False. Defualt Value is False | False | |

+| aggregateRouteConfiguration|List of Ipv4 and Ipv6 route configurations | | |

+## Internal Network Creation

+| Parameter|Description|Example|Required|

+|||||

+|vlan-Id |Vlan identifier with range from 501 to 4095|1001|True|

+|resource-group|Use the corresponding NFC resource group name| NFCresourcegroupname | True

+|l3-isolation-domain-name|Resource Name of the l3isolationDomain|example-l3domain | True

+|location|AODS Azure Region used during NFC Creation|eastus | True

+## Options to create Internal Networks

+|Parameter|Description|Example|Required|

+|||||

+|connectedIPv4Subnets |IPv4 subnet used by the HAKS cluster's workloads|10.0.0.0/24||

+|connectedIPv6Subnets |IPv6 subnet used by the HAKS cluster's workloads|10:101:1::1/64||

+|staticRouteConfiguration |IPv4 Prefix of the static route|10.0.0.0/24|

+|bgpConfiguration|IPv4 nexthop address|10.0.0.0/24| |

+|defaultRouteOriginate | True/False "Enables default route to be originated when advertising routes via BGP" | True | |

+|peerASN |Peer ASN of Network Function|65047||

+|allowAS |Allows for routes to be received and processed even if the router detects its own ASN in the AS-Path. Input as 0 is disable, Possible values are 1-10, default is 2.|2||

+|allowASOverride |Enable Or Disable allowAS|Enable||

+|ipv4ListenRangePrefixes| BGP IPv4 listen range, maximum range allowed in /28| 10.1.0.0/26 | |

+|ipv6ListenRangePrefixes| BGP IPv6 listen range, maximum range allowed in /127| 3FFE:FFFF:0:CD30::/126| |

+|ipv4ListenRangePrefixes| BGP IPv4 listen range, maximum range allowed in /28| 10.1.0.0/26 | |

+|ipv4NeighborAddress| IPv4 neighbor address|10.0.0.11| |

+|ipv6NeighborAddress| IPv6 neighbor address|10:101:1::11| |

+az nf internalnetwork create

+--resource-group "ResourceGroupName"

+--l3-isolation-domain-name "example-l3domain"

+--resource-name "example-internalnetwork"

+--vlan-id 805

+--connected-ipv4-subnets '[{"prefix":"10.1.2.0/24"}]'

+--mtu 1500

+--bgp-configuration '{"defaultRouteOriginate": "True", "allowAS": 2, "allowASOverride": "Enable", "PeerASN": 65535, "ipv4ListenRangePrefixes": ["10.1.2.0/28"]}'

+{

+ "administrativeState": "Enabled",

+ "annotation": null,

+ "bfdDisabledOnResources": null,

+ "bfdForStaticRoutesDisabledOnResources": null,

+ "bgpConfiguration": {

+ "allowAs": 2,

+ "allowAsOverride": "Enable",

+ "annotation": null,

+ "bfdConfiguration": null,

+ "defaultRouteOriginate": "True",

+ "fabricAsn": 65046,

+ "ipv4ListenRangePrefixes": [

+ "10.1.2.0/28"

+ ],

+ "ipv4NeighborAddress": null,

+ "ipv6ListenRangePrefixes": null,

+ "ipv6NeighborAddress": null,

+ "peerAsn": 65535

+ },

+ "bgpDisabledOnResources": null,

+ "connectedIPv4Subnets": [

+ {

+ "annotation": null,

+ "prefix": "10.1.2.0/24"

+ }

+ ],

+ "connectedIPv6Subnets": null,

+ "disabledOnResources": null,

+ "exportRoutePolicyId": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain",

+ "importRoutePolicyId": null,

+ "mtu": 1500,

+ "name": "internalnetwork805",

+ "provisioningState": "Accepted",

+ "resourceGroup": "ResourceGroupName",

+ "staticRouteConfiguration": null,

+ "systemData": {

+ "createdAt": "2023-XX-XXT05:26:33.547816+00:00",

+ "createdBy": "email@example.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XXT05:26:33.547816+00:00",

+ "lastModifiedBy": "email@example.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/l3isolationdomains/internalnetworks",

+ "vlanId": 805

+## Multiple static routes with single next hop

+az nf internalnetwork create

+--resource-name "example-internalnetwork"

+--l3domain "example-l3domain"

+--resource-group "ResourceGroupName"

+--location "eastus"

+--vlan-id "2028"

+--mtu "1500"

+--connected-ipv4-subnets '[{"prefix":"10.18.34.0/24","gateway":"10.18.34.2"}]' --bgp-configuration '{"defaultRouteOriginate":true,"peerASN":65510,"ipv4Prefix":"10.18.34.0/24"}'

+--static-route-configuration '{"ipv4Routes":[{"prefix":"10.23.0.0/19","nextHop":["10.20.0.1"]},{"prefix":"10.24.0.0/19","nextHop":["10.20.0.1"]}]}'

+```

+Expected Output

+```json

+{

+ "administrativeState": "Enabled",

+ "annotation": null,

+ "bfdDisabledOnResources": null,

+ "bfdForStaticRoutesDisabledOnResources": null,

+ "bgpConfiguration": {

+ "allowAs": 2,

+ "allowAsOverride": "Enable",

+ "annotation": null,

+ "bfdConfiguration": null,

+ "defaultRouteOriginate": "True",

+ "fabricAsn": 65046,

+ "ipv4ListenRangePrefixes": null,

+ "ipv4NeighborAddress": null,

+ "ipv6ListenRangePrefixes": null,

+ "ipv6NeighborAddress": null,

+ "peerAsn": 65510

+ },

+ "bgpDisabledOnResources": null,

+ "connectedIPv4Subnets": [

+ {

+ "annotation": null,

+ "prefix": "10.18.34.0/24"

+ }

+ ],

+ "connectedIPv6Subnets": null,

+ "disabledOnResources": null,

+ "exportRoutePolicyId": null,

+ "id": "/subscriptions//xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx7/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain/internalNetworks/example-internalnetwor",

+ "importRoutePolicyId": null,

+ "mtu": 1500,

+ "name": "example-internalnetwork",

+ "provisioningState": "Accepted",

+ "resourceGroup": "ResourceGroupName",

+ "staticRouteConfiguration": {

+ "bfdConfiguration": null,

+ "ipv4Routes": [

+ {

+ "nextHop": [

+ "10.20.0.1"

+ ],

+ "prefix": "10.23.0.0/19"

+ },

+ {

+ "nextHop": [

+ "10.20.0.1"

+ ],

+ "prefix": "10.24.0.0/19"

+ }

+ ],

+ "ipv6Routes": null

+ },

+ "systemData": {

+ "createdAt": "2023-XX-XXT13:46:26.394343+00:00",

+ "createdBy": "email@example.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XXT13:46:26.394343+00:00",

+ "lastModifiedBy": "email@example.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/l3isolationdomains/internalnetworks",

+ "vlanId": 2028

+}

+az nf internalnetwork create

+--resource-group "ResourceGroupName"

+--l3-isolation-domain-name "example-l3domain"

+--resource-name "example-internalipv6network"

+--location "eastus"

+--vlan-id 1090

+--connected-ipv6-subnets '[{"prefix":"10:101:1::0/64", "gateway":"10:101:1::1"}]'

+--mtu 1500 --bgp-configuration '{"defaultRouteOriginate":true,"peerASN": 65020,"ipv6NeighborAddress":[{"address": "10:101:1::11"}]}'

+{

+ "administrativeState": "Enabled",

+ "annotation": null,

+ "bfdDisabledOnResources": null,

+ "bfdForStaticRoutesDisabledOnResources": null,

+ "bgpConfiguration": {

+ "allowAs": 2,

+ "allowAsOverride": "Enable",

+ "annotation": null,

+ "bfdConfiguration": null,

+ "defaultRouteOriginate": "True",

+ "fabricAsn": 65046,

+ "ipv4ListenRangePrefixes": null,

+ "ipv4NeighborAddress": null,

+ "ipv6ListenRangePrefixes": null,

+ "ipv6NeighborAddress": [

+ {

+ "address": "10:101:1::11",

+ "operationalState": "Disabled"

+ }

+ ],

+ "peerAsn": 65020

+ },

+ "bgpDisabledOnResources": null,

+ "connectedIPv4Subnets": null,

+ "connectedIPv6Subnets": [

+ {

+ "annotation": null,

+ "prefix": "10:101:1::0/64"

+ }

+ ],

+ "disabledOnResources": null,

+ "exportRoutePolicyId": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/l3domain2/internalNetworks/internalipv6network",

+ "importRoutePolicyId": null,

+ "mtu": 1500,

+ "name": "internalipv6network",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "ResourceGroupName",

+ "staticRouteConfiguration": null,

+ "systemData": {

+ "createdAt": "2023-XX-XXT10:34:33.933814+00:00",

+ "createdBy": "email@example.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XXT10:34:33.933814+00:00",

+ "lastModifiedBy": "email@example.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/l3isolationdomains/internalnetworks",

+ "vlanId": 1090

+|Parameter|Description|Example|Required|

+|||||

+|peeringOption |Peering using either optionA or optionb. Possible values OptionA and OptionB |OptionB| True|

+|optionBProperties | OptionB properties configuration. To specify use exportRouteTargets or importRouteTargets|"exportRouteTargets": ["1234:1234"]}}||

+|optionAProperties | Configuration of OptionA properties. Please refer to OptionA example in section below |||

+|external|This is an optional Parameter to input MPLS Option 10 (B) connectivity to external networks via PE devices. Using this Option, a user can Input Import and Export Route Targets as shown in the example| ||

+**Note:** For Option A You need to create an external network before you enable the L3 isolation Domain. An external is dependent on Internal network, so an external can't be enabled without an internal network. The vlan-id value should be between 501 and 4095.

+## External Network Creation using Option B

+az nf externalnetwork create

+--resource-group "ResourceGroupName"

+--l3domain "examplel3domain"

+--resource-name "examplel3-externalnetwork"

+--location "eastus"

+--peering-option "OptionB" --option-b-properties '{"importRouteTargets": ["65541:2001"], "exportRouteTargets": ["65531:2001"]}'

+ "administrativeState": "Enabled",

+ "exportRoutePolicyId": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxxX/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/examplel3isolationdomain/externalNetworks/example-externalnetwork",

+ "importRoutePolicyId": null,

+ "name": "examplel3-externalnetwork",

+ "networkToNetworkInterconnectId": null,

+ "optionAProperties": null,

+ "optionBProperties": {

+ "exportRouteTargets": [

+ "65531:2001"

+ ],

+ "importRouteTargets": [

+ "65541:2001"

+ ]

+ },

+ "peeringOption": "OptionB",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2023-XX-XXT15:45:31.938216+00:00",

+ "lastModifiedAt": "2023-XX-XXT15:45:31.938216+00:00",

+ "type": "microsoft.managednetworkfabric/l3isolationdomains/externalnetworks"

+}

+```

+## External Network creation with Option A

+```azurecli

+az nf externalnetwork create

+--resource-group "ResourceGroupName"

+--l3domain "example-l3domain"

+--resource-name "example-externalipv4network"

+--location "eastus" --peering-option "OptionA"

+--option-a-properties '{"peerASN": 65026,"vlanId": 2423, "mtu": 1500, "primaryIpv4Prefix": "10.18.0.148/30", "secondaryIpv4Prefix": "10.18.0.152/30"}'

+```

+Expected Output

+```json

+{

+ "administrativeState": "Enabled",

+ "annotation": null,

+ "disabledOnResources": null,

+ "exportRoutePolicyId": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxxX/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/examplel3isolationdomain/externalNetworks/example-externalnetwork",

+ "importRoutePolicyId": null,

+ "name": "example-externalipv4network",

+ "networkToNetworkInterconnectId": null,

+ "optionAProperties": {

+ "bfdConfiguration": null,

+ "fabricAsn": 65026,

+ "mtu": 1500,

+ "peerAsn": 65026,

+ "primaryIpv4Prefix": "10.18.0.148/30",

+ "primaryIpv6Prefix": null,

+ "secondaryIpv4Prefix": "10.18.0.152/30",

+ "secondaryIpv6Prefix": null,

+ "vlanId": 2423

+ },

+ "optionBProperties": null,

+ "peeringOption": "OptionA",

+ "provisioningState": "Accepted",

+ "resourceGroup": "ResourceGroupName",

+ "systemData": {

+ "createdAt": "2023-XX-XXT07:23:54.396679+00:00",

+ "createdBy": "email@address.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XX1T07:23:54.396679+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/l3isolationdomains/externalnetworks"

+az nf externalnetwork create

+--resource-group "ResourceGroupName"

+--l3-isolation-domain-name "example-l3domain"

+--resource-name "example-externalipv6network"

+--location "eastus"

+--vlan-id 506

+--primary-ipv6-prefix "10:101:2::0/127"

+ "fabricAsn": 65026,

+ "id": "/subscriptions//xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain/externalNetworks/example-externalipv6network",

+ "resourceGroup": "ResourceGroupName",

+ "createdAt": "2022-XX-XXT07:52:26.366069+00:00",

+ "lastModifiedAt": "2022-XX-XXT07:52:26.366069+00:00",

+ "vlanId": 506

+az nf l3domain update-admin-state --resource-group "ResourceGroupName" --resource-name "example-l3domain" --state Enable/Disable

+ "administrativeState": "Enabled",

+ "annotation": null,

+ "description": null,

+ "disabledOnResources": null,

+ "external": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedNetworkFabric/l3IsolationDomains/example-l3domain",

+ "internal": null,

+ "location": "eastus",

+ "name": "example-l3domain",

+ "networkFabricId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/NFresourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "optionBDisabledOnResources": null,

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroupName",

+ "systemData": {

+ "createdAt": "2022-XX-XXT06:23:43.372461+00:00",

+ "createdBy": "email@address.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2022-XX-XXT06:25:53.240975+00:00",

+ "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "lastModifiedByType": "Application"

+ },

+ "tags": null,

+ "type": "microsoft.managednetworkfabric/l3isolationdomains"

+ }

+ az nf l3domain delete --resource-group "ResourceGroupName" --resource-name "example-l3domain"

+## Create L3 Isolation Untrust

+az nf l3domain create --resource-group "ResourceGroupName" --resource-name "l3untrust" --location "eastus" --nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName"

+## Create L3 Isolation domain Trust

+az nf l3domain create --resource-group "ResourceGroupName" --resource-name "l3trust" --location "eastus" --nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName"

+## Create L3 Isolation domain Mgmt

+az nf l3domain create --resource-group "ResourceGroupName" --resource-name "l3mgmt" --location "eastus" --nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName"

+## Internal Network Untrust L3 ISD

+az nf internalnetwork create --resource-group "ResourceGroupName" --l3-isolation-domain-name l3untrust --resource-name untrustnetwork --location "eastus" --vlan-id 502 --fabric-asn 65048 --peer-asn 65047--connected-i-pv4-subnets prefix="10.151.3.11/24" --mtu 1500

+## Internal Network Trust ISD

+az nf internalnetwork create --resource-group "ResourceGroupName" --l3-isolation-domain-name l3trust --resource-name trustnetwork --location "eastus" --vlan-id 503 --fabric-asn 65048 --peer-asn 65047--connected-i-pv4-subnets prefix="10.151.1.11/24" --mtu 1500

+## Internal Network Mgmt ISD

+az nf internalnetwork create --resource-group "ResourceGroupName" --l3-isolation-domain-name l3mgmt --resource-name mgmtnetwork --location "eastus" --vlan-id 504 --fabric-asn 65048 --peer-asn 65047--connected-i-pv4-subnets prefix="10.151.2.11/24" --mtu 1500

+## Enable ISD Untrust

+az nf l3domain update-admin-state --resource-group "ResourceGroupName" --resource-name "l3untrust" --state Enable

+## Enable ISD Trust

+az nf l3domain update-admin-state --resource-group "ResourceGroupName" --resource-name "l3trust" --state Enable

+## Enable ISD Mgmt

+az nf l3domain update-admin-state --resource-group "ResourceGroupName" --resource-name "l3mgmt" --state Enable

+#### Below example is used to create any L2 Isolation needed by workload

+## L2 Isolation domain

+az nf l2domain create --resource-group "ResourceGroupName" --resource-name "l2HAnetwork" --location "eastus" --nf-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName" --vlan-id 505 --mtu 1500

+## Enable L2 Isolation Domain

+az nf l2domain update-administrative-state --resource-group "ResourceGroupName" --resource-name "l2HAnetwork" --state Enable

+# Create and Provision a Network Fabric using Azure CLI

+* An Azure account with an active subscription.

+* Install the latest version of the CLI commands (2.0 or later). For information about installing the CLI commands, see [Install Azure CLI](./howto-install-cli-extensions.md)

+* A Network Fabric controller manages multiple Network Fabrics on the same Azure region.

+* Physical Operator-Nexus instance with cabling as per BoM.

+* Express Route connectivity between NFC and Operator-Nexus instances.

+* Terminal server pre-configured with username and password [installed and configured](./howto-platform-prerequisites.md#set-up-terminal-server)

+* PE devices pre-configured with necessary VLANs, Route-Targets and IP addresses.

+* Supported SKUs from NFA Release 1.5 and beyond for Fabric are **M4-A400-A100-C16-aa** and **M8-A400-A100-C16-aa**.

+ * M4-A400-A100-C16-aa - Up to four Compute Racks

+ * M8-A400-A100-C16-aa - Up to eight Compute Racks

+## Steps to Provision a Fabric & Racks

+* Create a Network Fabric by providing racks, server count, SKU & network configuration.

+* Create a Network to Network Interconnect by providing Layer2 & Layer 3 Parameters

+* Update the serial number in the networkDevice resource with the actual serial number on the device.

+* Configure the terminal server with the serial numbers of all the devices.

+* Provision the Network Fabric.

+## Fabric Configuration

+The following table specifies parameters used to create Network Fabric

+| Parameter | Description | Example | Required |

+|--|-||-|

+| resource-group | Name of the resource group | "NFResourceGroup" |True |

+| location | Operator-Nexus Azure region | "eastus" |True |

+| resource-name | Name of the FabricResource | NF-ResourceName |True |

+| nf-sku |Fabric SKU ID is the SKU of the ordered BoM. Two SKUs are supported (**M4-A400-A100-C16-aa** and **M8-A400-A100-C16-aa**). | M4-A400-A100-C16-aa |True | String|

+|nfc-id|Network Fabric Controller ARM resource id|/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName|True |

+|rackcount|Number of compute racks per fabric. Possible values are 2-8|8|True |

+|serverCountPerRack|Number of compute servers per rack. Possible values are 4, 8, 12 or 16|16|True |

+|ipv4Prefix|IPv4 Prefix of the management network. This Prefix should be unique across all Network Fabrics in a Network Fabric Controller. Prefix length should be at least 19 (/20 isn't allowed, /18 and lower are allowed) | 10.246.0.0/19|True |

+|ipv6Prefix|IPv6 Prefix of the management network. This Prefix should be unique across all Network Fabrics in a Network Fabric Controller. | 10:5:0:0::/59|True |

+|**management-network-config**| Details of management network ||True |

+|**infrastructureVpnConfiguration**| Details of management VPN connection between Network Fabric and infrastructure services in Network Fabric Controller||True

+|*optionBProperties*| Details of MPLS option 10B is used for connectivity between Network Fabric and Network Fabric Controller||True

+|importRouteTargets|Values of import route targets to be configured on CEs for exchanging routes between CE & PE via MPLS option 10B|e.g., 65048:10039|True(If OptionB enabled)|

+|exportRouteTargets|Values of export route targets to be configured on CEs for exchanging routes between CE & PE via MPLS option 10B|e.g., 65048:10039|True(If OptionB enabled)|

+|**workloadVpnConfiguration**| Details of workload VPN connection between Network Fabric and workload services in Network Fabric Controller||

+|*optionBProperties*| Details of MPLS option 10B is used for connectivity between Network Fabric and Network Fabric Controller||

+|importRouteTargets|Values of import route targets to be configured on CEs for exchanging routes between CE & PE via MPLS option 10B|e.g., 65048:10050|True(If OptionB enabled)|

+|exportRouteTargets|Values of export route targets to be configured on CEs for exchanging routes between CE & PE via MPLS option 10B|e.g., 65048:10050|True(If OptionB enabled)|

+|**ts-config**| Terminal Server Configuration Details||True

+|primaryIpv4Prefix| The terminal server Net1 interface should be assigned the first usable IP from the prefix and the corresponding interface on PE should be assigned the second usable address|20.0.10.0/30, TS Net1 interface should be assigned 20.0.10.1 and PE interface 20.0.10.2|True|

+|secondaryIpv4Prefix|IPv4 Prefix for connectivity between TS and PE2. The terminal server Net2 interface should be assigned the first usable IP from the prefix and the corresponding interface on PE should be assigned the second usable address|20.0.0.4/30, TS Net2 interface should be assigned 20.0.10.5 and PE interface 20.0.10.6|True|

+|username| Username configured on the terminal server that the services use to configure TS|username|True|

+|password| Password configured on the terminal server that the services use to configure TS|password|True|

+|serialNumber| Serial number of Terminal Server|SN of the Terminal Server||

+## Create a Network Fabric

+Resource group must be created before Network Fabric creation. It's recommended to create a separate resource group for each Network Fabric. Resource group can be created by the following command:

+az nf fabric create \

+--resource-group "NFResourceGroupName"

+--nfc-id "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName"

+--fabric-asn 65048

+--ipv4-prefix 10.2.0.0/19

+--ipv6-prefix fda0:d59c:da02::/59

+--rack-count 4

+--server-count-per-rack 8

+--ts-config '{"primaryIpv4Prefix":"20.0.1.0/30", "secondaryIpv4Prefix":"20.0.0.0/30", "username":"****", "password": "****", "serialNumber":"TerminalServerSerialNumber"}'

+--managed-network-config '{"infrastructureVpnConfiguration":{"peeringOption":"OptionB","optionBProperties":{"importRouteTargets":["65048:10039"],"exportRouteTargets":["65048:10039"]}}, "workloadVpnConfiguration":{"peeringOption": "OptionB", "optionBProperties": {"importRouteTargets": ["65048:10050"], "exportRouteTargets": ["65048:10050"]}}}'

+> [!Note]

+> * if it's a four racks set up then the rack count would be 4

+> * if it's an eight rack set up then the rack count would be 8

+```output

+ "fabricAsn": 65048,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "ipv4Prefix": "10.2.0.0/19",

+ "ipv6Prefix": "fda0:d59c:da02::/59",

+ "65048:10039"

+ "65048:10039"

+ "65048:10050"

+ "65048:10050"

+ "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+ "rackCount": 3,

+ "resourceGroup": "NFResourceGroupName",

+ "serverCountPerRack": 7,

+ "createdAt": "2023-XX-X-6T12:52:11.769525+00:00",

+ "lastModifiedAt": "2023-XX-XX-6T12:52:11.769525+00:00",

+ "primaryIpv4Prefix": "20.0.1.0/30",

+ "secondaryIpv4Prefix": "20.0.0.0/30",

+ "serialNumber": "TerminalServerSerialNumber",

+ "username": "****"

+## show fabric

+az nf fabric show --resourcegroup "NFResourceGroupName" --resource-name "NFName"

+```output

+{

+ "annotation": null,

+ "fabricAsn": 65048,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "ipv4Prefix": "10.2.0.0/19",

+ "ipv6Prefix": "fda0:d59c:da02::/59",

+ "65048:10039"

+ "65048:10039"

+ "65048:10050"

+ "65048:10050"

+ "name": "nffab1031623",

+ "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroupName/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+ "rackCount": 3,

+ "racks": [

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-aggrack",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack1",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack2"

+ ],

+ "serverCountPerRack": 7,

+ "createdAt": "2023-XX-XXT12:52:11.769525+00:00",

+ "lastModifiedAt": "2023-XX-XXT12:53:02.504974+00:00",

+ "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "lastModifiedByType": "Application"

+ "primaryIpv4Prefix": "20.0.1.0/30",

+ "secondaryIpv4Prefix": "20.0.0.0/30",

+ "serialNumber": "TerminalServerSerialNumber",

+ "username": "****"

+```

+## List or Get Network Fabric

+```azurecli

+az nf fabric list --resource-group "NFResourceGroup"

+```

+Expected output:

+```output

+{

+ "annotation": null,

+ "fabricAsn": 65048,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "ipv4Prefix": "10.2.0.0/19",

+ "ipv6Prefix": "fda0:d59c:da02::/59",

+ "l2IsolationDomains": [Null],

+ "l3IsolationDomains": [Null],

+ "location": "eastus",

+ "managementNetworkConfiguration": {

+ "infrastructureVpnConfiguration": {

+ "administrativeState": "Enabled",

+ "networkToNetworkInterconnectId": null,

+ "optionAProperties": null,

+ "optionBProperties": {

+ "exportRouteTargets": [

+ "65048:10039"

+ ],

+ "importRouteTargets": [

+ "65048:10039"

+ ]

+ },

+ "peeringOption": "OptionB"

+ },

+ "workloadVpnConfiguration": {

+ "administrativeState": "Enabled",

+ "networkToNetworkInterconnectId": null,

+ "optionAProperties": null,

+ "optionBProperties": {

+ "exportRouteTargets": [

+ "65048:10050"

+ ],

+ "importRouteTargets": [

+ "65048:10050"

+ ]

+ },

+ "peeringOption": "OptionB"

+ }

+ },

+ "name": "NFName",

+ "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+ "networkFabricSku": "NFSKU",

+ "operationalState": "Provisioned",

+ "provisioningState": "Succeeded",

+ "rackCount": 3,

+ "racks": [

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-aggrack",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack1",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack2"

+ ],

+ "resourceGroup": "NFResourceGroup",

+ "routerId": null,

+ "serverCountPerRack": 7,

+ "systemData": {

+ "createdAt": "2023-XX-XXT12:52:11.769525+00:00",

+ "createdBy": "email@address.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XXT02:05:44.043591+00:00",

+ "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "lastModifiedByType": "Application"

+ },

+ "tags": null,

+ "terminalServerConfiguration": {

+ "networkDeviceId": null,

+ "password": null,

+ "primaryIpv4Prefix": "20.0.1.0/30",

+ "primaryIpv6Prefix": null,

+ "secondaryIpv4Prefix": "20.0.0.0/30",

+ "secondaryIpv6Prefix": null,

+ "serialNumber": "TerminalServerSerialNumber",

+ "username": "****"

+ },

+ "type": "microsoft.managednetworkfabric/networkfabrics"

+ }

+## NNI Configuration

+The following table specifies parameters used to create Network to Network Interconnect

+| Parameter | Description | Example | Required |

+|--|-||-|

+|isMangementType| Configuration to make NNI to be used for management of Fabric. Default value is true. Possible values are True/False |True|True

+|useOptionB| Configuration to enable optionB. Possible values are True/False |True|True

+||

+|*layer2Configuration*| Layer 2 configuration ||

+||

+|portCount| Number of ports that are part of the port-channel. Maximum value is based on Fabric SKU|2||

+|mtu| Maximum transmission unit between CE and PE. |1500||

+||

+|*layer3Configuration*| Layer 3 configuration between CEs and PEs||True

+||

+|primaryIpv4Prefix|IPv4 Prefix for connectivity between CE1 and PE1. CE1 port-channel interface is assigned the first usable IP from the prefix and the corresponding interface on PE1 should be assigned the second usable address|10.246.0.124/31, CE1 port-channel interface is assigned 10.246.0.125 and PE1 port-channel interface should be assigned 10.246.0.126||String

+|secondaryIpv4Prefix|IPv4 Prefix for connectivity between CE2 and PE2. CE2 port-channel interface is assigned the first usable IP from the prefix and the corresponding interface on PE2 should be assigned the second usable address|10.246.0.128/31, CE2 port-channel interface should be assigned 10.246.0.129 and PE2 port-channel interface 10.246.0.130||String

+|primaryIpv6Prefix|IPv6 Prefix for connectivity between CE1 and PE1. CE1 port-channel interface is assigned the first usable IP from the prefix and the corresponding interface on PE1 should be assigned the second usable address|3FFE:FFFF:0:CD30::a1 is assigned to CE1 and 3FFE:FFFF:0:CD30::a2 is assigned to PE1. Default value is 3FFE:FFFF:0:CD30::a0/126||String

+|secondaryIpv6Prefix|IPv6 Prefix for connectivity between CE2 and PE2. CE2 port-channel interface is assigned the first usable IP from the prefix and the corresponding interface on PE2 should be assigned the second usable address|3FFE:FFFF:0:CD30::a5 is assigned to CE2 and 3FFE:FFFF:0:CD30::a6 is assigned to PE2. Default value is 3FFE:FFFF:0:CD30::a4/126.||String

+|fabricAsn|ASN number assigned on CE for BGP peering with PE|65048||

+|peerAsn|ASN number assigned on PE for BGP peering with CE. For iBGP between PE/CE, the value should be same as fabricAsn, for eBGP the value should be different from fabricAsn |65048|True|

+|fabricAsn|ASN number assigned on CE for BGP peering with PE|65048||

+|vlan-Id|Vlan for NNI.Range is between 501-4095 |501||

+|importRoutePolicy|Details to import route policy.|||

+|exportRoutePolicy|Details to export route policy.|||

+||||

+## Create a Network to Network Interconnect

+Resource group & Network Fabric must be created before Network to Network Interconnect creation.

+Run the following command to create the Network to Network Interconnect:

+```azurecli

+az nf nni create \

+--resource-group "NFResourceGroup" \

+--resource-name "NFNNIName" \

+--fabric "NFFabric" \

+--layer2-configuration '{"portCount": 3, "mtu": 1500}' \

+--layer3-configuration '{"peerASN": 65048, "vlanId": 501, "primaryIpv4Prefix": "10.2.0.124/30", "secondaryIpv4Prefix": "10.2.0.128/30", "primaryIpv6Prefix": "10:2:0:124::400/127", "secondaryIpv6Prefix": "10:2:0:124::402/127"}'

+```output

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/nffab1031623/networkToNetworkInterconnects/NFNNIName",

+ "portCount": 3

+ "peerAsn": 65048,

+ "primaryIpv4Prefix": "10.2.0.124/30",

+ "primaryIpv6Prefix": "10:2:0:124::400/127",

+ "secondaryIpv4Prefix": "10.2.0.128/30",

+ "secondaryIpv6Prefix": "10:2:0:124::402/127",

+ "vlanId": 501

+ "name": "NFNNIName",

+ "createdAt": "2023-XX-XXT13:13:22.514644+00:00",

+ "lastModifiedAt": "2023-XX-XXT13:13:22.514644+00:00",

+## Show Network Fabric NNI (Network to Network Interface)

+```azurecli

+az nf nni show -g "NFResourceGroup" --resource-name "NFNNIName" --fabric "NFFabric"

+```

+Expected output:

+```output

+{

+ "administrativeState": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFFabric/networkToNetworkInterconnects/NFNNIName",

+ "isManagementType": "True",

+ "layer2Configuration": {

+ "interfaces": null,

+ "mtu": 1500,

+ "portCount": 3

+ },

+ "layer3Configuration": {

+ "exportRoutePolicyId": null,

+ "fabricAsn": null,

+ "importRoutePolicyId": null,

+ "peerAsn": 65048,

+ "primaryIpv4Prefix": "10.2.0.124/30",

+ "primaryIpv6Prefix": "10:2:0:124::400/127",

+ "secondaryIpv4Prefix": "10.2.0.128/30",

+ "secondaryIpv6Prefix": "10:2:0:124::402/127",

+ "vlanId": 501

+ },

+ "name": "NFNNIName",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "systemData": {

+ "createdAt": "2023-XX-XXT13:13:22.514644+00:00",

+ "createdBy": "email@address.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XX13:13:22.514644+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/networkfabrics/networktonetworkinterconnects",

+ "useOptionB": "True"

+```

+## List or Get Network Fabric NNI (Network to Network Interface)

+az nf nni list -g NFResourceGroup --fabric NFFabric

+```output

+{

+ "administrativeState": null,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFFabric/networkToNetworkInterconnects/NFNNIName",

+ "isManagementType": "True",

+ "layer2Configuration": {

+ "interfaces": null,

+ "mtu": 1500,

+ "portCount": 3

+ },

+ "layer3Configuration": {

+ "exportRoutePolicyId": null,

+ "fabricAsn": null,

+ "importRoutePolicyId": null,

+ "peerAsn": 65048,

+ "primaryIpv4Prefix": "10.2.0.124/30",

+ "primaryIpv6Prefix": "10:2:0:124::400/127",

+ "secondaryIpv4Prefix": "10.2.0.128/30",

+ "secondaryIpv6Prefix": "10:2:0:124::402/127",

+ "vlanId": 501

+ },

+ "name": "NFNNIName",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "systemData": {

+ "createdAt": "2023-XX-XXT13:13:22.514644+00:00",

+ "createdBy": "email@address.com.com",

+ "createdByType": "User",

+ "lastModifiedAt": "2023-XX-XXT13:13:22.514644+00:00",

+ "lastModifiedBy": "email@address.com.com",

+ "lastModifiedByType": "User"

+ },

+ "type": "microsoft.managednetworkfabric/networkfabrics/networktonetworkinterconnects",

+ "useOptionB": "True"

+ }

+```

+## Next Steps

+* Update the serial number in the networkDevice resource with the actual serial number on the device. The device sends the serial number as part of DHCP request.

+* Configure the terminal server with the serial numbers of all the devices (which also hosts DHCP server)

+* Provision the network devices via zero-touch provisioning mode, Based on the serial number in the DHCP request, the DHCP server responds with the boot configuration file for the corresponding device

+## Update Network Fabric Devices

+Run the following command to update Network Fabric Devices:

+```azurecli

+az nf device update \

+--resource-group "NFResourceGroup" \

+--resource-name "Network-Device-Name" \

+--location "eastus" \

+--serial-number "xxxx"

+```

+Expected output:

+```output

+ "hostName": "AggrRack-CE01",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/Network-Device-Name",

+ "location": "eastus2euap",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "CE1",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "AXXXX;DCS-XXXXX-24;XX.XX;JXXXXXXX",

+ "createdAt": "2023-XX-XXT12:52:42.270551+00:00",

+ "lastModifiedAt": "2023-XX-XXT13:30:24.098335+00:00",

+> [!Note]

+> The above snapshot only serves as an example. You should update all the devices that are part of both AggRack and computeRacks.

+For example, AggRack consists of

+* CE01

+* CE02

+* TOR17

+* TOR18

+* Mgmnt Switch01

+* Mgmnt Switch02 and etc.

+## List or Get Network Fabric Devices

+Run the following command to List Network Fabric Devices:

+```output

+ "hostName": "AggrRack-CE01",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-CE1",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "CE1",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "createdAt": "2023-XX-XXT12:52:42.270551+00:00",

+ "lastModifiedAt": "2023-XX-XXT13:30:24.098335+00:00",

+ "hostName": "AggrRack-CE02",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-CE2",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "CE2",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "createdAt": "2023-XX-XXT12:52:43.489256+00:00",

+ "lastModifiedAt": "2023-XX-XXT13:30:40.923567+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "tags": null,

+ "type": "microsoft.managednetworkfabric/networkdevices",

+ "version": null

+ },

+ {

+ "annotation": null,

+ "hostName": "AggRack-TOR17",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-TOR17",

+ "location": "eastus2euap",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "TOR17",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "systemData": {

+ "createdAt": "2023-XX-XXT12:52:44.676759+00:00",

+ "createdBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "createdByType": "Application",

+ "lastModifiedAt": "2023-XX-XXT13:31:59.650758+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "tags": null,

+ "type": "microsoft.managednetworkfabric/networkdevices",

+ "version": null

+ },

+ {

+ "annotation": null,

+ "hostName": "AggRack-TOR18",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-TOR18",

+ "location": "eastus",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "TOR18",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "systemData": {

+ "createdAt": "2023-03-16T12:52:45.801778+00:00",

+ "createdBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "createdByType": "Application",

+ "lastModifiedAt": "2023-XX-XXT13:32:13.369591+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "tags": null,

+ "type": "microsoft.managednetworkfabric/networkdevices",

+ "version": null

+ },

+ {

+ "annotation": null,

+ "hostName": "AggRack-MGMT1",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-MgmtSwitch1",

+ "location": "eastus",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "MgmtSwitch1",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "systemData": {

+ "createdAt": "2023-XX-XXT12:52:46.911202+00:00",

+ "createdBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "createdByType": "Application",

+ "lastModifiedAt": "2023-XX-XXT13:31:00.836730+00:00",

+ "lastModifiedBy": "email@address.com",

+ "lastModifiedByType": "User"

+ },

+ "tags": null,

+ "type": "microsoft.managednetworkfabric/networkdevices",

+ "version": null

+ },

+ {

+ "annotation": null,

+ "hostName": "AggRack-MGMT2",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-MgmtSwitch2",

+ "location": "eastus",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "MgmtSwitch2",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "provisioningState": "Succeeded",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "ArXXX;DCS-7XXXXXX-24;12.05;JPXXXXXXXX",

+ "systemData": {

+ "createdAt": "2023-XX-XXT12:52:48.020528+00:00",

+ "createdBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+ "createdByType": "Application",

+ "lastModifiedAt": "2023-XX-XXT13:31:42.173645+00:00",

+Run the following command to Get or Show details of a Network Fabric Device:

+az nf device show --resource-group "NFResourceGroup" --resource-name "Network-Device-Name"

+```output

+ "hostName": "AggrRack-CE01",

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkDevices/NFName-AggrRack-CE1",

+ "name": "Network-Device-Name",

+ "networkDeviceRole": "CE1",

+ "networkDeviceSku": "DefaultSku",

+ "networkRackId": "/subscriptions/61065ccc-9543-4b91-b2d1-0ce42a914507/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkRacks/Network-Device-Name",

+ "resourceGroup": "NFResourceGroup",

+ "serialNumber": "AXXXX;DCS-XXXXX-24;XX.XX;JXXXXXXX",

+ "createdAt": "2023-XX-XXT12:52:42.270551+00:00",

+ "lastModifiedAt": "2023-XX-XXT13:30:24.098335+00:00",

+## Provision fabric

+After updating the device serial number, the fabric needs to be provisioned by executing the following command

+az nf fabric provision --resource-group "NFResourceGroup" --resource-name "NFName"

+az nf fabric show --resource-group "NFResourceGroup" --resource-name "NFName"

+```output

+ "fabricAsn": 65048,

+ "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+ "ipv4Prefix": "10.2.0.0/19",

+ "ipv6Prefix": "fda0:d59c:da02::/59",

+ "infrastructureVpnConfiguration": {

+ "administrativeState": "Enabled",

+ "networkToNetworkInterconnectId": null,

+ "peeringOption": "OptionB"

+ "administrativeState": "Enabled",

+ "networkToNetworkInterconnectId": null,

+ "peeringOption": "OptionB"

+ "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+ "networkFabricSku": "NFSKU",

+ "operationalState": "Provisioning",

+ "rackCount": 3,

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-aggrack",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack1",

+ "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack2"

+ "routerId": null,

+ "serverCountPerRack": 7,

+ "createdAt": "2023-XX-XXT12:52:11.769525+00:00",

+ "createdBy": "email@address.com",

+ "lastModifiedAt": "2023-XX-XXT14:47:59.424826+00:00",

+ "primaryIpv4Prefix": "20.0.1.0/30",

+ "secondaryIpv4Prefix": "20.0.0.0/30",

+ "serialNumber": "XXXXXXXXXXXX",

+ "username": "XXXX"

+## Deprovision a Fabric

+To deprovision a fabric ensure Fabric operational state should be in provisioned state

+az nf fabric deprovision --resource-group "NFResourceGroup" --resource-name "NFName"

+```output

+  "annotation": null,

+  "fabricAsn": 65046,

+  "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+  "ipv4Prefix": "10.18.0.0/19",

+  "ipv6Prefix": null,

+  "l2IsolationDomains": [],

+  "l3IsolationDomains": null,

+  "location": "eastus",

+  "managementNetworkConfiguration": {

+    "infrastructureVpnConfiguration": {

+      "administrativeState": "Enabled",

+      "networkToNetworkInterconnectId": null,

+      "optionAProperties": null,

+      "optionBProperties": {

+        "exportRouteTargets": [

+          "65048:10039"

+        ],

+        "importRouteTargets": [

+          "65048:10039"

+        ]

+      },

+      "peeringOption": "OptionB"

+    },

+    "workloadVpnConfiguration": {

+      "administrativeState": "Enabled",

+      "networkToNetworkInterconnectId": null,

+      "optionAProperties": null,

+      "optionBProperties": {

+        "exportRouteTargets": [

+          "65048:10050"

+        ],

+        "importRouteTargets": [

+          "65048:10050"

+        ]

+      },

+      "peeringOption": "OptionB"

+    }

+  },

+  "name": "NFName",

+  "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+  "networkFabricSku": "M4-A400-A100-C16-aa",

+  "operationalState": "Deprovisioned",

+  "provisioningState": "Succeeded",

+  "rackCount": 3,

+  "racks": [

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-aggrack",

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack1",

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack2"

+  ],

+  "resourceGroup": "NFResourceGroup",

+  "routerId": null,

+  "serverCountPerRack": 8,

+  "systemData": {

+    "createdAt": "2023-XX-XXT19:30:23.319643+00:00",

+    "createdBy": "email@address.com",

+    "createdByType": "User",

+    "lastModifiedAt": "2023-XX-XXT06:47:36.130713+00:00",

+    "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+    "lastModifiedByType": "Application"

+  },

+  "tags": null,

+  "terminalServerConfiguration": {

+    "networkDeviceId": null,

+    "password": null,

+    "primaryIpv4Prefix": "20.0.1.12/30",

+    "primaryIpv6Prefix": null,

+    "secondaryIpv4Prefix": "20.0.0.12/30",

+    "secondaryIpv6Prefix": null,

+    "serialNumber": "XXXXXXXXXXXXX",

+    "username": "XXXX"

+  },

+  "type": "microsoft.managednetworkfabric/networkfabrics"

+## Deleting Fabric

+To delete the fabric the operational state of Fabric shouldn't be "Provisioned". To change the operational state from Provisioned to Deprovision, run the deprovision command. Ensure there are no racks associated before deleting fabric.

+az nf fabric delete --resource-group "NFResourceGroup" --resource-name "NFName"

+```

+Expected output:

+```output

+{

+  "annotation": null,

+  "fabricAsn": 65044,

+  "id": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabrics/NFName",

+  "ipv4Prefix": "10.21.0.0/16",

+  "ipv6Prefix": "10:15:0:0::/59",

+  "l2IsolationDomains": null,

+  "l3IsolationDomains": null,

+  "location": "eastus",

+  "managementNetworkConfiguration": {

+    "infrastructureVpnConfiguration": {

+      "administrativeState": "Enabled",

+      "networkToNetworkInterconnectId": null,

+      "optionAProperties": null,

+      "optionBProperties": {

+        "exportRouteTargets": [

+          "65044:10039"

+        ],

+        "importRouteTargets": [

+          "65044:10039"

+        ]

+      },

+      "peeringOption": "OptionB"

+    },

+    "workloadVpnConfiguration": {

+      "administrativeState": "Enabled",

+      "networkToNetworkInterconnectId": null,

+      "optionAProperties": null,

+      "optionBProperties": {

+        "exportRouteTargets": [

+          "65044:10050"

+        ],

+        "importRouteTargets": [

+          "65044:10050"

+        ]

+      },

+      "peeringOption": "OptionB"

+    }

+  },

+  "name": "nffab2030823",

+  "networkFabricControllerId": "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourceGroups/NFCResourceGroup/providers/Microsoft.ManagedNetworkFabric/networkFabricControllers/NFCName",

+  "networkFabricSku": "SKU-Name",

+  "operationalState": "Deprovisioned",

+  "provisioningState": "Deleting",

+  "rackCount": 3,

+  "racks": [

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-aggrack",

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack1",

+    "/subscriptions/xxxxxx-xxxxxx-xxxx-xxxx-xxxxxx/resourcegroups/NFResourceGroup/providers/microsoft.managednetworkfabric/networkracks/NFName-comprack2"

+  ],

+  "resourceGroup": "NFResourceGroup",

+  "routerId": null,

+  "serverCountPerRack": 7,

+  "systemData": {

+    "createdAt": "2023-XX-XXT10:31:22.423399+00:00",

+    "createdBy": "email@address.com",

+    "createdByType": "User",

+    "lastModifiedAt": "2023-XX-XXT06:31:41.675991+00:00",

+    "lastModifiedBy": "d1bd24c7-b27f-477e-86dd-939e107873d7",

+    "lastModifiedByType": "Application"

+  },

+  "tags": null,

+  "terminalServerConfiguration": {

+    "networkDeviceId": null,

+    "password": null,

+    "primaryIpv4Prefix": "20.0.1.68/30",

+    "primaryIpv6Prefix": null,

+    "secondaryIpv4Prefix": "20.0.0.68/30",

+    "secondaryIpv6Prefix": null,

+    "serialNumber": "XXXXXXXXXXXXX",

+    "username": "XXXX"

+  },

+  "type": "microsoft.managednetworkfabric/networkfabrics"

+}

+```

+After successfully deleting the Network Fabric, when you run a show of the same fabric, you won't find any resources available.

+```azurecli

+az nf fabric show --resource-group "NFResourceGroup" --resource-name "NFName"

+```

+Expected output:

+```output

+Command group 'nf' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus

+(ResourceNotFound) The Resource 'Microsoft.ManagedNetworkFabric/NetworkFabrics/NFName' under resource group 'NFResourceGroup' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix

+Code: ResourceNotFound

+Before you link the subscription to a Dynatrace environment,[complete the pre-deployment configuration.](dynatrace-how-to-configure-prereqs.md).

+- [Manage the Dynatrace resource](dynatrace-how-to-manage.md)

+> You only need to add a Co-Administrator if the user needs to manage Azure classic deployments by using [Azure Service Management PowerShell Module](/powershell/azure/servicemanagement/install-azure-ps). If the user only uses the Azure portal to manage the classic resources, you wonΓÇÖt need to add the classic administrator for the user.

+- You can use `latest` if you want to use the latest image and not a specific older version. If the *latest* image version is newly released in marketplace and has an unforeseen issue, the deployment may fail. If you are using Portal for deployment, we recommend choosing a different image *sku train* (e.g. 12-SP4 instead of 15-SP3) till the issues are resolved. However, if deploying via API/CLI, you can provide any other *image version* which is available. To view and select the available image versions from a publisher, use below commands

+1. Create an **Ubuntu 20.04** VM in Azure. For more information, see [how to create a Linux VM in the Azure portal](../../virtual-machines/linux/quick-create-portal.md).

+1. Where `orchestration_ansible_user` is the user with **admin** privileges like (e.g. root).

+1. Create a container within the storage account. You can choose any container name, such as `sapbits`.

+1. Create a folder within the container, named `sapfiles`.

+1. Go to the `sapfiles` folder.

+1. Create two subfolders named `archives` and `boms`.

+1. In the `boms` folder, create four subfolders with the following names, depending on the SAP version that you're using:

+ 1. [S41909SPS03_v0011ms.yaml](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/S41909SPS03_v0011ms.yaml)

+ 1. [S42020SPS03_v0003ms.yaml](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/S42020SPS03_v0003ms.yaml)

+ 1. [HANA_2_00_064_v0001ms.yaml](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/archives/HANA_2_00_064_v0001ms/HANA_2_00_064_v0001ms.yaml)

+ 1. [S4HANA_2021_ISS_v0001ms.yaml](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/S4HANA_2021_ISS_v0001ms.yaml)

+ 1. [HANA_2_00_064_v0001ms.yaml](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/archives/HANA_2_00_064_v0001ms/HANA_2_00_064_v0001ms.yaml)

+ 1. [HANA_2_00_055_v1_install.rsp.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/HANA_2_00_055_v1_install.rsp.j2)

+ 1. [S41909SPS03_v0011ms-app-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-app-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-dbload-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-dbload-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-ers-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-ers-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-generic-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-generic-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-pas-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-pas-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-scs-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-scs-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-scsha-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-scsha-inifile-param.j2)

+ 1. [S41909SPS03_v0011ms-web-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S41909SPS03_v0011ms/templates/S41909SPS03_v0011ms-web-inifile-param.j2)

+ 1. [HANA_2_00_055_v1_install.rsp.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/HANA_2_00_055_v1_install.rsp.j2)

+ 1. [HANA_2_00_install.rsp.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/HANA_2_00_install.rsp.j2)

+ 1. [S42020SPS03_v0003ms-app-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-app-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-dbload-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-dbload-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-ers-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-ers-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-generic-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-generic-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-pas-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-pas-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-scs-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-scs-inifile-param.j2)

+ 1. [S42020SPS03_v0003ms-scsha-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S42020SPS03_v0003ms/templates/S42020SPS03_v0003ms-scsha-inifile-param.j2)

+ 1. [HANA_2_00_055_v1_install.rsp.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/HANA_2_00_055_v1_install.rsp.j2)

+ 1. [HANA_2_00_install.rsp.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/HANA_2_00_install.rsp.j2)

+ 1. [NW_ABAP_ASCS_S4HANA2021.CORE.HDB.AB](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/NW_ABAP_ASCS_S4HANA2021.CORE.HDB.ABAP_Distributed.params)

+ 1. [NW_ABAP_CI-S4HANA2021.CORE.HDB.ABAP_Distributed.params](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/NW_ABAP_CI-S4HANA2021.CORE.HDB.ABAP_Distributed.params)

+ 1. [NW_ABAP_DB-S4HANA2021.CORE.HDB.ABAP_Distributed.params](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/NW_ABAP_DB-S4HANA2021.CORE.HDB.ABAP_Distributed.params)

+ 1. [NW_DI-S4HANA2021.CORE.HDB.PD_Distributed.params](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/NW_DI-S4HANA2021.CORE.HDB.PD_Distributed.params)

+ 1. [NW_Users_Create-GENERIC.HDB.PD_Distributed.params](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/NW_Users_Create-GENERIC.HDB.PD_Distributed.params)

+ 1. [S4HANA_2021_ISS_v0001ms-app-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-app-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-dbload-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-dbload-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-ers-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-ers-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-generic-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-generic-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-pas-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-pas-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-scs-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-scs-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-scsha-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-scsha-inifile-param.j2)

+ 1. [S4HANA_2021_ISS_v0001ms-web-inifile-param.j2](https://raw.githubusercontent.com/Azure/SAP-automation-samples/main/SAP/S4HANA_2021_ISS_v0001ms/templates/S4HANA_2021_ISS_v0001ms-web-inifile-param.j2)

+1. Upload all the files that you downloaded to the `templates` folder.

+1. Go back to the `sapfiles` folder, then go to the `archives` subfolder.

+ 1. [S41909SPS03_v0011ms.yaml](https://github.com/Azure/SAP-automation-samples/blob/main/SAP/S41909SPS03_v0011ms/S41909SPS03_v0011ms.yaml)

+ 1. [S42020SPS03_v0003ms.yaml](https://github.com/Azure/SAP-automation-samples/blob/main/SAP/S42020SPS03_v0003ms/S42020SPS03_v0003ms.yaml)

+ 1. [HANA_2_00_064_v0001ms.yaml](https://github.com/Azure/SAP-automation-samples/blob/main/SAP/archives/HANA_2_00_064_v0001ms/HANA_2_00_064_v0001ms.yaml)

+ 1. [S4HANA_2021_ISS_v0001ms.yaml](https://github.com/Azure/SAP-automation-samples/blob/main/SAP/S4HANA_2021_ISS_v0001ms/S4HANA_2021_ISS_v0001ms.yaml)

+ 1. [HANA_2_00_064_v0001ms.yaml](https://github.com/Azure/SAP-automation-samples/blob/main/SAP/archives/HANA_2_00_064_v0001ms/HANA_2_00_064_v0001ms.yaml)

+1. Upload all the packages that you downloaded to the `archives` folder. Don't rename the files.

+ 1. Upload the files to the `archives` folder.

+ 1. Save and reupload the YAML file. Make sure you only have one YAML file in the subfolder (`S41909SPS03_v0011ms` or `S42020SPS03_v0003ms` or `S4HANA_2021_ISS_v0001ms`) of the `boms` folder.

+ > This connector has been built on http trigger based Azure Function. And it provides an endpoint to which GitHub will be connected through it's webhook capability and posts the subscribed events into Microsoft Sentinel. This might result in additional data ingestion costs. Check the [Azure Functions pricing page](https://azure.microsoft.com/pricing/details/functions/) for details.

+*Now we are done with the GitHub Webhook configuration. Once the GitHub events triggered and after the delay of 20 to 30 mins (As there will be a dealy for LogAnalytics to spin up the resources for the first time), you should be able to see all the transactional events from the GitHub into LogAnalytics workspace table called "githubscanaudit_CL".*

+* Deployment: filter by deployment name

+> [!NOTE]

+> For spring boot applications, you need to [add spring-boot-starter-actuator dependency](concept-manage-monitor-app-spring-boot-actuator.md#add-actuator-dependency) to see metrics from spring boot actuator.

+ > [!NOTE]

+ > These metrics are available only for spring-boot applications, and you need to [add spring-boot-starter-actuator dependency](concept-manage-monitor-app-spring-boot-actuator.md#add-actuator-dependency) to enable these metrics.

+The unsupported client list above isn't exhaustive and may change over time.

+| ACLs | <li>`chgrp` - change group<li>`chmod` - change permissions/mode<li>`chown` - change owner<li>`put/get -p` - preserving properties such as permissions and timestamps |

+- Static IP addresses aren't supported for storage accounts. This isn't an SFTP specific limitation.

+- Internet routing isn't supported. Use Microsoft network routing.

+- There's a 2-minute time out for idle or inactive connections. OpenSSH will appear to stop responding and then disconnect. Some clients reconnect automatically.

+- To resolve the `Failed to update SFTP settings for account 'accountname'. Error: The value 'True' isn't allowed for property isSftpEnabled.` error, ensure that the following prerequisites are met at the storage account level:

+BlobServiceClient blobServiceClient = new BlobServiceClientBuilder()

+- For tutorials, samples, quickstarts, and other documentation, visit [Azure for Java developers](/azure/developer/java/sdk/overview).

+ Title: Azure Resource Graph sample queries

+- Bug fix for the PowerShell script FileSyncErrorsReport.ps1

+Premium SSD v2 support a 4k physical sector size by default, but can be configured to use a 512E sector size as well. While most applications are compatible with 4k sector sizes, some require 512 byte sector sizes. Oracle Database, for example, requires release 12.2 or later in order to support 4k native disks. For older versions of Oracle DB, 512 byte sector size is required.

+Create a Premium SSD v2 disk in an availability zone. Then create a VM in the same region and availability zone that supports Premium Storage and attach the disk to it. The following script creates a Premium SSD v2 with a 4k sector size, to deploy one with a 512 sector size, update the `$logicalSectorSize` parameter. Replace the values of all the variables with your own, then run the following script:

+##Replace 4096 with 512 to deploy a disk with 512 sector size

+Create a Premium SSD v2 disk in an availability zone. Then create a VM in the same region and availability zone that supports Premium Storage and attach the disk to it. The following script creates a Premium SSD v2 with a 4k sector size, to deploy one with a 512 sector size, update the `$logicalSectorSize` parameter. Replace the values of all the variables with your own, then run the following script:

+#To use a 512 sector size, replace 4096 with 512

+1. Select whether you'd like to deploy a 4k or 512 logical sector size.

+ :::image type="content" source="media/disks-deploy-premium-v2/premv2-sector-size.png" alt-text="Screenshot of deployment logical sector size deployment options." lightbox="media/disks-deploy-premium-v2/premv2-sector-size.png":::

+Premium SSD v2 support a 4k physical sector size by default, but can be configured to use a 512E sector size as well. While most applications are compatible with 4k sector sizes, some require 512 byte sector sizes. Oracle Database, for example, requires release 12.2 or later in order to support 4k native disks. For older versions of Oracle DB, 512 byte sector size is required.

+description: Learn how to configure the Azure Linux diagnostic extension (LAD) 4.0 to collect metrics and log events from Linux VMs in Azure.

+This article describes the latest versions of the Linux diagnostic extension (LAD).

+> For information about version 3.x, see [Use the Linux diagnostic extension 3.0 to monitor metrics and logs](./diagnostics-linux-v3.md).

+The Linux diagnostic extension helps you monitor the health of a Linux VM on Microsoft Azure. It has the following capabilities:

+| -- | | | |

+| Metrics | [Counter, Aggregation, Sample Rate, Specifiers](#performancecounters) | Azure Table Storage | EventHub, Azure Blob Storage (JSON format), Azure Monitor (new in LAD 4.0) |

+| Syslog | [Facility, Severity Level](#syslogevents) | Azure Table Storage | EventHub, Azure Blob Storage (JSON Format) |

+| Files | [Log Path, Destination Table](#filelogs) | Azure Table Storage | EventHub, Azure Blob Storage (JSON Format) |

+This extension works with both Azure deployment models: Azure Resource Manager and classic.

+## Prerequisites

+- **Azure Linux agent version 2.2.0 or later**. Most Azure VM Linux gallery images include version 2.2.7 or later. Run `/usr/sbin/waagent -version` to confirm the version installed on the VM. If the VM runs an older version of the guest agent, [update the Linux agent](./update-linux-agent.md).

+- **Azure CLI**. [Set up the Azure CLI](/cli/azure/install-azure-cli) environment on your machine.

+- **The `wget` command**. If you don't already have it, install it using the corresponding package manager.

+- **An Azure subscription and general purpose storage account** to store the data. General purpose storage accounts support table storage, which is required. A blob storage account doesn't work.

+- **Python 2**.

+The following distributions and versions include only Azure-endorsed Linux vendor images. The extension generally doesn't support third-party BYOL and BYOS images, like appliances.

+- SUSE Linux Enterprise Server 12 SP5

+- Red Hat Enterprise Linux (RHEL) 7.9

+A distribution that lists only major versions, like Debian 7, is also supported for all minor versions. If a specific minor version is specified, only that version is supported. If a plus sign (+) is appended, minor versions equal to or later than the specified version are supported.

+The Linux diagnostic extension requires Python 2. If your virtual machine uses a distribution that doesn't include Python 2, install it.

+> [!NOTE]

+> We are currently planning to converge all versions of the Linux Diagnostic Extensions (LAD) with the new Azure Monitoring Agent, which already supports Python 3. The LAD will be scheduled for deprecation pending announcement and approval.

+>

+To install Python 2, run one of the following sample commands:

+The `python2` executable file must be aliased to `python`.

+ ```bash

+ sudo update-alternatives --remove-all python

+ ```

+1. Run the following command to create the new alias.

+ ```bash

+ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 1

+ ```

+## Install the extension

+You can enable this extension for your VM and Virtual Machine Scale Set by using the Azure PowerShell cmdlets, Azure CLI scripts, Azure Resource Manager templates (ARM templates), or the Azure portal. For more information, see [Virtual machine extensions and features for Linux](features-linux.md).

+> [!NOTE]

+>

+> Some components of the Linux Diagnostic VM extension are also shipped in the [Log Analytics VM extension](./oms-linux.md). Conflicts can arise if both extensions are instantiated in the same ARM template.

+>

+>To avoid install-time conflicts, use the [`dependsOn` directive](../../azure-resource-manager/templates/resource-dependency.md#dependson) to install the extensions sequentially. The extensions can be installed in either order.

+Use the installation instructions and a [downloadable sample configuration](https://raw.githubusercontent.com/Azure/azure-linux-extensions/master/Diagnostic/tests/lad_2_3_compatible_portal_pub_settings.json) to configure LAD 4.0 to:

+- Capture and store the same metrics that LAD versions 2.3 and 3.x provided.

+- Send metrics to the Azure Monitor sink along with the usual sink to Azure Storage. This functionality is new in LAD 4.0.

+- Capture a useful set of file system metrics, as in LAD 3.0.

+- Capture the default syslog collection enabled by LAD 2.3.

+- Enable the Azure portal experience for charting and alerting on VM metrics.

+The downloadable configuration is just an example. Modify it to suit your needs.

+You can install and configure LAD 4.0 in the Azure CLI or in Azure PowerShell.

+az vm extension set --publisher Microsoft.Azure.Diagnostics \

+ --name LinuxDiagnostic --version 4.0 --resource-group <resource_group_name> \

+ --vm-name <vm_name> --protected-settings ProtectedSettings.json \

+ --settings PublicSettings.json

+The command assumes that you're using the Azure Resource Management mode of the Azure CLI. To configure LAD for classic deployment model VMs, switch to Service Management mode (`azure config mode asm`) and omit the resource group name in the command.

+# [Azure PowerShell](#tab/powershell)

+Set-AzVMExtension -ResourceGroupName <resource_group_name> -VMName <vm_name> `

+ -Location <vm_location> -ExtensionType LinuxDiagnostic `

+ -Publisher Microsoft.Azure.Diagnostics -Name LinuxDiagnostic `

+ -SettingString $publicSettings -ProtectedSettingString $protectedSettings `

+ -TypeHandlerVersion 4.0

+To enable automatic update of the agent, we recommend that you enable the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature:

+az vm extension set --publisher Microsoft.Azure.Diagnostics --name LinuxDiagnostic \

+ --version 4.0 --resource-group <resource_group_name> --vm-name <vm_name> \

+ --protected-settings ProtectedSettings.json --settings PublicSettings.json \

+ --enable-auto-upgrade true

+# [Azure PowerShell](#tab/powershell)

+Set-AzVMExtension -ResourceGroupName <resource_group_name> -VMName <vm_name> `

+ -Location <vm_location> -ExtensionType LinuxDiagnostic `

+ -Publisher Microsoft.Azure.Diagnostics -Name LinuxDiagnostic `

+ -SettingString $publicSettings -ProtectedSettingString $protectedSettings `

+ -TypeHandlerVersion 4.0 -EnableAutomaticUpgrade $true

+In these examples, the sample configuration collects a set of standard data and sends it to table storage. The URL for the sample configuration and its contents can change.

+In most cases, you should download a copy of the portal settings JSON file and customize it for your needs. Use templates or your own automation to use a customized version of the configuration file rather than downloading from the URL each time.

+When you enable the new Azure Monitor sink, the VMs need to have system-assigned identity enabled to generate Managed Service Identity (MSI) authentication tokens. You can add these settings during or after VM creation. For instructions for the Azure portal, the Azure CLI, PowerShell, and Azure Resource Manager, see [Configure managed identities](../../active-directory/managed-identities-azure-resources/qs-configure-portal-windows-vm.md).

+#### Installation sample - Azure CLI

+my_subscription_id=<your_azure_subscription_id>

+az account set --subscription $my_subscription_id

+az vm identity assign --resource-group $my_resource_group --name $my_linux_vm

+# Download the sample public settings. You could instead use curl or any web browser.

+my_vm_resource_id=$(az vm show --resource-group $my_resource_group \

+ --name $my_linux_vm --query "id" -o tsv)

+my_diagnostic_storage_account_sastoken=$(az storage account generate-sas \

+ --account-name $my_diagnostic_storage_account --expiry 2037-12-31T23:59:00Z \

+ --permissions wlacu --resource-types co --services bt -o tsv)

+my_lad_protected_settings="{'storageAccountName': '$my_diagnostic_storage_account', \

+ 'storageAccountSasToken': '$my_diagnostic_storage_account_sastoken'}"

+az vm extension set --publisher Microsoft.Azure.Diagnostics --name LinuxDiagnostic \

+ --version 4.0 --resource-group $my_resource_group --vm-name $my_linux_vm \

+ --protected-settings "${my_lad_protected_settings}" --settings portal_public_settings.json

+# [Azure PowerShell](#tab/powershell)

+#### Installation sample - PowerShell

+$sasToken = New-AzStorageAccountSASToken -Service Blob,Table `

+ -ResourceType Service,Container,Object -Permission "racwdlup" `

+ -Context (Get-AzStorageAccount -ResourceGroupName $storageAccountResourceGroup `

+ -AccountName $storageAccountName).Context -ExpiryTime $([System.DateTime]::Now.AddYears(10))

+$protectedSettings="{'storageAccountName': '$storageAccountName', `

+ 'storageAccountSasToken': '$sasToken'}"

+Set-AzVMExtension -ResourceGroupName $VMresourceGroup -VMName $vmName `

+ -Location $vm.Location -ExtensionType LinuxDiagnostic `

+ -Publisher Microsoft.Azure.Diagnostics -Name LinuxDiagnostic `

+ -SettingString $publicSettings -ProtectedSettingString $protectedSettings `

+ -TypeHandlerVersion 4.0

+#### Installation sample for Virtual Machine Scale Sets - Azure CLI

+my_subscription_id=<your_azure_subscription_id>

+az account set --subscription $my_subscription_id

+az vmss identity assign --resource-group $my_resource_group --name $my_linux_vmss

+# Download the sample public settings. You could also use curl or any web browser.

+my_vmss_resource_id=$(az vmss show --resource-group $my_resource_group \

+ --name $my_linux_vmss --query "id" -o tsv)

+my_diagnostic_storage_account_sastoken=$(az storage account generate-sas \

+ --account-name $my_diagnostic_storage_account --expiry 2037-12-31T23:59:00Z \

+ --permissions wlacu --resource-types co --services bt -o tsv)

+az vmss extension set --publisher Microsoft.Azure.Diagnostics --name LinuxDiagnostic

+ --version 4.0 --resource-group $my_resource_group --vmss-name $my_linux_vmss \

+ --protected-settings "${my_lad_protected_settings}" --settings portal_public_settings.json

+After you change your protected or public settings, run the same command to deploy them to the VM. If any settings changed, the updates are sent to the extension. LAD reloads the configuration and restarts itself.

+The latest version of the extension is 4.0, *which is currently in public preview*. Older versions of 3.x are still supported. 2.x versions have been deprecated since July 31, 2018.

+- On Azure Resource Manager deployment model VMs, include `"autoUpgradeMinorVersion": true` in the VM deployment template.

+- On classic deployment model VMs, specify version `4.*` if you're installing the extension through the Azure CLI or PowerShell.

+This set of configuration information contains sensitive information that should be protected from public view. It contains, for example, storage credentials. The settings are transmitted to the extension, which stores them in encrypted form.

+| Name | Value |

+| - | -- |

+| storageAccountName | The name of the storage account in which the extension writes data. |

+| storageAccountEndPoint | (Optional) The endpoint that identifies the cloud in which the storage account exists. If this setting is absent, by default, LAD uses the Azure public cloud, `https://core.windows.net`. To use a storage account in Azure Germany, Azure Government, or Azure China 21Vianet, set this value as required. |

+| storageAccountSasToken | An [Account SAS token](https://azure.microsoft.com/blog/sas-update-account-sas-now-supports-all-storage-services/) for blob and table services (`ss='bt'`). This token applies to containers and objects (`srt='co'`). It grants add, create, list, update, and write permissions (`sp='acluw'`). Do *not* include the leading question-mark (?). |

+| mdsdHttpProxy | (Optional) HTTP proxy information the extension needs to connect to the specified storage account and endpoint. |

+| sinksConfig | (Optional) Details of alternative destinations to which metrics and events can be delivered. The following sections provide details about each data sink the extension supports. |

+You can construct the required shared access signature token through the Azure portal:

+1. In the menu on the left, under **Security + networking**, select **Shared access signature**.

+1. Select **Generate SAS and connection string**.

+Copy the generated shared access signature into the `storageAccountSasToken` field. Remove the leading question mark (?).

+The `sinksConfig` optional section defines more destinations to which the extension sends collected information. The `"sink"` array contains an object for each extra data sink. The `"type"` attribute determines the other attributes in the object.

+| Element | Value |

+| - | -- |

+| name | A string used to refer to this sink elsewhere in the extension configuration. |

+| type | The type of sink being defined. Determines the other values, if any, in instances of this type. |

+The `"sasURL"` entry contains the full URL, including the shared access signature token, for the event hub to which data should be published. LAD requires a shared access signature to name a policy that enables the send claim. Here's an example:

+- Create an Event Hubs namespace called `contosohub`.

+- Create an event hub in the namespace called `syslogmsgs`.

+- Create a shared access policy on the event hub named `writer` that enables the send claim.

+If you create a SAS that's good until midnight UTC on January 1, 2018, the `sasURL` value might be like the following example.

+Blobs are stored in a container that has the same name as the sink. The Azure Storage rules for blob container names apply to the names of `JsonBlob` sinks. Names must have between 3 and 63 lowercase alphanumeric ASCII characters or dashes.

+| Element | Value |

+| - | -- |

+| StorageAccount | The name of the storage account in which the extension writes data. Must be the name specified in the [protected settings](#protected-settings). |

+| mdsdHttpProxy | (Optional) The proxy specified in the [protected settings](#protected-settings). If the private value is set, it overrides the public value. Put proxy settings that contain a secret, such as a password, in the [protected settings](#protected-settings). |

+If you don't want to enable syslog or metrics collection, specify an empty structure for the `ladCfg` element:

+| Element | Value |

+| - | -- |

+| eventVolume | (Optional) Controls the number of partitions created within the storage table. The value must be `"Large"`, `"Medium"`, or `"Small"`. The default value is `"Medium"`. |

+| sampleRateInSeconds | (Optional) The default interval between the collection of raw, that is, unaggregated, metrics. The smallest supported sample rate is 15 seconds. The default is `15`. |

+| Element | Value |

+| - | -- |

+| resourceId | The Azure Resource Manager resource ID of the VM or of the Virtual Machine Scale Set to which the VM belongs. Also specify this setting if the configuration uses any `JsonBlob` sink. |

+| scheduledTransferPeriod | The frequency at which aggregate metrics are computed and transferred to Azure Monitor Metrics. The frequency is expressed as an IS 8601 time interval. The smallest transfer period is 60 seconds, that is, PT1M. Specify at least one `scheduledTransferPeriod`. |

+The `performanceCounters` optional section controls the collection of metrics. Raw samples are aggregated for each [scheduledTransferPeriod](#metrics) to produce these values:

+- Mean

+- Minimum

+- Maximum

+- Last-collected value

+- Count of raw samples used to compute the aggregate

+| Element | Value |

+| - | -- |

+| sinks | (Optional) A comma-separated list of names of sinks to which LAD sends aggregated metric results. All aggregated metrics are published to each listed sink. For example, `"MyEventHubSink, MyJsonSink, MyAzMonSink"`. For more information, see [`sinksConfig` (protected settings)](#sinksconfig) and [`sinksConfig` (public settings)](#sinksconfig-1). |

+| type | Identifies the actual provider of the metric. |

+| class | Together with `"counter"`, identifies the specific metric within the provider's namespace. |

+| counter | Together with `"class"`, identifies the specific metric within the provider's namespace. [See a list of available counters](#metrics-supported-by-the-builtin-provider). |

+| counterSpecifier | Identifies the metric within the Azure Monitor Metrics namespace. |

+| condition | (Optional) Selects an instance of the object to which the metric applies. Or selects the aggregation across all instances of that object. |

+| sampleRate | The IS 8601 interval that sets the rate at which raw samples for this metric are collected. If the value isn't set, the value of [`sampleRateInSeconds`](#ladcfg) sets the collection interval. The shortest supported sample rate is 15 seconds (PT15S). |

+| unit | Defines the unit for the metric. Should be one of these strings: `"Count"`, `"Bytes"`, `"Seconds"`, `"Percent"`, `"CountPerSecond"`, `"BytesPerSecond"`, `"Millisecond"`. Consumers of the collected data expect the collected data values to match this unit. LAD ignores this field. |

+| displayName | The label to attach to the data in Azure Monitor Metrics when viewing in the `Guest (classic)` metrics namespace. This label is in the language specified by the associated locale setting. LAD ignores this field. **Note**: If viewing the same metric in the `azure.vm.linux.guestmetrics` Metrics Namespace, which is available if `AzMonSink` is configured, the display name depends entirely on the counter. To find the mapping between counters and names, see [Metrics supported by the builtin provider](#metrics-supported-by-the-builtin-provider). |

+The `counterSpecifier` is an arbitrary identifier. Consumers of metrics, like the Azure portal charting and alerting feature, use `counterSpecifier` as the key that identifies a metric or an instance of a metric.

+For `builtin` metrics, we recommend `counterSpecifier` values that begin with `/builtin/`. To collect a specific instance of a metric, attach the identifier of the instance to the `counterSpecifier` value. Here are some examples:

+- `/builtin/Processor/PercentIdleTime`. Idle time averaged across all vCPUs

+- `/builtin/Disk/FreeSpace(/mnt)`. Free space for the `/mnt` file system

+- `/builtin/Disk/FreeSpace`. Free space averaged across all mounted file systems

+When you specify `performanceCounters`, LAD always writes data to a table in Azure Storage. The same data can be written to JSON blobs or Event Hubs or both. You can't disable storing data to a table.

+- `WADMetrics`

+- The `"scheduledTransferPeriod"` for the aggregated values stored in the table

+- `P10DV2S`

+- A date, in the form *YYYYMMDD*, which changes every 10 days

+| Element | Value |

+| - | --

+facilityName | A syslog facility name, such as `"LOG_USER"` or `"LOG_LOCAL0"`. For more information, see *Values for facility* in the [syslog manual page](http://man7.org/linux/man-pages/man3/syslog.3.html).

+minSeverity | A syslog severity level, such as `"LOG_ERR"` or `"LOG_INFO"`. For more information, see *Values for level* in the [syslog manual page](http://man7.org/linux/man-pages/man3/syslog.3.html). The extension captures events sent to the facility at or above the specified level.

+When you specify `syslogEvents`, LAD always writes data to a table in Azure Storage. The same data can be written to JSON blobs or Event Hubs or both. You can't disable storing data to a table.

+- `LinuxSyslog`

+- A date, in the form *YYYYMMDD*, which changes every 10 days

+The optional public `sinksConfig` section enables sending metrics to the Azure Monitor sink in addition to the Storage account and the default Guest Metrics view.

+The `fileLogs` section controls the capture of log files. LAD captures new text lines as they're written to the file. It writes them to table rows and any specified sinks, such as `JsonBlob` and `EventHub`.

+> The `fileLogs` are captured by a subcomponent of LAD called `omsagent`. To collect `fileLogs`, ensure that the `omsagent` user has read permissions on the files you specify. It must also have execute permissions on all directories in the path to that file. After LAD is installed, to check permissions, run `sudo su omsagent -c 'cat /path/to/file'`.

+| Element | Value |

+| - | -- |

+| file | The full path of the log file to be watched and captured. The path can't specify a directory or contain wildcard characters. The `omsagent` user account must have read access to the file path. |

+| table | (Optional) The Azure Storage Table into which new lines from the tail of the file are written. The table must be in the designated storage account, as specified in the protected configuration. |

+| sinks | (Optional) A comma-separated list of names of more sinks to which log lines are sent. |

+The default metrics that LAD supports are aggregated across all file systems, disks, or names. For nonaggregated metrics, refer to the newer Azure Monitor sink metrics support.

+> The display names for each metric differ depending on the metrics namespace to which it belongs:

+>

+> - `Guest (classic)` populated from your storage account: the specified `displayName` in the `performanceCounters` section, or the default display name as seen in Azure Portal. For the VM, under **Monitoring** > **Diagnostic settings**, select **Metrics** tab.

+> - `azure.vm.linux.guestmetrics` populated from `AzMonSink`, if configured: the "`azure.vm.linux.guestmetrics` Display Name" specified in the following tables.

+> The metric values between `Guest (classic)` and `azure.vm.linux.guestmetrics` versions differ. While the classic metrics had certain aggregations applied in the agent, the new metrics are unaggregated counters, giving customers the flexibility to aggregate as desired at viewing/alerting time.

+- Processor

+- Memory

+- Network

+- File system

+- Disk

+| Counter | azure.vm.linux.guestmetrics Display Name | Meaning |

+| | - | -

+| PercentIdleTime | `cpu idle time` | Percentage of time during the aggregation window that processors ran the kernel idle loop |

+| PercentProcessorTime | `cpu percentage guest os` | Percentage of time running a not idle thread |

+| PercentIOWaitTime | `cpu io wait time` | Percentage of time waiting for I/O operations to finish |

+| PercentInterruptTime | `cpu interrupt time` | Percentage of time running hardware or software interrupts and deferred procedure calls (DPCs) |

+| PercentUserTime | `cpu user time` | Of not idle time during the aggregation window, the percentage of time spent in user mode at normal priority |

+| PercentNiceTime | `cpu nice time` | Of not idle time, the percentage spent at lowered (nice) priority |

+| PercentPrivilegedTime | `cpu privileged time` | Of not idle time, the percentage spent in privileged (kernel) mode |

+| Counter | azure.vm.linux.guestmetrics Display Name | Meaning |

+| | - | - |

+| AvailableMemory | `memory available` | Available physical memory in MiB |

+| PercentAvailableMemory | `mem. percent available` | Available physical memory as a percentage of total memory |

+| UsedMemory | `memory used` | In-use physical memory (MiB) |

+| PercentUsedMemory | `memory percentage` | In-use physical memory as a percentage of total memory |

+| PagesPerSec | `pages` | Total paging (read/write) |

+| PagesReadPerSec | `page reads` | Pages read from the backing store, such as swap file, program file, and mapped file |

+| PagesWrittenPerSec | `page writes` | Pages written to the backing store, such as swap file and mapped file |

+| AvailableSwap | `swap available` | Unused swap space (MiB) |

+| PercentAvailableSwap | `swap percent available` | Unused swap space as a percentage of the total swap |

+| UsedSwap | `swap used` | In-use swap space (MiB) |

+| PercentUsedSwap | `swap percent used` | In-use swap space as a percentage of the total swap |

+| Counter | azure.vm.linux.guestmetrics Display Name | Meaning |

+| | - | - |

+| BytesTransmitted | `network out guest os` | Total bytes sent since startup |

+| BytesReceived | `network in guest os` | Total bytes received since startup |

+| BytesTotal | `network total bytes` | Total bytes sent or received since startup |

+| PacketsTransmitted | `packets sent` | Total packets sent since startup |

+| PacketsReceived | `packets received` | Total packets received since startup |

+| TotalRxErrors | `packets received errors` | Number of receive errors since startup |

+| TotalTxErrors | `packets sent errors` | Number of transmit errors since startup |

+| TotalCollisions | `network collisions` | Number of collisions reported by the network ports since startup |

+The File system class of metrics provides information about file system usage. Absolute and percentage values are reported as they would be displayed to an ordinary user, not root.

+| Counter | azure.vm.linux.guestmetrics Display Name | Meaning |

+| | - | - |

+| FreeSpace | `filesystem free space` | Available disk space in bytes |

+| UsedSpace | `filesystem used space` | Used disk space in bytes |

+| PercentFreeSpace | `filesystem % free space` | Percentage of free space |

+| PercentUsedSpace | `filesystem % used space` | Percentage of used space |

+| PercentFreeInodes | `filesystem % free inodes` | Percentage of unused index nodes (inodes) |

+| PercentUsedInodes | `filesystem % used inodes` | Percentage of allocated (in use) inodes summed across all file systems |

+| BytesReadPerSecond | `filesystem read bytes/sec` | Bytes read per second |

+| BytesWrittenPerSecond | `filesystem write bytes/sec` | Bytes written per second |

+| BytesPerSecond | `filesystem bytes/sec` | Bytes read or written per second |

+| ReadsPerSecond | `filesystem reads/sec` | Read operations per second |

+| WritesPerSecond | `filesystem writes/sec` | Write operations per second |

+| TransfersPerSecond | `filesystem transfers/sec` | Read or write operations per second |

+| Counter | azure.vm.linux.guestmetrics Display Name | Meaning |

+| | - | - |

+| ReadsPerSecond | `disk reads` | Read operations per second |

+| WritesPerSecond | `disk writes` | Write operations per second |

+| TransfersPerSecond | `disk transfers` | Total operations per second |

+| AverageReadTime | `disk read time` | Average seconds per read operation |

+| AverageWriteTime | `disk write time` | Average seconds per write operation |

+| AverageTransferTime | `disk transfer time` | Average seconds per operation |

+| AverageDiskQueueLength | `disk queue length` | Average number of queued disk operations |

+| ReadBytesPerSecond | `disk read guest os` | Number of bytes read per second |

+| WriteBytesPerSecond | `disk write guest os` | Number of bytes written per second |

+| BytesPerSecond | `disk total bytes` | Number of bytes read or written per second |

+Based on the preceding definitions, this section provides a sample LAD 4.0 extension configuration and some explanation. To apply this sample, use your own storage account name, account shared access signature token, and Event Hubs SAS tokens.

+> Depending on whether you use the Azure CLI or Azure PowerShell to install LAD, the method for providing public and protected settings differs:

+> - If you're using the Azure CLI, save the following settings to *ProtectedSettings.json* and *PublicSettings.json* to use the preceding sample command.

+> - If you're using PowerShell, run `$protectedSettings = '{ ... }'` and `$publicSettings = '{ ... }'` to save the following settings to `$protectedSettings` and `$publicSettings`.

+### Protected settings configuration

+- A storage account.

+- A matching account shared access signature token.

+- Several sinks: `JsonBlob` or `EventHub` with SAS tokens.

+### Public settings configuration

+- Upload percent-processor-time metrics and used-disk-space metrics to the `WADMetrics*` table.

+- Upload messages from syslog facility `"user"` and severity `"info"` to the `LinuxSyslog*` table.

+- Upload appended lines in file `/var/log/myladtestlog` to the `MyLadTestLog` table.

+- Azure Blob Storage. The container name is as defined in the `JsonBlob` sink.

+- An Event Hubs endpoint, as specified in the `EventHub` sink.

+The `resourceId` in the configuration must match that of the VM or the Virtual Machine Scale Set.

+- Azure platform metrics charting and alerting knows the `resourceId` of the VM you're working on. It expects to find the data for your VM by using the `resourceId` as the lookup key.

+- If you use Azure autoscale, the `resourceId` in the autoscale configuration must match the `resourceId` that LAD uses.

+- The `resourceId` is built in to the names of JSON blobs written by LAD.

+Data sent to `JsonBlob` sinks is stored in blobs in the storage account named in the [protected settings](#protected-settings-configuration). You can consume the blob data in any Azure Blob Storage API.

+- Visual Studio Server Explorer

+- [Azure Storage Explorer](https://azure.microsoft.com/features/storage-explorer/)

+- In [Azure Monitor](../../azure-monitor/alerts/alerts-classic-portal.md), create alerts for the metrics you collect.

+- [Create monitoring charts](../../azure-monitor/data-platform.md) for your metrics.

+- [Create a Virtual Machine Scale Set](../linux/tutorial-create-vmss.md) by using your metrics to control autoscaling.

+> After you have run Sysprep on a VM, that VM is considered *generalized* and cannot be restarted. The process of generalizing a VM is not reversible. If you need to keep the original VM functioning, you should create a snapshot of the OS disk, create a VM from the snapshot, and then generalize that copy of the VM.

+```output

+```output

+```output

+```config

+```output

+ ```config

+sudo zypper install util-linux

+ - ["ephemeral0.1", "/mnt/resource"]

+ - ["ephemeral0.1", "/mnt/resource"]

+ - ["ephemeral0.1", "/mnt/resource"]

+ echo "Removing swapfile" #RHEL uses a swapfile by default

+ - ["ephemeral0.1", "/mnt/resource"]

+ - ["ephemeral0.1", "/mnt/resource"]

+ - ["ephemeral0.1", "/mnt/resource"]

+ - ["ephemeral0.1", "/mnt/ressource"]

+Once an event is scheduled, it will move into the `Started` state after it's been approved or the `NotBefore` time passes. However, in rare cases, the operation will be cancelled by Azure before it starts. In that case the event will be removed from the Events array, and the impact will not occur as previously scheduled.

+description: Learn how to use the Custom Script Extension to run scripts and deploy applications to Windows virtual machines in Azure.

+#Customer intent: As an IT administrator or developer, I want to learn about how to install applications on Windows VMs so that I can automate the process and reduce the risk of human error of manual configuration tasks.

+> * Use the Custom Script Extension to install IIS.

+> * Create a VM that uses the Custom Script Extension.

+> * View a running IIS site after the extension is applied.

+The Azure Cloud Shell is a free interactive shell that you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account.

+To open the Cloud Shell, select **Open Cloudshell** from the upper right corner of a code block. You can also launch Cloud Shell in a separate browser tab by going to [https://shell.azure.com/powershell](https://shell.azure.com/powershell). Select **Copy** to copy the blocks of code, paste it into the Cloud Shell, and press enter to run it.

+The Custom Script Extension downloads and executes scripts on Azure VMs. This extension is useful for post-deployment configuration, software installation, or any other configuration or management task. You can download scripts from Azure storage or GitHub, or you can provide scripts to the Azure portal at extension run time.

+The Custom Script extension integrates with Azure Resource Manager templates and can be run by using the Azure CLI, PowerShell, Azure portal, or the Azure Virtual Machine REST API.

+You can use the Custom Script Extension with both Linux and Windows VMs.

+Now you can create the VM with [New-AzVM](/powershell/module/az.compute/new-azvm). The following example creates a VM named *myVM* in the *EastUS* location. If they don't already exist, the resource group *myResourceGroupAutomate* and supporting network resources are created. To allow web traffic, the cmdlet also opens port *80*.

+The resources and VM take a few minutes to be created.

+Obtain the public IP address of your load balancer with [Get-AzPublicIPAddress](/powershell/module/az.network/get-azpublicipaddress). The following example obtains the IP address for `myPublicIPAddress` created earlier:

+> * Use the Custom Script Extension to install IIS.

+> * Create a VM that uses the Custom Script Extension.

+> * View a running IIS site after the extension is applied.

+ Title: "Tutorial: Secure a Windows web server with TLS certificates in Azure"

+description: Learn how to use Azure PowerShell to secure a Windows virtual machine that runs the IIS web server with TLS certificates stored in Azure Key Vault.

+#Customer intent: As an IT administrator or developer, I want to learn how to secure a web server with TLS certificates so that I can protect my customer data on web applications that I build and run.

+# Tutorial: Secure a web server on a Windows virtual machine in Azure with TLS certificates stored in Key Vault

+**Applies to:** :heavy_check_mark: Windows VMs :heavy_check_mark: Flexible scale sets

+> Currently, this doc only works for Generalized images. If you attempt this tutorial by using a Specialized disk you will receive an error.

+To secure web servers, a Transport Layer Security (TLS) certificate can be used to encrypt web traffic. TLS certificates can be stored in Azure Key Vault and allow secure deployments of certificates to Windows virtual machines (VMs) in Azure. In this tutorial you learn how to:

+> * Create an Azure Key Vault.

+> * Generate or upload a certificate to the Key Vault.

+> * Create a VM and install the IIS web server.

+> * Inject the certificate into the VM and configure IIS with a TLS binding.

+The Azure Cloud Shell is a free interactive shell that you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account.

+To open the Cloud Shell, just select **Open Cloudshell** from the upper right corner of a code block. You can also launch Cloud Shell in a separate browser tab by going to [https://shell.azure.com/powershell](https://shell.azure.com/powershell). Select **Copy** to copy the blocks of code, paste them into the Cloud Shell, and press enter to run them.

+Azure Key Vault safeguards cryptographic keys and secrets, such as certificates or passwords. Key Vault helps streamline the certificate management process and enables you to maintain control of keys that access those certificates. You can create a self-signed certificate inside Key Vault, or you can upload an existing, trusted certificate that you already own.

+Rather than by using a custom VM image that includes certificates baked-in, inject certificates into a running VM. This process ensures that the most up-to-date certificates are installed on a web server during deployment. If you renew or replace a certificate, you don't also have to create a new custom VM image. The latest certificates are automatically injected as you create more VMs. During the whole process, the certificates never leave the Azure platform or are exposed in a script, command-line history, or template.

+Next, create a Key Vault with [New-AzKeyVault](/powershell/module/az.keyvault/new-azkeyvault). Each Key Vault requires a unique name and should be all lower case. Replace `mykeyvault` with your own unique Key Vault name in the following example:

+## Generate a certificate and store it in Key Vault

+For production use, you should import a valid certificate signed by a trusted provider with [Import-AzKeyVaultCertificate](/powershell/module/az.keyvault/import-azkeyvaultcertificate). For this tutorial, the following example shows how you can generate a self-signed certificate with [Add-AzKeyVaultCertificate](/powershell/module/az.keyvault/add-azkeyvaultcertificate) that uses the default certificate policy from [New-AzKeyVaultCertificatePolicy](/powershell/module/az.keyvault/new-azkeyvaultcertificatepolicy).

+Now you can create the VM with [New-AzVM](/powershell/module/az.compute/new-azvm). The following example creates a VM named *myVM* in the *EastUS* location. If they don't already exist, the supporting network resources are created. To allow secure web traffic, the cmdlet also opens port *443*.

+In this tutorial, you secured an IIS web server with a TLS certificate stored in Azure Key Vault. You learned how to:

+> * Create an Azure Key Vault.

+> * Generate or upload a certificate to the Key Vault.

+> * Create a VM and install the IIS web server.

+> * Inject the certificate into the VM and configure IIS with a TLS binding.

+For prebuilt virtual machine script samples, see:

+> [!NOTE]

+> Be aware of versions that are End Of Life (EOL) and no longer supported by Redhat. Uploaded images that are, at or beyond EOL will be supported on a reasonable business effort basis. Link to Redhat's [Product Lifecycle](https://access.redhat.com/product-life-cycles/?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204)

+> [!NOTE]

+> Be aware of versions that are End Of Life (EOL) and no longer supported by Redhat. Uploaded images that are, at or beyond EOL will be supported on a reasonable business effort basis. Link to Redhat's [Product Lifecycle](https://access.redhat.com/product-life-cycles/?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204)

+ssh azureuser@<publicIpAddress>

+sudo su - oracle

+dbca -silent \

+ ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1; export ORACLE_HOME

+ ORACLE_SID=cdb1; export ORACLE_SID

+sqlplus / as sysdba

+sqlplus / as sysdba

+ lsnrctl stop

+ lsnrctl start

+ssh azureuser@<publicIpAddress>

+sudo su - oracle

+ lsnrctl stop

+ lsnrctl start

+ orapwd file=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwcdb1 password=OraPasswd1 entries=10

+ export ORACLE_SID=cdb1

+ sqlplus / as sysdba

+ rman TARGET sys/OraPasswd1@cdb1 AUXILIARY sys/OraPasswd1@cdb1_stby

+sqlplus / as sysdba

+ dgmgrl sys/OraPasswd1@cdb1

+sqlplus sys/OraPasswd1@cdb1

+dgmgrl sys/OraPasswd1@cdb1

+sqlplus sys/OraPasswd1@cdb1_stby

+dgmgrl sys/OraPasswd1@cdb1_stby

+sqlplus sys/OraPasswd1@cdb1

+> [!NOTE]

+> Be aware of versions that are End Of Life (EOL) and no longer supported by Redhat. Uploaded images that are, at or beyond EOL will be supported on a reasonable business effort basis. Link to Redhat's [Product Lifecycle](https://access.redhat.com/product-life-cycles/?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204)

+> [!NOTE]

+> Be aware of versions that are End Of Life (EOL) and no longer supported by Redhat. Uploaded images that are, at or beyond EOL will be supported on a reasonable business effort basis. Link to Redhat's [Product Lifecycle](https://access.redhat.com/product-life-cycles/?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204)

+ dbca -silent \

+ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1; export ORACLE_HOME

+ORACLE_SID=cdb1; export ORACLE_SID

+LD_LIBRARY_PATH=ORACLE_HOME/lib; export LD_LIBRARY_PATH

+lsnrctl start

+ dbca -silent \

+ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1; export ORACLE_HOME

+ORACLE_SID=cdb1; export ORACLE_SID

+LD_LIBRARY_PATH=ORACLE_HOME/lib; export LD_LIBRARY_PATH

+sudo su - oracle

+lsnrctl start

+sqlplus / as sysdba

+ scp fbo_ggs_Linux_x64_shiphome.zip <publicIpAddress>:<folder>

+ sudo su -

+ mv <folder>/*.zip /opt

+ yum install unzip

+ cd /opt

+ unzip fbo_ggs_Linux_x64_shiphome.zip

+ chown -R oracle:oinstall /opt/fbo_ggs_Linux_x64_shiphome

+ sudo su - oracle

+ mkdir .ssh (if not already created)

+ cd .ssh

+ cd /opt/fbo_ggs_Linux_x64_shiphome/Disk1

+ ./runInstaller

+ cd $ORACLE_HOME/network/admin

+ vi tnsnames.ora

+ sqlplus / as sysdba

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ sqlplus system/OraPasswd1@pdb1

+ sudo su - oracle

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ ./ggsci

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ ./ggsci

+ ./ggsci

+ sqlplus / as sysdba

+ ./ggsci

+ cd $ORACLE_HOME/network/admin

+ vi tnsnames.ora

+ sqlplus / as sysdba

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ sqlplus system/OraPasswd1@pdb1

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ ./ggsci

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ ./ggsci

+cd /u01/app/oracle/product/12.1.0/oggcore_1

+./ggsci

+ cd /u01/app/oracle/product/12.1.0/oggcore_1

+ ./ggsci

+[Explore VM deployment CLI samples](https://github.com/Azure-Samples/azure-cli-samples/tree/master/virtual-machine)

+ . oraenv

+ scp vmoracle19c_xxxxxx_xxxxxx_xxxxxx.py azureuser@<publicIpAddress>:/tmp

+```output

+scp vmoracle19c_xxxxxx_xxxxxx_xxxxxx.py azureuser@<publicIpAddress>:/tmp

+ ```output

+ sudo su - oracle

+ sqlplus / as sysdba

+ ```output

+This article demonstrates the use of Azure Files as a media to back up and restore an Oracle database running on an Azure VM. The steps in this article have been tested against Oracle 12.1 and higher. You'll back up the database using Oracle RMAN to an Azure file share mounted to the VM using the SMB protocol. Using Azure Files for backup media is extremely cost effective and performant. However, for very large databases, Azure Backup provides a better solution.

+6. Start the Oracle listener if it isn't already running:

+ lsnrctl start

+9. Start the database if it isn't already running:

+1. [Back up the database](#back-up-the-database).

+In this step, you'll back up the Oracle database using Oracle Recovery Manager (RMAN) to Azure Files. Azure file shares are fully managed file shares that live in the cloud. They can be accessed using either the Server Message Block (SMB) protocol or the Network File System (NFS) protocol. This step covers creating a file share that uses the SMB protocol to mount to your VM. For information about how to mount using NFS, see [How to create an NFS share](../../../storage/files/storage-files-how-to-create-nfs-shares.md).

+When mounting the Azure Files, we'll use the `cache=none` to disable caching of file share data. And to ensure files created in the share are owned by the oracle user set the `uid=oracle` and `gid=oinstall` options as well.

+5. Click on ***+ File share*** and in the ***New file share*** blade name your file share ***orabkup1***. Set ***Quota*** to ***10240*** GiB and check ***Transaction optimized*** as the tier. The quota reflects an upper boundary that the file share can grow to. As we're using Standard storage, resources are PAYG and not provisioned so setting it to 10 TiB will not incur costs beyond what you use. If your backup strategy requires more storage, you must set the quota to an appropriate level to hold all backups. When you have completed the New file share blade, click ***Create***.

+### Back up the database

+In this section, we'll be using Oracle Recovery Manager (RMAN) to take a full backup of the database and archive logs and write the backup as a backup set to the Azure File share mounted earlier.

+ rman target /

+2. In this example, we're limiting the size of RMAN backup pieces to 1 TiB. Please note the RMAN backup MAXPIECESIZE can go upto 4TiB as Azure standard file shares and Premium File Shares have a maximum file size limit of 4 TiB. For more information, see [Azure Files Scalability and Performance Targets](../../../storage/files/storage-files-scale-targets.md).)

+## Upgrade from RHEL 8 VMs to RHEL 9 VMs

+Instructions for an in-place upgrade from Red Hat Enterprise Linux 8 VMs to Red Hat Enterprise Linux 9 VMs on Azure is provided at the [Red Hat upgrading from RHEL 8 to RHEL 9 documentation here.](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/upgrading_from_rhel_8_to_rhel_9/index)

+## Upgrade SAP environments from RHEL 8 VMs to RHEL 9 VMs

+Instructions for an in-place upgrade from Red Hat Enterprise Linux 8 SAP VMs to Red Hat Enterprise Linux 9 SAP VMs on Azure is provided at the [Red Hat upgrading from RHEL 8 SAP to RHEL 9 SAP documentation here.](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_for_sap_solutions/9/html-single/how_to_in-place_upgrade_sap_environments_from_rhel_8_to_rhel_9/index)

+* To learn more about Red Hat support policies for all versions of RHEL, see [Red Hat Enterprise Linux life cycle](https://access.redhat.com/support/policy/updates/errata) in the Red Hat documentation.

+> [!IMPORTANT]

+> While Microsoft will not stop advertising the range after the specified date, it is strongly recommended to independently create a follow-up ROA if the original expiration date has passed to avoid external carriers from not accepting the advertisement.

+> It is also recommended to create a ROA for any existing ASN that is advertising the range to avoid any issues during migration.

+> [!IMPORTANT]

+> While Microsoft will not stop advertising the range after the specified date, it is strongly recommended to independently create a follow-up ROA if the original expiration date has passed to avoid external carriers from not accepting the advertisement.

+> It is also recommended to create a ROA for any existing ASN that is advertising the range to avoid any issues during migration.

+> [!IMPORTANT]

+> While Microsoft will not stop advertising the range after the specified date, it is strongly recommended to independently create a follow-up ROA if the original expiration date has passed to avoid external carriers from not accepting the advertisement.

+The following PowerShell example specifies the `-GatewaySku` as VpnGw1. When using PowerShell to create a gateway, you must first create the IP configuration, then use a variable to refer to it. In this example, the configuration variable is $gwipconfig.

+If you have a VPN gateway and you want to use a different gateway SKU, your options are to either resize your gateway SKU, or to change to another SKU. When you change to another gateway SKU, you delete the existing gateway entirely and build a new one. Creating a gateway can often take 45 minutes or more, depending on the selected gateway SKU. In comparison, when you resize a gateway SKU, there isn't much downtime because you don't have to delete and rebuild the gateway. While it's faster to resize your gateway SKU, there are rules regarding resizing:

+1. When working with the old gateway SKUs, you can resize between Standard and HighPerformance SKUs.

+1. You **cannot** resize from Basic/Standard/HighPerformance SKUs to VpnGw SKUs. You must instead, [change](#change) to the new SKUs.

+When you create the virtual network gateway for a VPN gateway configuration, you must specify a *VPN type*. The VPN type that you choose depends on the connection topology that you want to create. For example, a P2S connection requires a RouteBased VPN type. A VPN type can also depend on the hardware that you're using. S2S configurations require a VPN device. Some VPN devices only support a certain VPN type.

+The VPN type you select must satisfy all the connection requirements for the solution you want to create. For example, if you want to create a S2S VPN gateway connection and a P2S VPN gateway connection for the same virtual network, use the VPN type *RouteBased* because P2S requires a RouteBased VPN type. You also need to verify that your VPN device supported a RouteBased VPN connection.

+Once a virtual network gateway has been created, you can't change the VPN type. If you want a different VPN type, first delete the virtual network gateway, and then create a new gateway.

+Before you create a VPN gateway, you must create a gateway subnet. The gateway subnet contains the IP addresses that the virtual network gateway VMs and services use. When you create your virtual network gateway, gateway VMs are deployed to the gateway subnet and configured with the required VPN gateway settings. Never deploy anything else (for example, additional VMs) to the gateway subnet. The gateway subnet must be named 'GatewaySubnet' to work properly. Naming the gateway subnet 'GatewaySubnet' lets Azure know that this is the subnet to which it should deploy the virtual network gateway VMs and services.

+When you create the gateway subnet, you specify the number of IP addresses that the subnet contains. The IP addresses in the gateway subnet are allocated to the gateway VMs and gateway services. Some configurations require more IP addresses than others.

+When you're planning your gateway subnet size, refer to the documentation for the configuration that you're planning to create. For example, the ExpressRoute/VPN Gateway coexist configuration requires a larger gateway subnet than most other configurations. Additionally, you may want to make sure your gateway subnet contains enough IP addresses to accommodate possible future additional configurations. While you can create a gateway subnet as small as /29 (applicable to Basic SKU only), we recommend that you create a gateway subnet of /27 or larger (/27, /26 etc.). This accommodates most configurations.

+When you configure a local network gateway, you specify the name, the public IP address or the fully qualified domain name (FQDN) of the on-premises VPN device, and the address prefixes that are located on the on-premises location. Azure looks at the destination address prefixes for network traffic, consults the configuration that you've specified for your local network gateway, and routes packets accordingly. If you use Border Gateway Protocol (BGP) on your VPN device, you provide the BGP peer IP address of your VPN device and the autonomous system number (ASN) of your on-premises network. You also specify local network gateways for VNet-to-VNet configurations that use a VPN gateway connection.

+Location = East US

+AddressSpace = 10.1.0.0/16

+SubnetName = Frontend

+Subnet = 10.1.0.0/24

+LNG Public IP = <On-premises VPN device IP address>

+Local Address Prefixes = 10.0.0.0/24, 20.0.0.0/24

+Gateway IP Config = gwipconfig1

+VPNType = RouteBased

+GatewayType = Vpn

+ -Location 'East US' -GatewayIpAddress '23.99.221.164' -AddressPrefix '10.0.0.0/24'

+ -Location 'East US' -GatewayIpAddress '23.99.221.164' -AddressPrefix @('20.0.0.0/24','10.0.0.0/24')

+$gwpip= New-AzPublicIpAddress -Name VNet1GWPIP -ResourceGroupName TestRG1 -Location 'East US' -AllocationMethod Static -Sku Standard

+* The *-GatewayType* for a site-to-site configuration is *Vpn*. The gateway type is always specific to the configuration that you are implementing. For example, other gateway configurations may require -GatewayType ExpressRoute.