+## Deploying Azure AD provisioning agent

+The Azure AD Provisioning agent can be deployed on the same server hosting a SCIM enabled application, or a seperate server, providing it has line of sight to the application's SCIM endpoint. A single agent also supports provision to multiple applications hosted locally on the same server or seperate hosts, again as long as each SCIM endpoint is reachable by the agent.

+ 1. [Download](https://aka.ms/OnPremProvisioningAgent) the provisioning agent and copy it onto the virtual machine or server that your SCIM application endpoint is hosted on.

+ 2. Run the provisioning agent installer, agree to the terms of service, and select **Install**.

+ 3. Once installed, locate and launch the **AAD Connect Provisioning Agent wizard**, and when prompted for an extensions select **On-premises provisioning**

+ 4. For the agent to register itself with your tenant, provide credentials for an Azure AD admin with Hybrid administrator or global administrator permissions.

+ 5. Select **Confirm** to confirm the installation was successful.

+

+## Provisioning to SCIM-enabled application

+Once the agent is installed, no further configuration is necesary on-prem, and all provisioning configurations are then managed from the portal. Repeat the below steps for every on-premises application being provisioned via SCIM.

+

+ 1. In the Azure portal navigate to the Enterprise applications and add the **On-premises SCIM app** from the [gallery](../../active-directory/manage-apps/add-application-portal.md).

+ 2. From the left hand menu navigate to the **Provisioning** option and select **Get started**.

+ 3. Select **Automatic** from the dropdown list and expand the **On-Premises Connectivity** option.

+ 4. Select the agent that you installed from the dropdown list and select **Assign Agent(s)**.

+ 5. Now either wait 10 minutes or restart the **Microsoft Azure AD Connect Provisioning Agent** before proceeding to the next step & testing the connection.

+ 6. In the **Tenant URL** field, provide the SCIM endpoint URL for your application. The URL is typically unique to each target application and must be resolveable by DNS. An example for a scenario where the agent is installed on the same host as the application is https://localhost:8585/scim 

+ 7. Select **Test Connection**, and save the credentials. The application SCIM endpoint must be actively listening for inbound provisioning requests, otherwise the test will fail. Use the steps [here](on-premises-ecma-troubleshoot.md#troubleshoot-test-connection-issues) if you run into connectivity issues.

+ 8. Configure any [attribute mappings](customize-application-attributes.md) or [scoping](define-conditional-rules-for-provisioning-user-accounts.md) rules required for your application.

+ 9. Add users to scope by [assigning users and groups](../../active-directory/manage-apps/add-application-portal-assign-users.md) to the application.

+ 10. Test provisioning a few users [on demand](provision-on-demand.md).

+ 11. Add more users into scope by assigning them to your application.

+ 12. Go to the **Provisioning** pane, and select **Start provisioning**.

+ 13. Monitor using the [provisioning logs](../../active-directory/reports-monitoring/concept-provisioning-logs.md).

+* Use the `scp` claim to validate that the user has granted the calling app permission to call your API.

+* Ensure the calling client is allowed to call your API using the `appid` claim (for v1.0 tokens) or the `azp` claim (for v2.0 tokens).

+ * You only need to validate these claims (`appid`, `azp`) if you want to restrict your web API to be called only by pre-determined applications (e.g., line-of-business applications or web APIs called by well-known frontends). APIs intended to allow access from any calling application do not need to validate these claims.

+|`tid` | String, a GUID | Represents the tenant that the user is signing in to. For work and school accounts, the GUID is the immutable tenant ID of the organization that the user is signing in to. For sign-ins to the personal Microsoft account tenant (services like Xbox, Teams for Life, or Outlook), the value is `9188040d-6c67-4c5b-b112-36a304b66dad`.|

+> The Virtual Machine Administrator Login and Virtual Machine User Login roles use `dataActions` and can be assigned at the management group, subscription, resource group, or resource scope. We recommend that you assign the roles at the management group, subscription, or resource level and not at the individual VM level. This practice avoids the risk of reaching the [Azure role assignments limit](../../role-based-access-control/troubleshooting.md#limits) per subscription.

+If you see an "Azure role not assigned" error on your SSH prompt, verify that you've configured Azure RBAC policies for the VM that grants the user either the Virtual Machine Administrator Login role or the Virtual Machine User Login role. If you're having problems with Azure role assignments, see the article [Troubleshoot Azure RBAC](../../role-based-access-control/troubleshooting.md#limits).

+> If you're having problems with Azure role assignments, see [Troubleshoot Azure RBAC](../../role-based-access-control/troubleshooting.md#limits).

+| [Fiori](https://www.sap.com/products/fiori.html) | The web-based user experience of SAP (as opposed to the desktop-based experience). |

+Regardless of where the authorization information comes from, it can then be emitted as the `Groups` attribute inside the SAML token by configuring that attribute name as the [default partner claim type on the claims schema](../../active-directory-b2c/claimsschema.md#defaultpartnerclaimtypes) or by overriding the [partner claim type on the output claims](../../active-directory-b2c/relyingparty.md#outputclaims). Note however that BTP allows you to [map Role Collections to User Attributes](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/b3fbb1a9232d4cf99967a0b29dd85d4c.html), which means that *any* attribute name can be used for authorization decisions, even if you don't use the `Groups` attribute name.

+ Title: 'Create an enterprise application from a multi-tenant application'

+description: Create an enterprise application using the client ID for a multi-tenant application.

+zone_pivot_groups: enterprise-apps-cli

+#Customer intent: As an administrator of an Azure AD tenant, I want to create an enterprise application using client ID for a multi-tenant application provided by a service provider or independent software vendor.

+# Create an enterprise application from a multi-tenant application in Azure Active Directory

+In this article, you'll learn how to create an enterprise application in your tenant using the client ID for a multi-tenant application. An enterprise application refers to a service principal within a tenant. The service principal discussed in this article is the local representation, or application instance, of a global application object in a single tenant or directory.

+Before you proceed to add the application using any of these options, check whether the enterprise application is already in your tenant by attempting to sign in to the application. If the sign-in is successful, the enterprise application already exists in your tenant.

+If you have verified that the application isn't in your tenant, proceed with any of the following ways to add the enterprise application to your tenant using the appId

+## Prerequisites

+To add an enterprise application to your Azure AD tenant, you need:

+- An Azure AD user account. If you don't already have one, you can [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- One of the following roles: Global Administrator, Cloud Application Administrator, or Application Administrator.

+- The client ID of the multi-tenant application.

+## Create an enterprise application

+If you've been provided with the admin consent URL, navigate to the URL through a web browser to [grant tenant-wide admin consent](grant-admin-consent.md) to the application. Granting tenant-wide admin consent to the application will add it to your tenant. The tenant-wide admin consent URL has the following format:

+```http

+https://login.microsoftonline.com/common/oauth2/authorize?response_type=code&client_id=248e869f-0e5c-484d-b5ea1fba9563df41&redirect_uri=https://www.your-app-url.com

+```

+where:

+- `{client-id}` is the application's client ID (also known as appId).

+1. Run `connect-MgGraph -Scopes "Application.ReadWrite.All"` and sign in with a Global Admin user account.

+1. Run the following command to create the enterprise application:

+ ```powershell

+ New-MgServicePrincipal -AppId fc876dd1-6bcb-4304-b9b6-18ddf1526b62

+ ```

+1. To delete the enterprise application you created, run the command:

+ ```powershell

+ Remove-MgServicePrincipal

+ -ServicePrincipalId <objectID>

+ ```

+From the Microsoft Graph explorer window:

+1. To create the enterprise application, insert the following query:

+

+ ```http

+ POST /servicePrincipals.

+ ```

+1. Supply the following request in the **Request body**.

+ {

+ "appId": "fc876dd1-6bcb-4304-b9b6-18ddf1526b62"

+ }

+1. Grant the Application.ReadWrite.All permission under the **Modify permissions** tab and select **Run query**.

+1. To delete the enterprise application you created, run the query:

+ ```http

+ DELETE /servicePrincipals/{objectID}

+ ```

+1. To create the enterprise application, run the following command:

+

+ ```azurecli

+ az ad sp create --id fc876dd1-6bcb-4304-b9b6-18ddf1526b62

+ ```

+1. To delete the enterprise application you created, run the command:

+ ```azurecli

+ az ad sp delete --id

+ ```

+## Next steps

+- [Add RBAC role to the enterprise application](/azure/role-based-access-control/role-assignments-portal)

+- [Assign users to your application](add-application-portal-assign-users.md)

+Users often are unable to consent to the permissions an application is requesting. Configure the admin consent workflow to allow users to provide a justification and request an administrator's review and approval of an application. For training on how to configure admin consent workflow in your Azure AD tenant, see [Configure admin consent workflow](/learn/modules/configure-admin-consent-workflow).

+will be displayed with ΓÇ£Identity not foundΓÇ¥ when viewed in the portal. [Read more](../../role-based-access-control/troubleshooting.md#symptomrole-assignments-with-identity-not-found).

+- [Azure AD Privileged Identity Management API reference](/graph/api/resources/privilegedidentitymanagementv3-overview)

+- [Configure Azure AD role settings in Privileged Identity Management](pim-how-to-change-default-settings.md)

+Use Privileged Identity Management (PIM) in Azure Active Directory (Azure AD), part of Microsoft Entra, to allow eligible role members for Azure resources to schedule activation for a future date and time. They can also select a specific activation duration within the maximum (configured by administrators).

+- [Custom roles in Azure](../../role-based-access-control/custom-roles.md)

+- [Start an access review for Azure resource roles in Privileged Identity Management](./pim-create-azure-ad-roles-and-resource-roles-review.md)

+- [Assign Azure resource roles in Privileged Identity Management](pim-resource-roles-assign-roles.md)

+- [Deploy Privileged Identity Management](pim-deployment-plan.md)

+- [Create an access review of Azure AD roles in PIM](./pim-create-azure-ad-roles-and-resource-roles-review.md)

+> * Deactivate users in 8x8 when they do not require access anymore

+1. Sign in to [Admin Console](https://admin.8x8.com). Select **Identity and Security**.

+ [  ](./media/8x8-provisioning-tutorial/8x8-identity-and-security.png#lightbox)

+2. In the **User Provisioning Integration (SCIM)** pane, click the toggle to enable and then click **Save**.

+ [  ](./media/8x8-provisioning-tutorial/8x8-enable-user-provisioning.png#lightbox)

+ [  ](./media/8x8-provisioning-tutorial/8x8-copy-url-token.png#lightbox)

+ 

+ 

+ 

+ 

+ 

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)

+1. Articulate 360 application expects the default attributes to be replaced with the specific attributes as shown below. These attributes are also pre populated but you can review them as per your requirements.

+ 

+ | FirstName | user.givenname |

+ | LastName | user.surname |

+1. In the **SAML Signing Certificate** section, click the edit icon and change the **Signing Option** to **Sign SAML response and assertion**. Click **Save**.

+ 

+ Title: 'Tutorial: Azure AD SSO integration with Cheetah For Benelux'

+description: Learn how to configure single sign-on between Azure Active Directory and Cheetah For Benelux.

+# Tutorial: Azure AD SSO integration with Cheetah For Benelux

+In this tutorial, you'll learn how to integrate Cheetah For Benelux with Azure Active Directory (Azure AD). When you integrate Cheetah For Benelux with Azure AD, you can:

+* Control in Azure AD who has access to Cheetah For Benelux.

+* Enable your users to be automatically signed-in to Cheetah For Benelux with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+## Prerequisites

+To get started, you need the following items:

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* Cheetah For Benelux single sign-on (SSO) enabled subscription.

+* Along with Cloud Application Administrator, Application Administrator can also add or manage applications in Azure AD.

+For more information, see [Azure built-in roles](../roles/permissions-reference.md).

+## Scenario description

+In this tutorial, you configure and test Azure AD SSO in a test environment.

+* Cheetah For Benelux supports **SP** initiated SSO.

+* Cheetah For Benelux supports **Just In Time** user provisioning.

+> [!NOTE]

+> Identifier of this application is a fixed string value so only one instance can be configured in one tenant.

+## Add Cheetah For Benelux from the gallery

+To configure the integration of Cheetah For Benelux into Azure AD, you need to add Cheetah For Benelux from the gallery to your list of managed SaaS apps.

+1. Sign in to the Azure portal using either a work or school account, or a personal Microsoft account.

+1. On the left navigation pane, select the **Azure Active Directory** service.

+1. Navigate to **Enterprise Applications** and then select **All Applications**.

+1. To add new application, select **New application**.

+1. In the **Add from the gallery** section, type **Cheetah For Benelux** in the search box.

+1. Select **Cheetah For Benelux** from results panel and then add the app. Wait a few seconds while the app is added to your tenant.

+## Configure and test Azure AD SSO for Cheetah For Benelux

+Configure and test Azure AD SSO with Cheetah For Benelux using a test user called **B.Simon**. For SSO to work, you need to establish a link relationship between an Azure AD user and the related user in Cheetah For Benelux.

+To configure and test Azure AD SSO with Cheetah For Benelux, perform the following steps:

+1. **[Configure Azure AD SSO](#configure-azure-ad-sso)** - to enable your users to use this feature.

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** - to test Azure AD single sign-on with B.Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** - to enable B.Simon to use Azure AD single sign-on.

+1. **[Configure Cheetah For Benelux SSO](#configure-cheetah-for-benelux-sso)** - to configure the single sign-on settings on application side.

+ 1. **[Create Cheetah For Benelux test user](#create-cheetah-for-benelux-test-user)** - to have a counterpart of B.Simon in Cheetah For Benelux that is linked to the Azure AD representation of user.

+1. **[Test SSO](#test-sso)** - to verify whether the configuration works.

+## Configure Azure AD SSO

+Follow these steps to enable Azure AD SSO in the Azure portal.

+1. In the Azure portal, on the **Cheetah For Benelux** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ a. In the **Reply URL** textbox, type the URL:

+ `https://ups.eu.sso.cheetah.com/saml2/idpresponse`

+ b. In the **Sign-on URL** text box, type the URL:

+ `https://ups.eu.sso.cheetah.com/login?client_id=5c2m16mhv4cd4o5cpgekmsmlne&response_type=token&scope=aws.cognito.signin.user.admin+openid+profile&redirect_uri=https://prodeditor.eu.cheetah.com/CssWebTask/landing/?cheetah_client=BNLX`

+1. On the **Set-up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Federation Metadata XML** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up Cheetah For Benelux** section, copy the appropriate URL(s) based on your requirement.

+ 

+### Create an Azure AD test user

+In this section, you'll create a test user in the Azure portal called B.Simon.

+1. From the left pane in the Azure portal, select **Azure Active Directory**, select **Users**, and then select **All users**.

+1. Select **New user** at the top of the screen.

+1. In the **User** properties, follow these steps:

+ 1. In the **Name** field, enter `B.Simon`.

+ 1. In the **User name** field, enter the username@companydomain.extension. For example, `B.Simon@contoso.com`.

+ 1. Select the **Show password** check box, and then write down the value that's displayed in the **Password** box.

+ 1. Click **Create**.

+### Assign the Azure AD test user

+In this section, you'll enable B.Simon to use Azure single sign-on by granting access to Cheetah For Benelux.

+1. In the Azure portal, select **Enterprise Applications**, and then select **All applications**.

+1. In the applications list, select **Cheetah For Benelux**.

+1. In the app's overview page, find the **Manage** section and select **Users and groups**.

+1. Select **Add user**, then select **Users and groups** in the **Add Assignment** dialog.

+1. In the **Users and groups** dialog, select **B.Simon** from the Users list, then click the **Select** button at the bottom of the screen.

+1. If you are expecting a role to be assigned to the users, you can select it from the **Select a role** dropdown. If no role has been set up for this app, you see "Default Access" role selected.

+1. In the **Add Assignment** dialog, click the **Assign** button.

+## Configure Cheetah For Benelux SSO

+To configure single sign-on on **Cheetah For Benelux** side, you need to send the downloaded **Federation Metadata XML** and appropriate copied URLs from Azure portal to [Cheetah For Benelux support team](mailto:support@cheetah.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create Cheetah For Benelux test user

+In this section, a user called B.Simon is created in Cheetah For Benelux. Cheetah For Benelux supports just-in-time user provisioning, which is enabled by default. There is no action item for you in this section. If a user doesn't already exist in Cheetah For Benelux, a new one is created after authentication.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+* Click on **Test this application** in Azure portal. This will redirect to Cheetah For Benelux Sign-on URL where you can initiate the login flow.

+* Go to Cheetah For Benelux Sign-on URL directly and initiate the login flow from there.

+* You can use Microsoft My Apps. When you click the Cheetah For Benelux tile in the My Apps, this will redirect to Cheetah For Benelux Sign-on URL. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Next steps

+Once you configure Cheetah For Benelux you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+To enable SSO in Expensify, you first need to enable **Domain Control** in the application. For additional support, work with [Expensify Client support team](mailto:help@expensify.com). Once you have Domain Control enabled, follow these steps:

+ Title: 'Tutorial: Azure AD SSO integration with LUSID'

+description: Learn how to configure single sign-on between Azure Active Directory and LUSID.

+# Tutorial: Azure AD SSO integration with LUSID

+In this tutorial, you'll learn how to integrate LUSID with Azure Active Directory (Azure AD). When you integrate LUSID with Azure AD, you can:

+* Control in Azure AD who has access to LUSID.

+* Enable your users to be automatically signed-in to LUSID with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+## Prerequisites

+To get started, you need the following items:

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* LUSID single sign-on (SSO) enabled subscription.

+* Along with Cloud Application Administrator, Application Administrator can also add or manage applications in Azure AD.

+For more information, see [Azure built-in roles](../roles/permissions-reference.md).

+## Scenario description

+In this tutorial, you configure and test Azure AD SSO in a test environment.

+* LUSID supports **SP** and **IDP** initiated SSO.

+* LUSID supports **Just In Time** user provisioning.

+## Add LUSID from the gallery

+To configure the integration of LUSID into Azure AD, you need to add LUSID from the gallery to your list of managed SaaS apps.

+1. Sign in to the Azure portal using either a work or school account, or a personal Microsoft account.

+1. On the left navigation pane, select the **Azure Active Directory** service.

+1. Navigate to **Enterprise Applications** and then select **All Applications**.

+1. To add new application, select **New application**.

+1. In the **Add from the gallery** section, type **LUSID** in the search box.

+1. Select **LUSID** from results panel and then add the app. Wait a few seconds while the app is added to your tenant.

+## Configure and test Azure AD SSO for LUSID

+Configure and test Azure AD SSO with LUSID using a test user called **B.Simon**. For SSO to work, you need to establish a link relationship between an Azure AD user and the related user at LUSID.

+To configure and test Azure AD SSO with LUSID, perform the following steps:

+1. **[Configure Azure AD SSO](#configure-azure-ad-sso)** - to enable your users to use this feature.

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** - to test Azure AD single sign-on with B.Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** - to enable B.Simon to use Azure AD single sign-on.

+1. **[Configure LUSID SSO](#configure-lusid-sso)** - to configure the single sign-on settings on application side.

+ 1. **[Create LUSID test user](#create-lusid-test-user)** - to have a counterpart of B.Simon in LUSID that is linked to the Azure AD representation of user.

+1. **[Test SSO](#test-sso)** - to verify whether the configuration works.

+## Configure Azure AD SSO

+Follow these steps to enable Azure AD SSO in the Azure portal.

+1. In the Azure portal, on the **LUSID** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ a. In the **Identifier** textbox, type a URL using the following pattern:

+ `https://www.okta.com/saml2/service-provider/<ID>`

+ b. In the **Reply URL** textbox, type a URL using the following pattern:

+ `https://<CustomerDomain>.identity.lusid.com/sso/saml2/<ID>`

+1. Click **Set additional URLs** and perform the following steps, if you wish to configure the application in **SP** initiated mode:

+ a. In the **Sign-on URL** text box, type a URL using the following pattern:

+ `https://<CustomerDomain>.lusid.com/ `

+ b. In the **Relay State** text box, type a URL using the following pattern:

+ `https://<CustomerDomain>.lusid.com/app/home`

+ > [!Note]

+ > These values are not real. Update these values with the actual Identifier, Reply URL, Sign on URL and Relay State URL. Contact [LUSID support team](mailto:support@finbourne.com) to get these values. You can also refer to the patterns shown in the **Basic SAML Configuration** section in the Azure portal.

+1. LUSID application expects the SAML assertions in a specific format, which requires you to add custom attribute mappings to your SAML token attributes configuration. The following screenshot shows the list of default attributes.

+ 

+1. In addition to above, LUSID application expects few more attributes to be passed back in SAML response, which are shown below. These attributes are also pre populated but you can review them as per your requirements.

+ | Name | Source Attribute|

+ | | |

+ | email | user.mail |

+ | firstName | user.givenname |

+ | lastName | user.surname |

+ | fbn-groups | user.assignedroles |

+ > [!NOTE]

+ > Please click [here](../develop/howto-add-app-roles-in-azure-ad-apps.md#app-roles-ui) to know how to configure Role in Azure AD.

+1. On the **Set up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Certificate (Base64)** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up LUSID** section, copy the appropriate URL(s) based on your requirement.

+ 

+### Create an Azure AD test user

+In this section, you'll create a test user in the Azure portal called B.Simon.

+1. From the left pane in the Azure portal, select **Azure Active Directory**, select **Users**, and then select **All users**.

+1. Select **New user** at the top of the screen.

+1. In the **User** properties, follow these steps:

+ 1. In the **Name** field, enter `B.Simon`.

+ 1. In the **User name** field, enter the username@companydomain.extension. For example, `B.Simon@contoso.com`.

+ 1. Select the **Show password** check box, and then write down the value that's displayed in the **Password** box.

+ 1. Click **Create**.

+### Assign the Azure AD test user

+In this section, you'll enable B.Simon to use Azure single sign-on by granting access to LUSID.

+1. In the Azure portal, select **Enterprise Applications**, and then select **All applications**.

+1. In the applications list, select **LUSID**.

+1. In the app's overview page, find the **Manage** section and select **Users and groups**.

+1. Select **Add user**, then select **Users and groups** in the **Add Assignment** dialog.

+1. In the **Users and groups** dialog, select **B.Simon** from the Users list, then click the **Select** button at the bottom of the screen.

+1. If you are expecting a role to be assigned to the users, you can select it from the **Select a role** dropdown. If no role has been set up for this app, you see "Default Access" role selected.

+1. In the **Add Assignment** dialog, click the **Assign** button.

+## Configure LUSID SSO

+To configure single sign-on on **LUSID** side, you need to send the downloaded **Certificate (Base64)** and appropriate copied URLs from Azure portal to [LUSID support team](mailto:support@finbourne.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create LUSID test user

+In this section, a user called B.Simon is created in LUSID. LUSID supports just-in-time user provisioning, which is enabled by default. There is no action item for you in this section. If a user doesn't already exist in LUSID, a new one is created after authentication.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+#### SP initiated:

+* Click on **Test this application** in Azure portal. This will redirect to LUSID Sign-on URL where you can initiate the login flow.

+* Go to LUSID Sign-on URL directly and initiate the login flow from there.

+#### IDP initiated:

+* Click on **Test this application** in Azure portal and you should be automatically signed in to the LUSID for which you set up the SSO.

+You can also use Microsoft My Apps to test the application in any mode. When you click the LUSID tile in the My Apps, if configured in SP mode you would be redirected to the application sign-on page for initiating the login flow and if configured in IDP mode, you should be automatically signed in to the LUSID for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Next steps

+Once you configure LUSID you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+ Title: 'Tutorial: Azure AD SSO integration with Lytx DriveCam'

+description: Learn how to configure single sign-on between Azure Active Directory and Lytx DriveCam.

+# Tutorial: Azure AD SSO integration with Lytx DriveCam

+In this tutorial, you'll learn how to integrate Lytx DriveCam with Azure Active Directory (Azure AD). When you integrate Lytx DriveCam with Azure AD, you can:

+* Control in Azure AD who has access to Lytx DriveCam.

+* Enable your users to be automatically signed-in to Lytx DriveCam with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+## Prerequisites

+To get started, you need the following items:

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* Lytx DriveCam single sign-on (SSO) enabled subscription.

+* Along with Cloud Application Administrator, Application Administrator can also add or manage applications in Azure AD.

+For more information, see [Azure built-in roles](../roles/permissions-reference.md).

+## Scenario description

+In this tutorial, you configure and test Azure AD SSO in a test environment.

+* Lytx DriveCam supports **IDP** initiated SSO.

+## Add Lytx DriveCam from the gallery

+To configure the integration of Lytx DriveCam into Azure AD, you need to add Lytx DriveCam from the gallery to your list of managed SaaS apps.

+1. Sign in to the Azure portal using either a work or school account, or a personal Microsoft account.

+1. On the left navigation pane, select the **Azure Active Directory** service.

+1. Navigate to **Enterprise Applications** and then select **All Applications**.

+1. To add new application, select **New application**.

+1. In the **Add from the gallery** section, type **Lytx DriveCam** in the search box.

+1. Select **Lytx DriveCam** from results panel and then add the app. Wait a few seconds while the app is added to your tenant.

+## Configure and test Azure AD SSO for Lytx DriveCam

+Configure and test Azure AD SSO with Lytx DriveCam using a test user called **B.Simon**. For SSO to work, you need to establish a link relationship between an Azure AD user and the related user at Lytx DriveCam.

+To configure and test Azure AD SSO with Lytx DriveCam, perform the following steps:

+1. **[Configure Azure AD SSO](#configure-azure-ad-sso)** - to enable your users to use this feature.

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** - to test Azure AD single sign-on with B.Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** - to enable B.Simon to use Azure AD single sign-on.

+1. **[Configure Lytx DriveCam SSO](#configure-lytx-drivecam-sso)** - to configure the single sign-on settings on application side.

+ 1. **[Create Lytx DriveCam test user](#create-lytx-drivecam-test-user)** - to have a counterpart of B.Simon in Lytx DriveCam that is linked to the Azure AD representation of user.

+1. **[Test SSO](#test-sso)** - to verify whether the configuration works.

+## Configure Azure AD SSO

+Follow these steps to enable Azure AD SSO in the Azure portal.

+1. In the Azure portal, on the **Lytx DriveCam** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, the application is pre-configured and the necessary URLs are already pre-populated with Azure. The user needs to save the configuration by clicking the **Save** button.

+1. On the **Set up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Certificate (Base64)** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up Lytx DriveCam** section, copy the appropriate URL(s) based on your requirement.

+ 

+### Create an Azure AD test user

+In this section, you'll create a test user in the Azure portal called B.Simon.

+1. From the left pane in the Azure portal, select **Azure Active Directory**, select **Users**, and then select **All users**.

+1. Select **New user** at the top of the screen.

+1. In the **User** properties, follow these steps:

+ 1. In the **Name** field, enter `B.Simon`.

+ 1. In the **User name** field, enter the username@companydomain.extension. For example, `B.Simon@contoso.com`.

+ 1. Select the **Show password** check box, and then write down the value that's displayed in the **Password** box.

+ 1. Click **Create**.

+### Assign the Azure AD test user

+In this section, you'll enable B.Simon to use Azure single sign-on by granting access to Lytx DriveCam.

+1. In the Azure portal, select **Enterprise Applications**, and then select **All applications**.

+1. In the applications list, select **Lytx DriveCam**.

+1. In the app's overview page, find the **Manage** section and select **Users and groups**.

+1. Select **Add user**, then select **Users and groups** in the **Add Assignment** dialog.

+1. In the **Users and groups** dialog, select **B.Simon** from the Users list, then click the **Select** button at the bottom of the screen.

+1. If you are expecting a role to be assigned to the users, you can select it from the **Select a role** dropdown. If no role has been set up for this app, you see "Default Access" role selected.

+1. In the **Add Assignment** dialog, click the **Assign** button.

+## Configure Lytx DriveCam SSO

+To configure single sign-on on **Lytx DriveCam** side, you need to send the downloaded **Certificate (Base64)** and appropriate copied URLs from Azure portal to [Lytx DriveCam support team](mailto:support@lytx.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create Lytx DriveCam test user

+In this section, you create a user called Britta Simon at Lytx DriveCam. Work with [Lytx DriveCam support team](mailto:support@lytx.com) to add the users in the Lytx DriveCam platform. Users must be created and activated before you use single sign-on.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+* Click on Test this application in Azure portal and you should be automatically signed in to the Lytx DriveCam for which you set up the SSO.

+* You can use Microsoft My Apps. When you click the Lytx DriveCam tile in the My Apps, you should be automatically signed in to the Lytx DriveCam for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Next steps

+Once you configure Lytx DriveCam you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+ Title: 'Tutorial: Azure AD SSO integration with Mist Cloud Admin SSO'

+description: Learn how to configure single sign-on between Azure Active Directory and Mist Cloud Admin SSO.

+# Tutorial: Azure AD SSO integration with Mist Cloud Admin SSO

+In this tutorial, you'll learn how to integrate Mist Cloud Admin SSO with Azure Active Directory (Azure AD). When you integrate Mist Cloud Admin SSO with Azure AD, you can:

+* Control in Azure AD who has access to Mist Cloud Admin SSO.

+* Enable your users to be automatically signed-in to Mist Cloud Admin SSO with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+## Prerequisites

+To get started, you need the following items:

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* Mist Cloud Admin SSO single sign-on (SSO) enabled subscription.

+* Along with Cloud Application Administrator, Application Administrator can also add or manage applications in Azure AD.

+For more information, see [Azure built-in roles](../roles/permissions-reference.md).

+## Scenario description

+In this tutorial, you configure and test Azure AD SSO in a test environment.

+* Mist Cloud Admin SSO supports **SP** and **IDP** initiated SSO.

+## Add Mist Cloud Admin SSO from the gallery

+To configure the integration of Mist Cloud Admin SSO into Azure AD, you need to add Mist Cloud Admin SSO from the gallery to your list of managed SaaS apps.

+1. Sign in to the Azure portal using either a work or school account, or a personal Microsoft account.

+1. On the left navigation pane, select the **Azure Active Directory** service.

+1. Navigate to **Enterprise Applications** and then select **All Applications**.

+1. To add new application, select **New application**.

+1. In the **Add from the gallery** section, type **Mist Cloud Admin SSO** in the search box.

+1. Select **Mist Cloud Admin SSO** from results panel and then add the app. Wait a few seconds while the app is added to your tenant.

+## Configure and test Azure AD SSO for Mist Cloud Admin SSO

+Configure and test Azure AD SSO with Mist Cloud Admin SSO using a test user called **B.Simon**. For SSO to work, you need to establish a link relationship between an Azure AD user and the related user at Mist Cloud Admin SSO.

+To configure and test Azure AD SSO with Mist Cloud Admin SSO, perform the following steps:

+1. **[Configure Azure AD SSO](#configure-azure-ad-sso)** - to enable your users to use this feature.

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** - to test Azure AD single sign-on with B.Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** - to enable B.Simon to use Azure AD single sign-on.

+1. **[Configure Mist Cloud Admin SSO](#configure-mist-cloud-admin-sso)** - to configure the single sign-on settings on application side.

+ 1. **[Create Mist Cloud Admin SSO test user](#create-mist-cloud-admin-sso-test-user)** - to have a counterpart of B.Simon in Mist Cloud Admin SSO that is linked to the Azure AD representation of user.

+1. **[Test SSO](#test-sso)** - to verify whether the configuration works.

+## Configure Azure AD SSO

+Follow these steps to enable Azure AD SSO in the Azure portal.

+1. In the Azure portal, on the **Mist Cloud Admin SSO** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ a. In the **Identifier** textbox, type a value using the following pattern:

+ `https://api.<MISTCLOUDREGION>.mist.com/api/v1/saml/<SSOUNIQUEID>/login`

+ b. In the **Reply URL** textbox, type a URL using the following pattern:

+ `https://api.<MISTCLOUDREGION>.mist.com/api/v1/saml/<SSOUNIQUEID>/login`

+1. Click **Set additional URLs** and perform the following step, if you wish to configure the application in **SP** initiated mode:

+ In the **Sign-on URL** text box, type the URL:

+ `https://manage.mist.com`

+ > [!Note]

+ > These values are not real. Update these values with the actual Identifier and Reply URL. Contact [Mist Cloud Admin SSO support team](mailto:support@mist.com) to get these values. You can also refer to the patterns shown in the **Basic SAML Configuration** section in the Azure portal.

+1. Mist Cloud Admin SSO application expects the SAML assertions in a specific format, which requires you to add custom attribute mappings to your SAML token attributes configuration. The following screenshot shows the list of default attributes.

+ 

+1. In addition to above, Mist Cloud Admin SSO application expects few more attributes to be passed back in SAML response, which are shown below. These attributes are also pre populated but you can review them as per your requirements.

+ | Name | Source Attribute|

+ | | |

+ | FirstName | user.givenname |

+ | LastName | user.surname |

+ | Role | user.assignedroles |

+ > [!NOTE]

+ > Please click [here](../develop/howto-add-app-roles-in-azure-ad-apps.md#app-roles-ui) to know how to configure Role in Azure AD.

+1. On the **Set up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Certificate (Base64)** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up Mist Cloud Admin SSO** section, copy the appropriate URL(s) based on your requirement.

+ 

+### Create an Azure AD test user

+In this section, you'll create a test user in the Azure portal called B.Simon.

+1. From the left pane in the Azure portal, select **Azure Active Directory**, select **Users**, and then select **All users**.

+1. Select **New user** at the top of the screen.

+1. In the **User** properties, follow these steps:

+ 1. In the **Name** field, enter `B.Simon`.

+ 1. In the **User name** field, enter the username@companydomain.extension. For example, `B.Simon@contoso.com`.

+ 1. Select the **Show password** check box, and then write down the value that's displayed in the **Password** box.

+ 1. Click **Create**.

+### Assign the Azure AD test user

+In this section, you'll enable B.Simon to use Azure single sign-on by granting access to Mist Cloud Admin SSO.

+1. In the Azure portal, select **Enterprise Applications**, and then select **All applications**.

+1. In the applications list, select **Mist Cloud Admin SSO**.

+1. In the app's overview page, find the **Manage** section and select **Users and groups**.

+1. Select **Add user**, then select **Users and groups** in the **Add Assignment** dialog.

+1. In the **Users and groups** dialog, select **B.Simon** from the Users list, then click the **Select** button at the bottom of the screen.

+1. If you are expecting a role to be assigned to the users, you can select it from the **Select a role** dropdown. If no role has been set up for this app, you see "Default Access" role selected.

+1. In the **Add Assignment** dialog, click the **Assign** button.

+## Configure Mist Cloud Admin SSO

+1. Log in to your Mist Cloud Admin SSO company site as an administrator.

+1. Go to **Organization** > **Settings** > **Single Sign-On** > **Add IdP**.

+

+ 

+1. In the **Create Identity Provider** section, perform the following steps:

+ 

+ 1. In the **Issuer** textbox, paste the **Azure AD Identifier** value which you have copied from the Azure portal.

+ 1. Open the downloaded **Certificate (Base64)** from the Azure portal into Notepad and paste the content into the **Certificate** textbox.

+ 1. In the **SSO URL** textbox, paste the **Login URL** value which you have copied from the Azure portal.

+ 1. In the **Custom Logout URL** textbox, paste the **Logout URL** value which you have copied from the Azure portal.

+ 1. Copy **ACS URL** value, paste this value into the **Reply URL** text box in the **Basic SAML Configuration** section in the Azure portal.

+ 1. Click **Save**.

+### Create Mist Cloud Admin SSO test user

+In this section, you create a user called Britta Simon at Mist Cloud Admin SSO. Work with [Mist Cloud Admin SSO support team](mailto:support@mist.com) to add the users in the Mist Cloud Admin SSO platform. Users must be created and activated before you use single sign-on.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+#### SP initiated:

+* Click on **Test this application** in Azure portal. This will redirect to Mist Cloud Admin SSO Sign-on URL where you can initiate the login flow.

+* Go to Mist Cloud Admin SSO Sign-on URL directly and initiate the login flow from there.

+#### IDP initiated:

+* Click on **Test this application** in Azure portal and you should be automatically signed in to the Mist Cloud Admin SSO for which you set up the SSO.

+You can also use Microsoft My Apps to test the application in any mode. When you click the Mist Cloud Admin SSO tile in the My Apps, if configured in SP mode you would be redirected to the application sign-on page for initiating the login flow and if configured in IDP mode, you should be automatically signed in to the Mist Cloud Admin SSO for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Next steps

+Once you configure Mist Cloud Admin SSO you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+ Title: 'Tutorial: Azure AD SSO integration with myAOS'

+description: Learn how to configure single sign-on between Azure Active Directory and myAOS.

+# Tutorial: Azure AD SSO integration with myAOS

+In this tutorial, you'll learn how to integrate myAOS with Azure Active Directory (Azure AD). When you integrate myAOS with Azure AD, you can:

+* Control in Azure AD who has access to myAOS.

+* Enable your users to be automatically signed-in to myAOS with their Azure AD accounts.

+* Manage your accounts in one central location - the Azure portal.

+## Prerequisites

+To get started, you need the following items:

+* An Azure AD subscription. If you don't have a subscription, you can get a [free account](https://azure.microsoft.com/free/).

+* myAOS single sign-on (SSO) enabled subscription.

+* Along with Cloud Application Administrator, Application Administrator can also add or manage applications in Azure AD.

+For more information, see [Azure built-in roles](../roles/permissions-reference.md).

+## Scenario description

+In this tutorial, you configure and test Azure AD SSO in a test environment.

+* myAOS supports **IDP** initiated SSO.

+## Add myAOS from the gallery

+To configure the integration of myAOS into Azure AD, you need to add myAOS from the gallery to your list of managed SaaS apps.

+1. Sign in to the Azure portal using either a work or school account, or a personal Microsoft account.

+1. On the left navigation pane, select the **Azure Active Directory** service.

+1. Navigate to **Enterprise Applications** and then select **All Applications**.

+1. To add new application, select **New application**.

+1. In the **Add from the gallery** section, type **myAOS** in the search box.

+1. Select **myAOS** from results panel and then add the app. Wait a few seconds while the app is added to your tenant.

+## Configure and test Azure AD SSO for myAOS

+Configure and test Azure AD SSO with myAOS using a test user called **B.Simon**. For SSO to work, you need to establish a link relationship between an Azure AD user and the related user in myAOS.

+To configure and test Azure AD SSO with myAOS, perform the following steps:

+1. **[Configure Azure AD SSO](#configure-azure-ad-sso)** - to enable your users to use this feature.

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** - to test Azure AD single sign-on with B.Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** - to enable B.Simon to use Azure AD single sign-on.

+1. **[Configure myAOS SSO](#configure-myaos-sso)** - to configure the single sign-on settings on application side.

+ 1. **[Create myAOS test user](#create-myaos-test-user)** - to have a counterpart of B.Simon in myAOS that is linked to the Azure AD representation of user.

+1. **[Test SSO](#test-sso)** - to verify whether the configuration works.

+## Configure Azure AD SSO

+Follow these steps to enable Azure AD SSO in the Azure portal.

+1. In the Azure portal, on the **myAOS** application integration page, find the **Manage** section and select **single sign-on**.

+1. On the **Select a single sign-on method** page, select **SAML**.

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+ 

+1. On the **Basic SAML Configuration** section, the application is pre-configured and the necessary URLs are already pre-populated with Azure. The user needs to save the configuration by clicking the **Save** button.

+1. On the **Set-up single sign-on with SAML** page, in the **SAML Signing Certificate** section, find **Federation Metadata XML** and select **Download** to download the certificate and save it on your computer.

+ 

+1. On the **Set up myAOS** section, copy the appropriate URL(s) based on your requirement.

+ 

+### Create an Azure AD test user

+In this section, you'll create a test user in the Azure portal called B.Simon.

+1. From the left pane in the Azure portal, select **Azure Active Directory**, select **Users**, and then select **All users**.

+1. Select **New user** at the top of the screen.

+1. In the **User** properties, follow these steps:

+ 1. In the **Name** field, enter `B.Simon`.

+ 1. In the **User name** field, enter the username@companydomain.extension. For example, `B.Simon@contoso.com`.

+ 1. Select the **Show password** check box, and then write down the value that's displayed in the **Password** box.

+ 1. Click **Create**.

+### Assign the Azure AD test user

+In this section, you'll enable B.Simon to use Azure single sign-on by granting access to myAOS.

+1. In the Azure portal, select **Enterprise Applications**, and then select **All applications**.

+1. In the applications list, select **myAOS**.

+1. In the app's overview page, find the **Manage** section and select **Users and groups**.

+1. Select **Add user**, then select **Users and groups** in the **Add Assignment** dialog.

+1. In the **Users and groups** dialog, select **B.Simon** from the Users list, then click the **Select** button at the bottom of the screen.

+1. If you are expecting a role to be assigned to the users, you can select it from the **Select a role** dropdown. If no role has been set up for this app, you see "Default Access" role selected.

+1. In the **Add Assignment** dialog, click the **Assign** button.

+## Configure myAOS SSO

+To configure single sign-on on **myAOS** side, you need to send the downloaded **Federation Metadata XML** and appropriate copied URLs from Azure portal to [myAOS support team](mailto:support@vialto.com). They set this setting to have the SAML SSO connection set properly on both sides.

+### Create myAOS test user

+In this section, you create a user called Britta Simon in myAOS. Work with [myAOS support team](mailto:support@vialto.com) to add the users in the myAOS platform. Users must be created and activated before you use single sign-on.

+## Test SSO

+In this section, you test your Azure AD single sign-on configuration with following options.

+* Click on Test this application in Azure portal and you should be automatically signed in to the myAOS for which you set up the SSO.

+* You can use Microsoft My Apps. When you click the myAOS tile in the My Apps, you should be automatically signed in to the myAOS for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](../user-help/my-apps-portal-end-user-access.md).

+## Next steps

+Once you configure myAOS you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Cloud App Security](/cloud-app-security/proxy-deployment-aad).

+ d. In the **Identity Provider Issuer** textbox, paste the **Application ID** value which you have copied from Zylo's overview page in Azure portal.

+- Only Azure Files [SMB](/azure/storage/files/files-smb-protocol) are supported. Azure Files [NFS](/azure/storage/files/files-nfs-protocol) is not currently supported for Linux App Services.

+### Azure Public:

+| Region | Normal and dedicated host | Availability zone support |

+| -- | :-: | :-: |

+| Australia East | x | x |

+| Australia Southeast | x | |

+| Brazil South | x | x |

+| Canada Central | x | x |

+| Canada East | x | |

+| Central India | x | x |

+| Central US | x | x |

+| East Asia | x | x |

+| East US | x | x |

+| East US 2 | x | x |

+| France Central | x | x |

+| Germany West Central | x | x |

+| Japan East | x | x |

+| Korea Central | x | x |

+| North Central US | x | |

+| North Europe | x | x |

+| Norway East | x | x |

+| South Africa North | x | x |

+| South Central US | x | x |

+| Southeast Asia | x | x |

+| Switzerland North | x | |

+| UAE North | x | |

+| UK South | x | x |

+| UK West | x | |

+| West Central US | x | |

+| West Europe | x | x |

+| West US | x | |

+| West US 2 | x | x |

+| West US 3 | x | x |

+### Azure Government:

+| Region | Normal and dedicated host | Availability zone support |

+| -- | :-: | :-: |

+| US Gov Texas | x | |

+| US Gov Arizona | x | |

+| US Gov Virginia | x | |

+> [Whitepaper on Using App Service Environment v3 in Compliance-Oriented Industries](https://azure.microsoft.com/resources/using-app-service-environment-v3-in-compliance-oriented-industries/)

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../../cognitive-services/cognitive-services-security.md) article for more information.

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../../cognitive-services/cognitive-services-security.md) article for more information.

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../../cognitive-services/cognitive-services-security.md) article for more information.

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../../cognitive-services/cognitive-services-security.md) article for more information.

+> [!IMPORTANT]

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../../cognitive-services/cognitive-services-security.md) article for more information.

+- To assign an Azure role, you must have ```Microsoft.Authorization/roleAssignments/write``` permissions, such as [User Access Administrator](/azure/role-based-access-control/built-in-roles#user-access-administrator) or [Owner](/azure/role-based-access-control/built-in-roles#owner).

+ Title: Migrate Azure Application Gateway Standard and WAF v2 deployments to availability zone support

+[Application Gateway Standard v2](/azure/application-gateway/overview-v2) and Application Gateway with [WAF v2](/azure/web-application-firewall/ag/ag-overview) supports zonal and zone redundant deployments. For more information about zone redundancy, see [Regions and availability zones](az-overview.md).

+If you previously deployed **Azure Application Gateway Standard v2** or **Azure Application Gateway Standard v2 + WAF v2** without zonal support, you must redeploy these services to enable zone redundancy. Two migration options to redeploy these services are described in this article.

+- Your deployment must be Standard v2 or WAF v2 SKU. Earlier SKUs (Standard and WAF) don't support availability zones.

+2. Follow the steps in [Create an application gateway](../application-gateway/quick-create-portal.md#create-an-application-gateway) or [Create an application gateway with a Web Application Firewall](/azure/web-application-firewall/ag/application-gateway-web-application-firewall-portal) to create a new Application Gateway v2 or Application Gateway v2 + WAF v2, respectively. You can reuse your existing Virtual Network or create a new one, but you must create a new frontend Public IP address.

+4. Follow the steps in [Create an application gateway](../application-gateway/quick-create-portal.md#create-an-application-gateway) or [Create an application gateway with a Web Application Firewall](/azure/web-application-firewall/ag/application-gateway-web-application-firewall-portal) to create a new Application Gateway v2 or Application Gateway v2 + WAF v2, respectively, using the same Virtual Network, subnets, and Public IP address that you used previously.

+ Title: Uninstall Azure Arc-enabled data services

+description: Uninstall Azure Arc-enabled data services

+|WindRiver| v1.23.1|v1.9.0_2022-07-12 |16.0.312.4243|postgres 12.3 (Ubuntu 12.3-1) |

+2. Run the connect command with the `proxy-https` and `proxy-http` parameters specified. If your proxy server is set up with both HTTP and HTTPS, be sure to use `--proxy-http` for the HTTP proxy and `--proxy-https` for the HTTPS proxy. If your proxy server only uses HTTP, you can use that value for both parameters.

+> [!NOTE]

+> If you're using a proxy server, you'll need to modify the `Invoke-WebRequest` command in the script to include the `Proxy` parameter and web address, as in: `Invoke-WebRequest -Uri "https://aka.ms/azcmagent-windows" -Proxy "http://xx.x.x.xx:xxxx -TimeoutSec 30 -OutFile "$InstallationFolder\install_windows_azcmagent.ps1"`

+> [!NOTE]

+> Before applying the Group Policy Scheduled Task, you must first check the folder `ScheduledTasks` (located within the `Preferences` folder) and modify the `ScheduledTasks.xml` file by changing `<GroupId>NT AUTHORITY\SYSTEM<\GroupId>` to `<UserId>NT AUTHORITY\SYSTEM</UserId>`.

+## July 2022

+### Redis 6 becomes default for new cache instances

+On November 1, 2022, all the versions of Azure Cache for Redis REST API, PowerShell, Azure CLI, and Azure SDK will create Redis instances using the latest stable version of Redis offered by Azure Cache for Redis by default. Previously, Redis version 4.0 was the default version used. However, as of October 2021, the latest stable Redis version offered in Azure Cache for Redis is 6.0.

+>[!NOTE]

+> This change does not affect any existing instances. It is only applicable to new instances created from November 1, 2022, and onward.

+>

+> The default Redis version that is used when creating a cache instance can vary because it is based on the latest stable version offered in Azure Cache for Redis.

+If you need a specific version of Redis for your application, we recommend using latest artifact versions as shown in the table below. Then, choose the Redis version explicitly when you create the cache.

+| Artifact | Version that supports specifying Redis version |

+|||

+| REST API | 2020-06-01 and newer |

+| PowerShell | 6.3.0 and newer |

+| Azure CLI | 2.27.0 and newer |

+| Azure SDK for .NET | 7.0.0 and newer |

+| Azure SDK for Python | 13.0.0 and newer |

+| Azure SDK for Java | 2.2.0 and newer |

+| Azure SDK for JavaScript| 6.0.0 and newer |

+| Azure SDK for Go | v49.1.0 and newer |

+For more information, see [View cache metrics](cache-how-to-monitor.md#view-cache-metrics).

+| Custom telemetry type | Micrometer | Log4j, logback, JUL | 2.x SDK | opentelemetry-api |

+|--||||-|

+| Custom events | | | Yes | |

+| Custom metrics | Yes | | Yes | Yes |

+| Dependencies | | | Yes | Yes |

+| Exceptions | | Yes | Yes | Yes |

+| Page views | | | Yes | |

+| Requests | | | Yes | Yes |

+| Traces | | Yes | Yes | Yes |

+Application Insights Java 3.x is already listening for telemetry that's sent to the Application Insights Java 2.x SDK. This functionality is an important part of the upgrade story for existing 2.x users. And it fills an important gap in our custom telemetry support until all custom telemetry types are supported via the OpenTelemetry API.

+The following steps will provide more information to assist with troubleshooting.

+1. Navigate to the Monitor section of the Azure portal.

+1. Select **Metrics** on the left hand side.

+1. Under **Select a scope**, check the applicable subscription and resource groups.

+1. Under **Refine scope**, choose **Custom Metric Usage** and the desired location.

+1. Select the **Apply** button.

+1. Choose either **Active Time Series**, **Active Time Series Limit**, or **Throttled Time Series**.

+- **Cluster (default)**: Billing for ingested data is done at the cluster level. The ingested data quantities from each workspace associated to a cluster are aggregated to calculate the daily bill for the cluster. Per-node allocations from [Microsoft Defender for Cloud](../../security-center/index.yml) are applied at the workspace level prior to this aggregation of data across all workspaces in the cluster.

+The charge for maintaining restored logs is calculated based on the volume of data you restore, in GB, and the number or days for which you restore the data. Charges are prorated and subject to the minimum restore duration and data volume. There is no charge for querying against restored logs.

+For example, if your table holds 500 GB a day and you restore 10 days of data, you'll be charged for 5000 GB a day until you dismiss the restored data.

+- **Profiling web apps**:

+ - While you can use the Profiler at no extra cost, your web app must be hosted in the basic tier of the Web Apps feature of Azure App Service, at minimum.

+ - You can only attach 1 profiler to each web app.

+ * Comma-separated IP addresses. You can enter multiple host IPs or subnet masks in a single rule by separating them with commas. The length limit is 4096 characters. Example: `10.1.12.25,10.1.12.28,10.1.12.29,10.1.12.10/4`

+* Japan East

+* Switzerland North

+* West US

+|--|-|

+| Permission is denied error when mounting a dual-protocol volume. | A dual-protocol volume supports both the NFS and SMB protocols. When you try to access the mounted volume on the UNIX system, the system attempts to map the UNIX user you use to a Windows user. <br> Ensure that the `POSIX` attributes are properly set on the AD DS User object. |

+|-|-|

+> [!IMPORTANT]

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../key-vault/general/overview.md). See the Cognitive Services [security](../cognitive-services-security.md) article for more information.

+> [!IMPORTANT]

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../key-vault/general/overview.md). See the Cognitive Services [security](../cognitive-services-security.md) article for more information.

+> [!IMPORTANT]

+> Remember to remove the keys from your code when you're done, and never post them publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../key-vault/general/overview.md). See the Cognitive Services [security](../cognitive-services-security.md) article for more information.

+> **NOTE**

+> If you cannot get properly the log by using Application Insight, please confirm the Application Insights settings on the App Service resource.

+> Open App Service resource and go to Application Insights. And then please check whether it is Enabled or Disabled. If it is disabled, please enable it and then apply there.

+UPS pronunciations consist of a string of UPS phones, each separated by whitespace. UPS phone labels are all defined using ASCII character strings.

+You must provide the full locale with dash (`-`) separator, but language identification only uses one locale per base language. Do not include multiple locales (e.g., "en-US" and "en-GB") for the same language.

+Speech Translation supports different languages for speech-to-speech and speech-to-text translation. The available target languages depend on whether the translation target is speech or text.

+To set the input speech recognition language, specify the full locale with a dash (`-`) separator. See the [speech-to-text language table](#speech-to-text) above. The default language is `en-US` if you don't specify a language.

+To set the translation target language, with few exceptions you only specify the language code that precedes the locale dash (`-`) separator. For example, use `es` for Spanish (Spain) instead of `es-ES`. See the speech translation target language table below. The default language is `en` if you don't specify a language.

+> For the code samples below, you'll hard-code your Shared Access Signature (SAS) URL where indicated. Remember to remove the SAS URL from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../../key-vault/general/overview.md). See the Cognitive Services [security](../../cognitive-services-security.md) article for more information.

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../key-vault/general/overview.md). See the Cognitive Services [security](../cognitive-services-security.md) article for more information.

+> [!IMPORTANT]

+> Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like [Azure Key Vault](../../key-vault/general/overview.md). See the Cognitive Services [security](../cognitive-services-security.md) article for more information.

+- How to send emails with custom verified domains? [Add custom domains](../../quickstarts/email/add-custom-verified-domains.md)

+- How to send emails with Azure Managed Domains? [Add Azure Managed domains](../../quickstarts/email/add-azure-managed-domains.md)

+- How to send emails with custom verified domains? [Add custom domains](../../quickstarts/email/add-custom-verified-domains.md)

+- How to send emails with Azure Managed Domains? [Add Azure Managed domains](../../quickstarts/email/add-azure-managed-domains.md)

+- How to send emails with custom verified domains? [Add custom domains](../../quickstarts/email/add-custom-verified-domains.md)

+- How to send emails with Azure Managed Domains? [Add Azure Managed domains](../../quickstarts/email/add-azure-managed-domains.md)

+- How to send emails with custom verified domains? [Add custom domains](../../quickstarts/email/add-custom-verified-domains.md)

+- How to send emails with Azure Managed Domains? [Add Azure Managed domains](../../quickstarts/email/add-azure-managed-domains.md)

+- How to send emails with custom verified domains? [Add custom domains](../../quickstarts/email/add-custom-verified-domains.md)

+- How to send emails with Azure Managed Domains? [Add Azure Managed domains](../../quickstarts/email/add-azure-managed-domains.md)

+| Operation / Route | Description |

+| -- | - |

+| CreateIdentity | Creates an identity representing a single user. |

+| DeleteIdentity | Deletes an identity. |

+| CreateToken | Creates an access token. |

+| RevokeToken | Revokes all access tokens created for an identity before a time given. |

+| ExchangeTeamsUserAccessToken | Exchange an Azure Active Directory (Azure AD) access token of a Teams user for a new Communication Identity access token with a matching expiration time.|

+1. If the authentication is successful and you remain in the https://teams.microsoft.com/ domain, then your Teams License is eligible. If authentication fails or you're redirected to the https://teams.live.com/v2/ domain, then your Teams License isn't eligible to use Azure Communication Services support for Teams users.

+- [Service limits](service-limits.md)

+When Azure Managed Domain is provisioned to send mail, it has default Mail from address as donotreply@xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.azurecomm.net and the FROM display name would be the same. You'll able to configure and change the Mail from address and FROM display name to more user friendly value.

+| `external` | The ingress IP and fully qualified domain name (FQDN) can either be accessible externally from the internet or a VNET, or internally within the app environment only. | `true` for external visibility from the internet or a VNET, `false` for internal visibility within app environment only (default) | Yes |

+The secret value is mapped to the secret name declared at the application level as described in the [defining secrets](#defining-secrets) section. The `passwordSecretRef` and `secretref` parameters are used to reference the secret names as environment variables at the container level. The `passwordSecretRef` provides a descriptive parameter name for secrets containing passwords.

+The following example shows an application that declares a connection string at the application level and is used throughout the configuration via `secretref`.

+Here, the environment variable named `connection-string` gets its value from the application-level `queue-connection-string` secret by using `secretref`.

+Here, the environment variable named `connection-string` gets its value from the application-level `queue-connection-string` secret by using `secretref`.

+| [A/B Testing your ASP.NET Core apps using Azure Container Apps](https://github.com/Azure-Samples/dotNET-Frontend-AB-Testing-on-Azure-Container-Apps)<br /> | Shows how to use Azure App Configuration, ASP.NET Core Feature Flags, and Azure Container Apps revisions together to gradually release features or perform A/B tests. |

+| [gRPC with ASP.NET Core on Azure Container Apps](https://github.com/Azure-Samples/dotNET-Workers-with-gRPC-messaging-on-Azure-Container-Apps) | This repository contains a simple scenario built to demonstrate how ASP.NET Core 6.0 can be used to build a cloud-native application hosted in Azure Container Apps that uses gRPC request/response transmission from Worker microservices. The gRPC service simultaneously streams sensor data to a Blazor server frontend, so you can watch the data be charted in real-time. |

+| [Deploy an Orleans Cluster to Container Apps](https://github.com/Azure-Samples/Orleans-Cluster-on-Azure-Container-Apps) | An end-to-end sample and tutorial for getting a Microsoft Orleans cluster running on Azure Container Apps. Worker microservices rapidly transmit data to a back-end Orleans cluster for monitoring and storage, emulating thousands of physical devices in the field. |

+| [Deploy a shopping cart Orleans app to Container Apps](https://github.com/Azure-Samples/orleans-blazor-server-shopping-cart-on-container-apps) | An end-to-end example shopping cart app built in ASP.NET Core Blazor Server with Orleans deployed to Azure Container Apps. |

+On the Azure portal homepage, select **Create a resource**.

+Select **Containers** > **Container Instances**.

+On the **Basics** page, choose a subscription and enter the following values for **Resource group**, **Container name**, **Image source**, and **Container image**.

+* Container image: `mcr.microsoft.com/azuredocs/aci-helloworld:latest` (Linux)

+> [!NOTE]

+> For this quickstart, you use default settings to deploy the public Microsoft `aci-helloworld:latest` image. This sample Linux image packages a small web app written in Node.js that serves a static HTML page. You can also bring your own container images stored in Azure Container Registry, Docker Hub, or other registries.

+Leave the other values as their defaults, then select **Next: Networking**.

+An auto-generated hash is added as a DNS name label to your container instance's fully qualified domain name (FQDN), which prevents malicious subdomain takeover. Specify the **DNS name label scope reuse** for the FQDN. You can choose one of these options:

+* Tenant

+* Subscription

+* Resource Group

+* No reuse

+* Any reuse (This option is the least secure.)

+For this example, select **Tenant**.

+Leave all other settings as their defaults, then select **Review + create**.

+When deployment starts, a notification appears that indicates the deployment is in progress. Another notification is displayed when the container group has been deployed.

+Open the overview for the container group by navigating to **Resource Groups** > **myresourcegroup** > **mycontainer**. Make a note of the **FQDN** of the container instance and its **Status**.

+To view the container's logs, under **Settings**, select **Containers** > **Logs**. You should see the HTTP GET request generated when you viewed the application in your browser.

+- The commands listCollections, listDatabases, killCursors, and currentOp are excluded from RBAC in the preview.

+* If you have any custom roles that reference this subscription in `AssignableScopes`, you should update those custom roles to remove the subscription. If you try to update a custom role after you cancel a subscription, you might get an error. For more information, see [Troubleshoot problems with custom roles](../../role-based-access-control/troubleshooting.md#custom-roles) and [Azure custom roles](../../role-based-access-control/custom-roles.md).

+MACC functionality in the Azure portal is only available for direct MCA and direct EA customers. A direct agreement is between Microsoft and a customer. An indirect agreement is one where a customer signs an agreement with a Microsoft partner.

+- **Enterprise Agreement**: A billing account for an Enterprise Agreement (EA) is created when your organization signs an [Enterprise Agreement](https://azure.microsoft.com/pricing/enterprise-agreement/) to use Azure. An EA enrollment can contain an unlimited number of EA accounts. However, an EA account has a subscription limit of 5000. *Regardless of a subscription's state, its included in the limit. So, deleted and disabled subscriptions are included in the limit*. If you need more subscriptions than the limit, create more EA accounts. Generally speaking, a subscription is a billing container. We recommend that you avoid creating multiple subscriptions to implement access boundaries. To separate resources with an access boundary, consider using a resource group. For more information about resource groups, see [Manage Azure resource groups by using the Azure portal](../../azure-resource-manager/management/manage-resource-groups-portal.md).

+[User Access Administrator](../../role-based-access-control/built-in-roles.md#user-access-administrator) rights are required before you can grant users or groups the Reservation Administrator and Reservation Reader roles at the tenant level. In order to get User Access Administrator rights at the tenant level, follow [Elevate access](../../role-based-access-control/elevate-access-global-admin.md) steps.

+description: Learn about using managed identities in Azure Data Factory.

+# Managed identity for Azure Data Factory

+This article helps you understand managed identity (formerly known as Managed Service Identity/MSI) and how it works in Azure Data Factory.

+- **System-assigned:** You can enable a managed identity directly on a service instance. When you allow a system-assigned managed identity during the creation of the service, an identity is created in Azure AD tied to that service instance's lifecycle. By design, only that Azure resource can use this identity to request tokens from Azure AD. So when the resource is deleted, Azure automatically deletes the identity for you.

+-

+- **User-assigned:** You may also create a managed identity as a standalone Azure resource. You can [create a user-assigned managed identity](../active-directory/managed-identities-azure-resources/how-to-manage-ua-identity-portal.md) and assign it to one or more instances of a data factory. In user-assigned managed identities, the identity is managed separately from the resources that use it.

+> System-assigned managed identity is also referred to as 'Managed identity' elsewhere in the documentation and in the Data Factory Studio for backward compatibility purpose. We will explicitly mention 'User-assigned managed identity' when referring to it.

+- When creating a data factory through **Azure portal or PowerShell**, managed identity will always be created automatically.

+- When creating data factory through **SDK**, managed identity will be created only if you specify "Identity = new FactoryIdentity()" in the factory object for creation." See example in [.NET Quickstart - Create data factory](quickstart-create-data-factory-dot-net.md#create-a-data-factory).

+- When creating a data factory through **REST API**, managed identity will be created only if you specify "identity" section in request body. See example in [REST quickstart - create data factory](quickstart-create-data-factory-rest-api.md#create-a-data-factory).

+>- If you update a service instance which already has a managed identity without specifying the "identity" parameter in the factory objects or without specifying "identity" section in REST request body, you will get an error.

+You can find the managed identity information from Azure portal -> your data factory -> Properties.

+- Managed Identity Tenant

+See [Managed Identities for Azure Resources Overview](../active-directory/managed-identities-azure-resources/overview.md) for more background on managed identities for Azure resources, on which managed identity in Azure Data Factory is based.

+| Tactic | ATT&CK Version | Description |

+| | | -- |

+| **PreAttack** | | [PreAttack](https://attack.mitre.org/matrices/enterprise/pre/) could be either an attempt to access a certain resource regardless of a malicious intent, or a failed attempt to gain access to a target system to gather information prior to exploitation. This step is usually detected as an attempt, originating from outside the network, to scan the target system and identify an entry point. |

+| **Initial Access** | V7, V9 | Initial Access is the stage where an attacker manages to get a foothold on the attacked resource. This stage is relevant for compute hosts and resources such as user accounts, certificates etc. Threat actors will often be able to control the resource after this stage. |

+| **Persistence** | V7, V9 | Persistence is any access, action, or configuration change to a system that gives a threat actor a persistent presence on that system. Threat actors will often need to maintain access to systems through interruptions such as system restarts, loss of credentials, or other failures that would require a remote access tool to restart or provide an alternate backdoor for them to regain access. |

+| **Privilege Escalation** | V7, V9 | Privilege escalation is the result of actions that allow an adversary to obtain a higher level of permissions on a system or network. Certain tools or actions require a higher level of privilege to work and are likely necessary at many points throughout an operation. User accounts with permissions to access specific systems or perform specific functions necessary for adversaries to achieve their objective may also be considered an escalation of privilege. |

+| **Defense Evasion** | V7, V9 | Defense evasion consists of techniques an adversary may use to evade detection or avoid other defenses. Sometimes these actions are the same as (or variations of) techniques in other categories that have the added benefit of subverting a particular defense or mitigation. |

+| **Discovery** | V7, V9 | Discovery consists of techniques that allow the adversary to gain knowledge about the system and internal network. When adversaries gain access to a new system, they must orient themselves to what they now have control of and what benefits operating from that system give to their current objective or overall goals during the intrusion. The operating system provides many native tools that aid in this post-compromise information-gathering phase. |

+| **LateralMovement** | V7, V9 | Lateral movement consists of techniques that enable an adversary to access and control remote systems on a network and could, but does not necessarily, include execution of tools on remote systems. The lateral movement techniques could allow an adversary to gather information from a system without needing additional tools, such as a remote access tool. An adversary can use lateral movement for many purposes, including remote Execution of tools, pivoting to additional systems, access to specific information or files, access to additional credentials, or to cause an effect. |

+| **Execution** | V7, V9 | The execution tactic represents techniques that result in execution of adversary-controlled code on a local or remote system. This tactic is often used in conjunction with lateral movement to expand access to remote systems on a network. |

+| **Collection** | V7, V9 | Collection consists of techniques used to identify and gather information, such as sensitive files, from a target network prior to exfiltration. This category also covers locations on a system or network where the adversary may look for information to exfiltrate. |

+| **Command and Control** | V7, V9 | The command and control tactic represents how adversaries communicate with systems under their control within a target network. |

+| **Exfiltration** | V7, V9 | Exfiltration refers to techniques and attributes that result or aid in the adversary removing files and information from a target network. This category also covers locations on a system or network where the adversary may look for information to exfiltrate. |

+| **Impact** | V7, V9 | Impact events primarily try to directly reduce the availability or integrity of a system, service, or network; including manipulation of data to impact a business or operational process. This would often refer to techniques such as ransomware, defacement, data manipulation, and others. |

+ Title: Microsoft Defender for Azure SQL - the benefits and features

+description: Learn how Microsoft Defender for Azure SQL protects your Azure SQL databases.

+# Overview of Microsoft Defender for Azure SQL

+Microsoft Defender for Azure SQL includes two Microsoft Defender plans that extend Microsoft Defender for Cloud's [data security package](/azure/azure-sql/database/azure-defender-for-sql) to protect your SQL estate regardless of where it is located (Azure, multicloud or hybrid environments). Microsoft Defender for Azure SQL includes functions that can be used to discover and mitigate potential database vulnerabilities. Defender for Azure SQL can also detect anomalous activities that may be an indication of a threat to your databases.

+|Release state:|Generally available (GA)|

+|Pricing:|**Microsoft Defender for Azure SQL** is billed as shown on the [pricing page](https://azure.microsoft.com/pricing/details/defender-for-cloud/)|

+|Protected SQL versions:|Azure SQL [single databases](/azure/azure-sql/database/single-database-overview) and [elastic pools](/azure/azure-sql/database/elastic-pool-overview)<br>[Azure SQL Managed Instance](/azure/azure-sql/managed-instance/sql-managed-instance-paas-overview)<br>[Azure Synapse Analytics (formerly SQL DW) dedicated SQL pool](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is.md)|

+## What does Microsoft Defender for Azure SQL protect?

+Microsoft Defender for Azure SQL databases protects:

+- [Azure SQL Database](/azure/azure-sql/database/sql-database-paas-overview)

+- [Azure SQL Managed Instance](/azure/azure-sql/managed-instance/sql-managed-instance-paas-overview)

+- [Dedicated SQL pool in Azure Synapse](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is.md)

+When you enabled **Microsoft Defender for Azure SQL**, all supported resources that exist within the subscription are protected. Future resources created on the same subscription are also be protected.

+> Microsoft Defender for Azure SQL database currently works for read-write replicas only.

+## What are the benefits of Microsoft Defender for Azure SQL?

+This plan includes functionality for identifying and mitigating potential database vulnerabilities and detecting anomalous activities that could indicate threats to your databases.

+Learn more about [vulnerability assessment for Azure SQL Database](/azure/azure-sql/database/sql-vulnerability-assessment).

+> [!TIP]

+> View the list of security alerts for SQL servers [in the alerts reference page](alerts-reference.md#alerts-sql-db-and-warehouse).

+## What kind of alerts does Microsoft Defender for Azure SQL provide?

+In this article, you learned about Microsoft Defender for Azure SQL.

+For related information, see these resources:

+- [How Microsoft Defender for Azure SQL can protect SQL servers anywhere](https://www.youtube.com/watch?v=V7RdB6RSVpc).

+- [Set up email notifications for security alerts](configure-email-notifications.md)

+- [Learn more about Microsoft Sentinel](../sentinel/index.yml)

+ Title: How to enable Microsoft Defender for SQL servers on machines

+description: Learn how to protect your Microsoft SQL servers on Azure VMs, on-premises, and in hybrid and multicloud environments with Microsoft Defender for Cloud.

+Microsoft Defender for SQL servers on machines extends the protections for your Azure-native SQL servers to fully support hybrid environments and protect SQL servers hosted in Azure, multicloud environments, and even on-premises machines:

+- [SQL Server on Virtual Machines](https://azure.microsoft.com/services/virtual-machines/sql-server/)

+- On-premises SQL servers:

+ - [Azure Arc-enabled SQL Server (preview)](/sql/sql-server/azure-arc/overview)

+

+ - [SQL Server running on Windows machines without Azure Arc](../azure-monitor/agents/agent-windows.md)

+

+- Multicloud SQL servers:

+ - [Connect your AWS accounts to Microsoft Defender for Cloud](quickstart-onboard-aws.md)

+ - [Connect your GCP project to Microsoft Defender for Cloud](quickstart-onboard-gcp.md)

+ > [!NOTE]

+ > Enable database protection for your multicloud SQL servers through the AWS connector](quickstart-onboard-aws.md?pivots=env-settings#connect-your-aws-account) or the [GCP connector](quickstart-onboard-gcp.md?pivots=env-settings#configure-the-databases-plan).

+This plan includes functionality for identifying and mitigating potential database vulnerabilities and detecting anomalous activities that could indicate threats to your databases.

+A vulnerability assessment service discovers, tracks, and helps you remediate potential database vulnerabilities. Assessment scans provide an overview of your SQL machines' security state, and details of any security findings.

+Learn more about [vulnerability assessment for Azure SQL servers on machines](defender-for-sql-on-machines-vulnerability-assessment.md).

+|Protected SQL versions:|[SQL Server versions currently supported by Microsoft](/mem/configmgr/core/plan-design/configs/support-for-sql-server-versions) in:

+<br>- [SQL on Azure virtual machines](/azure/azure-sql/virtual-machines/windows/sql-server-on-azure-vm-iaas-what-is-overview)<br>- [SQL Server on Azure Arc-enabled servers](/sql/sql-server/azure-arc/overview)<br>- On-premises SQL servers on Windows machines without Azure Arc<br>|

+<a name="auto-provision-mma"></a>

+- **SQL Server on Azure VM** - If your SQL machine is hosted on an Azure VM, you can [enable auto provisioning of the Log Analytics agent](enable-data-collection.md#auto-provision-mma). Alternatively, you can follow the manual procedure for [Onboard your Azure Stack Hub VMs](quickstart-onboard-machines.md?pivots=azure-portal#onboard-your-azure-stack-hub-vms).

+- **SQL Server on-premises** - If your SQL Server is hosted on an on-premises Windows machine without Azure Arc, you can connect the machine to Azure by either:

+ - If you're using **Microsoft Defender for Cloud's default workspace** (named ΓÇ£default workspace-\<your subscription ID>-\<region>ΓÇ¥), select the relevant **subscription**.

+Microsoft Defender for SQL alerts are available in:

+- The Defender for Cloud's alerts page

+- The machine's security page

+- The [workload protections dashboard](workload-protections-dashboard.md)

+- Through the direct link in the alert emails

+To view alerts:

+1. Select **Security alerts** from Defender for Cloud's menu and select an alert.

+ * To improve your security posture, use Defender for Cloud's recommendations for the host machine indicated in each alert to reduce the risks of future attacks.

+No. To defend a SQL Server deployment on an Azure virtual machine, or a SQL Server running on an Azure Arc-enabled machine, Defender for Cloud requires:

+### Is there a performance effect from deploying Microsoft Defender for Azure SQL on machines?

+The focus of **Microsoft Defender for SQL on machines** is obviously security. But we also care about your business and so we've prioritized performance to ensure the minimal effect on your SQL servers.

+The service has a split architecture to balance data uploading and speed with performance:

+- Some of our detectors, including an [extended events trace](/azure/azure-sql/database/xevent-db-diff-from-svr) named `SQLAdvancedThreatProtectionTraffic`, run on the machine for real-time speed advantages.

+- Other detectors run in the cloud to spare the machine from heavy computational loads.

+Lab tests of our solution showed CPU usage averaging 3% for peak slices, comparing it against benchmark loads. An analysis of our current user data shows a negligible effect on CPU and memory usage.

+Of course, performance always varies between environments, machines, and loads. The statements and numbers above are provided as a general guideline, not a guarantee for any individual deployment.

+For related information, see these resources:

+- [How Microsoft Defender for Azure SQL can protect SQL servers anywhere](https://www.youtube.com/watch?v=V7RdB6RSVpc).

+[The MDE unified solution](/microsoft-365/security/defender-endpoint/configure-server-endpoints#new-windows-server-2012-r2-and-2016-functionality-in-the-modern-unified-solution) doesn't use or require installation of the Log Analytics agent. The unified solution is automatically deployed for all Windows servers connected through Azure Arc and multicloud servers connected through the multicloud connectors, except for Windows 2012 R2 and 2016 servers on Azure that are protected by Defender for Servers Plan 2. You can choose to deploy the MDE unified solution to those machines.

+To deploy the MDE unified solution, you'll need to use the [REST API call](#enable-the-mde-unified-solution-at-scale) or the Azure portal:

+To deploy the MDE unified solution, you'll need to use the [REST API call](#enable-the-mde-unified-solution-at-scale) or the Azure portal:

+### Enable the MDE unified solution at scale

+You can also enable the MDE unified solution at scale through the supplied REST API version 2022-05-01. For full details see the [API documentation](/rest/api/securitycenter/settings/update?tabs=HTTP).

+This is an example request body for the PUT request to enable the MDE unified solution:

+URI: `https://management.microsoft.com/subscriptions/<subscriptionId>/providers/Microsoft.Security/settings&api-version=2022-05-01-preview`

+```json

+{

+ "name": "WDATP_UNIFIED_SOLUTION",

+ "type": "Microsoft.Security/settings",

+ "kind": "DataExportSettings",

+ "properties": {

+ "enabled": true

+ }

+}

+```

+The capability to exempt some accounts that donΓÇÖt use MFA isn't currently supported. There are plans to add this capability, and the information can be viewed in our [Important upcoming changes](/azure/defender-for-cloud/upcoming-changes#multiple-changes-to-identity-recommendations) page.

+- Identity recommendations aren't available for subscriptions with more than 6,000 accounts. In these cases, these types of subscriptions will be listed under Not applicable tab.

+This article explains how to enable Microsoft Defender for Cloud's database protections for the most common database types, Azure, hybrid, and multicloud environments.

+Defender for Cloud database protections lets you protect your entire database estate with attack detection and threat response for the most popular database types in Azure. Defender for Cloud provides protection for the database engines and for data types, according to their attack surface and security risks.

+Database protection includes:

+- [Microsoft Defender for Azure SQL databases](defender-for-sql-introduction.md)

+- [Microsoft Defender for SQL servers on machines](defender-for-sql-usage.md)

+- [Microsoft Defender for open-source relational databases](defender-for-databases-introduction.md)

+- [Microsoft Defender for Azure Cosmos DB](concept-defender-for-cosmos.md)

+When you turn on database protection, you enable all of these Defender plans and protect all of the supported databases in your subscription. If you only want to protect specific types of databases, you can also turn on the database plans individually and exclude specific database resource types.

+Defender for CloudΓÇÖs database protection detects unusual and potentially harmful attempts to access or exploit your databases. Advanced threat detection capabilities and [Microsoft Threat Intelligence](https://www.microsoft.com/insidetrack/microsoft-uses-threat-intelligence-to-protect-detect-and-respond-to-threats) data are used to provide contextual security alerts. The alerts include steps to mitigate the detected threats and prevent future attacks.

+- To protect SQL databases in hybrid and multicloud environments, you have to connect your AWS account or GCP project to Defender for Cloud. Defender for Cloud uses Azure Arc to communicate with your hybrid and multicloud machines. Check out the following articles for more information:

+ - [Connect your AWS accounts to Microsoft Defender for Cloud](quickstart-onboard-aws.md)

+ - [Connect your GCP project to Microsoft Defender for Cloud](quickstart-onboard-gcp.md)

+ - [Connect your on-premises and other cloud machines to Microsoft Defender for Cloud](quickstart-onboard-machines.md)

+1. Either:

+ - **Protect all database types** - Select **On** in the Databases section to protect all database types.

+ - **Protect specific database types**:

+

+ 1. Select **Select types** to see the list of Defender plans for databases.

+ :::image type="content" source="media/quickstart-enable-database-protections/select-type.png" alt-text="Screenshot showing the toggles to enable specific resource types.":::

+ 1. Select **On** for each database type that you want to protect.

+ :::image type="content" source="media/quickstart-enable-database-protections/resource-type.png" alt-text="Screenshot showing the types of resources available.":::

+ 1. Select **Continue**.

+1. Select **Save**.

+In this article, you learned how to enable Microsoft Defender for Cloud for all database types on your subscription. Next, read more about each of the resource types:

+- [Protect against the Operations Management Suite vulnerability CVE-2022-29149](#protect-against-the-operations-management-suite-vulnerability-cve-2022-29149)

+### Protect against the Operations Management Suite vulnerability CVE-2022-29149

+Operations Management Suite (OMS) is a collection of cloud-based services for managing on-premises and cloud environments from one single place. Rather than deploying and managing on-premises resources, OMS components are entirely hosted in Azure.

+Log Analytics integrated with Azure HDInsight running OMS version 13 requires a patch to remediate [CVE-2022-29149](https://nvd.nist.gov/vuln/detail/CVE-2022-29149). Review the report about this vulnerability in the [Microsoft Security Update guide](https://msrc.microsoft.com/update-guide/en-US/vulnerability/CVE-2022-29149) for information about how to identify resources that are affected by this vulnerability and remediation steps.

+If you have Defender for Servers enabled with Vulnerability Assessment, you can use [this workbook](https://github.com/Azure/Microsoft-Defender-for-Cloud/tree/main/Workbooks/OMI%20Vulnerability%20Dashboard) to identify affected resources.

+| [Multiple changes to identity recommendations](#multiple-changes-to-identity-recommendations) | September 2022 |

+### Deprecate API App policies for App Service

+**Estimated date for change:** July 2022

+We will be deprecating the following policies to corresponding policies that already exist to include API apps:

+| To be deprecated | Changing to |

+|--|--|

+|`Ensure API app has 'Client Certificates (Incoming client certificates)' set to 'On'` | `App Service apps should have 'Client Certificates (Incoming client certificates)' enabled` |

+| `Ensure that 'Python version' is the latest, if used as a part of the API app` | `App Service apps that use Python should use the latest 'Python version` |

+| `CORS should not allow every resource to access your API App` | `App Service apps should not have CORS configured to allow every resource to access your apps` |

+| `Managed identity should be used in your API App` | `App Service apps should use managed identity` |

+| `Remote debugging should be turned off for API Apps` | `App Service apps should have remote debugging turned off` |

+| `Ensure that 'PHP version' is the latest, if used as a part of the API app` | `App Service apps that use PHP should use the latest 'PHP version'`|

+| `FTPS only should be required in your API App` | `App Service apps should require FTPS only` |

+| `Ensure that 'Java version' is the latest, if used as a part of the API app` | `App Service apps that use Java should use the latest 'Java version` |

+| `Latest TLS version should be used in your API App` | `App Service apps should use the latest TLS version` |

+### Change in pricing of runtime protection for Arc-enabled Kubernetes clusters

+**Estimated date for change:** August 2022

+Runtime protection is currently a preview feature for Arc-enabled Kubernetes clusters. In August, Arc-enabled Kubernetes clusters will be charged for runtime protection. You can view pricing details on the [pricing page](https://azure.microsoft.com/pricing/details/defender-for-cloud/). Subscriptions with Kubernetes clusters already onboarded to Arc, will begin to incur charges in August.

+> If you're setting up network monitoring for Enterprise IoT systems, you can skip directly to [Add a Defender for IoT plan for Enterprise IoT networks to an Azure subscription](#add-a-defender-for-iot-plan-for-enterprise-iot-networks-to-an-azure-subscription).

+## Add a Defender for IoT plan for OT networks to an Azure subscription

+This procedure describes how to add a Defender for IoT plan for OT networks to an Azure subscription.

+**To onboard a Defender for IoT plan for OT networks:**

+1. In the Azure portal, go to **Defender for IoT** > **Pricing**.

+1. In the **Purchase** pane, define the plan:

+ - **Purchase method**. Select a monthly or annual commitment, or a [trial](how-to-manage-subscriptions.md#about-defender-for-iot-trials). Microsoft Defender for IoT provides a 30-day free trial for the first 1,000 committed devices for evaluation purposes.

+

+ For more information, see the [Microsoft Defender for IoT pricing page](https://azure.microsoft.com/pricing/details/iot-defender/).

+ - **Number of sites** (for annual commitment only). Enter the number of committed sites.

+ - **Committed devices**. If you selected a monthly or annual commitment, enter the number of assets you'll want to monitor. If you selected a trial, this section doesn't appear as you have a default of 1000 devices.

+ For example:

+ :::image type="content" source="media/how-to-manage-subscriptions/onboard-plan-2.png" alt-text="Screenshot of adding a plan for OT networks to your subscription.":::

+1. Select **I accept the terms** option, and then select **Save**.

+Your OT networks plan will be shown under the associated subscription in the **Plans** grid.

+## Add a Defender for IoT plan for Enterprise IoT networks to an Azure subscription

+Onboard your Defender for IoT plan for Enterprise IoT networks in the Defender for Endpoint portal. For more information, see [Onboard Microsoft Defender for IoT](/microsoft-365/security/defender-endpoint/enable-microsoft-defender-for-iot-integration) in the Defender for Endpoint documentation.

+Once you've onboarded a plan for Enterprise IoT networks from Defender for Endpoint, you'll see the plan in Defender for IoT in the Azure portal, under the associated subscription in the **Plans** grid, on the **Defender for IoT** > **Pricing** page.

+ If you haven't already onboarded Defender for IoT to a subscription, see [Onboard a Defender for IoT plan for OT networks](how-to-manage-subscriptions.md#onboard-a-defender-for-iot-plan-for-ot-networks).

+|**On-premises management console** | Apply a new activation file on your on-premises management console if you've [modified the number of committed devices](how-to-manage-subscriptions.md#edit-a-plan-for-ot-networks) in your subscription. |

+Your Defender for IoT deployment is managed through a Microsoft Defender for IoT plan on your Azure subscription.

+- **For OT networks**, onboard, edit, and cancel Defender for IoT plans from Defender for IoT in the Azure portal.

+- **For Enterprise IoT networks**, onboard and cancel Defender for IoT plans in Microsoft Defender for Endpoint.

+Azure **Security admin**, **Subscription owners** and **Subscription contributors** can onboard, update, and remove Defender for IoT plans. For more information on user permissions, see [Defender for IoT user permissions](getting-started.md#permissions).

+After you've set up your network sensor and have full visibility into all devices, you can [Edit a plan](#edit-a-plan-for-ot-networks) to update the number of committed devices as needed.

+## Onboard a Defender for IoT plan for OT networks

+This procedure describes how to add a Defender for IoT plan for OT networks to an Azure subscription.

+**To onboard a Defender for IoT plan for OT networks:**

+1. In the Azure portal, go to **Defender for IoT** > **Pricing**.

+1. In the **Purchase** pane, define the plan:

+ - **Purchase method**. Select a monthly or annual commitment, or a [trial](#about-defender-for-iot-trials). Microsoft Defender for IoT provides a 30-day free trial for the first 1,000 committed devices for evaluation purposes.

+ - **Subscription**. Select the subscription where you would like to add a plan.

+ - **Number of sites** (for annual commitment only). Enter the number of committed sites.

+ - **Committed devices**. If you selected a monthly or annual commitment, enter the number of assets you'll want to monitor. If you selected a trial, this section doesn't appear as you have a default of 1000 devices.

+ For example:

+ :::image type="content" source="media/how-to-manage-subscriptions/onboard-plan-2.png" alt-text="Screenshot of adding a plan for OT networks to your subscription.":::

+1. Select the **I accept the terms** option, and then select **Save**.

+Your OT networks plan will be shown under the associated subscription in the **Plans** grid.

+## Onboard a Defender for IoT plan for Enterprise IoT networks

+Onboard your Defender for IoT plan for Enterprise IoT networks in the Defender for Endpoint portal. For more information, see [Onboard Microsoft Defender for IoT](/microsoft-365/security/defender-endpoint/enable-microsoft-defender-for-iot-integration) in the Defender for Endpoint documentation.

+Once you've onboarded a plan for Enterprise IoT networks from Defender for Endpoint, you'll see the plan in Defender for IoT in the Azure portal, under the associated subscription in the **Plans** grid, on the **Defender for IoT** > **Pricing** page.

+## Edit a plan for OT networks

+You can make changes to your OT networks plan, such as to change your plan commitment, update the number of committed devices, or committed sites.

+For example, you may have more devices that require monitoring if you're increasing existing site coverage, have discovered more devices than expected, or there are network changes such as adding switches. If the actual number of devices exceeds the number of committed devices on your plan, you'll see a warning on the **Pricing** page, and will need to adjust the number of committed devices on your plan accordingly.

+1. In the Azure portal, go to **Defender for IoT** > **Pricing**.

+ - Change your purchase method

+ - Update the number of committed devices

+ - Update the number of sites (annual commitments only)

+1. Select the **I accept the terms** option, and then select **Save**.

+1. In the Azure portal, go to **Defender for IoT** > **Pricing**.

+1. In the plan cancellation dialog, confirm that you've removed all associated sensors, and then select **Confirm cancellation** to cancel the Defender for IoT plan from the subscription.

+> [!NOTE]

+> To remove Enterprise IoT only from your plan, cancel your plan from Microsoft Defender for Endpoint. For more information, see the [Defender for Endpoint documentation](/microsoft-365/security/defender-endpoint/enable-microsoft-defender-for-iot-integration#cancel-your-defender-for-iot-plan).

+1. Onboard a new plan to the new subscription you want to use. For more information, see:

+ [Onboard a plan for OT networks](#onboard-a-defender-for-iot-plan-for-ot-networks) in the Azure portal

+ [Onboard a plan for Enterprise IoT networks](#onboard-a-defender-for-iot-plan-for-enterprise-iot-networks) in Defender for Endpoint

+## Microsoft Defender for Endpoint integration

+Once youΓÇÖve onboarded a plan and set up your sensor, your device data integrates automatically with Microsoft Defender for Endpoint. Discovered devices appear in both the Defender for IoT and Defender for Endpoint portals. Use this integration to extend security analytics capabilities for your Enterprise IoT devices and providing complete coverage.

+In Defender for Endpoint, you can view discovered IoT devices and related alerts, vulnerabilities, and recommendations. For more information, see:

+- [Microsoft Defender for IoT integration](/microsoft-365/security/defender-endpoint/enable-microsoft-defender-for-iot-integration)

+- [Defender for Endpoint device inventory](/microsoft-365/security/defender-endpoint/machines-view-overview)

+- [View and organize the Microsoft Defender for Endpoint Alerts queue](/microsoft-365/security/defender-endpoint/alerts-queue)

+- [Vulnerabilities in my organization](/microsoft-365/security/defender-vulnerability-management/)

+- [Security recommendations](/microsoft-365/security/defender-vulnerability-management/tvm-security-recommendation)

+## Prerequisites

+Before you start, make sure that you have:

+- Added a Defender for IoT plan for Enterprise IoT networks to your Azure subscription from the Microsoft Defender for Endpoint portal.

+To onboard a plan, see [Onboard with Microsoft Defender for IoT](/microsoft-365/security/defender-endpoint/enable-microsoft-defender-for-iot-integration).

+ - [Check Point: Configure Microsoft Azure Virtual WAN integration](https://www.checkpoint.com/cloudguard/microsoft-azure-security/wan).

+- [Tutorial: Secure your cloud network with Azure Firewall Manager using the Azure portal](secure-cloud-network.md)

+ Title: Use Azure Firewall to protect Azure Kubernetes Service (AKS) clusters

+description: Learn how to use Azure Firewall to protect Azure Kubernetes Service (AKS) clusters

+# Use Azure Firewall to protect Azure Kubernetes Service (AKS) clusters

+This article shows you how you can protect Azure Kubernetes Service (AKS) clusters by using Azure Firewall to secure outbound and inbound traffic.

+## Background

+Azure Kubernetes Service (AKS) offers a managed Kubernetes cluster on Azure. For more information, see [Azure Kubernetes Service](../aks/intro-kubernetes.md).

+Despite AKS being a fully managed solution, it does not offer a built-in solution to secure ingress and egress traffic between the cluster and external networks. Azure Firewall offers a solution to this.

+AKS clusters are deployed on a virtual network. This network can be managed (created by AKS) or custom (pre-configured by the user beforehand). In either case, the cluster has outbound dependencies on services outside of that virtual network (the service has no inbound dependencies). For management and operational purposes, nodes in an AKS cluster need to access certain ports and fully qualified domain names (FQDNs) describing these outbound dependencies. This is required for various functions including, but not limited to, the nodes that communicate with the Kubernetes API server. They download and install core Kubernetes cluster components and node security updates, or pull base system container images from Microsoft Container Registry (MCR), and so on. These outbound dependencies are almost entirely defined with FQDNs, which don't have static addresses behind them. The lack of static addresses means that Network Security Groups can't be used to lock down outbound traffic from an AKS cluster. For this reason, by default, AKS clusters have unrestricted outbound (egress) Internet access. This level of network access allows nodes and services you run to access external resources as needed.

+

+However, in a production environment, communications with a Kubernetes cluster should be protected to prevent against data exfiltration along with other vulnerabilities. All incoming and outgoing network traffic must be monitored and controlled based on a set of security rules. If you want to do this, you will have to restrict egress traffic, but a limited number of ports and addresses must remain accessible to maintain healthy cluster maintenance tasks and satisfy those outbound dependencies previously mentioned.

+

+The simplest solution uses a firewall device that can control outbound traffic based on domain names. A firewall typically establishes a barrier between a trusted network and an untrusted network, such as the Internet. Azure Firewall, for example, can restrict outbound HTTP and HTTPS traffic based on the FQDN of the destination, giving you fine-grained egress traffic control, but at the same time allows you to provide access to the FQDNs encompassing an AKS clusterΓÇÖs outbound dependencies (something that NSGs cannot do). Likewise, you can control ingress traffic and improve security by enabling threat intelligence-based filtering on an Azure Firewall deployed to a shared perimeter network. This filtering can provide alerts, and deny traffic to and from known malicious IP addresses and domains.

+See the following video by Abhinav Sriram for a quick overview on how this works in practice on a sample environment:

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE529Qc]

+You can download a zip file from the [Microsoft Download Center](https://download.microsoft.com/download/0/1/3/0131e87a-c862-45f8-8ee6-31fa103a03ff/aks-azfw-protection-setup.zip) that contains a bash script file and a yaml file to automatically configure the sample environment used in the video. It configures Azure Firewall to protect both ingress and egress traffic. The following guides walk through each step of the script in more detail so you can set up a custom configuration.

+The following diagram shows the sample environment from the video that the script and guide configure:

+There is one difference between the script and the following guide. The script uses managed identities, but the guide uses a service principal. This shows you two different ways to create an identity to manage and create cluster resources.

+## Restrict egress traffic using Azure Firewall

+### Set configuration via environment variables

+Define a set of environment variables to be used in resource creations.

+```bash

+PREFIX="aks-egress"

+RG="${PREFIX}-rg"

+LOC="eastus"

+PLUGIN=azure

+AKSNAME="${PREFIX}"

+VNET_NAME="${PREFIX}-vnet"

+AKSSUBNET_NAME="aks-subnet"

+# DO NOT CHANGE FWSUBNET_NAME - This is currently a requirement for Azure Firewall.

+FWSUBNET_NAME="AzureFirewallSubnet"

+FWNAME="${PREFIX}-fw"

+FWPUBLICIP_NAME="${PREFIX}-fwpublicip"

+FWIPCONFIG_NAME="${PREFIX}-fwconfig"

+FWROUTE_TABLE_NAME="${PREFIX}-fwrt"

+FWROUTE_NAME="${PREFIX}-fwrn"

+FWROUTE_NAME_INTERNET="${PREFIX}-fwinternet"

+```

+### Create a virtual network with multiple subnets

+Provision a virtual network with two separate subnets, one for the cluster, one for the firewall. Optionally you could also create one for internal service ingress.

+

+Create a resource group to hold all of the resources.

+```azurecli

+# Create Resource Group

+az group create --name $RG --location $LOC

+```

+Create a virtual network with two subnets to host the AKS cluster and the Azure Firewall. Each will have their own subnet. Let's start with the AKS network.

+```

+# Dedicated virtual network with AKS subnet

+az network vnet create \

+ --resource-group $RG \

+ --name $VNET_NAME \

+ --location $LOC \

+ --address-prefixes 10.42.0.0/16 \

+ --subnet-name $AKSSUBNET_NAME \

+ --subnet-prefix 10.42.1.0/24

+# Dedicated subnet for Azure Firewall (Firewall name cannot be changed)

+az network vnet subnet create \

+ --resource-group $RG \

+ --vnet-name $VNET_NAME \

+ --name $FWSUBNET_NAME \

+ --address-prefix 10.42.2.0/24

+```

+### Create and set up an Azure Firewall with a UDR

+Azure Firewall inbound and outbound rules must be configured. The main purpose of the firewall is to enable organizations to configure granular ingress and egress traffic rules into and out of the AKS Cluster.

+

+> [!IMPORTANT]

+> If your cluster or application creates a large number of outbound connections directed to the same or small subset of destinations, you might require more firewall frontend IPs to avoid maxing out the ports per frontend IP.

+> For more information on how to create an Azure firewall with multiple IPs, see [**here**](../firewall/quick-create-multiple-ip-template.md)

+Create a standard SKU public IP resource that will be used as the Azure Firewall frontend address.

+```azurecli

+az network public-ip create -g $RG -n $FWPUBLICIP_NAME -l $LOC --sku "Standard"

+```

+Register the preview cli-extension to create an Azure Firewall.

+```azurecli

+# Install Azure Firewall preview CLI extension

+az extension add --name azure-firewall

+# Deploy Azure Firewall

+az network firewall create -g $RG -n $FWNAME -l $LOC --enable-dns-proxy true

+```

+The IP address created earlier can now be assigned to the firewall frontend.

+> [!NOTE]

+> Set up of the public IP address to the Azure Firewall may take a few minutes.

+> To leverage FQDN on network rules we need DNS proxy enabled, when enabled the firewall will listen on port 53 and will forward DNS requests to the DNS server specified above. This will allow the firewall to translate that FQDN automatically.

+```azurecli

+# Configure Firewall IP Config

+az network firewall ip-config create -g $RG -f $FWNAME -n $FWIPCONFIG_NAME --public-ip-address $FWPUBLICIP_NAME --vnet-name $VNET_NAME

+```

+When the previous command has succeeded, save the firewall frontend IP address for configuration later.

+```azurecli

+# Capture Firewall IP Address for Later Use

+FWPUBLIC_IP=$(az network public-ip show -g $RG -n $FWPUBLICIP_NAME --query "ipAddress" -o tsv)

+FWPRIVATE_IP=$(az network firewall show -g $RG -n $FWNAME --query "ipConfigurations[0].privateIpAddress" -o tsv)

+```

+> [!NOTE]

+> If you use secure access to the AKS API server with [authorized IP address ranges](../aks/api-server-authorized-ip-ranges.md), you need to add the firewall public IP into the authorized IP range.

+### Create a UDR with a hop to Azure Firewall

+Azure automatically routes traffic between Azure subnets, virtual networks, and on-premises networks. If you want to change any of Azure's default routing, you do so by creating a route table.

+Create an empty route table to be associated with a given subnet. The route table will define the next hop as the Azure Firewall created above. Each subnet can have zero or one route table associated to it.

+```azurecli

+# Create UDR and add a route for Azure Firewall

+az network route-table create -g $RG -l $LOC --name $FWROUTE_TABLE_NAME

+az network route-table route create -g $RG --name $FWROUTE_NAME --route-table-name $FWROUTE_TABLE_NAME --address-prefix 0.0.0.0/0 --next-hop-type VirtualAppliance --next-hop-ip-address $FWPRIVATE_IP

+az network route-table route create -g $RG --name $FWROUTE_NAME_INTERNET --route-table-name $FWROUTE_TABLE_NAME --address-prefix $FWPUBLIC_IP/32 --next-hop-type Internet

+```

+See [virtual network route table documentation](../virtual-network/virtual-networks-udr-overview.md#user-defined) about how you can override Azure's default system routes or add additional routes to a subnet's route table.

+### Adding firewall rules

+> [!NOTE]

+> For applications outside of the kube-system or gatekeeper-system namespaces that needs to talk to the API server, an additional network rule to allow TCP communication to port 443 for the API server IP in addition to adding application rule for fqdn-tag AzureKubernetesService is required.

+Below are three network rules you can use to configure on your firewall, you may need to adapt these rules based on your deployment. The first rule allows access to port 9000 via TCP. The second rule allows access to port 1194 and 123 via UDP. Both these rules will only allow traffic destined to the Azure Region CIDR that we're using, in this case East US.

+Finally, we'll add a third network rule opening port 123 to `ntp.ubuntu.com` FQDN via UDP (adding an FQDN as a network rule is one of the specific features of Azure Firewall, and you'll need to adapt it when using your own options).

+After setting the network rules, we'll also add an application rule using the `AzureKubernetesService` that covers all needed FQDNs accessible through TCP port 443 and port 80.

+```

+# Add FW Network Rules

+az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'apiudp' --protocols 'UDP' --source-addresses '*' --destination-addresses "AzureCloud.$LOC" --destination-ports 1194 --action allow --priority 100

+az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'apitcp' --protocols 'TCP' --source-addresses '*' --destination-addresses "AzureCloud.$LOC" --destination-ports 9000

+az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'time' --protocols 'UDP' --source-addresses '*' --destination-fqdns 'ntp.ubuntu.com' --destination-ports 123

+# Add FW Application Rules

+az network firewall application-rule create -g $RG -f $FWNAME --collection-name 'aksfwar' -n 'fqdn' --source-addresses '*' --protocols 'http=80' 'https=443' --fqdn-tags "AzureKubernetesService" --action allow --priority 100

+```

+See [Azure Firewall documentation](overview.md) to learn more about the Azure Firewall service.

+### Associate the route table to AKS

+To associate the cluster with the firewall, the dedicated subnet for the cluster's subnet must reference the route table created above. Association can be done by issuing a command to the virtual network holding both the cluster and firewall to update the route table of the cluster's subnet.

+```azurecli

+# Associate route table with next hop to Firewall to the AKS subnet

+az network vnet subnet update -g $RG --vnet-name $VNET_NAME --name $AKSSUBNET_NAME --route-table $FWROUTE_TABLE_NAME

+```

+### Deploy AKS with outbound type of UDR to the existing network

+Now an AKS cluster can be deployed into the existing virtual network. We'll also use [outbound type `userDefinedRouting`](../aks/egress-outboundtype.md), this feature ensures any outbound traffic will be forced through the firewall and no other egress paths will exist (by default the Load Balancer outbound type could be used).

+

+The target subnet to be deployed into is defined with the environment variable, `$SUBNETID`. We didn't define the `$SUBNETID` variable in the previous steps. To set the value for the subnet ID, you can use the following command:

+```azurecli

+SUBNETID=$(az network vnet subnet show -g $RG --vnet-name $VNET_NAME --name $AKSSUBNET_NAME --query id -o tsv)

+```

+You'll define the outbound type to use the UDR that already exists on the subnet. This configuration will enable AKS to skip the setup and IP provisioning for the load balancer.

+> [!IMPORTANT]

+> For more information on outbound type UDR including limitations, see [**egress outbound type UDR**](../aks/egress-outboundtype.md#limitations).

+> [!TIP]

+> Additional features can be added to the cluster deployment such as [**Private Cluster**](../aks/private-clusters.md).

+>

+> The AKS feature for [**API server authorized IP ranges**](../aks/api-server-authorized-ip-ranges.md) can be added to limit API server access to only the firewall's public endpoint. The authorized IP ranges feature is denoted in the diagram as optional. When enabling the authorized IP range feature to limit API server access, your developer tools must use a jumpbox from the firewall's virtual network or you must add all developer endpoints to the authorized IP range.

+```azurecli

+az aks create -g $RG -n $AKSNAME -l $LOC \

+ --node-count 3 \

+ --network-plugin $PLUGIN \

+ --outbound-type userDefinedRouting \

+ --vnet-subnet-id $SUBNETID \

+ --api-server-authorized-ip-ranges $FWPUBLIC_IP

+```

+> [!NOTE]

+> For creating and using your own VNet and route table where the resources are outside of the worker node resource group, the CLI will add the role assignment automatically. If you are using an ARM template or other client, you need to use the Principal ID of the cluster managed identity to perform a [role assignment.][add role to identity]

+>

+> If you are not using the CLI but using your own VNet or route table which are outside of the worker node resource group, it's recommended to use [user-assigned control plane identity][Bring your own control plane managed identity]. For system-assigned control plane identity, we cannot get the identity ID before creating cluster, which causes delay for role assignment to take effect.

+### Enable developer access to the API server

+If you used authorized IP ranges for the cluster on the previous step, you must add your developer tooling IP addresses to the AKS cluster list of approved IP ranges in order to access the API server from there. Another option is to configure a jumpbox with the needed tooling inside a separate subnet in the Firewall's virtual network.

+Add another IP address to the approved ranges with the following command

+```bash

+# Retrieve your IP address

+CURRENT_IP=$(dig @resolver1.opendns.com ANY myip.opendns.com +short)

+# Add to AKS approved list

+az aks update -g $RG -n $AKSNAME --api-server-authorized-ip-ranges $CURRENT_IP/32

+```

+Use the [az aks get-credentials][az-aks-get-credentials] command to configure `kubectl` to connect to your newly created Kubernetes cluster.

+```azurecli

+az aks get-credentials -g $RG -n $AKSNAME

+```

+## Restrict ingress traffic using Azure Firewall

+You can now start exposing services and deploying applications to this cluster. In this example, we'll expose a public service, but you may also choose to expose an internal service via [internal load balancer](../aks/internal-lb.md).

+

+Deploy the Azure voting app application by copying the yaml below to a file named `example.yaml`.

+```yaml

+# voting-storage-deployment.yaml

+apiVersion: apps/v1

+kind: Deployment

+metadata:

+ name: voting-storage

+spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-storage

+ template:

+ metadata:

+ labels:

+ app: voting-storage

+ spec:

+ containers:

+ - name: voting-storage

+ image: mcr.microsoft.com/aks/samples/voting/storage:2.0

+ args: ["--ignore-db-dir=lost+found"]

+ resources:

+ requests:

+ cpu: 100m

+ memory: 128Mi

+ limits:

+ cpu: 250m

+ memory: 256Mi

+ ports:

+ - containerPort: 3306

+ name: mysql

+ volumeMounts:

+ - name: mysql-persistent-storage

+ mountPath: /var/lib/mysql

+ env:

+ - name: MYSQL_ROOT_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_ROOT_PASSWORD

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+ volumes:

+ - name: mysql-persistent-storage

+ persistentVolumeClaim:

+ claimName: mysql-pv-claim

+# voting-storage-secret.yaml

+apiVersion: v1

+kind: Secret

+metadata:

+ name: voting-storage-secret

+type: Opaque

+data:

+ MYSQL_USER: ZGJ1c2Vy

+ MYSQL_PASSWORD: UGFzc3dvcmQxMg==

+ MYSQL_DATABASE: YXp1cmV2b3Rl

+ MYSQL_ROOT_PASSWORD: UGFzc3dvcmQxMg==

+# voting-storage-pv-claim.yaml

+apiVersion: v1

+kind: PersistentVolumeClaim

+metadata:

+ name: mysql-pv-claim

+spec:

+ accessModes:

+ - ReadWriteOnce

+ resources:

+ requests:

+ storage: 1Gi

+# voting-storage-service.yaml

+apiVersion: v1

+kind: Service

+metadata:

+ name: voting-storage

+ labels:

+ app: voting-storage

+spec:

+ ports:

+ - port: 3306

+ name: mysql

+ selector:

+ app: voting-storage

+# voting-app-deployment.yaml

+apiVersion: apps/v1

+kind: Deployment

+metadata:

+ name: voting-app

+spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-app

+ template:

+ metadata:

+ labels:

+ app: voting-app

+ spec:

+ containers:

+ - name: voting-app

+ image: mcr.microsoft.com/aks/samples/voting/app:2.0

+ imagePullPolicy: Always

+ ports:

+ - containerPort: 8080

+ name: http

+ env:

+ - name: MYSQL_HOST

+ value: "voting-storage"

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+ - name: ANALYTICS_HOST

+ value: "voting-analytics"

+# voting-app-service.yaml

+apiVersion: v1

+kind: Service

+metadata:

+ name: voting-app

+ labels:

+ app: voting-app

+spec:

+ type: LoadBalancer

+ ports:

+ - port: 80

+ targetPort: 8080

+ name: http

+ selector:

+ app: voting-app

+# voting-analytics-deployment.yaml

+apiVersion: apps/v1

+kind: Deployment

+metadata:

+ name: voting-analytics

+spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-analytics

+ version: "2.0"

+ template:

+ metadata:

+ labels:

+ app: voting-analytics

+ version: "2.0"

+ spec:

+ containers:

+ - name: voting-analytics

+ image: mcr.microsoft.com/aks/samples/voting/analytics:2.0

+ imagePullPolicy: Always

+ ports:

+ - containerPort: 8080

+ name: http

+ env:

+ - name: MYSQL_HOST

+ value: "voting-storage"

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+# voting-analytics-service.yaml

+apiVersion: v1

+kind: Service

+metadata:

+ name: voting-analytics

+ labels:

+ app: voting-analytics

+spec:

+ ports:

+ - port: 8080

+ name: http

+ selector:

+ app: voting-analytics

+```

+Deploy the service by running:

+```bash

+kubectl apply -f example.yaml

+```

+### Add a DNAT rule to Azure Firewall

+> [!IMPORTANT]

+> When you use Azure Firewall to restrict egress traffic and create a user-defined route (UDR) to force all egress traffic, make sure you create an appropriate DNAT rule in Firewall to correctly allow ingress traffic. Using Azure Firewall with a UDR breaks the ingress setup due to asymmetric routing. (The issue occurs if the AKS subnet has a default route that goes to the firewall's private IP address, but you're using a public load balancer - ingress or Kubernetes service of type: LoadBalancer). In this case, the incoming load balancer traffic is received via its public IP address, but the return path goes through the firewall's private IP address. Because the firewall is stateful, it drops the returning packet because the firewall isn't aware of an established session. To learn how to integrate Azure Firewall with your ingress or service load balancer, see [Integrate Azure Firewall with Azure Standard Load Balancer](../firewall/integrate-lb.md).

+To configure inbound connectivity, a DNAT rule must be written to the Azure Firewall. To test connectivity to your cluster, a rule is defined for the firewall frontend public IP address to route to the internal IP exposed by the internal service.

+The destination address can be customized as it's the port on the firewall to be accessed. The translated address must be the IP address of the internal load balancer. The translated port must be the exposed port for your Kubernetes service.

+You'll need to specify the internal IP address assigned to the load balancer created by the Kubernetes service. Retrieve the address by running:

+```bash

+kubectl get services

+```

+The IP address needed will be listed in the EXTERNAL-IP column, similar to the following.

+```bash

+NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

+kubernetes ClusterIP 10.41.0.1 <none> 443/TCP 10h

+voting-analytics ClusterIP 10.41.88.129 <none> 8080/TCP 9m

+voting-app LoadBalancer 10.41.185.82 20.39.18.6 80:32718/TCP 9m

+voting-storage ClusterIP 10.41.221.201 <none> 3306/TCP 9m

+```

+Get the service IP by running:

+```bash

+SERVICE_IP=$(kubectl get svc voting-app -o jsonpath='{.status.loadBalancer.ingress[*].ip}')

+```

+Add the NAT rule by running:

+```azurecli

+az network firewall nat-rule create --collection-name exampleset --destination-addresses $FWPUBLIC_IP --destination-ports 80 --firewall-name $FWNAME --name inboundrule --protocols Any --resource-group $RG --source-addresses '*' --translated-port 80 --action Dnat --priority 100 --translated-address $SERVICE_IP

+```

+### Validate connectivity

+Navigate to the Azure Firewall frontend IP address in a browser to validate connectivity.

+You should see the AKS voting app. In this example, the Firewall public IP was `52.253.228.132`.

+

+## Clean up resources

+To clean up Azure resources, delete the AKS resource group.

+```azurecli

+az group delete -g $RG

+```

+description: An introduction to HDInsight, and the Apache Hadoop and Apache Spark technology stack and components, including Kafka, Hive, and HBase for big data analysis.

+Azure HDInsight is a managed, full-spectrum, open-source analytics service in the cloud for enterprises. With HDInsight, you can use open-source frameworks such as Hadoop, Apache Spark, Apache Hive, LLAP, Apache Kafka, and more, in your Azure environment.

+Azure HDInsight is a cloud distribution of Hadoop components. Azure HDInsight makes it easy, fast, and cost-effective to process massive amounts of data in a customizable environment. You can use the most popular open-source frameworks such as Hadoop, Spark, Hive, LLAP, Kafka and more. With these frameworks, you can enable a broad range of scenarios such as extract, transform, and load (ETL), data warehousing, machine learning, and IoT.

+|Cloud native | Azure HDInsight enables you to create optimized clusters for Hadoop, Spark, [Interactive query (LLAP)](./interactive-query/apache-interactive-query-get-started.md), Kafka, HBase on Azure. HDInsight also provides an end-to-end SLA on all your production workloads. |

+|Productivity | Azure HDInsight enables you to use rich productive tools for Hadoop and Spark with your preferred development environments. These development environments include Visual Studio, VSCode, Eclipse, and IntelliJ for Scala, Python, Java, and .NET support. |

+Azure HDInsight enables you to create clusters with open-source frameworks such as Hadoop, Spark, Hive, LLAP, Kafka, and HBase. These clusters, by default, come with other open-source components that are included on the cluster such as Apache Ambari, Avro, Apache Hive3, HCatalog, Apache Hadoop MapReduce, Apache Hadoop YARN, Apache Phoenix, Apache Pig, Apache Sqoop, Apache Tez, Apache Oozie, and Apache ZooKeeper.

+Spark, Hadoop, and LLAP don't store customer data, so these services automatically satisfy in-region data residency requirements including those specified in the [Trust Center](https://azuredatacentermap.azurewebsites.net/).

+> [!IMPORTANT]

+> IoT Central expects to receive UTF-8 encoded JSON data.

+To learn more about the DTDL telemetry naming rules, see [DTDL > Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry). You can't start a telemetry name using the `_` character.

+Don't create telemetry types with the following names. IoT Central uses these reserved names internally. If you try to use these names, IoT Central will ignore your data:

+* `EventEnqueuedUtcTime`

+* `EventProcessedUtcTime`

+* `PartitionId`

+* `EventHub`

+* `User`

+* `$metadata`

+* `$version`

+To learn more about the DTDL property naming rules, see [DTDL > Property](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#property). You can't start a property name using the `_` character.

+To learn more about the DTDL command naming rules, see [DTDL > Command](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#command). You can't start a command name using the `_` character.

+| Node.js|[npm](https://www.npmjs.com/package/azure-iot-provisioning-device) |[GitHub](https://github.com/Azure/azure-iot-sdk-node/tree/main/provisioning)|[Samples](https://github.com/Azure/azure-iot-sdk-node/tree/main/provisioning/device/samples)|[Quickstart](./quick-create-simulated-device-x509.md?pivots=programming-language-nodejs&tabs=windows)|[Reference](/javascript/api/azure-iot-provisioning-device) |

+| Node.js|[npm](https://www.npmjs.com/package/azure-iot-provisioning-service)|[GitHub](https://github.com/Azure/azure-iot-sdk-node/tree/main/provisioning)|[Samples](https://github.com/Azure/azure-iot-sdk-node/tree/main/provisioning/service/samples)|[Quickstart](./quick-enroll-device-tpm.md?pivots=programming-language-nodejs&tabs=symmetrickey)|[Reference](/javascript/api/azure-iot-provisioning-service) |

+| Node.js|[npm](https://www.npmjs.com/package/@azure/arm-deviceprovisioningservices)|[GitHub](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/deviceprovisioningservices/arm-deviceprovisioningservices)|[Reference](/javascript/api/overview/azure/arm-deviceprovisioningservices-readme) |

+ > To SSH into this VM after setup, use the associated **DNS name** with the following command:

+ 1. Open the configuration details (See how to [set up configuration file here](device-update-configuration-file.md) with the command below. Set your connectionType as 'AIS' and connectionData as empty string.

+ ```markdown

+ /etc/adu/du-config.json

+ ```

+ 5. Restart the Device Update agent by running the following command:

+ ```markdown

+ sudo systemctl restart adu-agent

+ ```

+Device Update for Azure IoT Hub software packages are subject to the following license terms:

+ * [Device update for IoT Hub license](https://github.com/Azure/iot-hub-device-update/blob/main/LICENSE)

+ * [Delivery optimization client license](https://github.com/microsoft/do-client/blob/main/LICENSE)

+Read the license terms before you use a package. Your installation and use of a package constitutes your acceptance of these terms. If you don't agree with the license terms, don't use that package.

+5. Restart the Device Update agent by running the following command:

+- [Device Update for Azure IoT Hub tutorial for Azure real-time operating system](device-update-azure-real-time-operating-system.md).

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+from azure.ai.ml.entities import Workspace

+ml_client.workspaces.begin_delete(name=ws.name, delete_dependent_resources=True)

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+| __\*.workspace.\<region\>.api.azureml.ms__<br>__\<region\>.experiments.azureml.net__<br>__\<region\>.api.azureml.ms__ | https:443 | Azure Machine Learning service API.|**&check;**|**&check;**|

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ * Using Azure AD groups helps you to avoid reaching the [subscription limit](../role-based-access-control/troubleshooting.md#limits) on role assignments.

+> [!div class="op_single_selector" title1="Select the version of Azure Machine Learning SDK or CLI extension you are using:"]

+> * [v1](./v1/how-to-create-attach-kubernetes.md)

+> * [v2 (current version)](how-to-attach-kubernetes-anywhere.md)

+> [!IMPORTANT]

+> Note that it takes about 30 minutes to an hour or more for changing v1_legacy_mode parameter from __true__ to __false__ to be reflected in the workspace. Therefore, if you set the parameter to __false__ but receive an error that the parameter is __true__ in a subsequent operation, please try after a few more minutes.

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+* Using a private endpoint does not affect Azure control plane (management operations) such as deleting the workspace or managing compute resources. For example, creating, updating, or deleting a compute target. These operations are performed over the public Internet as normal. Data plane operations, such as using Azure Machine Learning studio, APIs (including published pipelines), or the SDK use the private endpoint.

+| Network configuration | Select **Advanced** to create the compute within an existing virtual network. For more information about AKS in a virtual network, see [Network isolation during training and inference with private endpoints and virtual networks](./v1/how-to-secure-inferencing-vnet.md). |

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ Title: Log & view parameters, metrics and files with MLflow

+# Log & view metrics and log files with MLflow

+curl https://management.azure.com/subscriptions/<YOUR-SUBSCRIPTION-ID>/resourceGroups?api-version=2021-04-01 -H "Authorization:Bearer <YOUR-ACCESS-TOKEN>"

+Across Azure, many REST APIs are published. Each service provider updates their API on their own cadence, but does so without breaking existing programs. The service provider uses the `api-version` argument to ensure compatibility.

+> [!IMPORTANT]

+> The `api-version` argument varies from service to service. For the Machine Learning Service, for instance, the current API version is `2022-05-01`. To find the latest API version for other Azure services, see the [Azure REST API reference](/rest/api/azure/) for the specific service.

+All REST calls should set the `api-version` argument to the expected value. You can rely on the syntax and semantics of the specified version even as the API continues to evolve. If you send a request to a provider without the `api-version` argument, the response will contain a human-readable list of supported values.

+curl https://management.azure.com/subscriptions/<YOUR-SUBSCRIPTION-ID>/resourceGroups/<YOUR-RESOURCE-GROUP>/providers/Microsoft.MachineLearningServices/workspaces/?api-version=2022-05-01 \

+providers/Microsoft.MachineLearningServices/workspaces/<YOUR-WORKSPACE-NAME>/experiments?api-version=2022-05-01 \

+providers/Microsoft.MachineLearningServices/workspaces/<YOUR-WORKSPACE-NAME>/models?api-version=2022-05-01 \

+providers/Microsoft.MachineLearningServices/workspaces/<YOUR-WORKSPACE-NAME>/computes?api-version=2022-05-01 \

+ 'https://management.azure.com/subscriptions/<YOUR-SUBSCRIPTION-ID>/resourceGroups/<YOUR-RESOURCE-GROUP>/providers/Microsoft.MachineLearningServices/workspaces/<YOUR-WORKSPACE-NAME>/computes/<YOUR-COMPUTE-NAME>?api-version=2022-05-01' \

+/providers/Microsoft.MachineLearningServices/workspaces/<YOUR-NEW-WORKSPACE-NAME>?api-version=2022-05-01' \

+/providers/Microsoft.MachineLearningServices/workspaces/<YOUR-NEW-WORKSPACE-NAME>?api-version=2022-05-01' \

+/providers/Microsoft.MachineLearningServices/workspaces/<YOUR-NEW-WORKSPACE-NAME>?api-version=2022-05-01' \

+'https://<REGIONAL-API-SERVER>/modelmanagement/v1.0/subscriptions/<YOUR-SUBSCRIPTION-ID>/resourceGroups/<YOUR-RESOURCE-GROUP>/providers/Microsoft.MachineLearningServices/workspaces/<YOUR-WORKSPACE-NAME>/models/<YOUR-MODEL-ID>?api-version=2022-05-01' \

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+For detailed instructions on how to add default and private clusters, see [Secure an inferencing environment](./v1/how-to-secure-inferencing-vnet.md).

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+> * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+ * If using CLI v1 or SDK v1 - [Secure inference environment](./v1/how-to-secure-inferencing-vnet.md)

+> If you're looking for information on monitoring models deployed as web services, see [Collect model data](how-to-enable-data-collection.md) and [Monitor with Application Insights](how-to-enable-app-insights.md).

+The service provider uses the `api-version` argument to ensure compatibility. The `api-version` argument varies from service to service. Set the API version as a variable to accommodate future versions:

+This tutorial shows you how to upload and use your own data to train machine learning models in Azure Machine Learning. This tutorial is *part 3 of a three-part tutorial series*.

+In [Part 2: Train a model](tutorial-1st-experiment-sdk-train.md), you trained a model in the cloud, using sample data from `PyTorch`. You also downloaded that data through the `torchvision.datasets.CIFAR10` method in the PyTorch API. In this tutorial, you'll use the downloaded data to learn the workflow for working with your own data in Azure Machine Learning.

+>

+> * Understand the new Azure Machine Learning concepts (passing parameters, data inputs).

+You'll need the data that was downloaded in the previous tutorial. Make sure you have completed these steps:

+1. [Create the training script](tutorial-1st-experiment-sdk-train.md#create-training-scripts).

+> [!NOTE]

+ ```python

+ import mlflow

+ mlflow.log_metric('loss', loss)

+ ```

+1. **Save** the file. Close the tab if you wish.

+> [!NOTE]

+> Azure Machine Learning allows you to connect other cloud-based storages that store your data. For more details, see the [data documentation](./concept-data.md).

+There is no additional step needed for uploading data, the control script will define and upload the CIFAR10 training data.

+from azure.ai.ml import MLClient, command, Input

+from azure.identity import DefaultAzureCredential

+from azure.ai.ml.entities import Environment

+from azure.ai.ml import command, Input

+from azure.ai.ml.entities import Data

+from azure.ai.ml.constants import AssetTypes

+ # get details of the current Azure ML workspace

+ # default authentication flow for Azure applications

+ default_azure_credential = DefaultAzureCredential()

+ subscription_id = ws.subscription_id

+ resource_group = ws.resource_group

+ workspace = ws.name

+ # client class to interact with Azure ML services and resources, e.g. workspaces, jobs, models and so on.

+ ml_client = MLClient(

+ default_azure_credential,

+ subscription_id,

+ resource_group,

+ workspace)

+ # the key here should match the key passed to the command

+ my_job_inputs = {

+ "data_path": Input(type=AssetTypes.URI_FOLDER, path="./data")

+ }

+ env_name = "pytorch-env"

+ env_docker_image = Environment(

+ image="pytorch/pytorch:latest",

+ name=env_name,

+ conda_file="pytorch-env.yml",

+ )

+ ml_client.environments.create_or_update(env_docker_image)

+ # target name of compute where job will be executed

+ computeName="cpu-cluster"

+ job = command(

+ code="./src",

+ # the parameter will match the training script argument name

+ # inputs.data_path key should match the dictionary key

+ command="python train.py --data_path ${{inputs.data_path}}",

+ inputs=my_job_inputs,

+ environment=f"{env_name}@latest",

+ compute=computeName,

+ display_name="day1-experiment-data",

+ returned_job = ml_client.create_or_update(job)

+ aml_url = returned_job.studio_url

+ print("Monitor your job at", aml_url)

+> If you used a different name when you created your compute cluster, make sure to adjust the name in the code `computeName='cpu-cluster'` as well.

+The control script is similar to the one from [part 2 of this series](tutorial-1st-experiment-sdk-train.md), with the following new lines:

+ `my_job_inputs = { "data_path": Input(type=AssetTypes.URI_FOLDER, path="./data")}`

+ An [Input](/python/api/azure-ai-ml/azure.ai.ml.input) is used to reference inputs to your job. These can encompass data, either uploaded as part of the job or references to previously registered data assets. URI\*FOLDER tells that the reference points to a folder of data. The data will be mounted by default to the compute for the job.

+ `command="python train.py --data_path ${{inputs.data_path}}"`

+ `--data_path` matches the argument defined in the updated training script. `${{inputs.data_path}}` passes the input defined by the input dictionary, and the keys must match.

+## <a name="submit-to-cloud"></a> Submit the job to Azure Machine Learning

+Select **Save and run script in terminal** to run the *run-pytorch-data.py* script. This job will train the model on the compute cluster using the data you uploaded.

+DATA PATH: /mnt/azureml/cr/j/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/cap/data-capability/wd/INPUT_data_path

+['.amlignore', 'cifar-10-batches-py', 'cifar-10-python.tar.gz']

+================

+epoch=1, batch= 2000: loss 2.20

+epoch=1, batch= 4000: loss 1.90

+epoch=1, batch= 6000: loss 1.70

+epoch=1, batch= 8000: loss 1.58

+epoch=1, batch=10000: loss 1.54

+epoch=1, batch=12000: loss 1.48

+epoch=2, batch= 2000: loss 1.41

+epoch=2, batch= 4000: loss 1.38

+epoch=2, batch= 6000: loss 1.33

+epoch=2, batch= 8000: loss 1.30

+epoch=2, batch=10000: loss 1.29

+epoch=2, batch=12000: loss 1.25

+Finished Training

+- Azure Machine Learning has mounted Blob Storage to the compute cluster automatically for you, passing the mount point into `--data_path`. Compared to the previous job, there is no on the fly data download.

+- The `inputs=my_job_inputs` used in the control script resolves to the mount point.

+ Title: "Tutorial: Get started with a Python script"

+- The Azure Machine Learning Python SDK v2 (preview) installed.

+ To install the SDK you can either,

+ * Create a compute instance, which automatically installs the SDK and is pre-configured for ML workflows. For more information, see [Create and manage an Azure Machine Learning compute instance](how-to-create-manage-compute-instance.md).

+ * Use the following commands to install Azure ML Python SDK v2:

+ * Uninstall previous preview version:

+ ```python

+ pip uninstall azure-ai-ml

+ ```

+ * Install the Azure ML Python SDK v2:

+ ```python

+ pip install azure-ai-ml

+ ```

+This tutorial will use the compute instance as your development computer. First create a few folders and the script:

+1. Name the new folder **src**. Use the **Edit location** link if the file location is not correct.

+1. To the right of the **src** folder, use the **...** to create a new file in the **src** folder.

+Your project folder structure will now look like:

+You can run your code locally, which in this case means on the compute instance. Running code locally has the benefit of interactive debugging of code.

+If you have previously stopped your compute instance, start it now with the **Start compute** tool to the right of the compute dropdown. Wait about a minute for state to change to *Running*.

+A *control script* allows you to run your `hello.py` script on different compute resources. You use the control script to control how and where your machine learning code is run.

+Select the **...** at the end of **get-started** folder to create a new file. Create a Python file called *run-hello.py* and copy/paste the following code into that file:

+from azure.ai.ml import MLClient, command, Input

+from azure.identity import DefaultAzureCredential

+from azureml.core import Workspace

+# get details of the current Azure ML workspace

+# default authentication flow for Azure applications

+default_azure_credential = DefaultAzureCredential()

+subscription_id = ws.subscription_id

+resource_group = ws.resource_group

+workspace = ws.name

+# client class to interact with Azure ML services and resources, e.g. workspaces, jobs, models and so on.

+ml_client = MLClient(

+ default_azure_credential,

+ subscription_id,

+ resource_group,

+ workspace)

+# target name of compute where job will be executed

+computeName="cpu-cluster"

+job = command(

+ code="./src",

+ command="python hello.py",

+ environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest",

+ compute=computeName,

+ display_name="hello-world-example",

+)

+returned_job = ml_client.create_or_update(job)

+aml_url = returned_job.studio_url

+print("Monitor your job at", aml_url)

+> If you used a different name when you created your compute cluster, make sure to adjust the name in the code `computeName='cpu-cluster'` as well.

+ `ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)`

+ [MLClient](/python/api/azure-ai-ml/azure.ai.ml.mlclient) manages your Azure Machine Learning workspace and it's assets and resources.

+ `job = command(...)`

+ [command](/python/api/azure-ai-ml/azure.ai.ml#azure-ai-ml-command) provides a simple way to construct a standalone command job or one as part of a dsl.pipeline.

+ `returned_job = ml_client.create_or_update(job)`

+ Submits your script. This submission is called a [job](/python/api/azure-ai-ml/azure.ai.ml.entities.job). A job encapsulates a single execution of your code. Use a job to monitor the script progress, capture the output, analyze the results, visualize metrics, and more.

+ `aml_url = returned_job.studio_url`

+ The `job` object provides a handle on the execution of your code. Monitor its progress from the Azure Machine Learning studio with the URL that's printed from the Python script.

+1. In the terminal, you may be asked to sign in to authenticate. Copy the code and follow the link to complete this step.

+ [!INCLUDE [amlinclude-info](../../includes/machine-learning-py38-ignore.md)]

+`https://ml.azure.com/runs/<run-id>?wsid=/subscriptions/<subscription-id>/resourcegroups/<resource-group>/workspaces/<workspace-name>`.

+Follow the link. At first, you'll see a status of **Queued** or **Preparing**. The very first run will take 5-10 minutes to complete. This is because the following occurs:

+* The docker image is downloaded to the compute.

+Wait about 10 minutes. You'll see a message that the run has completed. Then use **Refresh** to see the status change to _Completed_. Once the job completes, go to the **Outputs + logs** tab.

+The `std_log.txt` file contains the standard output from a run. This file can be useful when you're debugging remote runs in the cloud.

+hello world

+> [!NOTE]

+This tutorial shows you how to train a machine learning model in Azure Machine Learning. This tutorial is _part 2 of a three-part tutorial series_.

+In [Part 1: Run "Hello world!"](tutorial-1st-experiment-hello-world.md) of the series, you learned how to use a control script to run a job in the cloud.

+1. On the toolbar, select **Save** to save the file. Close the tab if you wish.

+After the script completes, select **Refresh** above the file folders. You'll see the new data folder called **get-started/data** Expand this folder to view the downloaded data.

+Azure Machine Learning provides the concept of an [environment](/python/api/azureml-core/azureml.core.environment.environment) to represent a reproducible, versioned Python environment for running experiments. It's easy to create an environment from a local Conda or pip environment.

+ - python=3.8.5

+from azure.ai.ml import MLClient, command, Input

+from azure.identity import DefaultAzureCredential

+from azure.ai.ml.entities import Environment

+ # get details of the current Azure ML workspace

+ # default authentication flow for Azure applications

+ default_azure_credential = DefaultAzureCredential()

+ subscription_id = ws.subscription_id

+ resource_group = ws.resource_group

+ workspace = ws.name

+ # client class to interact with Azure ML services and resources, e.g. workspaces, jobs, models and so on.

+ ml_client = MLClient(

+ default_azure_credential,

+ subscription_id,

+ resource_group,

+ workspace)

+ env_name = "pytorch-env"

+ env_docker_image = Environment(

+ image="pytorch/pytorch:latest",

+ name=env_name,

+ conda_file="pytorch-env.yml",

+ )

+ ml_client.environments.create_or_update(env_docker_image)

+ # target name of compute where job will be executed

+ computeName="cpu-cluster"

+ job = command(

+ code="./src",

+ command="python train.py",

+ environment=f"{env_name}@latest",

+ compute=computeName,

+ display_name="day1-experiment-train",

+ )

+ returned_job = ml_client.create_or_update(job)

+ aml_url = returned_job.studio_url

+ print("Monitor your job at", aml_url)

+> If you used a different name when you created your compute cluster, make sure to adjust the name in the code `computeName='cpu-cluster'` as well.

+ `env_docker_image = ...`

+ Creates the custom environment against the pytorch base image, with additional conda file to install.

+ `environment=f"{env_name}@latest"`

+ Adds the environment to [command](/python/api/azure-ai-ml/azure.ai.ml#azure-ai-ml-command).

+## <a name="submit"></a> Submit the job to Azure Machine Learning

+ [!INCLUDE [amlinclude-info](../../includes/machine-learning-py38-ignore.md)]

+1. In the page that opens, you'll see the job status. The first time you run this script, Azure Machine Learning will build a new Docker image from your PyTorch environment. The whole job might around 10 minutes to complete. This image will be reused in future jobs to make them job much quicker.

+epoch=1, batch= 2000: loss 2.30

+epoch=1, batch= 4000: loss 2.17

+epoch=1, batch= 6000: loss 1.99

+epoch=1, batch= 8000: loss 1.87

+epoch=1, batch=10000: loss 1.72

+epoch=1, batch=12000: loss 1.63

+epoch=2, batch= 2000: loss 1.54

+epoch=2, batch= 4000: loss 1.53

+epoch=2, batch= 6000: loss 1.50

+epoch=2, batch= 8000: loss 1.46

+epoch=2, batch=10000: loss 1.44

+epoch=2, batch=12000: loss 1.41

+Select the **...** at the end of the folder, then select **Move** to move **data** to the **get-started** folder.

+Now that you have a model training script in Azure Machine Learning, let's start tracking some performance metrics.

+The current training script prints metrics to the terminal. Azure Machine Learning supports logging and tracking experiments using [MLflow tracking](https://www.mlflow.org/docs/latest/tracking.html). By adding a few lines of code, you gain the ability to visualize metrics in the studio and to compare metrics between multiple jobs.

+ import mlflow

+ # ADDITIONAL CODE: OPTIONAL: turn on autolog

+ # mlflow.autolog()

+ mlflow.log_metric('loss', loss)

+# ADDITIONAL CODE: OPTIONAL: turn on autolog

+mlflow.autolog()

+```

+With Azure Machine Learning and MLFlow, users can log metrics, model parameters and model artifacts automatically when training a model.

+```python

+mlflow.log_metric('loss', loss)

+You can log individual metrics as well.

+- Organized by experiment and job, so it's easy to keep track of and compare metrics.

+ - python=3.8.5

+ - mlflow

+ - azureml-mlflow

+Make sure you save this file before you submit the job.

+### <a name="submit-again"></a> Submit the job to Azure Machine Learning

+Select the tab for the *run-pytorch.py* script, then select **Save and run script in terminal** to re-run the *run-pytorch.py* script. Make sure you've saved your changes to `pytorch-env.yml` first.

+This time when you visit the studio, go to the **Metrics** tab where you can now see live updates on the model training loss! It may take a 1 to 2 minutes before the training begins.

+> [!NOTE]

+The steps in this article put Azure Container Registry behind the VNet. In this configuration, you can't deploy models to Azure Container Instances inside the VNet. For more information, see [Secure the inference environment](./v1/how-to-secure-inferencing-vnet.md).

+> The steps in this article put Azure Container Registry behind the VNet. In this configuration, you cannot deploy a model to Azure Container Instances inside the VNet. We do not recommend using Azure Container Instances with Azure Machine Learning in a virtual network. For more information, see [Secure the inference environment](./v1/how-to-secure-inferencing-vnet.md).

+ Title: Create and attach Azure Kubernetes Service

+description: 'Learn how to create a new Azure Kubernetes Service cluster through Azure Machine Learning, or how to attach an existing AKS cluster to your workspace.'

+# Create and attach an Azure Kubernetes Service cluster

+> [!div class="op_single_selector" title1="Select the version of Azure Machine Learning SDK or CLI extension you are using:"]

+> * [v1](how-to-create-attach-kubernetes.md)

+> * [v2 (current version)](../how-to-attach-kubernetes-anywhere.md)

+Azure Machine Learning can deploy trained machine learning models to Azure Kubernetes Service. However, you must first either __create__ an Azure Kubernetes Service (AKS) cluster from your Azure ML workspace, or __attach__ an existing AKS cluster. This article provides information on both creating and attaching a cluster.

+## Prerequisites

+- An Azure Machine Learning workspace. For more information, see [Create an Azure Machine Learning workspace](../how-to-manage-workspace.md).

+- The [Azure CLI extension for Machine Learning service](reference-azure-machine-learning-cli.md), [Azure Machine Learning Python SDK](/python/api/overview/azure/ml/intro), or the [Azure Machine Learning Visual Studio Code extension](../how-to-setup-vs-code.md).

+- If you plan on using an Azure Virtual Network to secure communication between your Azure ML workspace and the AKS cluster, your workspace and its associated resources (storage, key vault, Azure Container Registry) must have private endpoints or service endpoints in the same VNET as AKS cluster's VNET. Please follow tutorial [create a secure workspace](../tutorial-create-secure-workspace.md) to add those private endpoints or service endpoints to your VNET.

+## Limitations

+- If you need a **Standard Load Balancer(SLB)** deployed in your cluster instead of a Basic Load Balancer(BLB), create a cluster in the AKS portal/CLI/SDK and then **attach** it to the AzureML workspace.

+- If you have an Azure Policy that restricts the creation of Public IP addresses, then AKS cluster creation will fail. AKS requires a Public IP for [egress traffic](../../aks/limit-egress-traffic.md). The egress traffic article also provides guidance to lock down egress traffic from the cluster through the Public IP, except for a few fully qualified domain names. There are 2 ways to enable a Public IP:

+ - The cluster can use the Public IP created by default with the BLB or SLB, Or

+ - The cluster can be created without a Public IP and then a Public IP is configured with a firewall with a user defined route. For more information, see [Customize cluster egress with a user-defined-route](../../aks/egress-outboundtype.md).

+

+ The AzureML control plane does not talk to this Public IP. It talks to the AKS control plane for deployments.

+- To attach an AKS cluster, the service principal/user performing the operation must be assigned the __Owner or contributor__ Azure role-based access control (Azure RBAC) role on the Azure resource group that contains the cluster. The service principal/user must also be assigned [Azure Kubernetes Service Cluster Admin Role](../../role-based-access-control/built-in-roles.md#azure-kubernetes-service-cluster-admin-role) on the cluster.

+- If you **attach** an AKS cluster, which has an [Authorized IP range enabled to access the API server](../../aks/api-server-authorized-ip-ranges.md), enable the AzureML control plane IP ranges for the AKS cluster. The AzureML control plane is deployed across paired regions and deploys inference pods on the AKS cluster. Without access to the API server, the inference pods cannot be deployed. Use the [IP ranges](https://www.microsoft.com/download/confirmation.aspx?id=56519) for both the [paired regions](../../availability-zones/cross-region-replication-azure.md) when enabling the IP ranges in an AKS cluster.

+ Authorized IP ranges only works with Standard Load Balancer.

+- If you want to use a private AKS cluster (using Azure Private Link), you must create the cluster first, and then **attach** it to the workspace. For more information, see [Create a private Azure Kubernetes Service cluster](../../aks/private-clusters.md).

+- Using a [public fully qualified domain name (FQDN) with a private AKS cluster](../../aks/private-clusters.md) is __not supported__ with Azure Machine learning.

+- The compute name for the AKS cluster MUST be unique within your Azure ML workspace. It can include letters, digits and dashes. It must start with a letter, end with a letter or digit, and be between 3 and 24 characters in length.

+

+ - If you want to deploy models to **GPU** nodes or **FPGA** nodes (or any specific SKU), then you must create a cluster with the specific SKU. There is no support for creating a secondary node pool in an existing cluster and deploying models in the secondary node pool.

+

+- When creating or attaching a cluster, you can select whether to create the cluster for __dev-test__ or __production__. If you want to create an AKS cluster for __development__, __validation__, and __testing__ instead of production, set the __cluster purpose__ to __dev-test__. If you do not specify the cluster purpose, a __production__ cluster is created.

+ > [!IMPORTANT]

+ > A __dev-test__ cluster is not suitable for production level traffic and may increase inference times. Dev/test clusters also do not guarantee fault tolerance.

+- When creating or attaching a cluster, if the cluster will be used for __production__, then it must contain at least __3 nodes__. For a __dev-test__ cluster, it must contain at least 1 node.

+- The Azure Machine Learning SDK does not provide support scaling an AKS cluster. To scale the nodes in the cluster, use the UI for your AKS cluster in the Azure Machine Learning studio. You can only change the node count, not the VM size of the cluster. For more information on scaling the nodes in an AKS cluster, see the following articles:

+ - [Manually scale the node count in an AKS cluster](../../aks/scale-cluster.md)

+ - [Set up cluster autoscaler in AKS](../../aks/cluster-autoscaler.md)

+- __Do not directly update the cluster by using a YAML configuration__. While Azure Kubernetes Services supports updates via YAML configuration, Azure Machine Learning deployments will override your changes. The only two YAML fields that will not overwritten are __request limits__ and __cpu and memory__.

+- Creating an AKS cluster using the Azure Machine Learning studio UI, SDK, or CLI extension is __not__ idempotent. Attempting to create the resource again will result in an error that a cluster with the same name already exists.

+

+ - Using an Azure Resource Manager template and the [Microsoft.MachineLearningServices/workspaces/computes](/azure/templates/microsoft.machinelearningservices/2019-11-01/workspaces/computes) resource to create an AKS cluster is also __not__ idempotent. If you attempt to use the template again to update an already existing resource, you will receive the same error.

+## Azure Kubernetes Service version

+Azure Kubernetes Service allows you to create a cluster using a variety of Kubernetes versions. For more information on available versions, see [supported Kubernetes versions in Azure Kubernetes Service](../../aks/supported-kubernetes-versions.md).

+When **creating** an Azure Kubernetes Service cluster using one of the following methods, you *do not have a choice in the version* of the cluster that is created:

+* Azure Machine Learning studio, or the Azure Machine Learning section of the Azure portal.

+* Machine Learning extension for Azure CLI.

+* Azure Machine Learning SDK.

+These methods of creating an AKS cluster use the __default__ version of the cluster. *The default version changes over time* as new Kubernetes versions become available.

+When **attaching** an existing AKS cluster, we support all currently supported AKS versions.

+> [!IMPORTANT]

+> Azure Kubernetes Service uses [Blobfuse FlexVolume driver](https://github.com/Azure/kubernetes-volume-drivers/blob/master/flexvolume/blobfuse/README.md) for the versions <=1.16 and [Blob CSI driver](https://github.com/kubernetes-sigs/blob-csi-driver/blob/master/README.md) for the versions >=1.17.

+> Therefore, it is important to re-deploy or [update the web service](../how-to-deploy-update-web-service.md) after cluster upgrade in order to deploy to correct blobfuse method for the cluster version.

+> [!NOTE]

+> There may be edge cases where you have an older cluster that is no longer supported. In this case, the attach operation will return an error and list the currently supported versions.

+>

+> You can attach **preview** versions. Preview functionality is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. Support for using preview versions may be limited. For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

+### Available and default versions

+To find the available and default AKS versions, use the [Azure CLI](/cli/azure/install-azure-cli) command [az aks get-versions](/cli/azure/aks#az-aks-get-versions). For example, the following command returns the versions available in the West US region:

+```azurecli-interactive

+az aks get-versions -l westus -o table

+```

+The output of this command is similar to the following text:

+```text

+KubernetesVersion Upgrades

+- -

+1.18.6(preview) None available

+1.18.4(preview) 1.18.6(preview)

+1.17.9 1.18.4(preview), 1.18.6(preview)

+1.17.7 1.17.9, 1.18.4(preview), 1.18.6(preview)

+1.16.13 1.17.7, 1.17.9

+1.16.10 1.16.13, 1.17.7, 1.17.9

+1.15.12 1.16.10, 1.16.13

+1.15.11 1.15.12, 1.16.10, 1.16.13

+```

+To find the default version that is used when **creating** a cluster through Azure Machine Learning, you can use the `--query` parameter to select the default version:

+```azurecli-interactive

+az aks get-versions -l westus --query "orchestrators[?default == `true`].orchestratorVersion" -o table

+```

+The output of this command is similar to the following text:

+```text

+Result

+--

+1.16.13

+```

+If you'd like to **programmatically check the available versions**, use the Container Service Client - List Orchestrators REST API. To find the available versions, look at the entries where `orchestratorType` is `Kubernetes`. The associated `orchestrationVersion` entries contain the available versions that can be **attached** to your workspace.

+To find the default version that is used when **creating** a cluster through Azure Machine Learning, find the entry where `orchestratorType` is `Kubernetes` and `default` is `true`. The associated `orchestratorVersion` value is the default version. The following JSON snippet shows an example entry:

+```json

+...

+ {

+ "orchestratorType": "Kubernetes",

+ "orchestratorVersion": "1.16.13",

+ "default": true,

+ "upgrades": [

+ {

+ "orchestratorType": "",

+ "orchestratorVersion": "1.17.7",

+ "isPreview": false

+ }

+ ]

+ },

+...

+```

+## Create a new AKS cluster

+**Time estimate**: Approximately 10 minutes.

+Creating or attaching an AKS cluster is a one time process for your workspace. You can reuse this cluster for multiple deployments. If you delete the cluster or the resource group that contains it, you must create a new cluster the next time you need to deploy. You can have multiple AKS clusters attached to your workspace.

+The following example demonstrates how to create a new AKS cluster using the SDK and CLI:

+# [Python](#tab/python)

+```python

+from azureml.core.compute import AksCompute, ComputeTarget

+# Use the default configuration (you can also provide parameters to customize this).

+# For example, to create a dev/test cluster, use:

+# prov_config = AksCompute.provisioning_configuration(cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)

+prov_config = AksCompute.provisioning_configuration()

+# Example configuration to use an existing virtual network

+# prov_config.vnet_name = "mynetwork"

+# prov_config.vnet_resourcegroup_name = "mygroup"

+# prov_config.subnet_name = "default"

+# prov_config.service_cidr = "10.0.0.0/16"

+# prov_config.dns_service_ip = "10.0.0.10"

+# prov_config.docker_bridge_cidr = "172.17.0.1/16"

+aks_name = 'myaks'

+# Create the cluster

+aks_target = ComputeTarget.create(workspace = ws,

+ name = aks_name,

+ provisioning_configuration = prov_config)

+# Wait for the create process to complete

+aks_target.wait_for_completion(show_output = True)

+```

+For more information on the classes, methods, and parameters used in this example, see the following reference documents:

+* [AksCompute.ClusterPurpose](/python/api/azureml-core/azureml.core.compute.aks.akscompute.clusterpurpose)

+* [AksCompute.provisioning_configuration](/python/api/azureml-core/azureml.core.compute.akscompute#provisioning-configuration-agent-count-none--vm-size-none--ssl-cname-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--location-none--vnet-resourcegroup-name-none--vnet-name-none--subnet-name-none--service-cidr-none--dns-service-ip-none--docker-bridge-cidr-none--cluster-purpose-none--load-balancer-type-none--load-balancer-subnet-none-)

+* [ComputeTarget.create](/python/api/azureml-core/azureml.core.compute.computetarget#create-workspace--name--provisioning-configuration-)

+* [ComputeTarget.wait_for_completion](/python/api/azureml-core/azureml.core.compute.computetarget#wait-for-completion-show-output-false-)

+# [Azure CLI](#tab/azure-cli)

+```azurecli

+az ml computetarget create aks -n myaks

+```

+For more information, see the [az ml computetarget create aks](/cli/azure/ml(v1)/computetarget/create#az-ml-computetarget-create-aks) reference.

+# [Portal](#tab/azure-portal)

+For information on creating an AKS cluster in the portal, see [Create compute targets in Azure Machine Learning studio](../how-to-create-attach-compute-studio.md#inference-clusters).

+## Attach an existing AKS cluster

+**Time estimate:** Approximately 5 minutes.

+If you already have AKS cluster in your Azure subscription, you can use it with your workspace.

+> [!TIP]

+> The existing AKS cluster can be in a Azure region other than your Azure Machine Learning workspace.

+> [!WARNING]

+> Do not create multiple, simultaneous attachments to the same AKS cluster from your workspace. For example, attaching one AKS cluster to a workspace using two different names. Each new attachment will break the previous existing attachment(s).

+>

+> If you want to re-attach an AKS cluster, for example to change TLS or other cluster configuration setting, you must first remove the existing attachment by using [AksCompute.detach()](/python/api/azureml-core/azureml.core.compute.akscompute#detach--).

+For more information on creating an AKS cluster using the Azure CLI or portal, see the following articles:

+* [Create an AKS cluster (CLI)](/cli/azure/aks?bc=%2fazure%2fbread%2ftoc.json&toc=%2fazure%2faks%2fTOC.json#az-aks-create)

+* [Create an AKS cluster (portal)](../../aks/learn/quick-kubernetes-deploy-portal.md)

+* [Create an AKS cluster (ARM Template on Azure Quickstart templates)](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.containerinstance/aks-azml-targetcompute)

+The following example demonstrates how to attach an existing AKS cluster to your workspace:

+# [Python](#tab/python)

+```python

+from azureml.core.compute import AksCompute, ComputeTarget

+# Set the resource group that contains the AKS cluster and the cluster name

+resource_group = 'myresourcegroup'

+cluster_name = 'myexistingcluster'

+# Attach the cluster to your workgroup. If the cluster has less than 12 virtual CPUs, use the following instead:

+# attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+# cluster_name = cluster_name,

+# cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)

+attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+ cluster_name = cluster_name)

+aks_target = ComputeTarget.attach(ws, 'myaks', attach_config)

+# Wait for the attach process to complete

+aks_target.wait_for_completion(show_output = True)

+```

+For more information on the classes, methods, and parameters used in this example, see the following reference documents:

+* [AksCompute.attach_configuration()](/python/api/azureml-core/azureml.core.compute.akscompute#attach-configuration-resource-group-none--cluster-name-none--resource-id-none--cluster-purpose-none-)

+* [AksCompute.ClusterPurpose](/python/api/azureml-core/azureml.core.compute.aks.akscompute.clusterpurpose)

+* [AksCompute.attach](/python/api/azureml-core/azureml.core.compute.computetarget#attach-workspace--name--attach-configuration-)

+# [Azure CLI](#tab/azure-cli)

+To attach an existing cluster using the CLI, you need to get the resource ID of the existing cluster. To get this value, use the following command. Replace `myexistingcluster` with the name of your AKS cluster. Replace `myresourcegroup` with the resource group that contains the cluster:

+```azurecli

+az aks show -n myexistingcluster -g myresourcegroup --query id

+```

+This command returns a value similar to the following text:

+```text

+/subscriptions/{GUID}/resourcegroups/{myresourcegroup}/providers/Microsoft.ContainerService/managedClusters/{myexistingcluster}

+```

+To attach the existing cluster to your workspace, use the following command. Replace `aksresourceid` with the value returned by the previous command. Replace `myresourcegroup` with the resource group that contains your workspace. Replace `myworkspace` with your workspace name.

+```azurecli

+az ml computetarget attach aks -n myaks -i aksresourceid -g myresourcegroup -w myworkspace

+```

+For more information, see the [az ml computetarget attach aks](/cli/azure/ml(v1)/computetarget/attach#az-ml-computetarget-attach-aks) reference.

+# [Portal](#tab/azure-portal)

+For information on attaching an AKS cluster in the portal, see [Create compute targets in Azure Machine Learning studio](../how-to-create-attach-compute-studio.md#inference-clusters).

+## Create or attach an AKS cluster with TLS termination

+When you [create or attach an AKS cluster](how-to-create-attach-kubernetes.md), you can enable TLS termination with **[AksCompute.provisioning_configuration()](/python/api/azureml-core/azureml.core.compute.akscompute#provisioning-configuration-agent-count-none--vm-size-none--ssl-cname-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--location-none--vnet-resourcegroup-name-none--vnet-name-none--subnet-name-none--service-cidr-none--dns-service-ip-none--docker-bridge-cidr-none--cluster-purpose-none--load-balancer-type-none--load-balancer-subnet-none-)** and **[AksCompute.attach_configuration()](/python/api/azureml-core/azureml.core.compute.akscompute#attach-configuration-resource-group-none--cluster-name-none--resource-id-none--cluster-purpose-none-)** configuration objects. Both methods return a configuration object that has an **enable_ssl** method, and you can use **enable_ssl** method to enable TLS.

+Following example shows how to enable TLS termination with automatic TLS certificate generation and configuration by using Microsoft certificate under the hood.

+```python

+ from azureml.core.compute import AksCompute, ComputeTarget

+

+ # Enable TLS termination when you create an AKS cluster by using provisioning_config object enable_ssl method

+ # Leaf domain label generates a name using the formula

+ # "<leaf-domain-label>######.<azure-region>.cloudapp.azure.com"

+ # where "######" is a random series of characters

+ provisioning_config.enable_ssl(leaf_domain_label = "contoso")

+

+ # Enable TLS termination when you attach an AKS cluster by using attach_config object enable_ssl method

+ # Leaf domain label generates a name using the formula

+ # "<leaf-domain-label>######.<azure-region>.cloudapp.azure.com"

+ # where "######" is a random series of characters

+ attach_config.enable_ssl(leaf_domain_label = "contoso")

+```

+Following example shows how to enable TLS termination with custom certificate and custom domain name. With custom domain and certificate, you must update your DNS record to point to the IP address of scoring endpoint, please see [Update your DNS](../how-to-secure-web-service.md#update-your-dns)

+```python

+ from azureml.core.compute import AksCompute, ComputeTarget

+ # Enable TLS termination with custom certificate and custom domain when creating an AKS cluster

+

+ provisioning_config.enable_ssl(ssl_cert_pem_file="cert.pem",

+ ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")

+

+ # Enable TLS termination with custom certificate and custom domain when attaching an AKS cluster

+ attach_config.enable_ssl(ssl_cert_pem_file="cert.pem",

+ ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")

+```

+>[!NOTE]

+> For more information about how to secure model deployment on AKS cluster, please see [use TLS to secure a web service through Azure Machine Learning](../how-to-secure-web-service.md)

+## Create or attach an AKS cluster to use Internal Load Balancer with private IP

+When you create or attach an AKS cluster, you can configure the cluster to use an Internal Load Balancer. With an Internal Load Balancer, scoring endpoints for your deployments to AKS will use a private IP within the virtual network. Following code snippets show how to configure an Internal Load Balancer for an AKS cluster.

+# [Create](#tab/akscreate)

+To create an AKS cluster that uses an Internal Load Balancer, use the `load_balancer_type` and `load_balancer_subnet` parameters:

+```python

+from azureml.core.compute.aks import AksUpdateConfiguration

+from azureml.core.compute import AksCompute, ComputeTarget

+# Change to the name of the subnet that contains AKS

+subnet_name = "default"

+# When you create an AKS cluster, you can specify Internal Load Balancer to be created with provisioning_config object

+provisioning_config = AksCompute.provisioning_configuration(load_balancer_type = 'InternalLoadBalancer', load_balancer_subnet = subnet_name)

+# Create the cluster

+aks_target = ComputeTarget.create(workspace = ws,

+ name = aks_name,

+ provisioning_configuration = provisioning_config)

+# Wait for the create process to complete

+aks_target.wait_for_completion(show_output = True)

+```

+# [Attach](#tab/aksattach)

+To attach an AKS cluster and use an internal load balancer (no public IP for the cluster), use the `load_balancer_type` and `load_balancer_subnet` parameters:

+```python

+from azureml.core.compute import AksCompute, ComputeTarget

+# Set the resource group that contains the AKS cluster and the cluster name

+resource_group = 'myresourcegroup'

+cluster_name = 'myexistingcluster'

+# Change to the name of the subnet that contains AKS

+subnet_name = "default"

+# Attach the cluster to your workgroup. If the cluster has less than 12 virtual CPUs, use the following instead:

+# attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+# cluster_name = cluster_name,

+# cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)

+attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+ cluster_name = cluster_name,

+ load_balancer_type = 'InternalLoadBalancer',

+ load_balancer_subnet = subnet_name)

+aks_target = ComputeTarget.attach(ws, 'myaks', attach_config)

+# Wait for the attach process to complete

+aks_target.wait_for_completion(show_output = True)

+```

+>[!IMPORTANT]

+> If your AKS cluster is configured with an Internal Load Balancer, using a Microsoft provided certificate is not supported and you must use [custom certificate to enable TLS](../how-to-secure-web-service.md#deploy-on-azure-kubernetes-service).

+>[!NOTE]

+> For more information about how to secure inferencing environment, please see [Secure an Azure Machine Learning Inferencing Environment](how-to-secure-inferencing-vnet.md)

+## Detach an AKS cluster

+To detach a cluster from your workspace, use one of the following methods:

+> [!WARNING]

+> Using the Azure Machine Learning studio, SDK, or the Azure CLI extension for machine learning to detach an AKS cluster **does not delete the AKS cluster**. To delete the cluster, see [Use the Azure CLI with AKS](../../aks/learn/quick-kubernetes-deploy-cli.md#delete-the-cluster).

+# [Python](#tab/python)

+```python

+aks_target.detach()

+```

+# [Azure CLI](#tab/azure-cli)

+To detach the existing cluster to your workspace, use the following command. Replace `myaks` with the name that the AKS cluster is attached to your workspace as. Replace `myresourcegroup` with the resource group that contains your workspace. Replace `myworkspace` with your workspace name.

+```azurecli

+az ml computetarget detach -n myaks -g myresourcegroup -w myworkspace

+```

+# [Portal](#tab/azure-portal)

+In Azure Machine Learning studio, select __Compute__, __Inference clusters__, and the cluster you wish to remove. Use the __Detach__ link to detach the cluster.

+## Troubleshooting

+### Update the cluster

+Updates to Azure Machine Learning components installed in an Azure Kubernetes Service cluster must be manually applied.

+You can apply these updates by detaching the cluster from the Azure Machine Learning workspace and reattaching the cluster to the workspace.

+```python

+compute_target = ComputeTarget(workspace=ws, name=clusterWorkspaceName)

+compute_target.detach()

+compute_target.wait_for_completion(show_output=True)

+```

+Before you can re-attach the cluster to your workspace, you need to first delete any `azureml-fe` related resources. If there is no active service in the cluster, you can delete your `azureml-fe` related resources with the following code.

+```shell

+kubectl delete sa azureml-fe

+kubectl delete clusterrole azureml-fe-role

+kubectl delete clusterrolebinding azureml-fe-binding

+kubectl delete svc azureml-fe

+kubectl delete svc azureml-fe-int-http

+kubectl delete deploy azureml-fe

+kubectl delete secret azuremlfessl

+kubectl delete cm azuremlfeconfig

+```

+If TLS is enabled in the cluster, you will need to supply the TLS/SSL certificate and private key when reattaching the cluster.

+```python

+attach_config = AksCompute.attach_configuration(resource_group=resourceGroup, cluster_name=kubernetesClusterName)

+# If SSL is enabled.

+attach_config.enable_ssl(

+ ssl_cert_pem_file="cert.pem",

+ ssl_key_pem_file="key.pem",

+ ssl_cname=sslCname)

+attach_config.validate_configuration()

+compute_target = ComputeTarget.attach(workspace=ws, name=args.clusterWorkspaceName, attach_configuration=attach_config)

+compute_target.wait_for_completion(show_output=True)

+```

+If you no longer have the TLS/SSL certificate and private key, or you are using a certificate generated by Azure Machine Learning, you can retrieve the files prior to detaching the cluster by connecting to the cluster using `kubectl` and retrieving the secret `azuremlfessl`.

+```bash

+kubectl get secret/azuremlfessl -o yaml

+```

+> [!NOTE]

+> Kubernetes stores the secrets in Base64-encoded format. You will need to Base64-decode the `cert.pem` and `key.pem` components of the secrets prior to providing them to `attach_config.enable_ssl`.

+### Webservice failures

+Many webservice failures in AKS can be debugged by connecting to the cluster using `kubectl`. You can get the `kubeconfig.json` for an AKS cluster by running

+```azurecli-interactive

+az aks get-credentials -g <rg> -n <aks cluster name>

+```

+### Delete azureml-fe related resources

+After detaching cluster, if there is none active service in cluster, please delete the `azureml-fe` related resources before attaching again:

+```shell

+kubectl delete sa azureml-fe

+kubectl delete clusterrole azureml-fe-role

+kubectl delete clusterrolebinding azureml-fe-binding

+kubectl delete svc azureml-fe

+kubectl delete svc azureml-fe-int-http

+kubectl delete deploy azureml-fe

+kubectl delete secret azuremlfessl

+kubectl delete cm azuremlfeconfig

+```

+### Load balancers should not have public IPs

+When trying to create or attach an AKS cluster, you may receive a message that the request has been denied because "Load Balancers should not have public IPs". This message is returned when an administrator has applied a policy that prevents using an AKS cluster with a public IP address.

+To resolve this problem, create/attach the cluster by using the `load_balancer_type` and `load_balancer_subnet` parameters. For more information, see [Internal Load Balancer (private IP)](#create-or-attach-an-aks-cluster-to-use-internal-load-balancer-with-private-ip).

+## Next steps

+* [Use Azure RBAC for Kubernetes authorization](../../aks/manage-azure-rbac.md)

+* [How and where to deploy a model](how-to-deploy-and-where.md)

+ Title: Deploy machine learning models

+description: 'Learn how and where to deploy machine learning models. Deploy to Azure Container Instances, Azure Kubernetes Service, and FPGA.'

+adobe-target: true

+# Deploy machine learning models to Azure

+Learn how to deploy your machine learning or deep learning model as a web service in the Azure cloud.

+## Workflow for deploying a model

+The workflow is similar no matter where you deploy your model:

+1. Register the model.

+1. Prepare an entry script.

+1. Prepare an inference configuration.

+1. Deploy the model locally to ensure everything works.

+1. Choose a compute target.

+1. Deploy the model to the cloud.

+1. Test the resulting web service.

+For more information on the concepts involved in the machine learning deployment workflow, see [Manage, deploy, and monitor models with Azure Machine Learning](concept-model-management-and-deployment.md).

+## Prerequisites

+# [Azure CLI](#tab/azcli)

+- An Azure Machine Learning workspace. For more information, see [Create workspace resources](../quickstart-create-resources.md).

+- A model. The examples in this article use a pre-trained model.

+- A machine that can run Docker, such as a [compute instance](../how-to-create-manage-compute-instance.md).

+# [Python](#tab/python)

+- An Azure Machine Learning workspace. For more information, see [Create workspace resources](../quickstart-create-resources.md).

+- A model. The examples in this article use a pre-trained model.

+- The [Azure Machine Learning software development kit (SDK) for Python](/python/api/overview/azure/ml/intro).

+- A machine that can run Docker, such as a [compute instance](../how-to-create-manage-compute-instance.md).

+## Connect to your workspace

+# [Azure CLI](#tab/azcli)

+To see the workspaces that you have access to, use the following commands:

+```azurecli-interactive

+az login

+az account set -s <subscription>

+az ml workspace list --resource-group=<resource-group>

+```

+# [Python](#tab/python)

+```python

+from azureml.core import Workspace

+ws = Workspace(subscription_id="<subscription_id>",

+ resource_group="<resource_group>",

+ workspace_name="<workspace_name>")

+```

+For more information on using the SDK to connect to a workspace, see the [Azure Machine Learning SDK for Python](/python/api/overview/azure/ml/intro#workspace) documentation.

+## <a id="registermodel"></a> Register the model

+A typical situation for a deployed machine learning service is that you need the following components:

+

+Azure Machine Learnings allows you to separate the deployment into two separate components, so that you can keep the same code, but merely update the model. We define the mechanism by which you upload a model _separately_ from your code as "registering the model".

+When you register a model, we upload the model to the cloud (in your workspace's default storage account) and then mount it to the same compute where your webservice is running.

+The following examples demonstrate how to register a model.

+# [Azure CLI](#tab/azcli)

+The following commands download a model and then register it with your Azure Machine Learning workspace:

+```azurecli-interactive

+wget https://aka.ms/bidaf-9-model -O model.onnx --show-progress

+az ml model register -n bidaf_onnx \

+ -p ./model.onnx \

+ -g <resource-group> \

+ -w <workspace-name>

+```

+Set `-p` to the path of a folder or a file that you want to register.

+For more information on `az ml model register`, see the [reference documentation](/cli/azure/ml(v1)/model).

+### Register a model from an Azure ML training job

+If you need to register a model that was created previously through an Azure Machine Learning training job, you can specify the experiment, run, and path to the model:

+```azurecli-interactive

+az ml model register -n bidaf_onnx --asset-path outputs/model.onnx --experiment-name myexperiment --run-id myrunid --tag area=qna

+```

+The `--asset-path` parameter refers to the cloud location of the model. In this example, the path of a single file is used. To include multiple files in the model registration, set `--asset-path` to the path of a folder that contains the files.

+For more information on `az ml model register`, see the [reference documentation](/cli/azure/ml(v1)/model).

+# [Python](#tab/python)

+### Register a model from a local file

+You can register a model by providing the local path of the model. You can provide the path of either a folder or a single file on your local machine.

+<!-- pyhton nb call -->

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=register-model-from-local-file-code)]

+To include multiple files in the model registration, set `model_path` to the path of a folder that contains the files.

+For more information, see the documentation for the [Model class](/python/api/azureml-core/azureml.core.model.model).

+### Register a model from an Azure ML training job

+ When you use the SDK to train a model, you can receive either a [Run](/python/api/azureml-core/azureml.core.run.run) object or an [AutoMLRun](/python/api/azureml-train-automl-client/azureml.train.automl.run.automlrun) object, depending on how you trained the model. Each object can be used to register a model created by an experiment run.

+ + Register a model from an `azureml.core.Run` object:

+

+ [!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]

+ ```python

+ model = run.register_model(model_name='bidaf_onnx',

+ tags={'area': 'qna'},

+ model_path='outputs/model.onnx')

+ print(model.name, model.id, model.version, sep='\t')

+ ```

+ The `model_path` parameter refers to the cloud location of the model. In this example, the path of a single file is used. To include multiple files in the model registration, set `model_path` to the path of a folder that contains the files. For more information, see the [Run.register_model](/python/api/azureml-core/azureml.core.run.run#register-model-model-name--model-path-none--tags-none--properties-none--model-framework-none--model-framework-version-none--description-none--datasets-none--sample-input-dataset-none--sample-output-dataset-none--resource-configuration-none-kwargs-) documentation.

+ + Register a model from an `azureml.train.automl.run.AutoMLRun` object:

+ [!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]

+ ```python

+ description = 'My AutoML Model'

+ model = run.register_model(description = description,

+ tags={'area': 'qna'})

+ print(run.model_id)

+ ```

+ In this example, the `metric` and `iteration` parameters aren't specified, so the iteration with the best primary metric will be registered. The `model_id` value returned from the run is used instead of a model name.

+ For more information, see the [AutoMLRun.register_model](/python/api/azureml-train-automl-client/azureml.train.automl.run.automlrun#register-model-model-name-none--description-none--tags-none--iteration-none--metric-none-) documentation.

+ To deploy a registered model from an `AutoMLRun`, we recommend doing so via the [one-click deploy button in Azure Machine Learning studio](../how-to-use-automated-ml-for-ml-models.md#deploy-your-model).

+## Define a dummy entry script

+The entry script receives data submitted to a deployed web service and passes it to the model. It then returns the model's response to the client. *The script is specific to your model*. The entry script must understand the data that the model expects and returns.

+The two things you need to accomplish in your entry script are:

+1. Loading your model (using a function called `init()`)

+1. Running your model on input data (using a function called `run()`)

+For your initial deployment, use a dummy entry script that prints the data it receives.

+Save this file as `echo_score.py` inside of a directory called `source_dir`. This dummy script returns the data you send to it, so it doesn't use the model. But it is useful for testing that the scoring script is running.

+## Define an inference configuration

+An inference configuration describes the Docker container and files to use when initializing your web service. All of the files within your source directory, including subdirectories, will be zipped up and uploaded to the cloud when you deploy your web service.

+The inference configuration below specifies that the machine learning deployment will use the file `echo_score.py` in the `./source_dir` directory to process incoming requests and that it will use the Docker image with the Python packages specified in the `project_environment` environment.

+You can use any [Azure Machine Learning inference curated environments](../concept-prebuilt-docker-images-inference.md#list-of-prebuilt-docker-images-for-inference) as the base Docker image when creating your project environment. We will install the required dependencies on top and store the resulting Docker image into the repository that is associated with your workspace.

+> [!NOTE]

+> Azure machine learning [inference source directory](/python/api/azureml-core/azureml.core.model.inferenceconfig?view=azure-ml-py#constructor&preserve-view=true) upload does not respect **.gitignore** or **.amlignore**

+# [Azure CLI](#tab/azcli)

+A minimal inference configuration can be written as:

+Save this file with the name `dummyinferenceconfig.json`.

+[See this article](reference-azure-machine-learning-cli.md#inference-configuration-schema) for a more thorough discussion of inference configurations.

+# [Python](#tab/python)

+The following example demonstrates how to create a minimal environment with no pip dependencies, using the dummy scoring script you defined above.

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=inference-configuration-code)]

+For more information on environments, see [Create and manage environments for training and deployment](../how-to-use-environments.md).

+For more information on inference configuration, see the [InferenceConfig](/python/api/azureml-core/azureml.core.model.inferenceconfig) class documentation.

+## Define a deployment configuration

+A deployment configuration specifies the amount of memory and cores your webservice needs in order to run. It also provides configuration details of the underlying webservice. For example, a deployment configuration lets you specify that your service needs 2 gigabytes of memory, 2 CPU cores, 1 GPU core, and that you want to enable autoscaling.

+The options available for a deployment configuration differ depending on the compute target you choose. In a local deployment, all you can specify is which port your webservice will be served on.

+# [Azure CLI](#tab/azcli)

+For more information, see the [deployment schema](reference-azure-machine-learning-cli.md#deployment-configuration-schema).

+# [Python](#tab/python)

+The following Python demonstrates how to create a local deployment configuration:

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=deployment-configuration-code)]

+## Deploy your machine learning model

+You are now ready to deploy your model.

+# [Azure CLI](#tab/azcli)

+Replace `bidaf_onnx:1` with the name of your model and its version number.

+```azurecli-interactive

+az ml model deploy -n myservice \

+ -m bidaf_onnx:1 \

+ --overwrite \

+ --ic dummyinferenceconfig.json \

+ --dc deploymentconfig.json \

+ -g <resource-group> \

+ -w <workspace-name>

+```

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=deploy-model-code)]

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=deploy-model-print-logs)]

+For more information, see the documentation for [Model.deploy()](/python/api/azureml-core/azureml.core.model.model#deploy-workspace--name--models--inference-config-none--deployment-config-none--deployment-target-none--overwrite-false-) and [Webservice](/python/api/azureml-core/azureml.core.webservice.webservice).

+## Call into your model

+Let's check that your echo model deployed successfully. You should be able to do a simple liveness request, as well as a scoring request:

+# [Azure CLI](#tab/azcli)

+```azurecli-interactive

+curl -v http://localhost:32267

+curl -v -X POST -H "content-type:application/json" \

+ -d '{"query": "What color is the fox", "context": "The quick brown fox jumped over the lazy dog."}' \

+ http://localhost:32267/score

+```

+# [Python](#tab/python)

+<!-- python nb call -->

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=call-into-model-code)]

+## Define an entry script

+Now it's time to actually load your model. First, modify your entry script:

+Save this file as `score.py` inside of `source_dir`.

+Notice the use of the `AZUREML_MODEL_DIR` environment variable to locate your registered model. Now that you've added some pip packages.

+# [Azure CLI](#tab/azcli)

+Save this file as `inferenceconfig.json`

+# [Python](#tab/python)

+```python

+env = Environment(name='myenv')

+python_packages = ['nltk', 'numpy', 'onnxruntime']

+for package in python_packages:

+ env.python.conda_dependencies.add_pip_package(package)

+inference_config = InferenceConfig(environment=env, source_directory='./source_dir', entry_script='./score.py')

+```

+For more information, see the documentation for [LocalWebservice](/python/api/azureml-core/azureml.core.webservice.local.localwebservice), [Model.deploy()](/python/api/azureml-core/azureml.core.model.model#deploy-workspace--name--models--inference-config-none--deployment-config-none--deployment-target-none--overwrite-false-), and [Webservice](/python/api/azureml-core/azureml.core.webservice.webservice).

+## Deploy again and call your service

+Deploy your service again:

+# [Azure CLI](#tab/azcli)

+Replace `bidaf_onnx:1` with the name of your model and its version number.

+```azurecli-interactive

+az ml model deploy -n myservice \

+ -m bidaf_onnx:1 \

+ --overwrite \

+ --ic inferenceconfig.json \

+ --dc deploymentconfig.json \

+ -g <resource-group> \

+ -w <workspace-name>

+```

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=re-deploy-model-code)]

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=re-deploy-model-print-logs)]

+For more information, see the documentation for [Model.deploy()](/python/api/azureml-core/azureml.core.model.model#deploy-workspace--name--models--inference-config-none--deployment-config-none--deployment-target-none--overwrite-false-) and [Webservice](/python/api/azureml-core/azureml.core.webservice.webservice).

+Then ensure you can send a post request to the service:

+# [Azure CLI](#tab/azcli)

+```azurecli-interactive

+curl -v -X POST -H "content-type:application/json" \

+ -d '{"query": "What color is the fox", "context": "The quick brown fox jumped over the lazy dog."}' \

+ http://localhost:32267/score

+```

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=send-post-request-code)]

+## Choose a compute target

+## Deploy to cloud

+Once you've confirmed your service works locally and chosen a remote compute target, you are ready to deploy to the cloud.

+Change your deploy configuration to correspond to the compute target you've chosen, in this case Azure Container Instances:

+# [Azure CLI](#tab/azcli)

+The options available for a deployment configuration differ depending on the compute target you choose.

+Save this file as `re-deploymentconfig.json`.

+For more information, see [this reference](reference-azure-machine-learning-cli.md#deployment-configuration-schema).

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=deploy-model-on-cloud-code)]

+Deploy your service again:

+# [Azure CLI](#tab/azcli)

+Replace `bidaf_onnx:1` with the name of your model and its version number.

+```azurecli-interactive

+az ml model deploy -n myservice \

+ -m bidaf_onnx:1 \

+ --overwrite \

+ --ic inferenceconfig.json \

+ --dc re-deploymentconfig.json \

+ -g <resource-group> \

+ -w <workspace-name>

+```

+To view the service logs, use the following command:

+```azurecli-interactive

+az ml service get-logs -n myservice \

+ -g <resource-group> \

+ -w <workspace-name>

+```

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=re-deploy-service-code)]

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=re-deploy-service-print-logs)]

+For more information, see the documentation for [Model.deploy()](/python/api/azureml-core/azureml.core.model.model#deploy-workspace--name--models--inference-config-none--deployment-config-none--deployment-target-none--overwrite-false-) and [Webservice](/python/api/azureml-core/azureml.core.webservice.webservice).

+## Call your remote webservice

+When you deploy remotely, you may have key authentication enabled. The example below shows how to get your service key with Python in order to make an inference request.

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=call-remote-web-service-code)]

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=call-remote-webservice-print-logs)]

+See the article on [client applications to consume web services](../how-to-consume-web-service.md) for more example clients in other languages.

+ [!INCLUDE [Email Notification Include](../../../includes/machine-learning-email-notifications.md)]

+### Understanding service state

+During model deployment, you may see the service state change while it fully deploys.

+The following table describes the different service states:

+| Webservice state | Description | Final state?

+| -- | -- | -- |

+| Transitioning | The service is in the process of deployment. | No |

+| Unhealthy | The service has deployed but is currently unreachable. | No |

+| Unschedulable | The service cannot be deployed at this time due to lack of resources. | No |

+| Failed | The service has failed to deploy due to an error or crash. | Yes |

+| Healthy | The service is healthy and the endpoint is available. | Yes |

+> [!TIP]

+> When deploying, Docker images for compute targets are built and loaded from Azure Container Registry (ACR). By default, Azure Machine Learning creates an ACR that uses the *basic* service tier. Changing the ACR for your workspace to standard or premium tier may reduce the time it takes to build and deploy images to your compute targets. For more information, see [Azure Container Registry service tiers](../../container-registry/container-registry-skus.md).

+> [!NOTE]

+> If you are deploying a model to Azure Kubernetes Service (AKS), we advise you enable [Azure Monitor](../../azure-monitor/containers/container-insights-enable-existing-clusters.md) for that cluster. This will help you understand overall cluster health and resource usage. You might also find the following resources useful:

+>

+> * [Check for Resource Health events impacting your AKS cluster](../../aks/aks-resource-health.md)

+> * [Azure Kubernetes Service Diagnostics](../../aks/concepts-diagnostics.md)

+>

+> If you are trying to deploy a model to an unhealthy or overloaded cluster, it is expected to experience issues. If you need help troubleshooting AKS cluster problems please contact AKS Support.

+## Delete resources

+# [Azure CLI](#tab/azcli)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/2.deploy-local-cli.ipynb?name=delete-resource-code)]

+```azurecli-interactive

+az ml service delete -n myservice

+az ml service delete -n myaciservice

+az ml model delete --model-id=<MODEL_ID>

+```

+To delete a deployed webservice, use `az ml service delete <name of webservice>`.

+To delete a registered model from your workspace, use `az ml model delete <model id>`

+Read more about [deleting a webservice](/cli/azure/ml(v1)/computetarget/create#az-ml-service-delete) and [deleting a model](/cli/azure/ml/model#az-ml-model-delete).

+# [Python](#tab/python)

+[!Notebook-python[] (~/azureml-examples-main/python-sdk/tutorials/deploy-local/1.deploy-local.ipynb?name=delete-resource-code)]

+To delete a deployed web service, use `service.delete()`.

+To delete a registered model, use `model.delete()`.

+For more information, see the documentation for [WebService.delete()](/python/api/azureml-core/azureml.core.webservice%28class%29#delete--) and [Model.delete()](/python/api/azureml-core/azureml.core.model.model#delete--).

+## Next steps

+* [Troubleshoot a failed deployment](../how-to-troubleshoot-deployment.md)

+* [Update web service](../how-to-deploy-update-web-service.md)

+* [One click deployment for automated ML runs in the Azure Machine Learning studio](../how-to-use-automated-ml-for-ml-models.md#deploy-your-model)

+* [Use TLS to secure a web service through Azure Machine Learning](../how-to-secure-web-service.md)

+* [Monitor your Azure Machine Learning models with Application Insights](../how-to-enable-app-insights.md)

+* [Create event alerts and triggers for model deployments](../how-to-use-event-grid.md)

+* [Secure inferencing environment with Azure Virtual Network](how-to-secure-inferencing-vnet.md)

+ Title: Update deployed web services

+description: Learn how to refresh a web service that is already deployed in Azure Machine Learning. You can update settings such as model, environment, and entry script.

+# Update a deployed web service (v1)

+In this article, you learn how to update a web service that was deployed with Azure Machine Learning.

+## Prerequisites

+- This article assumes you have already deployed a web service with Azure Machine Learning. If you need to learn how to deploy a web service, [follow these steps](how-to-deploy-and-where.md).

+- The code snippets in this article assume that the `ws` variable has already been initialized to your workspace by using the [Workflow()](/python/api/azureml-core/azureml.core.workspace.workspace#constructor) constructor or loading a saved configuration with [Workspace.from_config()](/python/api/azureml-core/azureml.core.workspace.workspace#azureml-core-workspace-workspace-from-config). The following snippet demonstrates how to use the constructor:

+ [!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]

+ ```python

+ from azureml.core import Workspace

+ ws = Workspace(subscription_id="mysubscriptionid",

+ resource_group="myresourcegroup",

+ workspace_name="myworkspace")

+ ```

+## Update web service

+To update a web service, use the `update` method. You can update the web service to use a new model, a new entry script, or new dependencies that can be specified in an inference configuration. For more information, see the documentation for [Webservice.update](/python/api/azureml-core/azureml.core.webservice.webservice.webservice#update--args-).

+See [AKS Service Update Method.](/python/api/azureml-core/azureml.core.webservice.akswebservice#update-image-none--autoscale-enabled-none--autoscale-min-replicas-none--autoscale-max-replicas-none--autoscale-refresh-seconds-none--autoscale-target-utilization-none--collect-model-data-none--auth-enabled-none--cpu-cores-none--memory-gb-none--enable-app-insights-none--scoring-timeout-ms-none--replica-max-concurrent-requests-none--max-request-wait-time-none--num-replicas-none--tags-none--properties-none--description-none--models-none--inference-config-none--gpu-cores-none--period-seconds-none--initial-delay-seconds-none--timeout-seconds-none--success-threshold-none--failure-threshold-none--namespace-none--token-auth-enabled-none-)

+See [ACI Service Update Method.](/python/api/azureml-core/azureml.core.webservice.aci.aciwebservice#update-image-none--tags-none--properties-none--description-none--auth-enabled-none--ssl-enabled-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--ssl-cname-none--enable-app-insights-none--models-none--inference-config-none-)

+> [!IMPORTANT]

+> When you create a new version of a model, you must manually update each service that you want to use it.

+>

+> You can not use the SDK to update a web service published from the Azure Machine Learning designer.

+> [!IMPORTANT]

+> Azure Kubernetes Service uses [Blobfuse FlexVolume driver](https://github.com/Azure/kubernetes-volume-drivers/blob/master/flexvolume/blobfuse/README.md) for the versions <=1.16 and [Blob CSI driver](https://github.com/kubernetes-sigs/blob-csi-driver/blob/master/README.md) for the versions >=1.17.

+>

+> Therefore, it is important to re-deploy or update the web service after cluster upgrade in order to deploy to correct blobfuse method for the cluster version.

+> [!NOTE]

+> When an operation is already in progress, any new operation on that same web service will respond with 409 conflict error. For example, If create or update web service operation is in progress and if you trigger a new Delete operation it will throw an error.

+**Using the SDK**

+The following code shows how to use the SDK to update the model, environment, and entry script for a web service:

+```python

+from azureml.core import Environment

+from azureml.core.webservice import Webservice

+from azureml.core.model import Model, InferenceConfig

+# Register new model.

+new_model = Model.register(model_path="outputs/sklearn_mnist_model.pkl",

+ model_name="sklearn_mnist",

+ tags={"key": "0.1"},

+ description="test",

+ workspace=ws)

+# Use version 3 of the environment.

+deploy_env = Environment.get(workspace=ws,name="myenv",version="3")

+inference_config = InferenceConfig(entry_script="score.py",

+ environment=deploy_env)

+service_name = 'myservice'

+# Retrieve existing service.

+service = Webservice(name=service_name, workspace=ws)

+# Update to new model(s).

+service.update(models=[new_model], inference_config=inference_config)

+service.wait_for_deployment(show_output=True)

+print(service.state)

+print(service.get_logs())

+```

+**Using the CLI**

+You can also update a web service by using the ML CLI. The following example demonstrates registering a new model and then updating a web service to use the new model:

+```azurecli

+```

+> [!TIP]

+> In this example, a JSON document is used to pass the model information from the registration command into the update command.

+>

+> To update the service to use a new entry script or environment, create an [inference configuration file](reference-azure-machine-learning-cli.md#inference-configuration-schema) and specify it with the `ic` parameter.

+For more information, see the [az ml service update](/cli/azure/ml(v1)/service#az-ml-v1--service-update) documentation.

+## Next steps

+* [Troubleshoot a failed deployment](../how-to-troubleshoot-deployment.md)

+* [Create client applications to consume web services](../how-to-consume-web-service.md)

+* [How to deploy a model using a custom Docker image](../how-to-deploy-custom-container.md)

+* [Use TLS to secure a web service through Azure Machine Learning](how-to-secure-web-service.md)

+* [Monitor your Azure Machine Learning models with Application Insights](../how-to-enable-app-insights.md)

+* [Collect data for models in production](../how-to-enable-data-collection.md)

+* [Create event alerts and triggers for model deployments](../how-to-use-event-grid.md)

+ Title: Secure inferencing environments with virtual networks

+description: Use an isolated Azure Virtual Network to secure your Azure Machine Learning inferencing environment.

+# Secure an Azure Machine Learning inferencing environment with virtual networks

+In this article, you learn how to secure inferencing environments with a virtual network in Azure Machine Learning.

+> [!TIP]

+> This article is part of a series on securing an Azure Machine Learning workflow. See the other articles in this series:

+>

+> * [Virtual network overview](../how-to-network-security-overview.md)

+> * [Secure the workspace resources](../how-to-secure-workspace-vnet.md)

+> * [Secure the training environment](../how-to-secure-training-vnet.md)

+> * [Enable studio functionality](../how-to-enable-studio-virtual-network.md)

+> * [Use custom DNS](../how-to-custom-dns.md)

+> * [Use a firewall](../how-to-access-azureml-behind-firewall.md)

+>

+> For a tutorial on creating a secure workspace, see [Tutorial: Create a secure workspace](../tutorial-create-secure-workspace.md) or [Tutorial: Create a secure workspace using a template](../tutorial-create-secure-workspace-template.md).

+In this article you learn how to secure the following inferencing resources in a virtual network:

+> [!div class="checklist"]

+> - Default Azure Kubernetes Service (AKS) cluster

+> - Private AKS cluster

+> - AKS cluster with private link

+## Prerequisites

+ - "Microsoft.Network/virtualNetworks/join/action" on the virtual network resource.

+ - "Microsoft.Network/virtualNetworks/subnet/join/action" on the subnet resource.

+ For more information on Azure RBAC with networking, see the [Networking built-in roles](../../role-based-access-control/built-in-roles.md#networking)

+## Limitations

+### Azure Container Instances

+When your Azure Machine Learning workspace is configured with a private endpoint, deploying to Azure Container Instances in a VNet is not supported. Instead, consider using a [Managed online endpoint with network isolation](../how-to-secure-online-endpoint.md).

+### Azure Kubernetes Service

+* If your AKS cluster is behind of a VNET, your workspace and its associated resources (storage, key vault, Azure Container Registry) must have private endpoints or service endpoints in the same VNET as AKS cluster's VNET. Please read tutorial [create a secure workspace](../tutorial-create-secure-workspace.md) to add those private endpoints or service endpoints to your VNET.

+* If your workspace has a __private endpoint__, the Azure Kubernetes Service cluster must be in the same Azure region as the workspace.

+* Using a [public fully qualified domain name (FQDN) with a private AKS cluster](../../aks/private-clusters.md) is __not supported__ with Azure Machine learning.

+<a id="aksvnet"></a>

+## Azure Kubernetes Service

+> [!IMPORTANT]

+> To use an AKS cluster in a virtual network, first follow the prerequisites in [Configure advanced networking in Azure Kubernetes Service (AKS)](../../aks/configure-azure-cni.md#prerequisites).

+To add AKS in a virtual network to your workspace, use the following steps:

+1. Sign in to [Azure Machine Learning studio](https://ml.azure.com/), and then select your subscription and workspace.

+1. Select __Compute__ on the left, __Inference clusters__ from the center, and then select __+ New__.

+ :::image type="content" source="./media/how-to-secure-inferencing-vnet/create-inference.png" alt-text="Screenshot of create inference cluster dialog.":::

+1. From the __Create inference cluster__ dialog, select __Create new__ and the VM size to use for the cluster. Finally, select __Next__.

+ :::image type="content" source="./media/how-to-secure-inferencing-vnet/create-inference-vm.png" alt-text="Screenshot of VM settings.":::

+1. From the __Configure Settings__ section, enter a __Compute name__, select the __Cluster Purpose__, __Number of nodes__, and then select __Advanced__ to display the network settings. In the __Configure virtual network__ area, set the following values:

+ * Set the __Virtual network__ to use.

+ > [!TIP]

+ > If your workspace uses a private endpoint to connect to the virtual network, the __Virtual network__ selection field is greyed out.

+ * Set the __Subnet__ to create the cluster in.

+ * In the __Kubernetes Service address range__ field, enter the Kubernetes service address range. This address range uses a Classless Inter-Domain Routing (CIDR) notation IP range to define the IP addresses that are available for the cluster. It must not overlap with any subnet IP ranges (for example, 10.0.0.0/16).

+ * In the __Kubernetes DNS service IP address__ field, enter the Kubernetes DNS service IP address. This IP address is assigned to the Kubernetes DNS service. It must be within the Kubernetes service address range (for example, 10.0.0.10).

+ * In the __Docker bridge address__ field, enter the Docker bridge address. This IP address is assigned to Docker Bridge. It must not be in any subnet IP ranges, or the Kubernetes service address range (for example, 172.18.0.1/16).

+ :::image type="content" source="./media/how-to-secure-inferencing-vnet/create-inference-settings.png" alt-text="Screenshot of configure network settings.":::

+1. When you deploy a model as a web service to AKS, a scoring endpoint is created to handle inferencing requests. Make sure that the network security group (NSG) that controls the virtual network has an inbound security rule enabled for the IP address of the scoring endpoint if you want to call it from outside the virtual network.

+ To find the IP address of the scoring endpoint, look at the scoring URI for the deployed service. For information on viewing the scoring URI, see [Consume a model deployed as a web service](../how-to-consume-web-service.md#connection-information).

+ > [!IMPORTANT]

+ > Keep the default outbound rules for the NSG. For more information, see the default security rules in [Security groups](../../virtual-network/network-security-groups-overview.md#default-security-rules).

+ ](./media/how-to-secure-inferencing-vnet/aks-vnet-inbound-nsg-scoring.png#lightbox)

+ > [!IMPORTANT]

+ > The IP address shown in the image for the scoring endpoint will be different for your deployments. While the same IP is shared by all deployments to one AKS cluster, each AKS cluster will have a different IP address.

+You can also use the Azure Machine Learning SDK to add Azure Kubernetes Service in a virtual network. If you already have an AKS cluster in a virtual network, attach it to the workspace as described in [How to deploy to AKS](how-to-deploy-and-where.md). The following code creates a new AKS instance in the `default` subnet of a virtual network named `mynetwork`:

+```python

+from azureml.core.compute import ComputeTarget, AksCompute

+# Create the compute configuration and set virtual network information

+config = AksCompute.provisioning_configuration(location="eastus2")

+config.vnet_resourcegroup_name = "mygroup"

+config.vnet_name = "mynetwork"

+config.subnet_name = "default"

+config.service_cidr = "10.0.0.0/16"

+config.dns_service_ip = "10.0.0.10"

+config.docker_bridge_cidr = "172.17.0.1/16"

+# Create the compute target

+aks_target = ComputeTarget.create(workspace=ws,

+ name="myaks",

+ provisioning_configuration=config)

+```

+When the creation process is completed, you can run inference, or model scoring, on an AKS cluster behind a virtual network. For more information, see [How to deploy to AKS](how-to-deploy-and-where.md).

+For more information on using Role-Based Access Control with Kubernetes, see [Use Azure RBAC for Kubernetes authorization](../../aks/manage-azure-rbac.md).

+## Network contributor role

+> [!IMPORTANT]

+> If you create or attach an AKS cluster by providing a virtual network you previously created, you must grant the service principal (SP) or managed identity for your AKS cluster the _Network Contributor_ role to the resource group that contains the virtual network.

+>

+> To add the identity as network contributor, use the following steps:

+1. To find the service principal or managed identity ID for AKS, use the following Azure CLI commands. Replace `<aks-cluster-name>` with the name of the cluster. Replace `<resource-group-name>` with the name of the resource group that _contains the AKS cluster_:

+ ```azurecli-interactive

+ az aks show -n <aks-cluster-name> --resource-group <resource-group-name> --query servicePrincipalProfile.clientId

+ ```

+ If this command returns a value of `msi`, use the following command to identify the principal ID for the managed identity:

+ ```azurecli-interactive

+ az aks show -n <aks-cluster-name> --resource-group <resource-group-name> --query identity.principalId

+ ```

+1. To find the ID of the resource group that contains your virtual network, use the following command. Replace `<resource-group-name>` with the name of the resource group that _contains the virtual network_:

+ ```azurecli-interactive

+ az group show -n <resource-group-name> --query id

+ ```

+1. To add the service principal or managed identity as a network contributor, use the following command. Replace `<SP-or-managed-identity>` with the ID returned for the service principal or managed identity. Replace `<resource-group-id>` with the ID returned for the resource group that contains the virtual network:

+ ```azurecli-interactive

+ az role assignment create --assignee <SP-or-managed-identity> --role 'Network Contributor' --scope <resource-group-id>

+ ```

+For more information on using the internal load balancer with AKS, see [Use internal load balancer with Azure Kubernetes Service](../../aks/internal-lb.md).

+## Secure VNet traffic

+There are two approaches to isolate traffic to and from the AKS cluster to the virtual network:

+* __Private AKS cluster__: This approach uses Azure Private Link to secure communications with the cluster for deployment/management operations.

+* __Internal AKS load balancer__: This approach configures the endpoint for your deployments to AKS to use a private IP within the virtual network.

+### Private AKS cluster

+By default, AKS clusters have a control plane, or API server, with public IP addresses. You can configure AKS to use a private control plane by creating a private AKS cluster. For more information, see [Create a private Azure Kubernetes Service cluster](../../aks/private-clusters.md).

+After you create the private AKS cluster, [attach the cluster to the virtual network](how-to-create-attach-kubernetes.md) to use with Azure Machine Learning.

+### Internal AKS load balancer

+By default, AKS deployments use a [public load balancer](../../aks/load-balancer-standard.md). In this section, you learn how to configure AKS to use an internal load balancer. An internal (or private) load balancer is used where only private IPs are allowed as frontend. Internal load balancers are used to load balance traffic inside a virtual network

+A private load balancer is enabled by configuring AKS to use an _internal load balancer_.

+#### Enable private load balancer

+> [!IMPORTANT]

+> You cannot enable private IP when creating the Azure Kubernetes Service cluster in Azure Machine Learning studio. You can create one with an internal load balancer when using the Python SDK or Azure CLI extension for machine learning.

+The following examples demonstrate how to __create a new AKS cluster with a private IP/internal load balancer__ using the SDK and CLI:

+# [Python](#tab/python)

+```python

+import azureml.core

+from azureml.core.compute import AksCompute, ComputeTarget

+# Verify that cluster does not exist already

+try:

+ aks_target = AksCompute(workspace=ws, name=aks_cluster_name)

+ print("Found existing aks cluster")

+except:

+ print("Creating new aks cluster")

+ # Subnet to use for AKS

+ subnet_name = "default"

+ # Create AKS configuration

+ prov_config=AksCompute.provisioning_configuration(load_balancer_type="InternalLoadBalancer")

+ # Set info for existing virtual network to create the cluster in

+ prov_config.vnet_resourcegroup_name = "myvnetresourcegroup"

+ prov_config.vnet_name = "myvnetname"

+ prov_config.service_cidr = "10.0.0.0/16"

+ prov_config.dns_service_ip = "10.0.0.10"

+ prov_config.subnet_name = subnet_name

+ prov_config.load_balancer_subnet = subnet_name

+ prov_config.docker_bridge_cidr = "172.17.0.1/16"

+ # Create compute target

+ aks_target = ComputeTarget.create(workspace = ws, name = "myaks", provisioning_configuration = prov_config)

+ # Wait for the operation to complete

+ aks_target.wait_for_completion(show_output = True)

+```

+# [Azure CLI](#tab/azure-cli)

+```azurecli

+az ml computetarget create aks -n myaks --load-balancer-type InternalLoadBalancer

+```

+To upgrade an existing AKS cluster to use an internal load balancer, use the following command:

+```azurecli

+az ml computetarget update aks \

+ -n myaks \

+ --load-balancer-subnet mysubnet \

+ --load-balancer-type InternalLoadBalancer \

+ --workspace-name myworkspace \

+ -g myresourcegroup

+```

+For more information, see the [az ml computetarget create aks](/cli/azure/ml(v1)/computetarget/create#az-ml-computetarget-create-aks) and [az ml computetarget update aks](/cli/azure/ml(v1)/computetarget/update#az-ml-computetarget-update-aks) reference.

+When __attaching an existing cluster__ to your workspace, use the `load_balancer_type` and `load_balancer_subnet` parameters of [AksCompute.attach_configuration()](/python/api/azureml-core/azureml.core.compute.aks.akscompute#azureml-core-compute-aks-akscompute-attach-configuration) to configure the load balancer.

+For information on attaching a cluster, see [Attach an existing AKS cluster](how-to-create-attach-kubernetes.md).

+## Limit outbound connectivity from the virtual network

+If you don't want to use the default outbound rules and you do want to limit the outbound access of your virtual network, you must allow access to Azure Container Registry. For example, make sure that your Network Security Groups (NSG) contains a rule that allows access to the __AzureContainerRegistry.RegionName__ service tag where `{RegionName} is the name of an Azure region.

+## Next steps

+This article is part of a series on securing an Azure Machine Learning workflow. See the other articles in this series:

+* [Virtual network overview](../how-to-network-security-overview.md)

+* [Secure the workspace resources](../how-to-secure-workspace-vnet.md)

+* [Secure the training environment](../how-to-secure-training-vnet.md)

+* [Enable studio functionality](../how-to-enable-studio-virtual-network.md)

+* [Use custom DNS](../how-to-custom-dns.md)

+* [Use a firewall](../how-to-access-azureml-behind-firewall.md)

+ Title: Secure web services using TLS

+description: Learn how to enable HTTPS with TLS version 1.2 to secure a web service that's deployed through Azure Machine Learning.

+# Use TLS to secure a web service through Azure Machine Learning

+This article shows you how to secure a web service that's deployed through Azure Machine Learning.

+You use [HTTPS](https://en.wikipedia.org/wiki/HTTPS) to restrict access to web services and secure the data that clients submit. HTTPS helps secure communications between a client and a web service by encrypting communications between the two. Encryption uses [Transport Layer Security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). TLS is sometimes still referred to as *Secure Sockets Layer* (SSL), which was the predecessor of TLS.

+> [!TIP]

+> The Azure Machine Learning SDK uses the term "SSL" for properties that are related to secure communications. This doesn't mean that your web service doesn't use *TLS*. SSL is just a more commonly recognized term.

+>

+> Specifically, web services deployed through Azure Machine Learning support TLS version 1.2 for AKS and ACI. For ACI deployments, if you are on older TLS version, we recommend re-deploying to get the latest TLS version.

+>

+> TLS version 1.3 for Azure Machine Learning - AKS Inference is unsupported.

+TLS and SSL both rely on *digital certificates*, which help with encryption and identity verification. For more information on how digital certificates work, see the Wikipedia topic [Public key infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure).

+> [!WARNING]

+> If you don't use HTTPS for your web service, data that's sent to and from the service might be visible to others on the internet.

+>

+> HTTPS also enables the client to verify the authenticity of the server that it's connecting to. This feature protects clients against [man-in-the-middle](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks.

+This is the general process to secure a web service:

+1. Get a domain name.

+2. Get a digital certificate.

+3. Deploy or update the web service with TLS enabled.

+4. Update your DNS to point to the web service.

+> [!IMPORTANT]

+> If you're deploying to Azure Kubernetes Service (AKS), you can purchase your own certificate or use a certificate that's provided by Microsoft. If you use a certificate from Microsoft, you don't need to get a domain name or TLS/SSL certificate. For more information, see the [Enable TLS and deploy](#enable) section of this article.

+There are slight differences when you secure across [deployment targets](how-to-deploy-and-where.md).

+## Get a domain name

+If you don't already own a domain name, purchase one from a *domain name registrar*. The process and price differ among registrars. The registrar provides tools to manage the domain name. You use these tools to map a fully qualified domain name (FQDN) (such as www\.contoso.com) to the IP address that hosts your web service.

+## Get a TLS/SSL certificate

+There are many ways to get an TLS/SSL certificate (digital certificate). The most common is to purchase one from a *certificate authority* (CA). Regardless of where you get the certificate, you need the following files:

+* A **certificate**. The certificate must contain the full certificate chain, and it must be "PEM-encoded."

+* A **key**. The key must also be PEM-encoded.

+When you request a certificate, you must provide the FQDN of the address that you plan to use for the web service (for example, www\.contoso.com). The address that's stamped into the certificate and the address that the clients use are compared to verify the identity of the web service. If those addresses don't match, the client gets an error message.

+> [!TIP]

+> If the certificate authority can't provide the certificate and key as PEM-encoded files, you can use a utility such as [OpenSSL](https://www.openssl.org/) to change the format.

+> [!WARNING]

+> Use *self-signed* certificates only for development. Don't use them in production environments. Self-signed certificates can cause problems in your client applications. For more information, see the documentation for the network libraries that your client application uses.

+## <a id="enable"></a> Enable TLS and deploy

+**For AKS deployment**, you can enable TLS termination when you [create or attach an AKS cluster](how-to-create-attach-kubernetes.md) in AzureML workspace. At AKS model deployment time, you can disable TLS termination with deployment configuration object, otherwise all AKS model deployment by default will have TLS termination enabled at AKS cluster create or attach time.

+For ACI deployment, you can enable TLS termination at model deployment time with deployment configuration object.

+### Deploy on Azure Kubernetes Service

+ > [!NOTE]

+ > The information in this section also applies when you deploy a secure web service for the designer. If you aren't familiar with using the Python SDK, see [What is the Azure Machine Learning SDK for Python?](/python/api/overview/azure/ml/intro).

+When you [create or attach an AKS cluster](how-to-create-attach-kubernetes.md) in AzureML workspace, you can enable TLS termination with **[AksCompute.provisioning_configuration()](/python/api/azureml-core/azureml.core.compute.akscompute#provisioning-configuration-agent-count-none--vm-size-none--ssl-cname-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--location-none--vnet-resourcegroup-name-none--vnet-name-none--subnet-name-none--service-cidr-none--dns-service-ip-none--docker-bridge-cidr-none--cluster-purpose-none--load-balancer-type-none--load-balancer-subnet-none-)** and **[AksCompute.attach_configuration()](/python/api/azureml-core/azureml.core.compute.akscompute#attach-configuration-resource-group-none--cluster-name-none--resource-id-none--cluster-purpose-none-)** configuration objects. Both methods return a configuration object that has an **enable_ssl** method, and you can use **enable_ssl** method to enable TLS.

+You can enable TLS either with Microsoft certificate or a custom certificate purchased from CA.

+* **When you use a certificate from Microsoft**, you must use the *leaf_domain_label* parameter. This parameter generates the DNS name for the service. For example, a value of "contoso" creates a domain name of "contoso\<six-random-characters>.\<azureregion>.cloudapp.azure.com", where \<azureregion> is the region that contains the service. Optionally, you can use the *overwrite_existing_domain* parameter to overwrite the existing *leaf_domain_label*. The following example demonstrates how to create a configuration that enables an TLS with Microsoft certificate:

+ ```python

+ from azureml.core.compute import AksCompute

+ # Config used to create a new AKS cluster and enable TLS

+ provisioning_config = AksCompute.provisioning_configuration()

+ # Leaf domain label generates a name using the formula

+ # "<leaf-domain-label>######.<azure-region>.cloudapp.azure.com"

+ # where "######" is a random series of characters

+ provisioning_config.enable_ssl(leaf_domain_label = "contoso")

+ # Config used to attach an existing AKS cluster to your workspace and enable TLS

+ attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+ cluster_name = cluster_name)

+ # Leaf domain label generates a name using the formula

+ # "<leaf-domain-label>######.<azure-region>.cloudapp.azure.com"

+ # where "######" is a random series of characters

+ attach_config.enable_ssl(leaf_domain_label = "contoso")

+ ```

+ > [!IMPORTANT]

+ > When you use a certificate from Microsoft, you don't need to purchase your own certificate or domain name.

+* **When you use a custom certificate that you purchased**, you use the *ssl_cert_pem_file*, *ssl_key_pem_file*, and *ssl_cname* parameters. The following example demonstrates how to use .pem files to create a configuration that uses a TLS/SSL certificate that you purchased:

+

+ ```python

+ from azureml.core.compute import AksCompute

+ # Config used to create a new AKS cluster and enable TLS

+ provisioning_config = AksCompute.provisioning_configuration()

+ provisioning_config.enable_ssl(ssl_cert_pem_file="cert.pem",

+ ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")

+ # Config used to attach an existing AKS cluster to your workspace and enable SSL

+ attach_config = AksCompute.attach_configuration(resource_group = resource_group,

+ cluster_name = cluster_name)

+ attach_config.enable_ssl(ssl_cert_pem_file="cert.pem",

+ ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")

+ ```

+For more information about *enable_ssl*, see [AksProvisioningConfiguration.enable_ssl()](/python/api/azureml-core/azureml.core.compute.aks.aksprovisioningconfiguration#enable-ssl-ssl-cname-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--leaf-domain-label-none--overwrite-existing-domain-false-) and [AksAttachConfiguration.enable_ssl()](/python/api/azureml-core/azureml.core.compute.aks.aksattachconfiguration#enable-ssl-ssl-cname-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--leaf-domain-label-none--overwrite-existing-domain-false-).

+### Deploy on Azure Container Instances

+When you deploy to Azure Container Instances, you provide values for TLS-related parameters, as the following code snippet shows:

+```python

+from azureml.core.webservice import AciWebservice

+aci_config = AciWebservice.deploy_configuration(

+ ssl_enabled=True, ssl_cert_pem_file="cert.pem", ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")

+```

+For more information, see [AciWebservice.deploy_configuration()](/python/api/azureml-core/azureml.core.webservice.aciwebservice#deploy-configuration-cpu-cores-none--memory-gb-none--tags-none--properties-none--description-none--location-none--auth-enabled-none--ssl-enabled-none--enable-app-insights-none--ssl-cert-pem-file-none--ssl-key-pem-file-none--ssl-cname-none--dns-name-label-none--primary-key-none--secondary-key-none--collect-model-data-none--cmk-vault-base-url-none--cmk-key-name-none--cmk-key-version-none-).

+## Update your DNS

+For either AKS deployment with custom certificate or ACI deployment, you must update your DNS record to point to the IP address of scoring endpoint.

+> [!IMPORTANT]

+> When you use a certificate from Microsoft for AKS deployment, you don't need to manually update the DNS value for the cluster. The value should be set automatically.

+You can follow following steps to update DNS record for your custom domain name:

+1. Get scoring endpoint IP address from scoring endpoint URI, which is usually in the format of `http://104.214.29.152:80/api/v1/service/<service-name>/score`. In this example, the IP address is 104.214.29.152.

+1. Use the tools from your domain name registrar to update the DNS record for your domain name. The record maps the FQDN (for example, www\.contoso.com) to the IP address. The record must point to the IP address of scoring endpoint.

+ > [!TIP]

+ > Microsoft does is not responsible for updating the DNS for your custom DNS name or certificate. You must update it with your domain name registrar.

+1. After DNS record update, you can validate DNS resolution using *nslookup custom-domain-name* command. If DNS record is correctly updated, the custom domain name will point to the IP address of scoring endpoint.

+ There can be a delay of minutes or hours before clients can resolve the domain name, depending on the registrar and the "time to live" (TTL) that's configured for the domain name.

+For more information on DNS resolution with Azure Machine Learning, see [How to use your workspace with a custom DNS server](../how-to-custom-dns.md).

+## Update the TLS/SSL certificate

+TLS/SSL certificates expire and must be renewed. Typically this happens every year. Use the information in the following sections to update and renew your certificate for models deployed to Azure Kubernetes Service:

+### Update a Microsoft generated certificate

+If the certificate was originally generated by Microsoft (when using the *leaf_domain_label* to create the service), **it will automatically renew** when needed. If you want to manually renew it, use one of the following examples to update the certificate:

+> [!IMPORTANT]

+> * If the existing certificate is still valid, use `renew=True` (SDK) or `--ssl-renew` (CLI) to force the configuration to renew it. For example, if the existing certificate is still valid for 10 days and you don't use `renew=True`, the certificate may not be renewed.

+> * When the service was originally deployed, the `leaf_domain_label` is used to create a DNS name using the pattern `<leaf-domain-label>######.<azure-region>.cloudapp.azure.com`. To preserve the existing name (including the 6 digits originally generated), use the original `leaf_domain_label` value. Do not include the 6 digits that were generated.

+**Use the SDK**

+```python

+from azureml.core.compute import AksCompute

+from azureml.core.compute.aks import AksUpdateConfiguration

+from azureml.core.compute.aks import SslConfiguration

+# Get the existing cluster

+aks_target = AksCompute(ws, clustername)

+# Update the existing certificate by referencing the leaf domain label

+ssl_configuration = SslConfiguration(leaf_domain_label="myaks", overwrite_existing_domain=True, renew=True)

+update_config = AksUpdateConfiguration(ssl_configuration)

+aks_target.update(update_config)

+```

+**Use the CLI**

+```azurecli

+az ml computetarget update aks -g "myresourcegroup" -w "myresourceworkspace" -n "myaks" --ssl-leaf-domain-label "myaks" --ssl-overwrite-domain True --ssl-renew

+```

+For more information, see the following reference docs:

+* [SslConfiguration](/python/api/azureml-core/azureml.core.compute.aks.sslconfiguration)

+* [AksUpdateConfiguration](/python/api/azureml-core/azureml.core.compute.aks.aksupdateconfiguration)

+### Update custom certificate

+If the certificate was originally generated by a certificate authority, use the following steps:

+1. Use the documentation provided by the certificate authority to renew the certificate. This process creates new certificate files.

+1. Use either the SDK or CLI to update the service with the new certificate:

+ **Use the SDK**

+ ```python

+ from azureml.core.compute import AksCompute

+ from azureml.core.compute.aks import AksUpdateConfiguration

+ from azureml.core.compute.aks import SslConfiguration

+

+ # Read the certificate file

+ def get_content(file_name):

+ with open(file_name, 'r') as f:

+ return f.read()

+ # Get the existing cluster

+ aks_target = AksCompute(ws, clustername)

+

+ # Update cluster with custom certificate

+ ssl_configuration = SslConfiguration(cname="myaks", cert=get_content('cert.pem'), key=get_content('key.pem'))

+ update_config = AksUpdateConfiguration(ssl_configuration)

+ aks_target.update(update_config)

+ ```

+ **Use the CLI**

+ [!INCLUDE [cli v1](../../../includes/machine-learning-cli-v1.md)]

+ ```azurecli

+ az ml computetarget update aks -g "myresourcegroup" -w "myresourceworkspace" -n "myaks" --ssl-cname "myaks"--ssl-cert-file "cert.pem" --ssl-key-file "key.pem"

+ ```

+For more information, see the following reference docs:

+* [SslConfiguration](/python/api/azureml-core/azureml.core.compute.aks.sslconfiguration)

+* [AksUpdateConfiguration](/python/api/azureml-core/azureml.core.compute.aks.aksupdateconfiguration)

+## Disable TLS

+To disable TLS for a model deployed to Azure Kubernetes Service, create an `SslConfiguration` with `status="Disabled"`, then perform an update:

+```python

+from azureml.core.compute import AksCompute

+from azureml.core.compute.aks import AksUpdateConfiguration

+from azureml.core.compute.aks import SslConfiguration

+# Get the existing cluster

+aks_target = AksCompute(ws, clustername)

+# Disable TLS

+ssl_configuration = SslConfiguration(status="Disabled")

+update_config = AksUpdateConfiguration(ssl_configuration)

+aks_target.update(update_config)

+```

+## Next steps

+Learn how to:

+> Azure role assignments are cached by Azure Storage, so there may be a delay of up to 30 minutes between when you grant access to your remote rendering account and when it can be used to access your storage account. See the [Azure role-based access control (Azure RBAC) documentation](../../role-based-access-control/troubleshooting.md#symptomrole-assignment-changes-are-not-being-detected) for details.

+To make role assignments more manageable, avoid assigning roles directly to users. Instead, assign roles to groups. Assigning roles to groups instead of users also helps minimize the number of role assignments, which has a [limit of role assignments per subscription](troubleshooting.md#limits).

+- [Principal does not appear in Attribute source](conditions-troubleshoot.md#symptomprincipal-does-not-appear-in-attribute-source)

+- [Principal does not appear in Attribute source](conditions-troubleshoot.md#symptomprincipal-does-not-appear-in-attribute-source)

+## General issues

+### Symptom - Condition is not enforced

+### Symptom - Condition is not valid error when adding a condition

+## Issues in the visual editor

+### Symptom - Principal does not appear in Attribute source

+### Symptom - Principal does not appear in Attribute source when using PIM

+## Error messages in visual editor

+### Symptom - Condition not recognized

+After using the code editor, you switch to the visual editor and get a message similar to the following:

+`The current expression cannot be recognized. Switch to the code editor to edit the expression or delete the expression and add a new one.`

+Updates were made to the condition that the visual editor is not able to parse.

+Fix any [condition format or syntax](conditions-format.md) issues. Alternatively, you can delete the condition and try again.

+### Symptom - Attribute does not apply error for previously saved condition

+When you open a previously saved condition in the visual editor, you get the following message:

+`Attribute does not apply for the selected actions. Select a different set of actions.`

+**Cause**

+In May 2022, the Read a blob action was changed from the following format:

+`!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read'})`

+To exclude the `Blob.List` suboperation:

+`!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read'} AND NOT SubOperationMatches{'Blob.List'})`

+If you created a condition with the Read a blob action prior to May 2022, you might see this error message in the visual editor.

+**Solution**

+Open the **Select an action** pane and reselect the **Read a blob** action.

+### Symptom - Attribute does not apply error

+When you select one or more actions in the visual editor with an existing expression, you get the following message and the previously selected attribute is removed:

+`Attribute does not apply for the selected actions. Select a different set of actions.`

+The previously selected attribute no longer applies to the currently selected actions.

+**Solution 1**

+In the **Add action** section, select an action that applies to the selected attribute. For a list of storage actions that each storage attribute supports, see [Actions and attributes for Azure role assignment conditions in Azure Storage (preview)](../storage/common/storage-auth-abac-attributes.md).

+**Solution 2**

+In the **Build expression** section, select an attribute that applies to the currently selected actions. For a list of storage attributes that each storage action supports, see [Actions and attributes for Azure role assignment conditions in Azure Storage (preview)](../storage/common/storage-auth-abac-attributes.md).

+### Symptom - Attribute does not apply in this context warning

+When you make edits in the code editor and then switch to the visual editor, you get the following message and the previously selected attribute is removed:

+`Attribute does not apply in this context. Use a different role assignment scope or remove the expression.`

+**Cause**

+The specified attribute is not available in the current scope, such as using `Version ID` in a storage account with hierarchical namespace enabled.

+If you want to use the currently specified attribute, create the role assignment condition at a different scope, such as resource group scope. Or remove and re-create the expression using the currently selected actions.

+### Symptom - Attribute is not recognized error

+When you make edits in the code editor and then switch to the visual editor, you get the following message and the previously selected attribute is removed:

+`Attribute is not recognized. Select a valid attribute or remove the expression.`

+The specified attribute is not recognized, possibly because of a typo.

+In the code editor, fix the typo. Or remove the existing expression and use the visual editor to select an attribute.

+### Symptom - Attribute value is invalid error

+When you make edits in the code editor and then switch to the visual editor, you get the following message and the previously selected attribute is removed:

+`Attribute value is invalid. Select another attribute or value.`

+The right side of the expression contains an attribute or value that is not valid.

+Use the visual editor to select an attribute or specify a value.

+### Symptom - No actions selected error

+When you remove all of the actions in the visual editor, you get the following message:

+`No actions selected. Select one or more actions to edit expressions.`

+There is an existing expression, but no actions have been selected as a target.

+In the **Add action** section, add one or more actions that the expression should target.

+## Error messages in Azure PowerShell

+### Symptom - Resource attribute is not valid error

+When you try to add a role assignment with a condition using Azure PowerShell, you get an error similar to:

+```

+New-AzRoleAssignment : Resource attribute

+Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags:Project<$> is not valid.

+```

+**Cause**

+If your condition includes a dollar sign ($), you must prefix it with a backtick (\`).

+**Solution**

+Add a backtick (\`) before each dollar sign. The following shows an example. For more information about rules for quotation marks in PowerShell, see [About Quoting Rules](/powershell/module/microsoft.powershell.core/about/about_quoting_rules).

+```azurepowershell

+$condition = "((!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read'} AND NOT SubOperationMatches{'Blob.List'})) OR (@Resource[Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags:Project<`$key_case_sensitive`$>] StringEquals 'Cascade'))"

+```

+### Symptom - Error when copying and pasting a condition

+## Error messages in Azure CLI

+### Symptom - Resource attribute is not valid error

+When you try to add a role assignment with a condition using Azure CLI, you get an error similar to:

+```

+Resource attribute Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags:Project<$> is not valid.

+```

+If your condition includes a dollar sign ($), you must prefix it with a backslash (\\).

+**Solution**

+Add a backslash (\\) before each dollar sign. The following shows an example. For more information about rules for quotation marks in Bash, see [Double Quotes](https://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html).

+```azurecli

+condition="((!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read'} AND NOT SubOperationMatches{'Blob.List'})) OR (@Resource[Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags:Project<\$key_case_sensitive\$>] StringEquals 'Cascade'))"

+```

+### Symptom - Unrecognized arguments error

+When you try to add a role assignment with a condition using Azure CLI, you get an error similar to:

+`az: error: unrecognized arguments: --description {description} --condition {condition} --condition-version 2.0`

+**Cause**

+You are likely using an earlier version of Azure CLI that does not support role assignment condition parameters.

+Update to the latest version of Azure CLI (2.18 or later). For more information, see [Install the Azure CLI](/cli/azure/install-azure-cli).

+### Symptom - Error when assigning a condition string to a variable in Bash

+When you try to assign a condition string to a variable in Bash, you get the `bash: !: event not found` message.

+**Cause**

+In Bash, if history expansion is enabled, you might see the message `bash: !: event not found` because of the exclamation point (!).

+**Solution**

+Disable history expansion with the command `set +H`. To re-enable history expansion, use `set -H`.

+The role assignments limit for a subscription is currently being increased. For more information, see [Troubleshoot Azure RBAC](troubleshooting.md#limits).

+If you are getting close to the maximum number and you try to add more role assignments, you'll see a warning in the **Add role assignment** pane. For ways that you can reduce the number of role assignments, see [Troubleshoot Azure RBAC](troubleshooting.md#limits).

+If you get the error message: "The provided information does not map to a role assignment", make sure that you also specify the `-Scope` or `-ResourceGroupName` parameters. For more information, see [Troubleshoot Azure RBAC](troubleshooting.md#symptomrole-assignments-with-identity-not-found).

+You can have up to **2000** role assignments in each subscription. This limit includes role assignments at the subscription, resource group, and resource scopes. You can have up to **500** role assignments in each management group. The role assignments limit for a subscription is currently being increased. For more information, see [Troubleshoot Azure RBAC](troubleshooting.md#limits).

+This article describes some common solutions for issues related to Azure role-based access control (Azure RBAC).

+## Limits

+### Symptom - No more role assignments can be created

+When you try to assign a role, you get the following error message:

+`No more role assignments can be created (code: RoleAssignmentLimitExceeded)`

+**Cause**

+Azure supports up to **2000** role assignments per subscription. This limit includes role assignments at the subscription, resource group, and resource scopes, but not at the management group scope.

+**Solution**

+Try to reduce the number of role assignments in the subscription. Here are some ways that you can reduce the number of role assignments:

+### Symptom - No more role assignments can be created at management group scope

+You are unable to assign a role at management group scope.

+**Cause**

+**Solution**

+Try to reduce the number of role assignments in the management group.

+## Azure role assignments

+### Symptom - Unable to assign a role

+You are unable to assign a role in the Azure portal on **Access control (IAM)** because the **Add** > **Add role assignment** option is disabled or because you get the following permissions error:

+`The client with object id does not have authorization to perform action`

+**Cause**

+You are currently signed in with a user that does not have permission to assign roles at the selected scope.

+**Solution**

+Check that you are currently signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleAssignments/write` permission such as [Owner](built-in-roles.md#owner) or [User Access Administrator](built-in-roles.md#user-access-administrator) at the scope you are trying to assign the role.

+### Symptom - Unable to assign a role using a service principal with Azure CLI

+You are using a service principal to assign roles with Azure CLI and you get the following error:

+`Insufficient privileges to complete the operation`

+For example, let's say that you have a service principal that has been assigned the Owner role and you try to create the following role assignment as the service principal using Azure CLI:

+```azurecli

+az login --service-principal --username "SPNid" --password "password" --tenant "tenantid"

+az role assignment create --assignee "userupn" --role "Contributor" --scope "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}"

+```

+**Cause**

+It is likely Azure CLI is attempting to look up the assignee identity in Azure AD and the service principal cannot read Azure AD by default.

+**Solution**

+There are two ways to potentially resolve this error. The first way is to assign the [Directory Readers](../active-directory/roles/permissions-reference.md#directory-readers) role to the service principal so that it can read data in the directory.

+The second way to resolve this error is to create the role assignment by using the `--assignee-object-id` parameter instead of `--assignee`. By using `--assignee-object-id`, Azure CLI will skip the Azure AD lookup. You will need to get the object ID of the user, group, or application that you want to assign the role to. For more information, see [Assign Azure roles using Azure CLI](role-assignments-cli.md#assign-a-role-for-a-new-service-principal-at-a-resource-group-scope).

+```azurecli

+az role assignment create --assignee-object-id 11111111-1111-1111-1111-111111111111 --role "Contributor" --scope "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}"

+```

+### Symptom - Assigning a role sometimes fails with REST API or ARM templates

+You create a new service principal and immediately try to assign a role to that service principal and the role assignment sometimes fails.

+**Cause**

+The reason is likely a replication delay. The service principal is created in one region; however, the role assignment might occur in a different region that hasn't replicated the service principal yet.

+**Solution**

+Set the `principalType` property to `ServicePrincipal` when creating the role assignment. You must also set the `apiVersion` of the role assignment to `2018-09-01-preview` or later. For more information, see [Assign Azure roles to a new service principal using the REST API](role-assignments-rest.md#new-service-principal) or [Assign Azure roles to a new service principal using Azure Resource Manager templates](role-assignments-template.md#new-service-principal).

+### Symptom - Role assignments with identity not found

+In the list of role assignments for the Azure portal, you notice that the security principal (user, group, service principal, or managed identity) is listed as **Identity not found** with an **Unknown** type.

+

+**Cause 1**

+You recently invited a user when creating a role assignment and this security principal is still in the replication process across regions.

+**Solution 1**

+Wait a few moments and refresh the role assignments list.

+**Cause 2**

+You deleted a security principal that had a role assignment. If you assign a role to a security principal and then you later delete that security principal without first removing the role assignment, the security principal will be listed as **Identity not found** and an **Unknown** type.

+**Solution 2**

+In PowerShell, if you try to remove the role assignments using the object ID and role definition name, and more than one role assignment matches your parameters, you will get the error message: `The provided information does not map to a role assignment`. The following output shows an example of the error message:

+### Symptom - Cannot delete the last Owner role assignment

+You attempt to remove the last Owner role assignment for a subscription and you see the following error:

+`Cannot delete the last RBAC admin assignment`

+**Cause**

+Removing the last Owner role assignment for a subscription is not supported to avoid orphaning the subscription.

+**Solution**

+If you want to cancel your subscription, see [Cancel your Azure subscription](../cost-management-billing/manage/cancel-azure-subscription.md).

+You are allowed to remove the last Owner (or User Access Administrator) role assignment at subscription scope, if you are the Global Administrator for the tenant. In this case, there is no constraint for deletion. However, if the call comes from some other principal, then you won't be able to remove the last Owner role assignment at subscription scope.

+### Symptom - Role assignment is not moved after moving a resource

+**Cause**

+If you move a resource that has an Azure role assigned directly to the resource (or a child resource), the role assignment is not moved and becomes orphaned.

+**Solution**

+After you move a resource, you must re-create the role assignment. Eventually, the orphaned role assignment will be automatically removed, but it is a best practice to remove the role assignment before moving the resource. For information about how to move resources, see [Move resources to a new resource group or subscription](../azure-resource-manager/management/move-resource-group-and-subscription.md).

+### Symptom - Role assignment changes are not being detected

+You recently added or updated a role assignment, but the changes are not being detected.

+**Cause**

+Azure Resource Manager sometimes caches configurations and data to improve performance. When you assign roles or remove role assignments, it can take up to 30 minutes for changes to take effect.

+**Solution**

+If you are using the Azure portal, Azure PowerShell, or Azure CLI, you can force a refresh of your role assignment changes by signing out and signing in. If you are making role assignment changes with REST API calls, you can force a refresh by refreshing your access token.

+## Custom roles

+### Symptom - Unable to update a custom role

+You are unable to update an existing custom role.

+**Cause**

+You are currently signed in with a user that does not have permission to update custom roles.

+**Solution**

+Check that you are currently signed in with a user that is assigned a role that has the `Microsoft.Authorization/roleDefinition/write` permission such as [Owner](built-in-roles.md#owner) or [User Access Administrator](built-in-roles.md#user-access-administrator).

+### Symptom - Unable to create or update a custom role

+When you try to create or update a custom role, you get an error similar to following:

+`The client '<clientName>' with object id '<objectId>' has permission to perform action 'Microsoft.Authorization/roleDefinitions/write' on scope '/subscriptions/<subscriptionId>'; however, it does not have permission to perform action 'Microsoft.Authorization/roleDefinitions/write' on the linked scope(s)'/subscriptions/<subscriptionId1>,/subscriptions/<subscriptionId2>,/subscriptions/<subscriptionId3>' or the linked scope(s)are invalid`

+**Cause**

+This error usually indicates that you do not have permissions to one or more of the [assignable scopes](role-definitions.md#assignablescopes) in the custom role.

+**Solution**

+Try the following:

+- Review [Who can create, delete, update, or view a custom role](custom-roles.md#who-can-create-delete-update-or-view-a-custom-role) and check that you have permissions to create or update the custom role for all assignable scopes.

+- If you don't have permissions, ask your administrator to assign you a role that has the `Microsoft.Authorization/roleDefinitions/write` action, such as [Owner](built-in-roles.md#owner) or [User Access Administrator](built-in-roles.md#user-access-administrator), at the scope of the assignable scope.

+- Check that all the assignable scopes in the custom role are valid. If not, remove any invalid assignable scopes.

+For more information, see the custom role tutorials using the [Azure portal](custom-roles-portal.md), [Azure PowerShell](tutorial-custom-role-powershell.md), or [Azure CLI](tutorial-custom-role-cli.md).

+### Symptom - Unable to delete a custom role

+You are unable to delete a custom role and get the following error message:

+`There are existing role assignments referencing role (code: RoleDefinitionHasAssignments)`

+**Cause**

+There are role assignments still using the custom role.

+**Solution**

+Remove those role assignments and try to delete the custom role again.

+### Symptom - Unable to add more than one management group as assignable scope

+When you try to create or update a custom role, you can't add more than one management group as assignable scope.

+**Cause**

+You can only define one management group in `AssignableScopes` of a custom role. Adding a management group to `AssignableScopes` is currently in preview.

+**Solution**

+Define one management group in `AssignableScopes` of your custom role. For more information about custom roles and management groups, see [Organize your resources with Azure management groups](../governance/management-groups/overview.md#azure-custom-role-definition-and-assignment).

+### Symptom - Unable to add data actions to custom role

+When you try to create or update a custom role, you can't add data actions or you see the following message:

+`You cannot add data action permissions when you have a management group as an assignable scope`

+**Cause**

+You are trying to create a custom role with data actions and a management group as assignable scope. Custom roles with `DataActions` cannot be assigned at the management group scope.

+**Solution**

+Create the custom role with one or more subscriptions as the assignable scope. For more information about custom roles and management groups, see [Organize your resources with Azure management groups](../governance/management-groups/overview.md#azure-custom-role-definition-and-assignment).

+### Symptom - No more role definitions can be created

+When you try to create a new custom role, you get the following message:

+`Role definition limit exceeded. No more role definitions can be created (code: RoleDefinitionLimitExceeded)`

+**Cause**

+Azure supports up to **5000** custom roles in a directory. (For Azure Germany and Azure China 21Vianet, the limit is 2000 custom roles.)

+**Solution**

+Try to reduce the number of custom roles.

+## Access denied or permission errors

+### Symptom - Authorization failed

+When you try to create a resource, you get the following error message:

+`The client with object id does not have authorization to perform action over scope (code: AuthorizationFailed)`

+**Cause**

+You are currently signed in with a user that does not have write permission to the resource at the selected scope.

+**Solution**

+Check that you are currently signed in with a user that is assigned a role that has write permission to the resource at the selected scope. For example, to manage virtual machines in a resource group, you should have the [Virtual Machine Contributor](built-in-roles.md#virtual-machine-contributor) role on the resource group (or parent scope). For a list of the permissions for each built-in role, see [Azure built-in roles](built-in-roles.md).

+### Symptom - Unable to create a support request

+When you try to create or update a support ticket, you get the following error message:

+`You don't have permission to create a support request`

+**Cause**

+You are currently signed in with a user that does not have permission to the create support requests.

+**Solution**

+Check that you are currently signed in with a user that is assigned a role that has the `Microsoft.Support/supportTickets/write` permission, such as [Support Request Contributor](built-in-roles.md#support-request-contributor).

+## Azure features are disabled

+### Symptom - Some web app features are disabled

+A user has read access to a web app and some features are disabled.

+**Cause**

+If you grant a user read access to a web app, some features are disabled that you might not expect. The following management capabilities require write access to a web app and aren't available in any read-only scenario.

+**Solution**

+Assign the [Contributor](built-in-roles.md#contributor) or another [Azure built-in role](built-in-roles.md) with write permissions for the web app.

+### Symptom - Some web app resources are disabled

+A user has write access to a web app and some features are disabled.

+**Cause**

+These items require write access to theApp Service plan that corresponds to your website:

+**Solution**

+Assign an [Azure built-in role](built-in-roles.md) with write permissions for the app service plan or resource group.

+### Symptom - Some virtual machine features are disabled

+A user has access to a virtual machine and some features are disabled.

+**Cause**

+These items require write access to the virtual machine:

+These require write access to both the virtual machine, and the resource group (along with the Domain name) that it is in:

+**Solution**

+Assign an [Azure built-in role](built-in-roles.md) with write permissions for the virtual machine or resource group.

+### Symptom - Some function app features are disabled

+A user has access to a function app and some features are disabled. For example, they can click the **Platform features** tab and then click **All settings** to view some settings related to a function app (similar to a web app), but they can't modify any of these settings.

+**Cause**

+**Solution**

+Assign an [Azure built-in role](built-in-roles.md) with write permissions for the function app or resource group.

+## Transferring a subscription to a different directory

+### Symptom - All role assignments are deleted after transferring a subscription

+**Cause**

+When you transfer an Azure subscription to a different Azure AD directory, all role assignments are **permanently** deleted from the source Azure AD directory and are not migrated to the target Azure AD directory.

+**Solution**

+You must re-create your role assignments in the target directory. You also have to manually recreate managed identities for Azure resources. For more information, see [Transfer an Azure subscription to a different Azure AD directory](transfer-subscription.md) and [FAQs and known issues with managed identities](../active-directory/managed-identities-azure-resources/known-issues.md).

+### Symptom - Unable to access subscription after transferring a subscription

+**Solution**

+If you are an Azure AD Global Administrator and you don't have access to a subscription after it was transferred between directories, use the **Access management for Azure resources** toggle to temporarily [elevate your access](elevate-access-global-admin.md) to get access to the subscription.

+## Classic subscription administrators

+If you are having issues with Service administrator or Co-administrators, see [Add or change Azure subscription administrators](../cost-management-billing/manage/add-change-subscription-administrator.md) and [Classic subscription administrator roles, Azure roles, and Azure AD roles](rbac-and-directory-admin-roles.md).

+ "storageConnectionString": "ResourceId=/subscriptions/{subscription-ID}/resourceGroups/{resource-group-name}/providers/Microsoft.Storage/storageAccounts/{storage-account-name};"

+| - [Add indicators in bulk to threat intelligence by file](../../sentinel/indicators-bulk-file-import.md) | Public Preview | Not Available |

+1. There is only one JSON template for all indicator types. The JSON template is based on STIX 2.1 format.

+ "pattern": "[ipv4-addr:value = '198.168.100.5']",

+- [Detect threats quickly with near-real-time (NRT) analytics rules in Microsoft Sentinel](near-real-time-rules.md)

+ Title: Monitor the health of your Microsoft Sentinel data connectors

+- ***SentinelHealth* data table**. (Public preview) Provides insights on health drifts, such as latest failure events per connector, or connectors with changes from success to failure states, which you can use to create alerts and other automated actions. The *SentinelHealth* data table is currently supported only for [selected data connectors](#supported-data-connectors).

+> [!IMPORTANT]

+>

+> The *SentinelHealth* data table is currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+>

+To get data connector health data from the *SentinelHealth* data table, you must first turn on the Microsoft Sentinel health feature for your workspace. For more information, see [Turn on health monitoring for Microsoft Sentinel](monitor-sentinel-health.md).

+The following table describes the columns and data generated in the SentinelHealth data table for data connectors:

+ Title: Turn on health monitoring in Microsoft Sentinel

+description: Monitor supported data connectors by using the SentinelHealth data table.

+# Turn on health monitoring for Microsoft Sentinel (preview)

+Monitor the health of supported data connectors by turning on health monitoring in Microsoft Sentinel. Get insights on health drifts, such as the latest failure events, or changes from success to failure states. Use this information to create alerts and other automated actions.

+To get health data from the *SentinelHealth* data table, you must first turn on the Microsoft Sentinel health feature for your workspace.

+When the health feature is turned on, the *SentinelHealth* data table is created at the first success or failure event generated for supported data connectors.

+To configure the retention time for your health events, see [Configure data retention and archive policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md).

+> [!IMPORTANT]

+>

+> The *SentinelHealth* data table is currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+>

+## Turn on health monitoring for your workspace

+1. In Microsoft Sentinel, under the **Configuration** menu on the left, select **Settings** and expand the **Health** section.

+1. Select **Configure Diagnostic Settings** and create a new diagnostic setting.

+ - In the **Diagnostic setting name** field, enter a meaningful name for your setting.

+ - In the **Category details** column, select the appropriate category like **Data Connector**.

+ - Under **Destination details**, select **Send to Log Analytics workspace**, and select your subscription and workspace from the dropdown menus.

+1. Select **Save** to save your new setting.

+The *SentinelHealth* data table is created at the first success or failure event generated for supported resources.

+## Access the *SentinelHealth* table

+In the Microsoft Sentinel **Logs** page, run a query on the *SentinelHealth* table. For example:

+```kusto

+SentinelHealth

+ | take 20

+```

+## Next steps

+[Monitor the health of your Microsoft Sentinel data connectors](monitor-data-connector-health.md)

+ - [SAP HANA Audit Trail - Best Practice](https://help.sap.com/docs/SAP_HANA_PLATFORM/b3ee5778bc2e4a089d3299b82ec762a7/35eb4e567d53456088755b8131b7ed1d.html?version=2.0.03)

+Azure Sentinel now provides the ability to enhance your data connector health monitoring with a new *SentinelHealth* table. The *SentinelHealth* table is created after you [turn on the Azure Sentinel health feature](monitor-sentinel-health.md) in your Azure Sentinel workspace, at the first success or failure health event generated.

+IMPORTANT: Microsoft Sentinel refreshes indicators every 12 days to make sure they are available for matching purposes through the analytic rules.

+> When autoforwarding is setup, the value for `AutoDeleteOnIdle` on the source entity is automatically set to the maximum value of the data type.

+> - On the source side, autoforwarding acts as a receive operation, so the source that has autoforwarding enabled is never really "idle" and hence it won't be automatically deleted.

+> - Autoforwarding doesn't make any changes to the destination entity. If `AutoDeleteOnIdle` is enabled on destination entity, the entity is automatically deleted if it's inactive for the specified idle interval. We recommend that you don't enable `AutoDeleteOnIdle` on the destination entity because if the destination entity is deleted, the souce entity will continually see exceptions when trying to forward messages that destination.

+| [Angular](https://angular.io/) | `dist/<APP_NAME>` <br><br>If you do not include an `<APP_NAME>`, remove the trailing slash. | `npm run build -- --configuration production` |

+1. From the _Settings_ menu, select **APIs**.

+1. From the _Production_ row, select **Link** to open the *Link new Backend* window.

+ Enter the following settings.

+ | Setting | Value |

+ |--|--|

+ | Backed resource type | Select **Function App**. |

+ | Subscription | Select your Azure subscription name. |

+ | Resource name | Select the Azure Functions app name. |

+The Azure Functions app is now mapped to the `/api` route of your static web app.

+| [Immutable storage](immutable-storage-overview.md) |  |  ² |  |  ² |

+| [Immutable storage](immutable-storage-overview.md) |  |  ² |  |  ² |

+If the exit code is any other non-zero exit code, it may be an exit code from the system. For example, OOMKilled. Check your operating system documentation for special exit codes.

+| FileCapacity | The amount of File storage used by the storage account. <br/><br/> Unit: Bytes <br/> Aggregation Type: Average <br/> Dimensions: FileShare, Tier <br/> Value example: 1024 |

+| FileCount | The number of files in the storage account. <br/><br/> Unit: Count <br/> Aggregation Type: Average <br/> Dimensions: FileShare, Tier <br/> Value example: 1024 |

+- [Azure Event Hubs](../../../event-hubs/event-hubs-about.md), [Azure Stream Analytics](../../../stream-analytics/stream-analytics-introduction.md), and [Apache Kafka](/azure/databricks/structured-streaming/kafka) to integrate live streaming data from Azure Synapse.

+To learn about migrating to a dedicated SQL pool, see [Migrate a data warehouse to a dedicated SQL pool in Azure Synapse Analytics](../migrate-to-synapse-analytics-guide.md).

+- [Azure Event Hubs](../../../event-hubs/event-hubs-about.md), [Azure Stream Analytics](../../../stream-analytics/stream-analytics-introduction.md), and [Apache Kafka](/azure/databricks/structured-streaming/kafka) to integrate live streaming data from Azure Synapse.

+To learn about migrating to a dedicated SQL pool, see [Migrate a data warehouse to a dedicated SQL pool in Azure Synapse Analytics](../migrate-to-synapse-analytics-guide.md).

+ Title: Managed identity

+description: Learn about using managed identities in Azure Synapse.

+# Managed identity for Azure Synapse

+This article helps you understand managed identity (formerly known as Managed Service Identity/MSI) and how it works in Azure Synapse.

+## Overview

+Managed identities eliminate the need to manage credentials. Managed identities provide an identity for the service instance when connecting to resources that support Azure Active Directory (Azure AD) authentication. For example, the service can use a managed identity to access resources like [Azure Key Vault](../key-vault/general/overview.md), where data admins can securely store credentials or access storage accounts. The service uses the managed identity to obtain Azure AD tokens.

+There are two types of supported managed identities:

+- **System-assigned:** You can enable a managed identity directly on a service instance. When you allow a system-assigned managed identity during the creation of the service, an identity is created in Azure AD tied to that service instance's lifecycle. By design, only that Azure resource can use this identity to request tokens from Azure AD. So when the resource is deleted, Azure automatically deletes the identity for you. Azure Synapse Analytics requires that a system-assigned managed identity must be created along with the Synapse workspace.

+- **User-assigned:** You may also create a managed identity as a standalone Azure resource. You can [create a user-assigned managed identity](../active-directory/managed-identities-azure-resources/how-to-manage-ua-identity-portal.md) and assign it to one or more instances of a Synapse workspace. In user-assigned managed identities, the identity is managed separately from the resources that use it.

+Managed identity provides the below benefits:

+- [Store credential in Azure Key Vault](../data-factory/store-credentials-in-key-vault.md), in which case-managed identity is used for Azure Key Vault authentication.

+- Access data stores or computes using managed identity authentication, including Azure Blob storage, Azure Data Explorer, Azure Data Lake Storage Gen1, Azure Data Lake Storage Gen2, Azure SQL Database, Azure SQL Managed Instance, Azure Synapse Analytics, REST, Databricks activity, Web activity, and more. Check the connector and activity articles for details.

+- Managed identity is also used to encrypt/decrypt data and metadata using the customer-managed key stored in Azure Key Vault, providing double encryption.

+## System-assigned managed identity

+>[!NOTE]

+> System-assigned managed identity is also referred to as 'Managed identity' elsewhere in the documentation and in the Synapse Studio UI for backward compatibility purpose. We will explicitly mention 'User-assigned managed identity' when referring to it.

+### <a name="generate-managed-identity"></a> Generate system-assigned managed identity

+System-assigned managed identity is generated as follows:

+- When creating a Synapse workspace through **Azure portal or PowerShell**, managed identity will always be created automatically.

+- When creating a workspace through **SDK**, managed identity will be created only if you specify Identity = new ManagedIdentity" in the Synapse workspace object for creation." See example in [.NET Quickstart - Create data factory](../data-factory/quickstart-create-data-factory-dot-net.md#create-a-data-factory).

+- When creating Synapse workspace through **REST API**, managed identity will be created only if you specify "identity" section in request body. See example in [REST quickstart - create data factory](../data-factory/quickstart-create-data-factory-rest-api.md#create-a-data-factory).

+If you find your service instance doesn't have a managed identity associated following [retrieve managed identity](#retrieve-managed-identity) instruction, you can explicitly generate one by updating it with identity initiator programmatically:

+- [Generate managed identity using PowerShell](#generate-system-assigned-managed-identity-using-powershell)

+- [Generate managed identity using REST API](#generate-system-assigned-managed-identity-using-rest-api)

+- [Generate managed identity using an Azure Resource Manager template](#generate-system-assigned-managed-identity-using-an-azure-resource-manager-template)

+- [Generate managed identity using SDK](#generate-system-assigned-managed-identity-using-sdk)

+>[!NOTE]

+>

+>- Managed identity cannot be modified. Updating a service instance which already has a managed identity won't have any impact, and the managed identity is kept unchanged.

+>- If you update a service instance which already has a managed identity without specifying the "identity" parameter in the factory or workspace objects or without specifying "identity" section in REST request body, you will get an error.

+>- When you delete a service instance, the associated managed identity will be deleted along.

+#### Generate system-assigned managed identity using PowerShell

+Call **New-AzSynapseWorkspace** command, then you see "Identity" fields being newly generated:

+```powershell

+PS C:\> $creds = New-Object System.Management.Automation.PSCredential ("ContosoUser", $password)

+PS C:\> New-AzSynapseWorkspace -ResourceGroupName <resourceGroupName> -Name <workspaceName> -Location <region> -DefaultDataLakeStorageAccountName <storageAccountName> -DefaultDataLakeStorageFileSystem <fileSystemName> -SqlAdministratorLoginCredential $creds

+DefaultDataLakeStorage : Microsoft.Azure.Commands.Synapse.Models.PSDataLakeStorageAccountDetails

+ProvisioningState : Succeeded

+SqlAdministratorLogin : ContosoUser

+VirtualNetworkProfile :

+Identity : Microsoft.Azure.Commands.Synapse.Models.PSManagedIdentity

+ManagedVirtualNetwork :

+PrivateEndpointConnections : {}

+WorkspaceUID : <workspaceUid>

+ExtraProperties : {[WorkspaceType, Normal], [IsScopeEnabled, False]}

+ManagedVirtualNetworkSettings :

+Encryption : Microsoft.Azure.Commands.Synapse.Models.PSEncryptionDetails

+WorkspaceRepositoryConfiguration :

+Tags :

+TagsTable :

+Location : <region>

+Id : /subscriptions/<subsID>/resourceGroups/<resourceGroupName>/providers/

+ Microsoft.Synapse/workspaces/<workspaceName>

+Name : <workspaceName>

+Type : Microsoft.Synapse/workspaces

+```

+#### Generate system-assigned managed identity using REST API

+> [!NOTE]

+> If you attempt to update a service instance that already has a managed identity without either specifying the **identity** parameter in the workspace object or providing an **identity** section in the REST request body, you will get an error.

+Call the API below with the "identity" section in the request body:

+```

+PATCH https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}?api-version=2018-06-01

+```

+**Request body**: add "identity": { "type": "SystemAssigned" }.

+```json

+{

+ "name": "<workspaceName>",

+ "location": "<region>",

+ "properties": {},

+ "identity": {

+ "type": "SystemAssigned"

+ }

+}

+```

+**Response**: managed identity is created automatically, and "identity" section is populated accordingly.

+```json

+{

+ "name": "<workspaceName>",

+ "tags": {},

+ "properties": {

+ "provisioningState": "Succeeded",

+ "loggingStorageAccountKey": "**********",

+ "createTime": "2021-09-26T04:10:01.1135678Z",

+ "version": "2018-06-01"

+ },

+ "identity": {

+ "type": "SystemAssigned",

+ "principalId": "765ad4ab-XXXX-XXXX-XXXX-51ed985819dc",

+ "tenantId": "72f988bf-XXXX-XXXX-XXXX-2d7cd011db47"

+ },

+ "id": "/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Synapse/workspaces/<workspaceName>",

+ "type": "Microsoft.Synapse/workspaces",

+ "location": "<region>"

+}

+```

+#### Generate system-assigned managed identity using an Azure Resource Manager template

+**Template**: add "identity": { "type": "SystemAssigned" }.

+```json

+{

+ "contentVersion": "1.0.0.0",

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "resources": [{

+ "name": "<workspaceName>",

+ "apiVersion": "2018-06-01",

+ "type": "Microsoft.Synapse/workspaces",

+ "location": "<region>",

+ "identity": {

+ "type": "SystemAssigned"

+ }

+ }]

+}

+```

+#### Generate system-assigned managed identity using SDK

+```csharp

+Workspace workspace = new Workspace

+{

+ Identity = new ManagedIdentity

+ {

+ Type = ResourceIdentityType.SystemAssigned

+ },

+ DefaultDataLakeStorage = new DataLakeStorageAccountDetails

+ {

+ AccountUrl = <defaultDataLakeStorageAccountUrl>,

+ Filesystem = <DefaultDataLakeStorageFilesystem>

+ },

+ SqlAdministratorLogin = <SqlAdministratorLoginCredentialUserName>

+ SqlAdministratorLoginPassword = <SqlAdministratorLoginCredentialPassword>,

+ Location = <region>

+};

+client.Workspaces.CreateOrUpdate(resourceGroupName, workspaceName, workspace);

+```

+### <a name="retrieve-managed-identity"></a> Retrieve system-assigned managed identity

+You can retrieve the managed identity from Azure portal or programmatically. The following sections show some samples.

+>[!TIP]

+> If you don't see the managed identity, [generate managed identity](#generate-managed-identity) by updating your service instance.

+#### Retrieve system-assigned managed identity using Azure portal

+You can find the managed identity information from Azure portal -> your Synapse workspace -> Properties.

+- Managed Identity Object ID

+The managed identity information will also show up when you create linked service, which supports managed identity authentication, like Azure Blob, Azure Data Lake Storage, Azure Key Vault, etc.

+To grant permissions, follow these steps. For detailed steps, see [Assign Azure roles using the Azure portal](../role-based-access-control/role-assignments-portal.md).

+1. Select **Access control (IAM)**.

+1. Select **Add** > **Add role assignment**.

+ :::image type="content" source="../../includes/role-based-access-control/media/add-role-assignment-menu-generic.png" alt-text="Screenshot that shows Access control (IAM) page with Add role assignment menu open.":::

+1. On the **Members** tab, select **Managed identity**, and then select **Select members**.

+1. Select your Azure subscription.

+1. Under **System-assigned managed identity**, select **Synapse workspace**, and then select a workspace. You can also use the object ID or workspace name (as the managed-identity name) to find this identity. To get the managed identity's application ID, use PowerShell.

+1. On the **Review + assign** tab, select **Review + assign** to assign the role.

+#### Retrieve system-assigned managed identity using PowerShell

+The managed identity principal ID and tenant ID will be returned when you get a specific service instance as follows. Use the **PrincipalId** to grant access:

+```powershell

+PS C:\> (Get-AzSynapseWorkspace -ResourceGroupName <resourceGroupName> -Name <workspaceName>).Identity

+IdentityType PrincipalId TenantId

+ -- --

+SystemAssigned cadadb30-XXXX-XXXX-XXXX-ef3500e2ff05 72f988bf-XXXX-XXXX-XXXX-2d7cd011db47

+```

+You can get the application ID by copying above principal ID, then running below Azure Active Directory command with principal ID as parameter.

+```powershell

+PS C:\> Get-AzADServicePrincipal -ObjectId cadadb30-XXXX-XXXX-XXXX-ef3500e2ff05

+ServicePrincipalNames : {76f668b3-XXXX-XXXX-XXXX-1b3348c75e02, https://identity.azure.net/P86P8g6nt1QxfPJx22om8MOooMf/Ag0Qf/nnREppHkU=}

+ApplicationId : 76f668b3-XXXX-XXXX-XXXX-1b3348c75e02

+DisplayName : <workspaceName>

+Id : cadadb30-XXXX-XXXX-XXXX-ef3500e2ff05

+Type : ServicePrincipal

+```

+#### Retrieve managed identity using REST API

+The managed identity principal ID and tenant ID will be returned when you get a specific service instance as follows.

+Call below API in the request:

+```

+GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}?api-version=2018-06-01

+```

+**Response**: You will get response like shown in below example. The "identity" section is populated accordingly.

+```json

+{

+ "properties": {

+ "defaultDataLakeStorage": {

+ "accountUrl": "https://exampledatalakeaccount.dfs.core.windows.net",

+ "filesystem": "examplefilesystem"

+ },

+ "encryption": {

+ "doubleEncryptionEnabled": false

+ },

+ "provisioningState": "Succeeded",

+ "connectivityEndpoints": {

+ "web": "https://web.azuresynapse.net?workspace=%2fsubscriptions%2{subscriptionId}%2fresourceGroups%2f{resourceGroupName}%2fproviders%2fMicrosoft.Synapse%2fworkspaces%2f{workspaceName}",

+ "dev": "https://{workspaceName}.dev.azuresynapse.net",

+ "sqlOnDemand": "{workspaceName}-ondemand.sql.azuresynapse.net",

+ "sql": "{workspaceName}.sql.azuresynapse.net"

+ },

+ "managedResourceGroupName": "synapseworkspace-managedrg-f77f7cf2-XXXX-XXXX-XXXX-c4cb7ac3cf4f",

+ "sqlAdministratorLogin": "sqladminuser",

+ "privateEndpointConnections": [],

+ "workspaceUID": "e56f5773-XXXX-XXXX-XXXX-a0dc107af9ea",

+ "extraProperties": {

+ "WorkspaceType": "Normal",

+ "IsScopeEnabled": false

+ },

+ "publicNetworkAccess": "Enabled",

+ "cspWorkspaceAdminProperties": {

+ "initialWorkspaceAdminObjectId": "3746a407-XXXX-XXXX-XXXX-842b6cf1fbcc"

+ },

+ "trustedServiceBypassEnabled": false

+ },

+ "type": "Microsoft.Synapse/workspaces",

+ "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}",

+ "location": "eastus",

+ "name": "{workspaceName}",

+ "identity": {

+ "type": "SystemAssigned",

+ "tenantId": "72f988bf-XXXX-XXXX-XXXX-2d7cd011db47",

+ "principalId": "cadadb30-XXXX-XXXX-XXXX-ef3500e2ff05"

+ },

+ "tags": {}

+}

+```

+> [!TIP]

+> To retrieve the managed identity from an ARM template, add an **outputs** section in the ARM JSON:

+```json

+{

+ "outputs":{

+ "managedIdentityObjectId":{

+ "type":"string",

+ "value":"[reference(resourceId('Microsoft.Synapse/workspaces', parameters('<workspaceName>')), '2018-06-01', 'Full').identity.principalId]"

+ }

+ }

+}

+```

+### Execute Azure Synapse Spark Notebooks with system assigned managed identity

+You can easily execute Synapse Spark Notebooks with the system assigned managed identity (or workspace managed identity) by enabling *Run as managed identity* from the *Configure session* menu. To execute Spark Notebooks with workspace managed identity, users need to have following RBAC roles:

+- Synapse Compute Operator on the workspace or selected Spark pool

+- Synapse Credential User on the workspace managed identity

+

+

+

+## User-assigned managed identity

+You can create, delete, manage user-assigned managed identities in Azure Active Directory. For more details refer to [Create, list, delete, or assign a role to a user-assigned managed identity using the Azure portal](../active-directory/managed-identities-azure-resources/how-to-manage-ua-identity-portal.md).

+In order to use a user-assigned managed identity, you must first [create credentials](../data-factory/credentials.md) in your service instance for the UAMI.

+## Next steps

+- [Create credentials](../data-factory/credentials.md).

+See the following topics that introduce when and how to use managed identity:

+- [Store credential in Azure Key Vault](../data-factory/store-credentials-in-key-vault.md).

+- [Copy data from/to Azure Data Lake Store using managed identities for Azure resources authentication](../data-factory/connector-azure-data-lake-store.md).

+See [Managed Identities for Azure Resources Overview](../active-directory/managed-identities-azure-resources/overview.md) for more background on managed identities for Azure resources, on which managed identity in Azure Synapse is based.

+>[!IMPORTANT]

+>To redirect a mass storage USB device connected to your local computer to the remote session host, you'll need to configure the **Drive/storage redirection** RDP property. Enabling the **USB redirection** RDP property by itself won't work. For more information, see [Local drive redirection](#local-drive-redirection).

+| Canonical | UbuntuServer | 18.04-LTS |

+| Canonical | 0001-com-ubuntu-server-focal | 20.04-LTS |

+| Canonical | 0001-com-ubuntu-server-focal | 20.04-LTS-Gen2 |

+| MicrosoftWindowsServer | WindowsServer | 2019-Datacenter-with-Containers |

+### [Azure PowerShell](#tab/azure-powershell)

+```powershell

+Register-AzProviderPreviewFeature -ProviderNamespace Microsoft.VirtualMachineImages -Name FairfaxPublicPreview

+```

+### [Azure CLI](#tab/azure-cli)

+ "apiVersion": "2022-02-14",

+The `type` is the resource type, which must be `"Microsoft.VirtualMachineImages/imageTemplates"`. The `apiVersion` will change over time as the API changes, but should be `"2022-02-14"` for now.

+"apiVersion": "2022-02-14",

+ "type":"Microsoft.Compute/virtualMachines/runCommands",

+ "name":"secondRunCommand",

+ "apiVersion":"2019-12-01",

+ "location":"[parameters('location')]",

+ "dependsOn":<full resourceID of the previous other Run Command>,

+ "properties":{

+ "source":{

+ "script":"echo Hello World!"

+ },

+ "timeoutInSeconds":60

+}

+> Unless you use [Resize without downtime (preview)](#resize-without-downtime-preview), resizing a data disk requires the VM to be deallocated.

+#Customer intent: As a developer, I want to configure alerts in Azure Monitor for SAP solutions so that I can receive alerts and notifications about my SAP systems.

+In this how-to guide, you'll learn how to configure alerts in Azure Monitor for SAP solutions (AMS). You can configure alerts and notifications from the [Azure portal](https://azure.microsoft.com/features/azure-portal) using its browser-based interface.

+- An Azure subscription.

+- A deployment of an AMS resource with at least one provider. You can configure providers for:

+ - The SAP application (NetWeaver)

+ - SAP HANA

+ - Microsoft SQL Server

+ - High availability (HA) Pacemaker clusters

+ - IBM Db2

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. In the Azure portal, browse and select your Azure Monitor for SAP solutions resource. Make sure you have at least one provider configured for this resource.

+1. Navigate to the workbook you want to use. For example, SAP HANA.

+1. Select a HANA instance.

+1. Select the **Alerts** button to view available **Alert Templates**.

+1. Select **Create rule** to configure an alert of your choice.

+1. For **Alert threshold**, enter your alert threshold.

+1. For **Provider instance**, select a provider instance.

+1. For **Action group**, select or create an [action group](../../../azure-monitor/alerts/action-groups.md) to configure the notification setting. You can edit frequency and severity information according to your requirements.

+1. Select **Enable alert rule** to create the alert rule.

+1. Select **Deploy alert rule** to finish your alert rule configuration. You can choose to see the alert template by selecting **View template**.

+1. Navigate to **Alert rules** to view the newly created alert rule. When and if alerts are fired, you can view them under **Fired alerts**.

+ :::image type="content" source="./media/ams-alerts/ams-alert-5.png" alt-text="Screenshot showing result of alert configuration." lightbox="./media/ams-alerts/ams-alert-5.png":::

+#Customer intent: As a developer, I want to learn what providers are available for Azure Monitor for SAP solutions so that I can connect to these providers.

+In the context of *Azure Monitor for SAP solutions (AMS)*, a *provider* contains the connection information for a corresponding component and helps to collect data from there. There are multiple provider types. For example, an SAP HANA provider is configured for a specific component within the SAP landscape, like an SAP HANA database. You can configure an AMS resource (also known as SAP monitor resource) with multiple providers of the same type or multiple providers of multiple types.

+This content applies to both versions of the service, *AMS* and *AMS (classic)*.

+

+You can choose to configure different provider types for data collection from the corresponding component in their SAP landscape. For example, you can configure one provider for the SAP HANA provider type, another provider for high availability cluster provider type, and so on.

+You can also configure multiple providers of a specific provider type to reuse the same SAP monitor resource and associated managed group. For more information, see [Manage Azure Resource Manager resource groups by using the Azure portal](../../../azure-resource-manager/management/manage-resource-groups-portal.md).

+

+It's recommended to configure at least one provider when you deploy an AMS resource. By configuring a provider, you start data collection from the corresponding component for which the provider is configured.

+If you don't configure any providers at the time of deployment, the AMS resource is still deployed, but no data is collected. You can add providers after deployment through the SAP monitor resource within the Azure portal. You can add or delete providers from the SAP monitor resource at any time.

+You can configure one or more providers of provider type SAP NetWeaver to enable data collection from SAP NetWeaver layer. AMS NetWeaver provider uses the existing [**SAPControl** Web service](https://www.sap.com/documents/2016/09/0a40e60d-8b7c-0010-82c7-eda71af511fa.html) interface to retrieve the appropriate information.

+| Web method | ABAP support | Java support | Metrics |

+| - | | | - |

+| **GetSystemInstanceList** | Yes | Yes | Instance availability, message server, gateway, ICM, ABAP availability |

+| **GetProcessList** | Yes | Yes | If instance list is red, you can find what process caused the issue |

+| **GetQueueStatistic** | Yes | Yes | Queue statistics (DIA, BATCH, UPD) |

+| **ABAPGetWPTable** | Yes | No | Work process utilization |

+| **EnqGetStatistic** | Yes | Yes | Locks |

+You can get the following data with the SAP NetWeaver provider:

+- SAP system and application server availability

+ - Instance process availability of dispatcher

+ - ICM

+ - Gateway

+ - Message server

+ - Enqueue Server

+ - IGS Watchdog

+- SMON Metrics (**/SDF/SMON**)

+- SWNC Workload, Memory, Transaction, User, RFC Usage (St03n)

+- Short Dumps (**ST22**)

+- Object Lock (**SM12**)

+- Failed Updates (**SM13**)

+- System Logs Analysis (**SM21**)

+- Batch Jobs Statistics (**SM37**)

+- Outbound Queues (**SMQ1**)

+- Inbound Queues (**SMQ2**)

+- Transactional RFC (**SM59**)

+- STMS Change Transport System Metrics (**STMS**)

+

+You can configure one or more providers of provider type *SAP HANA* to enable data collection from SAP HANA database. The SAP HANA provider connects to the SAP HANA database over SQL port, pulls data from the database, and pushes it to the Log Analytics workspace in your subscription. The SAP HANA provider collects data every 1 minute from the SAP HANA database.

+You can see the following data with the SAP HANA provider:

+- SAP HANA Backup data

+- The host IP address,

+- **SYSTEMDB** username and password

+It's recommended to configure the SAP HANA provider against **SYSTEMDB**. However, more providers can be configured against other database tenants.

+You can configure one or more Microsoft SQL Server providers to enable data collection from [SQL Server on Virtual Machines](https://azure.microsoft.com/services/virtual-machines/sql-server/). The SQL Server provider connects to Microsoft SQL Server over the SQL port. It then pulls data from the database and pushes it to the Log Analytics workspace in your subscription. Configure SQL Server for SQL authentication and for signing in with the SQL Server username and password. Set the SAP database as the default database for the provider. The SQL Server provider collects data from every 60 seconds up to every hour from the SQL server.

+You can configure one or more providers of provider type *High-availability cluster* to enable data collection from Pacemaker cluster within the SAP landscape. The High-availability cluster provider connects to Pacemaker using the [ha_cluster_exporter](https://github.com/ClusterLabs/ha_cluster_exporter) for **SUSE** based clusters and by using [Performance co-pilot](https://access.redhat.com/articles/6139852) for **RHEL** based clusters. AMS then pulls data from the database and pushes it to Log Analytics workspace in your subscription. The High-availability cluster provider collects data every 60 seconds from Pacemaker.

+You can configure one or more providers of provider type OS (Linux) to enable data collection from a BareMetal or VM node. The OS (Linux) provider connects to BareMetal or VM nodes using the [Node_Exporter](https://github.com/prometheus/node_exporter) endpoint. It then pulls data from the nodes and pushes it to Log Analytics workspace in your subscription. The OS (Linux) provider collects data every 60 seconds for most of the metrics from the nodes.

+1. Configure an OS (Linux) provider for each BareMetal or VM node instance in your environment.

+ - **Name**: a name for this provider, unique to the AMS instance.

+ - **Node Exporter endpoint**: usually `http://<servername or ip address>:9100/metrics`.

+Port 9100 is exposed for the **Node_Exporter** endpoint.

+> Make sure **Node-Exporter** keeps running after the node reboot.

+# Customer intent: As a developer, I want to deploy Azure Monitor for SAP solutions with PowerShell so that I can create resources with PowerShell.

+# Quickstart: deploy Azure Monitor for SAP solutions with PowerShell (preview)

+Get started with Azure Monitor for SAP solutions (AMS) by using the

+[Az.HanaOnAzure](/powershell/module/az.hanaonazure/#sap-hana-on-azure) PowerShell module to create AMS resources. You'll create a resource group, set up monitoring, and create a provider instance.

+## Prerequisites

+- If you don't have an Azure subscription, create a [free](https://azure.microsoft.com/free/) account before you begin.

+- If you choose to use PowerShell locally, this article requires that you install the Az PowerShell module. You'll also need to connect to your Azure account using the [Connect-AzAccount](/powershell/module/az.accounts/connect-azaccount) cmdlet. For more information about installing the Az PowerShell module, see [Install Azure PowerShell](/powershell/azure/install-az-ps). Alternately, you can use [Azure Cloud Shell](../../../cloud-shell/overview.md).

+- While the **Az.HanaOnAzure** PowerShell module is in preview, you must install it separately using the `Install-Module` cmdlet. Once this PowerShell module becomes generally available, it becomes part of future Az PowerShell module releases and available natively from within Azure Cloud Shell.

+ ```azurepowershell-interactive

+ Install-Module -Name Az.HanaOnAzure

+ ```

+- If you have multiple Azure subscriptions, choose the appropriate subscription in which the resources should be billed. Select a specific subscription using the

+ ```azurepowershell-interactive

+ Set-AzContext -SubscriptionId 00000000-0000-0000-0000-000000000000

+ ```

+# Customer intent: As a developer, I want to deploy Azure Monitor for SAP solutions in the Azure portal so that I can configure providers.

+# Quickstart: deploy Azure Monitor for SAP solutions in Azure portal (preview)

+Get started with Azure Monitor for SAP solutions (AMS) by using the [Azure portal](https://azure.microsoft.com/features/azure-portal) to deploy AMS resources and configure providers.

+## Prerequisites

+If you don't have an Azure subscription, create a [free](https://azure.microsoft.com/free/) account before you begin.

+## Create AMS monitoring resource

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. On the **Basics** tab, provide the required values. If applicable, you can use an existing Log Analytics workspace.

+## Create AMS (classic) monitoring resource

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. On the **Basics** tab, provide the required values. If applicable, you can use an existing Log Analytics workspace.

+#Customer intent: As a developer, I want to create an IBM Db2 provider so that I can monitor the resource through Azure Monitor for SAP solutions.

+In this how-to guide, you'll learn how to create an IBM Db2 provider for Azure Monitor for SAP solutions (AMS) through the Azure portal. This content applies only to AMS, not the AMS (classic) version.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+## Create IBM Db2 provider

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+#Customer intent: As a developer, I want to create a High Availability Pacemaker cluster so I can use the resource with Azure Monitor for SAP solutions.

+In this how-to guide, you'll learn to create a High Availability (HA) Pacemaker cluster provider for Azure Monitor for SAP solutions (AMS). You'll install the HA agent, then create the provider for AMS.

+This content applies to both AMS and AMS (classic) versions.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+ 

+1. For **Type**, select **High-availability cluster (Pacemaker)**.

+1. Configure providers for each node of the cluster by entering the endpoint URL for **HA Cluster Exporter Endpoint**.

+ 1. For SUSE-based clusters, enter `http://<'IP address'> :9664/metrics`.

+ 

+

+ 

+1. Continue to add more providers as needed.

+1. Select **Review + create** to review the settings.

+1. Select **Create** to finish creating the resource.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+#Customer intent: As a developer, I want to create an SAP HANA provider so that I can use the resource with Azure Monitor for SAP solutions.

+In this how-to guide, you'll learn to configure an SAP HANA provider for Azure Monitor for SAP solutions (AMS) through the Azure portal. There are instructions to set up the [current version](#configure-ams) and the [classic version](#configure-ams-classic) of AMS.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+ 1. Select **Add provider**.

+ 1. On the creation pane, for **Type**, select **SAP HANA**.

+ 

+ 1. For **IP address**, enter the IP address or hostname of the server that runs the SAP HANA instance that you want to monitor. If you're using a hostname, make sure there is connectivity within the virtual network.

+ 1. For **Database tenant**, enter the HANA database that you want to connect to. It's recommended to use **SYSTEMDB**, because tenant databases don't have all monitoring views. For legacy single-container HANA 1.0 instances, leave this field blank.

+ 1. For **Instance number**, enter the instance number of the database (0-99). The SQL port is automatically determined based on the instance number.

+ 1. For **Database username**, enter the dedicated SAP HANA database user. This user needs the **MONITORING** or **BACKUP CATALOG READ** role assignment. For non-production SAP HANA instances, use **SYSTEM** instead.

+ 1. For **Database password**, enter the password for the database username. You can either enter the password directly or use a secret inside Azure Key Vault.

+ 1. Select **Add provider**.

+ 1. For **Type**, select **SAP HANA**. Make sure that you configure an SAP HANA provider for the main node.

+ 1. For **IP address**, enter the private IP address for the HANA server.

+ 1. For **Database tenant**, enter the name of the tenant that you want to use. You can choose any tenant. However, it's recommended to use **SYSTEMDB**, because this tenant has more monitoring areas.

+ 1. For **SQL port**, enter the port number for your HANA database. The format begins with 3, includes the instance number, and ends in 13. For example, 30013 is the SQL port for the instance 001.

+ 1. For **Database username**, enter the username that you want to use. Make sure the database user has **monitoring** and **catalog read** role assignments.

+ 1. Select **Add provider** to finish adding the provider.

+1. Select **Create** to finish creating the AMS resource.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+#Customer intent: As a developer, I want to configure a Linux provider so that I can use Azure Monitor for SAP solutions for monitoring.

+In this how-to guide, you'll learn to create a Linux OS provider for *Azure Monitor for SAP solutions (AMS)* resources.

+This content applies to both versions of the service, *AMS* and *AMS (classic)*.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+- Install [node exporter version 1.3.0](https://prometheus.io/download/#node_exporter) in each SAP host that you want to monitor, either BareMetal or Azure virtual machine (Azure VM). For more information, see [the node exporter GitHub repository](https://github.com/prometheus/node_exporter).

+## Create Linux provider

+1. Select **Create** to finish creating the resource.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+#Customer intent: As a developer, I want to configure a SAP NetWeaver provider so that I can use Azure Monitor for SAP solutions.

+In this how-to guide, you'll learn to configure the SAP NetWeaver provider for use with *Azure Monitor for SAP solutions (AMS)*. You can use SAP NetWeaver with both versions of the service, *AMS* and *AMS (classic)*.

+The SAP start service provides multiple services, including monitoring the SAP system. Both versions of AMS use **SAPControl**, which is a SOAP web service interface that exposes these capabilities. The **SAPControl** interface [differentiates between protected and unprotected web service methods](https://wiki.scn.sap.com/wiki/display/SI/Protected+web+methods+of+sapstartsrv). It's necessary to unprotect some methods to use AMS with NetWeaver.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+To configure the NetWeaver provider for the current AMS version, you'll need to:

+1. [Unprotect methods for metrics](#unprotect-methods-for-metrics)

+1. [Check that the rules have updated properly](#check-updated-rules)

+1. [Set up RFC metrics](#set-up-rfc-metrics)

+1. [Add the NetWeaver provider](#add-netweaver-provider)

+1. Select the appropriate profile (*DEFAULT.PFL*).

+1. Change the value to:

+ ```text

+ SDEFAULT -GetQueueStatistic -ABAPGetWPTable -EnqGetStatistic -GetProcessList

+ ```

+ ```bash

+ RestartService

+ ```

+ ```bash

+ sapcontrol -nr <instance number> -function RestartService

+ ```

+ ```bash

+ sapcontrol -nr <instance number>; -function ParameterValue service/protectedwebmethods

+ ```

+ ```bash

+ sapcontrol -nr <instance number> -function ParameterValue service/protectedwebmethods -user "<admin user>" "<admin password>"

+ ```

+1. Review the output.

+1. Repeat the previous steps for each instance profile.

+1. For Linux, run the following commands. Replace `<your port>` with your configured port.

+ ```bash

+ sudo firewall-cmd --permanent --zone=public --add-port=<your port>/TCP

+ ```

+ ```bash

+ sudo firewall-cmd --reload

+ ```

+1. For Windows, open Windows Defender Firewall from the Start menu. Select **Advanced settings** in the side menu, then select **Inbound Rules**. To open a port, select **New Rule**. Add your port and set the protocol to TCP.

+To configure the NetWeaver provider for the AMS (classic) version:

+1. [Unprotect some methods](#unprotect-methods)

+1. [Restart the SAP start service](#restart-sap-start-service)

+1. [Check that your settings have been updated properly](#validate-changes)

+1. [Install the NetWeaver provider through the Azure portal](#install-netweaver-provider)

+### Unprotect methods

+1. Sign in with an administrative account.

+1. Execute transaction **RZ10**.

+1. Select the appropriate profile (*DEFAULT.PFL*).

+1. Select **Extended Maintenance** &gt; **Change**.

+1. Select the profile parameter `service/protectedwebmethods`.

+1. Change the value to `SDEFAULT -GetQueueStatistic -ABAPGetWPTable -EnqGetStatistic -GetProcessList`.

+1. Select **Copy**.

+1. Go back and select **Profile** &gt; **Save**.

+### Restart SAP start service

+After updating the parameter, restart the **SAPStartSRV** service on each of the instances in the SAP system. Restarting the services doesn't restart the SAP system. Only the **SAPStartSRV** service (in Windows) or daemon process (in Unix/Linux) is restarted.

+You must restart **SAPStartSRV** on each instance of the SAP system for the SAPControl web methods to be unprotected. These read-only SOAP API are required for the NetWeaver provider to fetch metric data from the SAP system. Failure to unprotect these methods leads to empty or missing visualizations on the NetWeaver metric workbook.

+On Windows, open the SAP Microsoft Management Console (MMC) / SAP Management Console (MC). Right-click on each instance and select **All Tasks** &gt; **Restart Service**.

+

+On Linux, run the command `sapcontrol -nr <NN> -function RestartService`. Replace `<NN>` with the SAP instance number to restart the host.

+## Validate changes

+After the SAP service restarts, check that the updated web method protection exclusion rules have been applied for each instance. Run one of the following commands. Again, replace `<NN>` with the SAP instance number.

+- If you're logged in as `<sidadm\>`, run `sapcontrol -nr <NN> -function ParameterValue service/protectedwebmethods`.

+- If you're logged in as another user, run `sapcontrol -nr <NN> -function ParameterValue service/protectedwebmethods -user "<adminUser>" "<adminPassword>"`.

+To validate your settings, run a test query against web methods. Replace the hostname, instance number, and method name with the appropriate values.

+```powershell

+$SAPHostName = "<hostname>"

+$InstanceNumber = "<instance-number>"

+$Function = "ABAPGetWPTable"

+[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

+$sapcntrluri = "https://" + $SAPHostName + ":5" + $InstanceNumber + "14/?wsdl"

+$sapcntrl = New-WebServiceProxy -uri $sapcntrluri -namespace WebServiceProxy -class sapcntrl

+$FunctionObject = New-Object ($sapcntrl.GetType().NameSpace + ".$Function")

+$sapcntrl.$Function($FunctionObject)

+```

+Repeat the previous steps for each instance profile.

+You can use an access control list (ACL) to filter the access to a server port. For more information, see [SAP note 1495075](https://launchpad.support.sap.com/#/notes/1495075).

+### Install NetWeaver provider

+To install the NetWeaver provider in the Azure portal:

+If the SAP application servers (VMs) are part of a network domain, such as an Azure Active Directory (Azure AD) managed domain, you must provide the corresponding subdomain. The AMS collector VM exists inside the virtual network, and isn't joined to the domain. AMS can't resolve the hostname of instances inside the SAP system unless the hostname is an FQDN. If you don't provide the subdomain, there can be missing or incomplete visualizations in the NetWeaver workbook.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+ Title: Configure Microsoft SQL Server provider for Azure Monitor for SAP solutions (preview)

+description: Learn how to configure a Microsoft SQL Server provider for use with Azure Monitor for SAP solutions (AMS).

+#Customer intent: As a developer, I want to configure a Microsoft SQL Server provider so that I can use Azure Monitor for SAP solutions for monitoring.

+In this how-to guide, you'll learn to configure a Microsoft SQL Server provider for Azure Monitor for SAP solutions (AMS) through the Azure portal.

+## Prerequisites

+- An Azure subscription.

+- An existing AMS resource. To create an AMS resource, see the [quickstart for the Azure portal](azure-monitor-sap-quickstart.md) or the [quickstart for PowerShell](azure-monitor-sap-quickstart-powershell.md).

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about AMS provider types](azure-monitor-providers.md)

+description: Learn how to set up an Azure virtual network for use with Azure Monitor for SAP solutions.

+#Customer intent: As a developer, I want to set up an Azure virtual network so that I can use Azure Monitor for SAP solutions.

+In this how-to guide, you'll learn how to configure an Azure virtual network so that you can deploy *Azure Monitor for SAP solutions (AMS)*. You'll learn to [create a new subnet](#create-new-subnet) for use with Azure Functions for both versions of the product, *AMS* and *AMS (classic)*. Then, if you're using the current version of AMS, you'll learn to [set up outbound internet access](#configure-outbound-internet-access) to the SAP environment that you want to monitor.

+## Create new subnet

+> [!NOTE]

+> This section applies to both AMS and AMS (classic).

+Azure Functions is the data collection engine for AMS. You'll need to create a new subnet to host Azure Functions.

+[Create a new subnet](../../../azure-functions/functions-networking-options.md#subnets) with an **IPv4/28** block or larger.

+For more information, see how to [integrate your app with an Azure virtual network](../../../app-service/overview-vnet-integration.md).

+> This section only applies to the current version of AMS. If you're using AMS (classic), skip this section.

+In many use cases, you might choose to restrict or block outbound internet access to your SAP network environment. However, AMS requires network connectivity between the [subnet that you configured](#create-new-subnet) and the systems that you want to monitor. Before you deploy an AMS resource, you need to configure outbound internet access, or the deployment will fail.

+**Route All** is a [standard feature of virtual network integration](../../../azure-functions/functions-networking-options.md#virtual-network-integration) in Azure Functions, which is deployed as part of AMS. Enabling or disabling this setting only affects traffic from Azure Functions. This setting doesn't affect any other incoming or outgoing traffic within your virtual network.

+You can only use this option before you deploy an AMS resource. It's not possible to change the **Route All** setting after you create the AMS resource.

+You can use this option after you've deployed an AMS resource.

+| 450 | allow_monitor | 443 | TCP | | Azure Monitor | Allow |

+| 501 | allow_keyVault | 443 | TCP | | Azure Key Vault | Allow |

+| 600 | allow_azure_controlplane | 443 | Any | | Azure Resource Manager | Allow |

+The AMS subnet IP address refers to the IP of the subnet associated with your AMS resource. To find the subnet, go to the AMS resource in the Azure portal. On the **Overview** page, review the **vNet/subnet** value.

+You can enable a private endpoint by creating a new subnet in the same virtual network as the system that you want to monitor. No other resources can use this subnet. It's not possible to use the same subnet as Azure Functions for your private endpoint.

+To create a private endpoint for AMS:

+1. Create a private endpoint connection for the following resources inside the managed resource group.

+ 1. [Azure Key Vault resources](#create-key-vault-endpoint)

+ 2. [Azure Storage resources](#create-storage-endpoint)

+ 3. [Azure Log Analytics workspaces](#create-log-analytics-endpoint)

+#### Create key vault endpoint

+You only need one private endpoint for all the Azure Key Vault resources (secrets, certificates, and keys). Once a private endpoint is created for key vault, the vault resources can't be accessed from systems outside the given vnet.

+1. On the **Resource** tab, enter or select all required information. For the key vault resource, there's only one subresource available, the vault.

+#### Create storage endpoint

+Repeat the following process for each type of storage subresource (table, queue, blob, and file):

+1. On the **Resource** tab, enter or select all required information. For the **Target sub-resource**, select one of the subresource types (table, queue, blob, or file).

+#### Create log analytics endpoint

+Select a scope for the private endpoint:

+Create the private endpoint:

+1. Go to the AMPLS resource in the Azure portal.

+1. In the resource menu, under **Configure**, select **Private Endpoint connections**.

+1. Select **Private Endpoint** to create a new endpoint.

+Configure the scope:

+Find and note important IP address ranges:

+1. Find the subnet for the log analytics private endpoint.

+ 1. Go to the private endpoint created for the AMPLS resource.

+ 2. On the private endpoint's menu, under **Settings**, select **DNS configuration**.

+ 3. On the **DNS configuration** page, note the associated IP addresses.

+ 4. Go to the AMS resource in the Azure portal.

+ 5. On the **Overview** page, select the **vNet/subnet** to go to that resource.

+ 6. On the virtual network page, select the subnet that you used to create the AMS resource.

+Add outbound security rules:

+1. Add the following required security rules.

+

+ | Priority | Description |

+ | -- | - |

+ | 550 | Allow the source IP for making calls to source system to be monitored. |

+ | 600 | Allow the source IP for making calls Azure Resource Manager service tag. |

+ | 650 | Allow the source IP to access key-vault resource using private endpoint IP. |

+ | 700 | Allow the source IP to access storage-account resources using private endpoint IP. (Include IPs for each of storage account sub resources: table, queue, file, and blob) |

+ | 800 | Allow the source IP to access log-analytics workspace resource using private endpoint IP. |

+

+## Next steps

+- [Quickstart: set up AMS through the Azure portal](azure-monitor-sap-quickstart.md)

+- [Quickstart: set up AMS with PowerShell](azure-monitor-sap-quickstart-powershell.md)

+description: Learn about how to monitor your SAP resources on Azure for availability, performance, and operation.

+#Customer intent: As a developer, I want to learn how to monitor my SAP resources on Azure so that I can better understand their availability, performance, and operation.

+When you have critical SAP applications and business processes that rely on Azure resources, you might want to monitor those resources for availability, performance, and operation. *Azure Monitor for SAP solutions (AMS)* is an Azure-native monitoring product for SAP landscapes that run on Azure. AMS uses specific parts of the [Azure Monitor](../../../azure-monitor/overview.md) infrastructure. You can use AMS with both [SAP on Azure Virtual Machines (Azure VMs)](./hana-get-started.md) and [SAP on Azure Large Instances](./hana-overview-architecture.md).

+There are currently two versions of this product, *AMS* and *AMS (classic)*.

+## What can you monitor?

+You can use AMS to collect data from Azure infrastructure and databases in one central location. Then, you can visually correlate the data for faster troubleshooting.

+To monitor different components of an SAP landscape (such as Azure VMs, high-availability clusters, SAP HANA databases, SAP NetWeaver, etc.), add the corresponding *[provider](./azure-monitor-providers.md)*. For more information, see [how to deploy AMS through the Azure portal](azure-monitor-sap-quickstart.md).

+The following table provides a quick comparison of the AMS (classic) and AMS.

+| AMS | AMS (classic) |

+AMS uses the [Azure Monitor](../../../azure-monitor/overview.md) capabilities of [Log Analytics](../../../azure-monitor/logs/log-analytics-overview.md) and [Workbooks](../../../azure-monitor/visualize/workbooks-overview.md). With it, you can:

+- Create [custom visualizations](../../../azure-monitor/visualize/workbooks-overview.md) by editing the default workbooks provided by AMS.

+## What data is collected?

+AMS doesn't collect Azure Monitor metrics or resource log data, like some other Azure resources do. Instead, AMS sends custom logs directly to the Azure Monitor Logs system. There, you can then use the built-in features of Log Analytics.

+Data collection in AMS depends on the providers that you configure. During public preview, the following data is collected.

+### Pacemaker cluster data

+High availability (HA) Pacemaker cluster data includes:

+Also see the [metrics specification](https://github.com/ClusterLabs/ha_cluster_exporter/blob/master/doc/metrics.md) for `ha_cluster_exporter`.

+### SAP HANA data

+SAP HANA data includes:

+### Microsoft SQL Server data

+Microsoft SQL server data includes:

+### OS (Linux) data

+OS (Linux) data includes:

+### SAP NetWeaver data

+SAP NetWeaver data includes:

+- SAP system and application server availability, including instance process availability of:

+ - Dispatcher

+ - ICM

+ - Gateway

+ - Message server

+ - Enqueue Server

+ - IGS Watchdog

+- Enqueue lock statistics and trends

+- SMON metrics (**/SDF/SMON**)

+- SWNC workload, memory, transaction, user, RFC usage (**St03n**)

+- Short dumps (**ST22**)

+- Object lock (**SM12**)

+- Failed updates (**SM13**)

+- System logs analysis (**SM21**)

+- Batch jobs statistics (**SM37**)

+- Outbound queues (**SMQ1**)

+- Inbound queues (**SMQ2**)

+- Transactional RFC (**SM59**)

+- STMS change transport system metrics (**STMS**)

+### IBM Db2 data

+IBM Db2 data includes:

+- DB availability

+- Number of connections, logical and physical reads

+- Waits and current locks

+- Top 20 runtime and executions

+## What is the architecture?

+There are separate explanations for the [AMS architecture](#ams-architecture) and the [AMS (classic) architecture](#ams-classic-architecture).

+Some important points about the architecture include:

+- The architecture is **multi-instance**. You can monitor multiple instances of a given component type across multiple SAP systems (SID) within a virtual network with a single resource of AMS. For example, you can monitor HANA databases, high availability (HA) clusters, Microsoft SQL server, SAP NetWeaver, etc.

+- The architecture is **multi-provider**. The architecture diagram shows the SAP HANA provider as an example. Similarly, you can configure more providers for corresponding components to collect data from those components. For example, HANA DB, HA cluster, Microsoft SQL server, and SAP NetWeaver.

+- The architecture has an **extensible query framework**. Write [SQL queries to collect data in JSON](https://github.com/Azure/AzureMonitorForSAPSolutions/blob/master/sapmon/content/SapHana.json). Easily add more SQL queries to collect other data.

+### AMS architecture

+The following diagram shows, at a high level, how AMS collects data from the SAP HANA database. The architecture is the same if SAP HANA is deployed on Azure VMs or Azure Large Instances.

+ Diagram of the new AMS architecture. The customer connects to the AMS resource through the Azure portal. There's a managed resource group containing Log Analytics, Azure Functions, Key Vault, and Storage queue. The Azure function connects to the providers. Providers include SAP NetWeaver (ABAP and JAVA), SAP HANA, Microsoft SQL Server, IBM Db2, Pacemaker clusters, and Linux OS.

+- The **Azure portal**, where you access the AMS service.

+- The **AMS resource**, where you view monitoring data.

+- The **managed resource group**, which is deployed automatically as part of the AMS resource's deployment. The resources inside the managed resource group help to collect data. Key resources include:

+ - An **Azure Functions resource** that hosts the monitoring code. This logic collects data from the source systems and transfers the data to the monitoring framework.

+ - An **[Azure Key Vault resource](../../../key-vault/general/basic-concepts.md)**, which securely holds the SAP HANA database credentials and stores information about providers.

+ - The **Log Analytics workspace**, which is the destination for storing data. Optionally, you can choose to use an existing workspace in the same subscription as your AMS resource at deployment.

+[Azure Workbooks](../../../azure-monitor/visualize/workbooks-overview.md) provides customizable visualization of the data in Log Analytics. To automatically refresh your workbooks or visualizations, pin the items to the Azure dashboard. The maximum refresh frequency is every 30 minutes.

+You can also use Kusto Query Language (KQL) to [run log queries](../../../azure-monitor/logs/log-query-overview.md) against the raw tables inside the Log Analytics workspace.

+### AMS (classic) architecture

+The following diagram shows, at a high level, how AMS (classic) collects data from the SAP HANA database. The architecture is the same if SAP HANA is deployed on Azure VMs or Azure Large Instances.

+ Diagram of the AMS (classic) architecture. The customer connects to the AMS resource through the Azure portal. There's a managed resource group containing Log Analytics, Azure Functions, Key Vault, and Storage queue. The Azure function connects to the providers. Providers include SAP NetWeaver (ABAP and JAVA), SAP HANA, Microsoft SQL Server, Pacemaker clusters, and Linux OS.

+- The **Azure portal**, which is your starting point. You can navigate to marketplace within the Azure portal and discover AMS.

+- The **AMS resource**, which is the landing place for you to view monitoring data.

+- **Managed resource group**, which is deployed automatically as part of the AMS resource's deployment. The resources deployed within the managed resource group help with the collection of data. Key resources deployed and their purposes are:

+ - **Azure VM**, also known as the *collector VM*, which is a **Standard_B2ms** VM. The main purpose of this VM is to host the *monitoring payload*. The monitoring payload is the logic of collecting data from the source systems and transferring the data to the monitoring framework. In the architecture diagram, the monitoring payload contains the logic to connect to the SAP HANA database over the SQL port. You're responsible for patching and maintaining the VM.

+ - **[Azure Key Vault](../../../key-vault/general/basic-concepts.md)**: which is deployed to securely hold SAP HANA database credentials and to store information about providers.

+ - **Log Analytics Workspace**, which is the destination where the data is stored.

+ - Visualization is built on top of data in Log Analytics using [Azure Workbooks](../../../azure-monitor/visualize/workbooks-overview.md). You can customize visualization. You can also pin your Workbooks or specific visualization within Workbooks to Azure dashboard for auto-refresh. The maximum frequency for refresh is every 30 minutes.

+ - You can use your existing workspace within the same subscription as SAP monitor resource by choosing this option at deployment.

+ - You can use KQL to run [queries](../../../azure-monitor/logs/log-query-overview.md) against the raw tables inside the Log Analytics workspace. Look at **Custom Logs**.

+ - You can use an existing Log Analytics workspace for data collection if it's deployed within the same Azure subscription as the resource for AMS.

+## Can you analyze metrics?

+AMS doesn't support metrics.

+### Analyze logs

+AMS doesn't support resource logs or activity logs. For a list of the tables used by Azure Monitor Logs that can be queried by Log Analytics, see [the data reference for monitoring SAP on Azure](monitor-sap-on-azure-reference.md#azure-monitor-logs-tables).

+### Make Kusto queries

+When you select **Logs** from the AMS menu, Log Analytics is opened with the query scope set to the current AMS. Log queries only include data from that resource. To run a query that includes data from other accounts or data from other Azure services, select **Logs** from the **Azure Monitor** menu. For more information, see [Log query scope and time range in Azure Monitor Log Analytics](../../../azure-monitor/logs/scope.md) for details.

+You can use Kusto queries to help you monitor your AMS resources. The following sample query gives you data from a custom log for a specified time range. You can specify the time range and the number of rows. In this example, you'll get five rows of data for your selected time range.

+```kusto

+## How do you get alerts?

+Azure Monitor alerts proactively notify you when important conditions are found in your monitoring data. You can then identify and address issues in your system before your customers notice them.

+You can configure alerts in AMS from the Azure portal. For more information, see [how to configure alerts in AMS with the Azure portal](azure-monitor-alerts-portal.md).

+### How can you create AMS resources?

+You have several options to deploy AMS and configure providers:

+- [Deploy AMS directly from the Azure portal](azure-monitor-sap-quickstart.md)

+- [Deploy AMS with Azure PowerShell](azure-monitor-sap-quickstart-powershell.md)

+- [Deploy AMS (classic) using the Azure Command-Line Interface (Azure CLI)](https://github.com/Azure/azure-hanaonazure-cli-extension#sapmonitor).

+## What is the pricing?

+AMS is a free product (no license fee). You're responsible for paying the cost of the underlying components in the managed resource group. You're also responsible for consumption costs associated with data use and retention. For more information, see standard Azure pricing documents:

+- [Azure VM pricing (applicable to AMS (classic))](https://azure.microsoft.com/pricing/details/virtual-machines/linux/)

+## How do you enable data sharing with Microsoft?

+> [!NOTE]

+> The following content only applies to the AMS (classic) version.

+AMS collects system metadata to provide improved support for SAP on Azure. No PII/EUII is collected.

+You can enable data sharing with Microsoft when you create AMS resource by choosing *Share* from the drop-down. We recommend that you enable data sharing. Data sharing gives Microsoft support and engineering teams information about your environment, which helps us provide better support for your mission-critical SAP on Azure solution.

+- For a list of custom logs relevant to AMS and information on related data types, see [Monitor SAP on Azure data reference](monitor-sap-on-azure-reference.md).

+- For information on providers available for AMS, see [AMS providers](azure-monitor-providers.md).

+ * Associate a public IP to any of the virtual machine's network interfaces (if there are multiple network interfaces, having a single one with a public IP will prevent default outbound access for the virtual machine).

+ Title: 'Configure P2S User VPN clients -certificate authentication - macOS and iOS'

+description: Learn how to configure the VPN client for Virtual WAN User VPN configurations that use certificate authentication and IKEv2 or OpenVPN tunnel. This article applies to macOS and iOS.

+# Configure User VPN P2S clients - certificate authentication - macOS and iOS

+This article helps you connect to Azure Virtual WAN from a macOS or iOS operating system over User VPN P2S for configurations that use Certificate Authentication. To connect from an iOS or macOS operating system over an OpenVPN tunnel, you use an OpenVPN client. To connect from a macOS operating system over an IKEv2 tunnel, you use the VPN client that is natively installed on your Mac.

+## Before you begin

+* Make sure you've completed the necessary configuration steps in the [Tutorial: Create a P2S User VPN connection using Azure Virtual WAN](virtual-wan-point-to-site-portal.md).

+* **Generate VPN client configuration files:** The VPN client configuration files that you generate are specific to the Virtual WAN User VPN profile that you download. Virtual WAN has two different types of configuration profiles: WAN-level (global), and hub-level. If there are any changes to the P2S VPN configuration after you generate the files, or you change to a different profile type, you need to generate new VPN client configuration files and apply the new configuration to all of the VPN clients that you want to connect. See [Generate User VPN client configuration files](about-vpn-profile-download.md).

+* **Obtain certificates:** The sections below require certificates. Make sure you have both the client certificate and the root server certificate information. For more information, see [Generate and export certificates](certificates-point-to-site.md) for more information.

+## <a name="ikev2-macOS"></a>IKEv2 - native client - macOS steps

+After you generate and download the VPN client configuration package, unzip it to view the folders. When you configure macOS native clients, you use the files in the **Generic** folder. The Generic folder is present if IKEv2 was configured on the gateway. You can find all of the information that you need to configure the native VPN client in the **Generic** folder. If you don't see the Generic folder, make sure IKEv2 is one of the tunnel types, then download the configuration package again.

+The **Generic** folder contains the following files.

+* **VpnSettings.xml**, which contains important settings like server address and tunnel type.

+* **VpnServerRoot.cer**, which contains the root certificate required to validate the Azure VPN gateway during P2S connection setup.

+Use the following steps to configure the native VPN client on Mac for certificate authentication. These steps must be completed on every Mac that you want to connect to Azure.

+### Install certificates

+#### Root certificate

+1. Copy to the root certificate file - **VpnServerRoot.cer** - to your Mac. Double-click the certificate. Depending on your operating system, the certificate will either automatically install, or you'll see the **Add Certificates** page.

+1. If you see the **Add Certificates** page, for **Keychain:** click the arrows and select **login** from the dropdown.

+1. Click **Add** to import the file.

+#### Client certificate

+The client certificate is used for authentication and is required. Typically, you can just click the client certificate to install. For more information about how to install a client certificate, see [Install a client certificate](install-client-certificates.md).

+#### Verify certificate install

+Verify that both the client and the root certificate are installed.

+1. Open **Keychain Access**.

+1. Go to the **Certificates** tab.

+1. Verify that both the client and the root certificate are installed.

+

+### Configure VPN client profile

+1. Go to **System Preferences -> Network**. On the Network page, click **'+'** to create a new VPN client connection profile for a P2S connection to the Azure virtual network.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/new.png" alt-text="Screenshot shows the Network window to click on +." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/new.png":::

+1. On the **Select the interface** page, click the arrows next to **Interface:**. From the dropdown, click **VPN**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/vpn.png" alt-text="Screenshot shows the Network window with the option to select an interface, VPN is selected." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/vpn.png":::

+1. For **VPN Type**, from the dropdown, click **IKEv2**. In the **Service Name** field, specify a friendly name for the profile, then click **Create**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/service-name.png" alt-text="Screenshot shows the Network window with the option to select an interface, select VPN type, and enter a service name." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/service-name.png":::

+1. Go to the VPN client profile that you downloaded. In the **Generic** folder, open the **VpnSettings.xml** file using a text editor. In the example, you can see that this VPN client profile connects to a WAN-level User VPN profile and that the VpnTypes are IKEv2 and OpenVPN. Even though there are two VPN types listed, this VPN client will connect over IKEv2. Copy the **VpnServer** tag value.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/vpn-server.png" alt-text="Screenshot shows the VpnSettings.xml file open with the VpnServer tag highlighted." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/vpn-server.png":::

+1. Paste the **VpnServer** tag value in both the **Server Address** and **Remote ID** fields of the profile. Leave **Local ID** blank. Then, click **Authentication Settings...**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/server-address.png" alt-text="Screenshot shows server info pasted to fields." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/server-address.png":::

+### Configure authentication settings

+#### Big Sur and later

+1. On the **Authentication Settings** page, for the Authentication settings field, click the arrows to select **Certificate**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/monterey/certificate.png" alt-text="Screenshot shows authentication settings with certificate selected." lightbox="./media/point-to-site-vpn-client-cert-mac/monterey/certificate.png":::

+1. Click **Select** to open the **Choose An Identity** page.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/monterey/select.png" alt-text="Screenshot to click Select." lightbox="./media/point-to-site-vpn-client-cert-mac/monterey/select.png":::

+1. The **Choose An Identity** page displays a list of certificates for you to choose from. If youΓÇÖre unsure which certificate to use, you can select **Show Certificate** to see more information about each certificate. Click the proper certificate, then click **Continue**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/monterey/choose-identity.png" alt-text="Screenshot shows certificate properties." lightbox="./media/point-to-site-vpn-client-cert-mac/monterey/choose-identity.png":::

+1. On the **Authentication Settings** page, verify that the correct certificate is shown, then click **OK**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/monterey/verify.png" alt-text="Screenshot shows the Choose An Identity dialog box where you can select the proper certificate." lightbox="./media/point-to-site-vpn-client-cert-mac/monterey/verify.png":::

+#### Catalina

+If you're using Catalina, use these authentication settings steps:

+1. For **Authentication Settings** choose **None**.

+1. Click **Certificate**, click **Select** and click the correct client certificate that you installed earlier. Then, click **OK**.

+### Specify certificate

+1. In the **Local ID** field, specify the name of the certificate. In this example, itΓÇÖs **P2SChildCertMac**.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/monterey/local-id.png" alt-text="Screenshot shows local ID value." lightbox="./media/point-to-site-vpn-client-cert-mac/monterey/local-id.png":::

+1. Click **Apply** to save all changes.

+### Connect

+1. Click **Connect** to start the P2S connection to the Azure virtual network. You may need to enter your "login" keychain password.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/select-connect.png" alt-text="Screenshot shows connect button." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/select-connect.png":::

+1. Once the connection has been established, the status shows as **Connected** and you can view the IP address that was pulled from the VPN client address pool.

+ :::image type="content" source="./media/point-to-site-vpn-client-cert-mac/mac/connected.png" alt-text="Screenshot shows Connected." lightbox="./media/point-to-site-vpn-client-cert-mac/mac/connected.png":::

+## <a name="openvpn-macOS"></a>OpenVPN Client - macOS steps

+The following example uses **TunnelBlick**.

+## <a name="OpenVPN-iOS"></a>OpenVPN Client - iOS steps

+The following example uses **OpenVPN Connect** from the App store.

+## Next steps

+[Tutorial: Create a P2S User VPN connection using Azure Virtual WAN](virtual-wan-point-to-site-portal.md).

+Unzip the file to view the folders. When you configure macOS native clients, you use the files in the **Generic** folder. The Generic folder is present if IKEv2 was configured on the gateway. You can find all of the information that you need to configure the native VPN client in the **Generic** folder. If you don't see the Generic folder, check the following items, then generate the zip file again.