-After the first failure, the first retry happens within the next 2 hours (usually in the next sync cycle).

-This cycle is repeated every 24 hours until the 30th day when retries are stopped and the job is disabled.

- ermissions Management across AWS, Azure, and GCP.

-CAE only has insight into [IP-based named locations](../conditional-access/location-condition.md#ip-address-ranges). CAE doesn't have insight into other location conditions like [MFA trusted IPs](../authentication/howto-mfa-mfasettings.md#trusted-ips) or country-based locations. When a user comes from an MFA trusted IP, trusted location that includes MFA Trusted IPs, or country location, CAE won't be enforced after that user moves to a different location. In those cases, Azure AD will issue a one-hour access token without instant IP enforcement check.

-With the location condition in Conditional Access, you can control access to your cloud apps based on the network location of a user. The location condition is commonly used to block access from countries/regions where your organization knows traffic shouldn't come from.

-description: Use the location condition to control access based on user physical or network location.

-As explained in the [overview article](overview.md) Conditional Access policies are at their most basic an if-then statement combining signals, to make decisions, and enforce organization policies. One of those signals that can be incorporated into the decision-making process is location.

-The location is determined by the public IP address a client provides to Azure Active Directory or GPS coordinates provided by the Microsoft Authenticator app. Conditional Access policies by default apply to all IPv4 and IPv6 addresses.

-Locations are named in the Azure portal under **Azure Active Directory** > **Security** > **Conditional Access** > **Named locations**. These named network locations may include locations like an organization's headquarters network ranges, VPN network ranges, or ranges that you wish to block. Named locations can be defined by IPv4/IPv6 address ranges or by countries.

-### IP address ranges

-To define a named location by IPv4/IPv6 address ranges, you'll need to provide:

-Administrators can name locations defined by IP address ranges to be trusted named locations.

-Sign-ins from trusted named locations improve the accuracy of Azure AD Identity Protection's risk calculation, lowering a user's sign-in risk when they authenticate from a location marked as trusted. Additionally, trusted named locations can be targeted in Conditional Access policies. For example, you may [restrict multifactor authentication registration to trusted locations](howto-conditional-access-policy-registration.md).

-To define a named location by country, you'll need to provide:

-If you select **Determine location by IP address (IPv4 only)**, the system will collect the IP address of the device the user is signing into. When a user signs in, Azure AD resolves the user's IPv4 address to a country or region, and the mapping is updated periodically. Organizations can use named locations defined by countries to block traffic from countries where they don't do business.

-> [!NOTE]

-> Sign-ins from IPv6 addresses cannot be mapped to countries or regions, and are considered unknown areas. Only IPv4 addresses can be mapped to countries or regions.

-If you select **Determine location by GPS coordinates**, the user will need to have the Microsoft Authenticator app installed on their mobile device. Every hour, the system will contact the userΓÇÖs Microsoft Authenticator app to collect the GPS location of the userΓÇÖs mobile device.

-The first time the user is required to share their location from the Microsoft Authenticator app, the user will receive a notification in the app. The user will need to open the app and grant location permissions.

-Every hour the user is accessing resources covered by the policy they will need to approve a push notification from the app.

->A Conditional Access policy with GPS-based named locations in report-only mode prompts users to share their GPS location, even though they aren't blocked from signing in.

-Multiple conditional access policies applications may prompt users for their GPS location before all Conditional Access policies are applied. Because of the way Conditional Access policies are applied, a user may be denied access if they pass the location check but fail another policy. For more information about policy enforcement, see the article [Building a Conditional Access policy](concept-conditional-access-policies.md).

-Some IP addresses aren't mapped to a specific country or region, including all IPv6 addresses. To capture these IP locations, check the box **Include unknown countries/regions** when defining a geographic location. This option allows you to choose if these IP addresses should be included in the named location. Use this setting when the policy using the named location should apply to unknown locations.

-### Configure MFA trusted IPs

-You can also configure IP address ranges representing your organization's local intranet in the [multifactor authentication service settings](https://account.activedirectory.windowsazure.com/usermanagement/mfasettings.aspx). This feature enables you to configure up to 50 IP address ranges. The IP address ranges are in CIDR format. For more information, see [Trusted IPs](../authentication/howto-mfa-mfasettings.md#trusted-ips).

-If you have Trusted IPs configured, they show up as **MFA Trusted IPs** in the list of locations for the location condition.

-#### Skipping multifactor authentication

-On the multifactor authentication service settings page, you can identify corporate intranet users by selecting **Skip multifactor authentication for requests from federated users on my intranet**. This setting indicates that the inside corporate network claim, which is issued by AD FS, should be trusted and used to identify the user as being on the corporate network. For more information, see [Enable the Trusted IPs feature by using Conditional Access](../authentication/howto-mfa-mfasettings.md#enable-the-trusted-ips-feature-by-using-conditional-access).

-After checking this option, including the named location **MFA Trusted IPs** will apply to any policies with this option selected.

-For mobile and desktop applications, which have long lived session lifetimes, Conditional Access is periodically reevaluated. The default is once an hour. When the inside corporate network claim is only issued at the time of the initial authentication, Azure AD may not have a list of trusted IP ranges. In this case, it's more difficult to determine if the user is still on the corporate network:

-1. Check if the userΓÇÖs IP address is in one of the trusted IP ranges.

-1. Check whether the first three octets of the userΓÇÖs IP address match the first three octets of the IP address of the initial authentication. The IP address is compared with the initial authentication when the inside corporate network claim was originally issued and the user location was validated.

-If both steps fail, a user is considered to be no longer on a trusted IP.

-By default, selecting **Any location** causes a policy to be applied to all IP addresses, which means any address on the Internet. This setting isn't limited to IP addresses you've configured as named location. When you select **Any location**, you can still exclude specific locations from a policy. For example, you can apply a policy to all locations except trusted locations to set the scope to all locations, except the corporate network.

-### Selected locations

-With this option, you can select one or more named locations. For a policy with this setting to apply, a user needs to connect from any of the selected locations. When you **Select** the named network selection control that shows the list of named networks opens. The list also shows if the network location has been marked as trusted. The named location called **MFA Trusted IPs** is used to include the IP settings that can be configured in the multifactor authentication service setting page.

-## IPv6 traffic

-By default, Conditional Access policies will apply to all IPv6 traffic. You can exclude specific IPv6 address ranges from a Conditional Access policy if you donΓÇÖt want policies to be enforced for specific IPv6 ranges. For example, if you want to not enforce a policy for uses on your corporate network, and your corporate network is hosted on public IPv6 ranges.

-### Identifying IPv6 traffic in the Azure AD Sign-in activity reports

-You can discover IPv6 traffic in your tenant by going the [Azure AD sign-in activity reports](../reports-monitoring/concept-sign-ins.md). After you have the activity report open, add the ΓÇ£IP addressΓÇ¥ column. This column will give you to identify the IPv6 traffic.

-You can also find the client IP by clicking a row in the report, and then going to the ΓÇ£LocationΓÇ¥ tab in the sign-in activity details.

-### When will my tenant have IPv6 traffic?

-Azure Active Directory (Azure AD) doesnΓÇÖt currently support direct network connections that use IPv6. However, there are some cases that authentication traffic is proxied through another service. In these cases, the IPv6 address will be used during policy evaluation.

-Most of the IPv6 traffic that gets proxied to Azure AD comes from Microsoft Exchange Online. When available, Exchange will prefer IPv6 connections. **So if you have any Conditional Access policies for Exchange, that have been configured for specific IPv4 ranges, youΓÇÖll want to make sure youΓÇÖve also added your organizations IPv6 ranges.** Not including IPv6 ranges will cause unexpected behavior for the following two cases:

-If you're using Azure VNets, you'll have traffic coming from an IPv6 address. If you have VNet traffic blocked by a Conditional Access policy, check your Azure AD sign-in log. Once youΓÇÖve identified the traffic, you can get the IPv6 address being used and exclude it from your policy.

-> [!NOTE]

-> If you want to specify an IP CIDR range for a single address, apply the /128 bit mask. If you see the IPv6 address 2001:db8:4a7d:3f57:a1e2:6b4a:8f3e:d17b and wanted to exclude that single address as a range, you would use 2001:db8:4a7d:3f57:a1e2:6b4a:8f3e:d17b/128.

-This check means for mobile and desktop applications using modern authentication, a change in location would be detected within an hour of changing the network location. For mobile and desktop applications that donΓÇÖt use modern authentication, the policy is applied on each token request. The frequency of the request can vary based on the application. Similarly, for web applications, the policy is applied at initial sign-in and is good for the lifetime of the session at the web application. Because of differences in session lifetimes across applications, the time between policy evaluation will also vary. Each time the application requests a new sign-in token, the policy is applied.

-By default, Azure AD issues a token on an hourly basis. After moving off the corporate network, within an hour the policy is enforced for applications using modern authentication.

-The IP address used in policy evaluation is the public IP address of the user. For devices on a private network, this IP address isn't the client IP of the userΓÇÖs device on the intranet, it's the address used by the network to connect to the public internet.

-When a cloud proxy is in place, a policy that is used to require a hybrid Azure AD joined device can be used, or the inside corpnet claim from AD FS.

-* What authentication contexts does this policy will be applied to?

-When active, the Backup Authentication Service doesn't evaluate authentication methods required by [authentication strengths](../authentication/concept-authentication-strengths.md). If you used a non-phishing-resistant authentication method before an outage, during an outage you aren't be prompted for multifactor authentication even if accessing a resource protected by a Conditional Access policy with a phishing-resistant authentication strength.

->

-}

-The ROPC flow is **not recommended** because asking a user for their password in your application is not secure. For more information about this problem, see [WhatΓÇÖs the solution to the growing problem of passwords?](https://news.microsoft.com/features/whats-solution-growing-problem-passwords-says-microsoft/).

-This `AcquireTokenByUsernamePassword` method takes the following parameters:

-If you're using Google as an identity provider, we recommend you use the system browser as Google doesn't allow [authentication from embedded webviews](https://developers.googleblog.com/2016/08/modernizing-oauth-interactions-in-native-apps.html). Currently, `login.microsoftonline.com` is a trusted authority with Google and will work with embedded webview. However, `b2clogin.com` is not a trusted authority with Google, so users will not be able to authenticate.

-We'll provide an update to this [issue](https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/issues/688) if things change.

-| Sample | Platform | Description|

-| | -- | --|

-|[active-directory-b2c-xamarin-native](https://github.com/Azure-Samples/active-directory-b2c-xamarin-native) | Xamarin iOS, Xamarin Android, UWP | A Xamarin Forms app that uses MSAL.NET to authenticate users via Azure AD B2C and then access a web API with the tokens returned.|

-az rest --method POST --uri 'https://graph.microsoft.com/applications/f6475511-fd81-4965-a00e-41e7792b7b9c/federatedIdentityCredentials' --body '{"name":"Testing","issuer":"https://token.actions.githubusercontent.com/","subject":"repo:octo-org/octo-repo:environment:Production","description":"Testing","audiences":["api://AzureADTokenExchange"]}'

- "issuer": "https://token.actions.githubusercontent.com/",

-*issuer*: The path to the GitHub OIDC provider: `https://token.actions.githubusercontent.com/`. This issuer will become trusted by your Azure application.

-| RedHat Enterprise Linux (RHEL) | RHEL 7.4 to RHEL 7.10, RHEL 8.3+ |

-| EXCHANGE_L_STANDARD | d42bdbd6-c335-4231-ab3d-c8f348d5aff5 |

-| EXCHANGE_S_ENTERPRISE | efb87545-963c-4e0d-99df-69c6916d9eb0 |

-See the following articles to learn more about securing external access to resources. We recommend you follow the listed order.

-3. [Create a security plan for external access](3-secure-access-plan.md)

-7. [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-9. [Secure external access to Microsoft Teams, SharePoint, and OneDrive with Azure AD](9-secure-access-teams-sharepoint.md)

-# Convert local guest accounts to Azure Active Directory B2B guest accounts

-Notify external users about migration timing. Communicate expectations, such as when external users must stop using a current password to enable authenticate by home and corporate credentials. Communications can include email campaigns and announcements.

-See the following articles on securing external access to resources. We recommend you take the actions in the listed order.

-1. [Determine your desired security posture for external access](1-secure-access-posture.md)

-1. [Discover your current state](2-secure-access-current-state.md)

-1. [Create a governance plan](3-secure-access-plan.md)

-1. [Use groups for security](4-secure-access-groups.md)

-1. [Transition to Azure AD B2B](5-secure-access-b2b.md)

-1. [Secure access with Entitlement Management](6-secure-access-entitlement-managment.md)

-1. [Secure access with Conditional Access policies](7-secure-access-conditional-access.md)

-1. [Secure access with Sensitivity labels](8-secure-access-sensitivity-labels.md)

-1. [Secure access to Microsoft Teams, OneDrive, and SharePoint](9-secure-access-teams-sharepoint.md)

-1. [Convert local guest accounts to B2B](10-secure-local-guest.md) (YouΓÇÖre here)

-* [Determine your security posture for external access](1-secure-access-posture.md)

-* [Create a security plan for external access](3-secure-access-plan.md)

-* [Securing external access with groups](4-secure-access-groups.md)

-* [Transition to governed collaboration with Azure Active Directory B2B collaboration](5-secure-access-b2b.md)

-* [Manage external access with entitlement management](6-secure-access-entitlement-managment.md)

-* [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-* [Control access with sensitivity labels](8-secure-access-sensitivity-labels.md)

-* [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

-### Next steps

-* [Determine your security posture for external access](1-secure-access-posture.md)

-* [Discover the current state of external collaboration in your organization](2-secure-access-current-state.md)

-* [Securing external access with groups](4-secure-access-groups.md)

-* [Transition to governed collaboration with Azure Active Directory B2B collaboration](5-secure-access-b2b.md)

-* [Manage external access with entitlement management](6-secure-access-entitlement-managment.md)

-* [Secure access with Conditional Access policies](7-secure-access-conditional-access.md)

-* [Control access with sensitivity labels](8-secure-access-sensitivity-labels.md)

-* [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

- * Are members from outside your organization.

-### Next steps

-See the following articles to learn more about securing external access to resources. We recommend you follow the listed order.

-1. [Determine your security posture for external access](1-secure-access-posture.md)

-3. [Create a security plan for external access](3-secure-access-plan.md)

-6. [Manage external access with Azure AD entitlement management](6-secure-access-entitlement-managment.md)

-7. [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-8. [Control access with sensitivity labels](8-secure-access-sensitivity-labels.md)

-9. [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

-For context and needed information we recommend you read the first four articles in the series of ten articles.

-* [Determine your security posture for external access](1-secure-access-posture.md)

-* [Discover the current state of external collaboration in your organization](2-secure-access-current-state.md)

-* [Create a security plan for external access](3-secure-access-plan.md)

-* [Securing external access with groups](4-secure-access-groups.md)

-### Next steps

-* [Determine your security posture for external access](1-secure-access-posture.md)

-* [Discover the current state of external collaboration in your organization](2-secure-access-current-state.md)

-* [Create a security plan for external access](3-secure-access-plan.md)

-* [Securing external access with groups](4-secure-access-groups.md)

-* [Manage external access with Entitlement Management](6-secure-access-entitlement-managment.md)

-* [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-* [Control access with sensitivity labels](8-secure-access-sensitivity-labels.md)

-* [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

-### Next steps

-See the following articles to learn more about securing external access to resources. We recommend you follow the listed order.

-1. [Determine your security posture for external access](1-secure-access-posture.md)

-3. [Create a security plan for external access](3-secure-access-plan.md)

-4. [Securing external access with groups](4-secure-access-groups.md)

-5. [Transition to governed collaboration with Azure AD B2B collaboration](5-secure-access-b2b.md)

-7. [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-8. [Control access with sensitivity labels](8-secure-access-sensitivity-labels.md)

-9. [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

-See the following articles on securing external access to resources. We recommend you take the actions in the listed order.

-1. [Determine your desired security posture for external access](1-secure-access-posture.md)

-1. [Discover your current state](2-secure-access-current-state.md)

-1. [Create a governance plan](3-secure-access-plan.md)

-1. [Use groups for security](4-secure-access-groups.md)

-1. [Transition to Azure AD B2B](5-secure-access-b2b.md)

-1. [Secure access with Entitlement Management](6-secure-access-entitlement-managment.md)

-1. [Secure access with Conditional Access policies](7-secure-access-conditional-access.md) (YouΓÇÖre here)

-1. [Secure access with Sensitivity labels](8-secure-access-sensitivity-labels.md)

-1. [Secure access to Microsoft Teams, OneDrive, and SharePoint](9-secure-access-teams-sharepoint.md)

-See the following articles to learn more about securing external access to resources. We recommend you follow the listed order.

-1. [Determine your security posture for external access](1-secure-access-posture.md)

-3. [Create a security plan for external access](3-secure-access-plan.md)

-4. [Secure external access with groups in Azure AD and Microsoft 365](4-secure-access-groups.md)

-5. [Transition to governed collaboration with Azure AD B2B collaboration](5-secure-access-b2b.md)

-6. [Manage external access with Azure AD entitlement management](6-secure-access-entitlement-managment.md)

-7. [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-9. [Secure external access to Microsoft Teams, SharePoint, and OneDrive for Business](9-secure-access-teams-sharepoint.md)

-Sharing in Microsoft 365 is partially governed by the **External Identities, External collaboration** settings in Azure Active Directory (Azure AD). If external sharing is disabled or restricted in Azure AD, it overrides sharing settings configured in Microsoft 365. An exception is if Azure AD B2B integration isn't enabled. You can configure SharePoint and OneDrive to support ad-hoc sharing via one-time password (OTP). The following screenshot shows the External Identities, External collaboration settings dialog.

-The External Identities collaboration feaure in Azure AD controls permissions. You can increase restrictions in Teams, but restrictions can't be lower than Azure AD settings.

-See the following articles to learn more about securing external access to resources. We recommend you follow the listed order.

-3. [Create a security plan for external access](3-secure-access-plan.md)

-4. [Secure external access with groups in Azure AD and Microsoft 365](4-secure-access-groups.md)

-5. [Transition to governed collaboration with Azure AD B2B collaboration](5-secure-access-b2b.md)

-6. [Manage external access with Azure AD entitlement management](6-secure-access-entitlement-managment.md)

-7. [Manage external access with Conditional Access policies](7-secure-access-conditional-access.md)

-8. [Control external access to resources in Azure AD with sensitivity labels](8-secure-access-sensitivity-labels.md)

-9. [Secure external access to Microsoft Teams, SharePoint, and OneDrive with Azure AD](9-secure-access-teams-sharepoint.md) (You're here)

-* [Where is my data? - Office 365 documentation](http://o365datacentermap.azurewebsites.net/)

-**Members can invite** | Setting this option to **Yes** allows non-admin members of your directory to invite guests. To learn more, see [Configure external collaboration settings](../external-identities/external-collaboration-settings-configure.md).

-**Admins and users in the guest inviter role can invite** | Setting this option to **Yes** allows admins and users in the guest inviter role to invite guests. When you set this option to **Yes**, users in the guest inviter role will still be able to invite guests, regardless of the **Members can invite** setting. To learn more, see [Configure external collaboration settings](../external-identities/external-collaboration-settings-configure.md).

-# Customize the schedule of workflows (Preview)

-## Customize the schedule of workflows using Microsoft Graph

-First, to view the current schedule interval of your workflows, run the following get call:

-```http

-GET https://graph.microsoft.com/beta/identityGovernance/lifecycleWorkflows/settings

-```

-To customize a workflow in Microsoft Graph, use the following request and body:

-```http

-PATCH https://graph.microsoft.com/beta/identityGovernance/lifecycleWorkflows/settings

-Content-type: application/json

-{

-"workflowScheduleIntervalInHours":<Interval between 0-24>

-}

-```

-In this article you'll learn how to configure permissions classifications in Azure Active Directory (Azure AD). Permission classifications allow you to identify the impact that different permissions have according to your organization's policies and risk evaluations. For example, you can use permission classifications in consent policies to identify the set of permissions that users are allowed to consent to.

-The minimum permissions needed to do basic sign in are `openid`, `profile`, `email`, and `offline_access`, which are all delegated permissions on the Microsoft Graph. With these permissions an app can read details of the signed-in user's profile, and can maintain this access even when the user is no longer using the app.

-# [Portal](#tab/azure-portal)

-# [PowerShell](#tab/azure-powershell)

-You can use the latest Azure AD PowerShell Preview module, [AzureADPreview](/powershell/module/azuread/?preserve-view=true&view=azureadps-2.0-preview), to classify permissions. Permission classifications are configured on the **ServicePrincipal** object of the API that publishes the permissions.

-#### List the current permission classifications for an API

-#### Classify a permission as "Low impact"

-#### Remove a delegated permission classification

-## Next steps

-To learn more:

-## Use Azure AD PowerShell to disable an unlisted app

-Ensure you've installed the AzureAD module (use the command Install-Module -Name AzureAD). In case you're prompted to install a NuGet module or the new Azure Active Directory V2 PowerShell module, type Y and press ENTER.

-You may know the AppId of an app that doesn't appear on the Enterprise apps list. For example, you may have deleted the app or the service principal hasn't yet been created due to the app being pre-authorized by Microsoft), you can manually create the service principal for the app and then disable it by using the following cmdlet.

-```azurepowershell-interactive

-The **Status** of a recommendation can be updated manually or automatically. If all resources are addressed according to the action plan, the status will automatically change to *Completed* the next time the recommendations service runs. The recommendation service runs every 24-48 hours, depending on the recommendation.

-

- 

- - Mark a recommendation as **Completed** if all impacted resources have been addressed.

- - Active resources may still appear in the list of resources for manually completed recommendations. If the resource is completed, the service will update the status the next time the service runs.

- - If the service identifies an active resource for a manually completed recommendation the next time the service runs, the recommendation will automatically change back to **Active**.

- - Completing a recommendation is the only action collected in the audit log. To view these logs, go to **Azure AD** > **Audit logs** and filter the service to "Azure AD recommendations."

- - Azure AD will ask for a reason why you dismissed the recommendation so we can improve the service.

- - The recommendation will become **Active** when the selected date occurs.

-# Azure AD recommendation: Migrate to Microsoft Authenticator

-This article covers the recommendation to migrate users to the Microsoft Authenticator app. This recommendation is called `useAuthenticatorApp` in the recommendations API in Microsoft Graph.

-|AC.L2-3.1.12<br><br>**Practice statement:** Monitor and control remote access sessions.<br><br>**Objectives:**<br>Determine if:<br>[a.] remote access sessions are permitted;<br>[b.] the types of permitted remote access are identified;<br>[c.] remote access sessions are controlled; and<br>[d.] remote access sessions are monitored. | In todayΓÇÖs world, users access cloud-based applications almost exclusively remotely from unknown or untrusted networks. It's critical to securing this pattern of access to adopt zero trust principals. To meet these controls requirements in a modern cloud world we must verify each access request explicitly, implement least privilege and assume breach.<br><br>Configure named locations to delineate internal vs external networks. Configure conditional access app control to route access via Microsoft Defender for Cloud Apps. Configure Defender for Cloud Apps to control and monitor all sessions.<br>[Zero Trust Deployment Guide for Microsoft Azure Active Directory](/security/blog/2020/04/30/zero-trust-deployment-guide-azure-active-directory/)<br>[Location condition in Azure Active Directory Conditional Access](../conditional-access/location-condition.md)<br>[Deploy Cloud App Security Conditional Access App Control for Azure AD apps](/cloud-app-security/proxy-deployment-aad.md)<br>[What is Microsoft Defender for Cloud Apps?](/cloud-app-security/what-is-cloud-app-security.md)<br>[Monitor alerts raised in Microsoft Defender for Cloud Apps](/cloud-app-security/monitor-alerts.md) |

-| AC.L2-3.1.14<br><br>**Practice statement:** Route remote access via managed access control points.<br><br>**Objectives:**<br>Determine if:<br>[a.] managed access control points are identified and implemented; and<br>[b.] remote access is routed through managed network access control points. | Configure named locations to delineate internal vs external networks. Configure conditional access app control to route access via Microsoft Defender for Cloud Apps. Configure Defender for Cloud Apps to control and monitor all sessions. Secure devices used by privileged accounts as part of the privileged access story.<br>[Location condition in Azure Active Directory Conditional Access](../conditional-access/location-condition.md)<br>[Session controls in Conditional Access policy](../conditional-access/concept-conditional-access-session.md)<br>[Securing privileged access overview](/security/compass/overview.md) |

-| AC.L2-3.1.15<br><br>**Practice statement:** Authorize remote execution of privileged commands and remote access to security-relevant information.<br><br>**Objectives:**<br>Determine if:<br>[a.] privileged commands authorized for remote execution are identified;<br>[b.] security-relevant information authorized to be accessed remotely is identified;<br>[c.] the execution of the identified privileged commands via remote access is authorized; and<br>[d.] access to the identified security-relevant information via remote access is authorized. | Conditional Access is the Zero Trust control plane to target policies for access to your apps when combined with authentication context. You can apply different policies in those apps. Secure devices used by privileged accounts as part of the privileged access story. Configure conditional access policies to require the use of these secured devices by privileged users when performing privileged commands.<br>[Cloud apps, actions, and authentication context in Conditional Access policy](../conditional-access/concept-conditional-access-cloud-apps.md)<br>[Securing privileged access overview](/security/compass/overview.md)<br>[Filter for devices as a condition in Conditional Access policy](../conditional-access/concept-condition-filters-for-devices.md) |

-| AC.L2-3.1.18<br><br>**Practice statement:** Control connection of mobile devices.<br><br>**Objectives:**<br>Determine if:<br>[a.] mobile devices that process, store, or transmit CUI are identified;<br>[b.] mobile device connections are authorized; and<br>[c.] mobile device connections are monitored and logged. | Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM) or group policy objects (GPO) to enforce mobile device configuration and connection profile. Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md)<br><br>**InTune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md)<br>[What is app management in Microsoft Intune?](/mem/intune/apps/app-management.md) |

-| AC.L2-3.1.21<br><br>**Practice statement:** Limit use of portable storage devices on external systems.<br><br>**Objectives:**<br>Determine if:<br>[a.] the use of portable storage devices containing CUI on external systems is identified and documented;<br>[b.] limits on the use of portable storage devices containing CUI on external systems are defined; and<br>[c.] the use of portable storage devices containing CUI on external systems is limited as defined. | Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM) or group policy objects (GPO) to control the use of portable storage devices on systems. Configure policy settings on the Windows device to completely prohibit or restrict use of portable storage at the OS level. For all other devices where you may be unable to granularly control access to portable storage block download entirely with Microsoft Defender for Cloud Apps. Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md)<br>[Configure authentication session management - Azure Active Directory](../conditional-access/howto-conditional-access-session-lifetime.md)<br><br>**InTune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md)<br>[Restrict USB devices using administrative templates in Microsoft Intune](/mem/intune/configuration/administrative-templates-restrict-usb.md)<br><br>**Microsoft Defender for Cloud Apps**<br>[Create session policies in Defender for Cloud Apps](/defender-cloud-apps/session-policy-aad.md)

-| CM.L2-3.4.2<br><br>**Practice statement:** Establish and enforce security configuration settings for information technology products employed in organizational systems.<br><br>**Objectives:**<br>Determine if:<br>[a.] security configuration settings for information technology products employed in the system are established and included in the baseline configuration; and<br>[b.] security configuration settings for information technology products employed in the system are enforced. | Adopt a zero-trust security posture. Use conditional access policies to restrict access to compliant devices. Configure policy settings on the device to enforce security configuration settings on the device with MDM solutions such as Microsoft Intune. Microsoft Endpoint Configuration Manager(MECM) or group policy objects can also be considered in hybrid deployments and combined with conditional access require hybrid Azure AD joined device.<br><br>**Zero-trust**<br>[Securing identity with Zero Trust](/security/zero-trust/identity.md)<br><br>**Conditional access**<br>[What is conditional access in Azure AD?](../conditional-access/overview.md)<br>[Grant controls in Conditional Access policy](../conditional-access/concept-conditional-access-grant.md)<br><br>**Device policies**<br>[What is Microsoft Intune?](/mem/intune/fundamentals/what-is-intune.md)<br>[What is Defender for Cloud Apps?](/cloud-app-security/what-is-cloud-app-security.md)<br>[What is app management in Microsoft Intune?](/mem/intune/apps/app-management.md)<br>[Microsoft Endpoint Manager overview](/mem/endpoint-manager-overview.md) |

-| CM.L2-3.4.6<br><br>**Practice statement:** Employ the principle of least functionality by configuring organizational systems to provide only essential capabilities.<br><br>**Objectives:**<br>Determine if:<br>[a.] essential system capabilities are defined based on the principle of least functionality; and<br>[b.] the system is configured to provide only the defined essential capabilities. | Configure device management solutions (Such as Microsoft Intune) to implement a custom security baseline applied to organizational systems to remove non-essential applications and disable unnecessary services. Leave only the fewest capabilities necessary for the systems to operate effectively. Configure conditional access to restrict access to compliant or hybrid Azure AD joined devices. <br>[What is Microsoft Intune](/mem/intune/fundamentals/what-is-intune.md)<br>[Require device to be marked as compliant](../conditional-access/require-managed-devices.md)<br>[Grant controls in Conditional Access policy - Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md) |

-| CM.L2-3.4.7<br><br>**Practice statement:** Restrict, disable, or prevent the use of nonessential programs, functions, ports, protocols, and services.<br><br>**Objectives:**<br>Determine if:<br>[a.]essential programs are defined;<br>[b.] the use of nonessential programs is defined;<br>[c.] the use of nonessential programs is restricted, disabled, or prevented as defined;<br>[d.] essential functions are defined;<br>[e.] the use of nonessential functions is defined;<br>[f.] the use of nonessential functions is restricted, disabled, or prevented as defined;<br>[g.] essential ports are defined;<br>[h.] the use of nonessential ports is defined;<br>[i.] the use of nonessential ports is restricted, disabled, or prevented as defined;<br>[j.] essential protocols are defined;<br>[k.] the use of nonessential protocols is defined;<br>[l.] the use of nonessential protocols is restricted, disabled, or prevented as defined;<br>[m.] essential services are defined;<br>[n.] the use of nonessential services is defined; and<br>[o.] the use of nonessential services is restricted, disabled, or prevented as defined. | Use Application Administrator role to delegate authorized use of essential applications. Use App Roles or group claims to manage least privilege access within application. Configure user consent to require admin approval and don't allow group owner consent. Configure Admin consent request workflows to enable users to request access to applications that require admin consent. Use Microsoft Defender for Cloud Apps to identify unsanctioned/unknown application use. Use this telemetry to then determine essential/non-essential apps.<br>[Azure AD built-in roles - Application Administrator](../roles/permissions-reference.md)<br>[Azure AD App Roles - App Roles vs. Groups ](../develop/howto-add-app-roles-in-azure-ad-apps.md)<br>[Configure how users consent to applications](../manage-apps/configure-user-consent.md?tabs=azure-portal.md)<br>[Configure group owner consent to apps accessing group data](../manage-apps/configure-user-consent-groups.md?tabs=azure-portal.md)<br>[Configure the admin consent workflow](../manage-apps/configure-admin-consent-workflow.md)<br>[What is Defender for Cloud Apps?](/defender-cloud-apps/what-is-defender-for-cloud-apps.d)<br>[Discover and manage Shadow IT tutorial](/defender-cloud-apps/tutorial-shadow-it.md) |

-| CM.L2-3.4.8<br><br>**Practice statement:** Apply deny-by-exception (blocklist) policy to prevent the use of unauthorized software or deny-all, permit-by-exception (allowlist) policy to allow the execution of authorized software.<br><br>**Objectives:**<br>Determine if:<br>[a.] a policy specifying whether allowlist or blocklist is to be implemented is specified;<br>[b.] the software allowed to execute under allowlist or denied use under blocklist is specified; and<br>[c.] allowlist to allow the execution of authorized software or blocklist to prevent the use of unauthorized software is implemented as specified.<br><br>CM.L2-3.4.9<br><br>**Practice statement:** Control and monitor user-installed software.<br><br>**Objectives:**<br>Determine if:<br>[a.] a policy for controlling the installation of software by users is established;<br>[b.] installation of software by users is controlled based on the established policy; and<br>[c.] installation of software by users is monitored. | Configure MDM/configuration management policy to prevent the use of unauthorized software. Configure conditional access grant controls to require compliant or hybrid joined device to incorporate device compliance with MDM/configuration management policy into the conditional access authorization decision.<br>[What is Microsoft Intune](/mem/intune/fundamentals/what-is-intune.md)<br>[Conditional Access - Require compliant or hybrid joined devices](../conditional-access/howto-conditional-access-policy-compliant-device.md) |

-| MP.L2-3.8.7<br><br>**Practice statement:** Control the use of removable media on system components.<br><br>**Objectives:**<br>Determine if:<br>[a.] the use of removable media on system components is controlled. | Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM) or group policy objects (GPO) to control the use of removable media on systems. Deploy and manage Removable Storage Access Control using Intune or Group Policy. Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](/conditional-access/concept-conditional-access-grant#require-hybrid-azure-ad-joined-device.md)<br><br>**Intune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md)<br><br>**Removable storage access control**<br>[Deploy and manage Removable Storage Access Control using Intune](/microsoft-365/security/defender-endpoint/deploy-manage-removable-storage-intune?view=o365-worldwide&preserve-view=true)<br>[Deploy and manage Removable Storage Access Control using group policy](/microsoft-365/security/defender-endpoint/deploy-manage-removable-storage-group-policy?view=o365-worldwide&preserve-view=true) |

-| SC.L2-3.13.3<br><br>**Practice statement:** Separate user functionality form system management functionality. <br><br>**Objectives:**<br>Determine if:<br>[a.] user functionality is identified;<br>[b.] system management functionality is identified; and<br>[c.] user functionality is separated from system management functionality. | Maintain separate user accounts in Azure Active Directory for everyday productivity use and administrative or system/privileged management. Privileged accounts should be cloud-only or managed accounts and not synchronized from on-premises to protect the cloud environment from on-premises compromise. System/privileged access should only be permitted from a security hardened privileged access workstation (PAW). Configure Conditional Access device filters to restrict access to administrative applications from PAWs that are enabled using Azure Virtual Desktops.<br>[Why are privileged access devices important](/security/compass/privileged-access-devices.md)<br>[Device Roles and Profiles](/security/compass/privileged-access-devices.md)<br>[Filter for devices as a condition in Conditional Access policy](../conditional-access/concept-condition-filters-for-devices.md)<br>[Azure Virtual Desktop](https://azure.microsoft.com/products/virtual-desktop/) |

-| SC.L2-3.13.4<br><br>**Practice statement:** Prevent unauthorized and unintended information transfer via shared system resources.<br><br>**Objectives:**<br>Determine if:<br>[a.] unauthorized and unintended information transfer via shared system resources is prevented. | Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM) or group policy objects (GPO) to ensure devices are compliant with system hardening procedures. Include compliance with company policy regarding software patches to prevent attackers from exploiting flaws.<br><br>Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md)<br><br>**InTune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md) |

-| SC.L2-3.13.13<br><br>**Practice statement:** Control and monitor the use of mobile code.<br><br>**Objectives:**<br>Determine if:<br>[a.] use of mobile code is controlled; and<br>[b.] use of mobile code is monitored. | Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM) or group policy objects (GPO) to disable the use of mobile code. Where use of mobile code is required monitor the use with endpoint security such as Microsoft Defender for Endpoint.<br><br>Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md)<br><br>**InTune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md)<br><br>**Defender for Endpoint**<br>[Microsoft Defender for Endpoint](/microsoft-365/security/defender-endpoint/microsoft-defender-endpoint?view=o365-worldwide&preserve-view=true) |

-| SI.L2-3.14.7<br><br>**Practice statement:**<br><br>**Objectives:** Identify unauthorized use of organizational systems.<br>Determine if:<br>[a.] authorized use of the system is defined; and<br>[b.] unauthorized use of the system is identified. | Consolidate telemetry: Azure AD logs to stream to SIEM, such as Azure Sentinel Configure device management policies via MDM (such as Microsoft Intune), Microsoft Endpoint Manager (MEM), or group policy objects (GPO) to require Intrusion Detection/Protection (IDS/IPS) such as Microsoft Defender for Endpoint is installed and in use. Use telemetry provided by the IDS/IPS to identify unusual activities or conditions related to inbound and outbound communications traffic or unauthorized use.<br><br>Configure Conditional Access policies to enforce device compliance.<br><br>**Conditional Access**<br>[Require device to be marked as compliant](../conditional-access/concept-conditional-access-grant.md)<br>[Require hybrid Azure AD joined device](../conditional-access/concept-conditional-access-grant.md)<br><br>**InTune**<br>[Device compliance policies in Microsoft Intune](/mem/intune/protect/device-compliance-get-started.md)<br><br>**Defender for Endpoint**<br>[Microsoft Defender for Endpoint](/microsoft-365/security/defender-endpoint/microsoft-defender-endpoint?view=o365-worldwide&preserve-view=true) |

-| IA.L2-3.5.6<br><br>**Practice statement:** Disable identifiers after a defined period of inactivity.<br><br>**Objectives:**<br>Determine if:<br>[a.] a period of inactivity after which an identifier is disabled is defined; and<br>[b.] identifiers are disabled after the defined period of inactivity. | Implement account management automation with Microsoft Graph and Azure AD PowerShell. Use Microsoft Graph to monitor sign-in activity and Azure AD PowerShell to take action on accounts within the required time frame.<br><br>**Determine inactivity**<br>[Manage inactive user accounts in Azure AD](../reports-monitoring/howto-manage-inactive-user-accounts.md)<br>[Manage stale devices in Azure AD](../devices/manage-stale-devices.md)<br><br>**Remove or disable accounts**<br>[Working with users in Microsoft Graph](/graph/api/resources/users.md)<br>[Get a user](/graph/api/user-get?tabs=http)<br>[Update user](/graph/api/user-update?tabs=http)<br>[Delete a user](/graph/api/user-delete?tabs=http)<br><br>**Work with devices in Microsoft Graph**<br>[Get device](/graph/api/device-get?tabs=http)<br>[Update device](/graph/api/device-update?tabs=http)<br>[Delete device](/graph/api/device-delete?tabs=http)<br><br>**[Use Azure AD PowerShell](/powershell/module/azuread/)**<br>[Get-AzureADUser](/powershell/module/azuread/get-azureaduser.md)<br>[Set-AzureADUser](/powershell/module/azuread/set-azureaduser)<br>[Get-AzureADDevice](/powershell/module/azuread/get-azureaddevice.md)<br>[Set-AzureADDevice](/powershell/module/azuread/set-azureaddevice.md) |

-| IA.L2-3.5.9<br><br>**Practice statement:** Allow temporary password use for system logons with an immediate change to a permanent password.<br><br>**Objectives:**<br>Determine if:<br>[a.] an immediate change to a permanent password is required when a temporary password is used for system sign-on. | An Azure AD user initial password is a temporary single use password that once successfully used is immediately required to be changed to a permanent password. Microsoft strongly encourages the adoption of passwordless authentication methods. Users can bootstrap Passwordless authentication methods using Temporary Access Pass (TAP). TAP is a time and use limited passcode issued by an admin that satisfies strong authentication requirements. Use of passwordless authentication along with the time and use limited TAP completely eliminates the use of passwords (and their reuse).<br>[Add or delete users - Azure Active Directory](../fundamentals/add-users-azure-active-directory.md)<br>[Configure a Temporary Access Pass in Azure AD to register Passwordless authentication methods](../authentication/howto-authentication-temporary-access-pass.md)<br>[Passwordless authentication](/security/business/solutions/passwordless-authentication?ef_id=369464fc2ba818d0bd6507de2cde3d58:G:s&OCID=AIDcmmdamuj0pc_SEM_369464fc2ba818d0bd6507de2cde3d58:G:s&msclkid=369464fc2ba818d0bd6507de2cde3d58) |

-> - East US 2

-> - West US 2

-With [Azure Container Networking Interface (CNI)][cni-networking], every pod gets an IP address from the subnet and can be accessed directly. These IP addresses must be unique across your network space and must be planned in advance. Each node has a configuration parameter for the maximum number of pods that it supports. The equivalent number of IP addresses per node are then reserved up front for that node. This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow.

-**Kubernetes service address range**: This parameter is the set of virtual IPs that Kubernetes assigns to internal [services][services] in your cluster. You can use any private address range that satisfies the following requirements:

-* AKS clusters may not use `169.254.0.0/16`, `172.30.0.0/16`, `172.31.0.0/16`, or `192.0.2.0/24` for the Kubernetes service address range, pod address range or cluster virtual network address range.

-As the name suggests, bridge mode Azure CNI, in a "just in time" fashion, will create a L2 bridge named "azure0". All the host side pod `veth` pair interfaces will be connected to this bridge. So Pod-Pod intra VM communication and the remaining traffic goes through this bridge. The bridge in question is a layer 2 virtual device that on its own cannot receive or transmit anything unless you bind one or more real devices to it. For this reason, eth0 of the Linux VM has to be converted into a subordinate to "azure0" bridge. This creates a complex network topology within the Linux VM and as a symptom CNI had to take care of other networking functions like DNS server update and so on.

-> There are two open source ingress controllers for Kubernetes based on Nginx: one is maintained by the Kubernetes community ([kubernetes/ingress-nginx][nginx-ingress]), and one is maintained by NGINX, Inc. ([nginxinc/kubernetes-ingress]). This article will be using the Kubernetes community ingress controller.

-This article uses [Helm 3][helm] to install the NGINX ingress controller on a [supported version of Kubernetes][aks-supported versions]. Make sure that you're using the latest release of Helm and have access to the *ingress-nginx* Helm repository. The steps outlined in this article may not be compatible with previous versions of the Helm chart, NGINX ingress controller, or Kubernetes.

-### [Azure CLI](#tab/azure-cli)

-This article also requires that you're running the Azure CLI version 2.0.64 or later. Run `az --version` to find the version. If you need to install or upgrade, see [Install Azure CLI][azure-cli-install].

-In addition, this article assumes you have an existing AKS cluster with an integrated Azure Container Registry (ACR). For more information on creating an AKS cluster with an integrated ACR, see [Authenticate with Azure Container Registry from Azure Kubernetes Service][aks-integrated-acr].

-### [Azure PowerShell](#tab/azure-powershell)

-This article also requires that you're running Azure PowerShell version 5.9.0 or later. Run `Get-InstalledModule -Name Az` to find the version. If you need to install or upgrade, see [Install Azure PowerShell][azure-powershell-install].

-In addition, this article assumes you have an existing AKS cluster with an integrated Azure Container Registry (ACR). For more information on creating an AKS cluster with an integrated ACR, see [Authenticate with Azure Container Registry from Azure Kubernetes Service][aks-integrated-acr-ps].

-To create a basic NGINX ingress controller without customizing the defaults, you'll use Helm.

-The above configuration uses the default configuration for simplicity. You can add parameters for customizing the deployment, for example, `--set controller.replicaCount=3`. The next section will show a highly customized example of the ingress controller.

-### Use an internal IP address

-By default, an NGINX ingress controller is created with a dynamic public IP address assignment. A common configuration requirement is to use an internal, private network and IP address. This approach allows you to restrict access to your services to internal users, with no external access.

-Create a file named `internal-ingress.yaml` using the following example manifest:

-```yaml

-controller:

- service:

- loadBalancerIP: 10.224.0.42

- annotations:

- service.beta.kubernetes.io/azure-load-balancer-internal: "true"

-```

-This example assigns *10.224.0.42* to the *loadBalancerIP* resource. Provide your own internal IP address for use with the ingress controller. Make sure that this IP address isn't already in use within your virtual network. Also, if you're using an existing virtual network and subnet, you must configure your AKS cluster with the correct permissions to manage the virtual network and subnet. For more information, see [Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS)][aks-configure-kubenet-networking] or [Configure Azure CNI networking in Azure Kubernetes Service (AKS)][aks-configure-advanced-networking].

-When you deploy the *nginx-ingress* chart with Helm, add the `-f internal-ingress.yaml` parameter.

- --set defaultBackend.image.digest="" \

- -f internal-ingress.yaml

-helm install nginx-ingress ingress-nginx/ingress-nginx `

- --set defaultBackend.image.digest="" `

- -f internal-ingress.yaml

-### Create an ingress controller

-To create the ingress controller, use Helm to install *nginx-ingress*. The ingress controller needs to be scheduled on a Linux node. Windows Server nodes shouldn't run the ingress controller. A node selector is specified using the `--set nodeSelector` parameter to tell the Kubernetes scheduler to run the NGINX ingress controller on a Linux-based node.

-For added redundancy, two replicas of the NGINX ingress controllers are deployed with the `--set controller.replicaCount` parameter. To fully benefit from running replicas of the ingress controller, make sure there's more than one node in your AKS cluster.

-The following example creates a Kubernetes namespace for the ingress resources named *ingress-basic* and is intended to work within that namespace. Specify a namespace for your own environment as needed. If your AKS cluster isn't Kubernetes role-based access control enabled, add `--set rbac.create=false` to the Helm commands.

-> [!NOTE]

-> If you would like to enable [client source IP preservation][client-source-ip] for requests to containers in your cluster, add `--set controller.service.externalTrafficPolicy=Local` to the Helm install command. The client source IP is stored in the request header under *X-Forwarded-For*. When you're using an ingress controller with client source IP preservation enabled, TLS pass-through won't work.

-helm install nginx-ingress ingress-nginx/ingress-nginx \

- --set defaultBackend.image.digest=""

-helm install nginx-ingress ingress-nginx/ingress-nginx `

- --set defaultBackend.image.digest=""

-```

-Create an `aks-helloworld-one.yaml` file and copy in the following example YAML:

-```yml

-apiVersion: apps/v1

-kind: Deployment

-metadata:

- name: aks-helloworld-one

-spec:

- replicas: 1

- selector:

- matchLabels:

- app: aks-helloworld-one

- template:

- labels:

- app: aks-helloworld-one

- containers:

- - name: aks-helloworld-one

- image: mcr.microsoft.com/azuredocs/aks-helloworld:v1

- ports:

- - containerPort: 80

- env:

- - name: TITLE

- value: "Welcome to Azure Kubernetes Service (AKS)"

-apiVersion: v1

-kind: Service

-metadata:

- name: aks-helloworld-one

-spec:

- type: ClusterIP

- ports:

- - port: 80

- selector:

- app: aks-helloworld-one

-```

-Create an `aks-helloworld-two.yaml` file and copy in the following example YAML:

-```yml

-apiVersion: apps/v1

-kind: Deployment

-metadata:

- name: aks-helloworld-two

-spec:

- replicas: 1

- selector:

- matchLabels:

- app: aks-helloworld-two

- template:

- labels:

- app: aks-helloworld-two

- containers:

- - name: aks-helloworld-two

- image: mcr.microsoft.com/azuredocs/aks-helloworld:v1

- ports:

- - containerPort: 80

- env:

- - name: TITLE

- value: "AKS Ingress Demo"

-apiVersion: v1

-kind: Service

-metadata:

- name: aks-helloworld-two

-spec:

- type: ClusterIP

- ports:

- - port: 80

- selector:

- app: aks-helloworld-two

-```

-Run the two demo applications using `kubectl apply`:

-```console

-kubectl apply -f aks-helloworld-one.yaml --namespace ingress-basic

-kubectl apply -f aks-helloworld-two.yaml --namespace ingress-basic

-```

-Create a file named `hello-world-ingress.yaml` and copy in the following example YAML.

-```yaml

-apiVersion: networking.k8s.io/v1

-kind: Ingress

-metadata:

- name: hello-world-ingress

- annotations:

- nginx.ingress.kubernetes.io/ssl-redirect: "false"

- nginx.ingress.kubernetes.io/use-regex: "true"

- nginx.ingress.kubernetes.io/rewrite-target: /$2

-spec:

- ingressClassName: nginx

- rules:

- - http:

- paths:

- - path: /hello-world-one(/|$)(.*)

- pathType: Prefix

- backend:

- service:

- name: aks-helloworld-one

- port:

- number: 80

- - path: /hello-world-two(/|$)(.*)

- pathType: Prefix

- backend:

- service:

- name: aks-helloworld-two

- port:

- number: 80

- - path: /(.*)

- pathType: Prefix

- backend:

- service:

- name: aks-helloworld-one

- port:

- number: 80

-apiVersion: networking.k8s.io/v1

-kind: Ingress

-metadata:

- name: hello-world-ingress-static

- annotations:

- nginx.ingress.kubernetes.io/ssl-redirect: "false"

- nginx.ingress.kubernetes.io/rewrite-target: /static/$2

-spec:

- ingressClassName: nginx

- rules:

- - http:

- paths:

- - path: /static(/|$)(.*)

- pathType: Prefix

- backend:

- service:

- name: aks-helloworld-one

- port:

- number: 80

-```

-Create the ingress resource using the `kubectl apply` command.

-```console

-kubectl apply -f hello-world-ingress.yaml --namespace ingress-basic

-```

-To test the routes for the ingress controller using an internal IP, create a test pod and attach a terminal session to it:

-```console

-kubectl run -it --rm aks-ingress-test --image=mcr.microsoft.com/dotnet/runtime-deps:6.0 --namespace ingress-basic

-```

-Install `curl` in the pod using `apt-get`:

-```console

-apt-get update && apt-get install -y curl

-```

-Now access the address of your Kubernetes ingress controller using `curl`, such as *http://10.224.0.42*. Provide your own internal IP address specified when you deployed the ingress controller.

-```console

-curl -L http://10.224.0.42

-```

-No path was provided with the address, so the ingress controller defaults to the */* route. The first demo application is returned, as shown in the following condensed example output:

-```

-$ curl -L http://10.224.0.42

-<!DOCTYPE html>

-<html xmlns="http://www.w3.org/1999/xhtml">

-<head>

- <link rel="stylesheet" type="text/css" href="/static/default.css">

- <title>Welcome to Azure Kubernetes Service (AKS)</title>

-[...]

-```

-Now add */hello-world-two* path to the address, such as *http://10.224.0.42/hello-world-two*. The second demo application with the custom title is returned, as shown in the following condensed example output:

-```

-$ curl -L -k http://10.224.0.42/hello-world-two

-<!DOCTYPE html>

-<html xmlns="http://www.w3.org/1999/xhtml">

-<head>

- <link rel="stylesheet" type="text/css" href="/static/default.css">

- <title>AKS Ingress Demo</title>

-[...]

-```

-Alternatively, a more granular approach is to delete the individual resources created. List the Helm releases with the `helm list` command. Look for charts named *nginx-ingress* and *aks-helloworld*, as shown in the following example output:

-```

-$ helm list --namespace ingress-basic

-NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION

-nginx-ingress ingress-basic 1 2020-01-06 19:55:46.358275 -0600 CST deployed nginx-ingress-1.27.1 0.26.1

-```

-Uninstall the releases with the `helm uninstall` command. The following example uninstalls the NGINX ingress deployment.

-```

-$ helm uninstall ingress-nginx --namespace ingress-basic

-release "ingress-nginx" uninstalled

-```

-Next, remove the two sample applications:

-```console

-kubectl delete -f aks-helloworld-one.yaml --namespace ingress-basic

-kubectl delete -f aks-helloworld-two.yaml --namespace ingress-basic

-```

-Remove the ingress route that directed traffic to the sample apps:

-```console

-kubectl delete -f hello-world-ingress.yaml

-```

-Finally, you can delete the itself namespace. Use the `kubectl delete` command and specify your namespace name:

-```console

-kubectl delete namespace ingress-basic

-```

- version: 14.7.1

-az k8s-extension create --cluster-name ${clusterName} --resource-group ${resourceGroup} --name ${adsExtensionName} --cluster-type connectedClusters --extension-type microsoft.arcdataservices --auto-upgrade false --scope cluster --release-namespace ${namespace} --config Microsoft.CustomLocation.ServiceAccount=sa-arc-bootstrapper

-az k8s-extension create --cluster-name $ENV:clusterName --resource-group $ENV:resourceGroup --name $ENV:adsExtensionName --cluster-type connectedClusters --extension-type microsoft.arcdataservices --auto-upgrade false --scope cluster --release-namespace $ENV:namespace --config Microsoft.CustomLocation.ServiceAccount=sa-arc-bootstrapper

-az k8s-extension create --cluster-name "<connected cluster name>" --resource-group "<resource group>" --name "<extension name>" --cluster-type connectedClusters --extension-type microsoft.arcdataservices --scope cluster --release-namespace "<namespace>" --config Microsoft.CustomLocation.ServiceAccount=sa-arc-bootstrapper --config imageCredentials.registry=<registry info> --config imageCredentials.username=<username> --config systemDefaultValues.image=<registry/repo/arc-bootstrapper:<imagetag>> --config-protected imageCredentials.password=$ENV:DOCKER_PASSWORD --debug

-az k8s-extension create --cluster-name "my-connected-cluster" --resource-group "my-resource-group" --name "arc-data-services" --cluster-type connectedClusters --extension-type microsoft.arcdataservices --scope cluster --release-namespace "arc" --config Microsoft.CustomLocation.ServiceAccount=sa-bootstrapper --config imageCredentials.registry=mcr.microsoft.com --config imageCredentials.username=arcuser --config systemDefaultValues.image=mcr.microsoft.com/arcdata/arc-bootstrapper:latest --config-protected imageCredentials.password=$ENV:DOCKER_PASSWORD --debug

-Like Azure Functions, there are templates to help you develop Durable Functions using [Visual Studio 2019](durable-functions-create-first-csharp.md), [Visual Studio Code](quickstart-js-vscode.md), and the [Azure portal](durable-functions-create-portal.md).

- durable_client = df.DurableOrchestrationClient(client)

- await durable_client.raise_event(instance_id, "ApprovalEvent", is_approved)

- > [!NOTE]

- > For more information on debugging, see [Durable Functions Diagnostics](durable-functions-diagnostics.md#debugging).

-1. Durable Functions require an Azure storage account to run. When Visual Studio Code prompts you to select a storage account, select **Select storage account**.

-1. Follow the prompts and provide the following information to create a new storage account in Azure:

-1. In the **Terminal** panel, copy the URL endpoint of your HTTP-triggered function.

-1. Use your browser, or a tool like [Postman](https://www.getpostman.com/) or [cURL](https://curl.haxx.se/), send an HTTP request to the URL endpoint. Replace the last segment with the name of the orchestrator function (`HelloOrchestrator`). The URL must be similar to `http://localhost:7071/api/orchestrators/HelloOrchestrator`.

-1. Copy the URL value for `statusQueryGetUri`, paste it in the browser's address bar, and execute the request. Alternatively, you can also continue to use Postman to issue the GET request.

- ```json

- {

- "name": "HelloOrchestrator",

- "instanceId": "9a528a9e926f4b46b7d3deaa134b7e8a",

- "runtimeStatus": "Completed",

- "input": null,

- "customStatus": null,

- "output": [

- "Hello Tokyo!",

- "Hello Seattle!",

- "Hello London!"

- ],

- "createdTime": "2020-03-18T21:54:49Z",

- "lastUpdatedTime": "2020-03-18T21:54:54Z"

- }

- ```

-1. To stop debugging, press <kbd>Shift+F5</kbd> in Visual Studio Code.

-> The Azure SQL trigger is only supported on **Premium and Dedicated** plans. Consumption is not supported.

-## Example usage

-<a id="example"></a>

-> [!NOTE]

-> In the current preview, Azure SQL triggers are only supported by [C# class library functions](functions-dotnet-class-library.md)

-## Attributes

-| **ConnectionStringSetting** | Required. The name of an app setting that contains the connection string for the database which contains the table being monitored for changes. The connection string setting name corresponds to the application setting (in `local.settings.json` for local development) that contains the [connection string](/dotnet/api/microsoft.data.sqlclient.sqlconnection.connectionstring?view=sqlclient-dotnet-core-5.&preserve-view=true#Microsoft_Data_SqlClient_SqlConnection_ConnectionString) to the Azure SQL or SQL Server instance.|

-|**Sql_Trigger_BatchSize** |This controls the number of changes processed at once before being sent to the triggered function. The default value is 100.|

-Setting up change tracking for use with the Azure SQL trigger requires two steps. These steps can be completed from any SQL tool that supports running queries, including [VS Code](/sql/tools/visual-studio-code/mssql-extensions), [Azure Data Studio](/sql/azure-data-studio/download-azure-data-studio) or [SQL Server Management Studio](/sql/ssms/download-sql-server-management-studio-ssms).

-Optionally, your functions can scale automatically based on the amount of changes that are pending to be processed in the user table. To allow your functions to scale properly on the Premium plan when using SQL triggers, you need to enable runtime scale monitoring.

-The extension NuGet package you install depends on the C# mode you're using in your function app:

-## Install bundle

-The SQL bindings extension is part of a preview [extension bundle], which is specified in your host.json project file.

-> Python language support for the SQL bindings extension is available starting with v4.5.0 of the [functions runtime](./set-runtime-version.md#view-and-update-the-current-runtime-version). You may need to update your install of Azure Functions [Core Tools](functions-run-local.md) for local development. Learn more about determining the runtime in Azure regions from the [functions runtime](./set-runtime-version.md#view-and-update-the-current-runtime-version) documentation. Please see the tracking [GitHub issue](https://github.com/Azure/azure-functions-sql-extension/issues/250) for the latest update on availability.

-The SQL bindings extension is part of a preview [extension bundle], which is specified in your host.json project file.

-## Update packages

-Support for the SQL bindings extension is available in the 1.11.3b1 version of the [Azure Functions Python library](https://pypi.org/project/azure-functions/). Add this version of the library to your functions project with an update to the line for `azure-functions==` in the `requirements.txt` file in your Python Azure Functions project as seen in the following snippet:

-```

-azure-functions==1.11.3b1

-```

-Following setting the library version, update your application settings to [isolate the dependencies](./functions-app-settings.md#python_isolate_worker_dependencies) by adding `PYTHON_ISOLATE_WORKER_DEPENDENCIES` with the value `1` to your application settings. Locally, this is set in the `local.settings.json` file as seen below:

-```json

-"PYTHON_ISOLATE_WORKER_DEPENDENCIES": "1"

-```

-Support for Python durable functions with SQL bindings isn't yet available.

-The SQL bindings extension is part of a preview [extension bundle], which is specified in your host.json project file.

-Azure SQL bindings for Azure Functions have a required property for connection string on both [input](./functions-bindings-azure-sql-input.md) and [output](./functions-bindings-azure-sql-output.md) bindings. SQL bindings passes the connection string to the Microsoft.Data.SqlClient library and supports the connection string as defined in the [SqlClient ConnectionString documentation](/dotnet/api/microsoft.data.sqlclient.sqlconnection.connectionstring?view=sqlclient-dotnet-core-5.0&preserve-view=true#Microsoft_Data_SqlClient_SqlConnection_ConnectionString). Notable keywords include:

-You can define a data collection rule to send data from multiple machines to multiple Log Analytics workspaces, including workspaces in a different region or tenant. Create the data collection rule in the *same region* as your VM / VMSS / Arc enabled server.

-The configuration change can take a few minutes to finish before taking effect. You must restart all Azure Monitor Agent pods manually. When the restarts are finished, a message appears that's similar to the following and includes the result `configmap "container-azm-ms-agentconfig" created`.

-* Different IP addresses for data and log volumes.

- * This setup will provide better performance and throughput for the SAP HANA database.

-Since AzAcSnap v5.0 was released as GA in April 2021, there have been 8 releases of AzAcSnap across two branches. Our goal with the new release model is to align with how Azure components are released. This change allows moving features from Preview to GA (without having to move an entire branch), and introduce new Preview features (without having to create a new branch). From AzAcSnap 6 we'll have a single branch with fully supported GA features and Preview features (which are subject to Microsoft's Preview Ts&Cs). ItΓÇÖs important to note customers can't accidentally use Preview features, and must enable them with the `--preview` command line option. This means the next release will be AzAcSnap 7, which could include; patches (if necessary) for GA features, current Preview features moving to GA, or new Preview features.

-

-Download the [latest release](https://aka.ms/azacsnapinstaller) of the installer and review how to [get started](azacsnap-get-started.md). For specific information on Preview features refer to the [AzAcSnap Preview](azacsnap-preview.md) page.

-### AzAcSnap v5.0 Preview (Build:20210318.30771)

-AzAcSnap v5.0 Preview (Build:20210318.30771) has been released with the following fixes and improvements:

- Azure NetApp Files volumes provide flexible, large and scalable storage shares for applications and users. Storage capacity and consumption by users is only limited by the size of the volume. In some scenarios, you may want to limit this storage consumption of users and groups within the volume. With Azure NetApp Files volume and group quotas, you can now do so. User and/or group quotas enable you to restrict the storage space that a user or group can use within a specific Azure NetApp Files volume. You can choose to set default (same for all users) or individual user quotas on all NFS, SMB, and dual protocol-enabled volumes. On all NFS-enabled volumes, you can set default (same for all users) or individual group quotas.

-### managementGrouopResourceID example

-Azure VMware Solution is currently supported in these [regions](https://azure.microsoft.com/global-infrastructure/services/?products=azure-vmware).

-1. Troubleshoot the problem using [Troubleshoot Azure Files problems in Windows Server Message Block (SMB)](../storage/files/storage-troubleshoot-windows-file-connection-problems.md).

-1. Troubleshoot the problem using [Troubleshoot Azure Files problems in Linux (SMB)](../storage/files/storage-troubleshoot-linux-file-connection-problems.md).

-1. Troubleshoot the problem using [Troubleshoot Azure Files problems in Windows (SMB)](../storage/files/storage-troubleshoot-windows-file-connection-problems.md) or [Troubleshoot Azure Files problems in Linux (SMB)](../storage/files/storage-troubleshoot-linux-file-connection-problems.md).

-Azure Content Delivery Network (CDN) offers two ways to control how your files are cached:

- - Global caching rules: You can set one global caching rule for each endpoint in your profile, which affects all requests to the endpoint. The global caching rule overrides any HTTP cache-directive headers, if set.

- - Custom caching rules: You can set one or more custom caching rules for each endpoint in your profile. Custom caching rules match specific paths and file extensions, are processed in order, and override the global caching rule, if set.

-Global and custom caching rules are processed in the following order:

- - Cache expiration duration: 1 day

- - Cache expiration duration: 2 days

- - Cache expiration duration: 3 days

-When these rules are set, a request for _&lt;endpoint hostname&gt;_.azureedge.net/home/https://docsupdatetracker.net/index.html triggers custom caching rule #2, which is set to: **Set if missing** and 3 days. Therefore, if the *https://docsupdatetracker.net/index.html* file has `Cache-Control` or `Expires` HTTP headers, they are honored; otherwise, if these headers are not set, the file is cached for 3 days.

-Similarly, caching is used by a content delivery network on edge servers close to the user to avoid requests traveling back to the origin and reducing end-user latency. Unlike a web browser cache, which is used only for a single user, the CDN has a shared cache. In a CDN shared cache, a file that is requested by one user can be accessed later by other users, which greatly decreases the number of requests to the origin server.

-Dynamic resources that change frequently or are unique to an individual user cannot be cached. Those types of resources, however, can take advantage of dynamic site acceleration (DSA) optimization on the Azure Content Delivery Network for performance improvements.

-Because a cached resource can potentially be out-of-date, or stale (as compared to the corresponding resource on the origin server), it is important for any caching mechanism to control when content is refreshed. To save time and bandwidth consumption, a cached resource is not compared to the version on the origin server every time it is accessed. Instead, as long as a cached resource is considered to be fresh, it is assumed to be the most current version and is sent directly to the client. A cached resource is considered to be fresh when its age is less than the age or period defined by a cache setting. For example, when a browser reloads a web page, it verifies that each cached resource on your hard drive is fresh and loads it. If the resource is not fresh (stale), an up-to-date copy is loaded from the server.

-If a resource is considered to be stale, the origin server is asked to validate it, that is, determine whether the data in the cache still matches whatΓÇÖs on the origin server. If the file has been modified on the origin server, the cache updates its version of the resource. Otherwise, if the resource is fresh, the data is delivered directly from the cache without validating it first.

-Similar to how caching is implemented in a web browser, you can control how caching is performed in a CDN by sending cache-directive headers. Cache-directive headers are HTTP headers, which are typically added by the origin server. Although most of these headers were originally designed to address caching in client browsers, they are now also used by all intermediate caches, such as CDNs.

-Not all resources can be cached. The following table shows what resources can be cached, based on the type of HTTP response. Resources delivered with HTTP responses that don't meet all of these conditions cannot be cached. For **Azure CDN Premium from Verizon** only, you can use the rules engine to customize some of these conditions.

-For **Azure CDN Standard from Microsoft** caching to work on a resource, the origin server must support any HEAD and GET HTTP requests and the content-length values must be the same for any HEAD and GET HTTP responses for the asset. For a HEAD request, the origin server must support the HEAD request, and must respond with the same headers as if it had received a GET request.

-| **CDN cache duration** | 2 days |7 days | None | 7 days | None | 1 day | 1 year |

-## Overview

-Azure CDN edge nodes will cache assets until the asset's time-to-live (TTL) expires. After the asset's TTL expires, when a client requests the asset from the edge node, the edge node will retrieve a new updated copy of the asset to serve the client request and store refresh the cache.

-The best practice to make sure your users always obtain the latest copy of your assets is to version your assets for each update and publish them as new URLs. CDN will immediately retrieve the new assets for the next client requests. Sometimes you may wish to purge cached content from all edge nodes and force them all to retrieve new updated assets. This might be due to updates to your web application, or to quickly update assets that contain incorrect information.

->

->

-This tutorial walks you through purging assets from all edge nodes of an endpoint.

-## Walkthrough

-1. In the [Azure Portal](https://portal.azure.com), browse to the CDN profile containing the endpoint you wish to purge.

-2. From the CDN profile blade, click the purge button.

-

- 

-

- The Purge blade opens.

-

- 

-3. On the Purge blade, select the service address you wish to purge from the URL dropdown.

- 

- > You can also get to the Purge blade by clicking the **Purge** button on the CDN endpoint blade. In that case, the **URL** field will be pre-populated with the service address of that specific endpoint.

- >

-4. Select what assets you wish to purge from the edge nodes. If you wish to clear all assets, click the **Purge all** checkbox. Otherwise, type the path of each asset you wish to purge in the **Path** textbox. Below formats are supported in the path.

- 1. **Single URL purge**: Purge individual asset by specifying the full URL, with or without the file extension, e.g.,`/pictures/strasbourg.png`; `/pictures/strasbourg`

- 2. **Wildcard purge**: Asterisk (\*) may be used as a wildcard. Purge all folders, sub-folders and files under an endpoint with `/*` in the path or purge all sub-folders and files under a specific folder by specifying the folder followed by `/*`, e.g.,`/pictures/*`. Note that wildcard purge is not supported by Azure CDN from Akamai currently.

-5. Click the **Purge** button.

-## See also

- az deployment group create --resource-group exampleRG --template-file main.bicep --parameters profileName=<profile-name> endpointName=<endpoint-name> originURL=<origin-url>

- New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile ./main.bicep -profileName "<profile-name>" -endpointName "<endpoint-name>" -originURL "<origin-url>"

-Pronunciation Assessment supports uninterrupted streaming mode. The demo on the Speech Studio supports up to 60 minutes of recording in streaming mode for evaluation. The Speech Studio demo allows for up to 60 minutes of recording in streaming mode for evaluation. As long as you do not press the stop recording button, the evaluation process does not finish and you can pause and resume evaluation conveniently.

-| ada | Yes | No | N/A | East US, South Central US, West Europe |

-| text-ada-001 | Yes | No | East US, South Central US, West Europe | N/A |

-| babbage | Yes | No | N/A | East US, South Central US, West Europe |

-| text-babbage-001 | Yes | No | East US, South Central US, West Europe | N/A |

-| curie | Yes | No | N/A | East US, South Central US, West Europe |

-| text-curie-001 | Yes | No | East US, South Central US, West Europe | N/A |

-| davinci¹ | Yes | No | N/A | East US, South Central US, West Europe |

-| code-cushman-001² | Yes | No | South Central US, West Europe | East US, South Central US, West Europe |

-| code-davinci-fine-tune-002² | Yes | No | N/A | East US, West Europe |

-² The model is available for fine-tuning by request only. Currently we aren't accepting new requests to fine-tune the model.

-| Fine-tuning | Ada <br> Babbage <br> Curie <br> Cushman* <br> Davinci* <br> \* Currently unavailable.|

-| Virtual network support | Yes |

-1. Migrate data by using the Azure Cosmos DB Live Data Migrator tool.

- To migrate data with near zero downtime, see [Azure Cosmos DB Live Data Migrator tool](https://github.com/Azure-Samples/azure-cosmosdb-live-data-migrator).

- With the Live Data Migrator tool still running, update the connection information in the new deployment of your application. You can retrieve the endpoints and keys for your application from the Azure portal.

- With requests now fully redirected to the new instance, you can delete the old Azure Cosmos DB account and the Live Data Migrator tool.

-* Review migration guidance for additional scenarios in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-This article applies to customers with a Microsoft Customer Agreement (MCA) and to customers who signed up for Azure through the Azure website, Azure.com (for a Microsoft Online Services Program account also called pay-as-you-go account).

-|Agreement type| Credit card | Wire transfer┬╣ | Check┬▓ |

-| | - | | |

-| Microsoft Customer Agreement<br>purchased through a Microsoft representative | Γ£ö (with a $50,000.00 USD limit) | Γ£ö | Γ£ö |

-| Enterprise Agreement | Γ£ÿ | Γ£ö | Γ£ö |

-| Azure.com | Γ£ö | Γ£ö if approved to pay by invoice | Γ£ÿ |

-┬▓ As noted previously, on April 1, 2023, Microsoft will stop accepting checks as a payment method for subscriptions that are paid by invoice.

-* Review the migration guidance in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-* Review the migration guidance in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-For information about additional migrating scenarios (source/target pairs), see the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-For information about additional migrating scenarios (source/target pairs), see the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-* Review the migration guidance in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-* Review the migration guidance for your scenario in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-* Review the migration guidance for your scenario in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

-* Review the migration guidance for your scenario in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

- * If all you know is the number of vcores and servers in your existing database cluster, read about [estimating request units using vCores or vCPUs](../cosmos-db/convert-vcore-to-request-unit.md)

-* Review migration guidance for additional scenarios in the Microsoft [Database Migration Guide](https://datamigration.microsoft.com/).

- * If all you know is the number of vcores and servers in your existing database cluster, read about [estimating request units using vCores or vCPUs](../cosmos-db/convert-vcore-to-request-unit.md)

-Review migration guidance for additional scenarios in the [Azure Database Migration Guide](https://datamigration.microsoft.com/).

-> Microsoft Graph API's ability to send events to Azure Event Grid is currently in **private preview**.

-> [!IMPORTANT]

-> In the following steps, you will follow instructions from [Node.js](https://github.com/microsoftgraph/nodejs-webhooks-sample), [Java](https://github.com/microsoftgraph/java-spring-webhooks-sample), and[.NET Core](https://github.com/microsoftgraph/aspnetcore-webhooks-sample) Webhook samples to enable flow of events from Microsoft Graph API. At some point in the sample, you will have an application registered with Azure AD. Email your application ID to <a href="mailto:ask-graph-and-grid@microsoft.com?subject=Please allow my application ID">mailto:ask-graph-and-grid@service.microsoft.com?subject=Please allow my Azure AD application with ID to send events through Graph API</a> so that the Microsoft Graph API team can add your application ID to allow list to use this new capability.

-Here are some of the key payload properties:

-> [!div class="checklist"]

-> - Ingest

-> - Normalize - Device mappings applied.

-> - Group - (Optional)

-> - Transform - FHIR destination mappings applied.

-> - Persist

-> [!div class="checklist"]

-> - Device identity

-> - Measurement type

-> - Time period

-`Device identity` and `measurement type` grouping is optional and enabled by the use of the [SampledData](https://www.hl7.org/fhir/datatypes.html#SampledData) measurement type. The SampledData measurement type provides a concise way to represent a time-based series of measurements from a device message into FHIR Observation resources. When you use the SampledData measurement type, measurements can be grouped into a single FHIR Observation resource that represents a 1-hour period or a 24-hour period.

-If no Device resource for a given device identifier exists in the FHIR service, the outcome depends upon the value of [Resolution Type](deploy-new-config.md#configure-the-destination-tab) set at the time of the MedTech service deployment. When set to `Lookup`, the specific message is ignored, and the pipeline continues to process other incoming device messages. If set to `Create`, the MedTech service creates minimal Device and Patient resources in the FHIR service.

-> The `Resolution Type` can also be adjusted post deployment of the MedTech service if a different `Resolution Type` is later required.

-description: Internet peering - FAQs

-# Internet peering - FAQs

-You may review information below for general questions.

-**What is the difference between Internet peering and Peering Service?**

-Peering Service is a service that intends to provide enterprise grade public IP connectivity to Microsoft for its enterprise customers. Enterprise grade Internet includes connectivity through ISPs that have high throughput connectivity to Microsoft and redundancy for a HA connectivity. Additionally, user traffic is optimized for latency to the nearest Microsoft Edge. Peering Service builds on peering connectivity with partner carrier. The peering connectivity with partner must be Direct peering as opposed to Exchange peering. Direct peering must have local and geo-redundancy.

-**What is legacy peering?**

-Peering connection set up using Azure PowerShell is managed as an Azure resource. Peering connections set up in the past are stored in our system as legacy peering which you may choose to convert to manage as an Azure resource.

-**When New-AzPeeringDirectConnectionObject is called, what IP addresses are given to Microsoft and Peer devices?**

-When calling New-AzPeeringDirectConnectionObject cmdlet, a `/31` address (`a.b.c.d/31`) or a `/30` address (`a.b.c.d/30`) is entered. The first IP address (`a.b.c.d+0`) is given to Peer's device and second IP address (`a.b.c.d+1`) is given to Microsoft device.

-**What is MaxPrefixesAdvertisedIPv4 and MaxPrefixesAdvertisedIPv6 parameters in New-AzPeeringDirectConnectionObject cmdlet?**

-MaxPrefixesAdvertisedIPv4 and MaxPrefixesAdvertisedIPv6 parameters represent the maximum number of IPv4 and IPv6 prefixes a Peer wants Microsoft to accept. These parameters can be modified anytime.

-description: Microsoft peering policy.

-Microsoft maintains a selective peering policy designed to ensure the best possible customer experience backed by industry standards and best practices, scaling for future demand and strategic placement of peering. As such, Microsoft reserves the right to make exceptions to the policy as deemed necessary. Microsoft's general requirements from your network are explained in the sections below. These are applicable to both Direct peering and Exchange peering requests.

-* Peer will have a publicly routable Autonomous System Number (ASN).

-* MD5 is not supported.

- * All parties peering with Microsoft agree not to accept routes from AS12076 (Express Route) under any circumstances and should filter out AS12076 on all peers.

- * Peer will have at least one publicly routable /24.

- * Microsoft will overwrite received Multi-Exit Discriminators (MED).

- * Microsoft prefers to receive BGP community-tags from peers to indicate route origination.

- * Peer are required to register their routes in a public Internet Routing Registry (IRR) database, for the purpose of filtering, and will keep this information up to date.

- * Peers will adhere to MANRS industry standards for route security. At its sole discretion, Microsoft may choose: 1.) not to establish peering with companies that do not have routes signed and registered; 2.) to remove invalid RPKI routes; 3.) not to accept routes from established peers that are not registered and signed.

-* A fully staffed 24x7 Network Operations Center (NOC), capable of assisting in the resolution of all technical and performance issues, security violations, denial of service attacks, or any other abuse originating within the peer or their customers.

-* Peers are expected to have a complete and up-to-date profile on [PeeringDB](https://www.peeringdb.com) including a 24x7 NOC email from corporate domain and phone number. We use this information to validate the peer's details such as NOC information, technical contact information, and their presence at the peering facilities etc. Personal Yahoo, Gmail, and hotmail accounts are not accepted.

- * Peers are expected to have at minimum a 10 Gb connection to the exchange.

- * Interconnection must be over single-mode fiber using 100 Gbps optics.

- * Microsoft will only establish Direct peering with ISP or Network Service providers.

- * Each Direct peering consists of two connections to two Microsoft edge routers from the Peer's routers located in Peer's edge. Microsoft requires dual BGP sessions across these connections. The peer may choose not to deploy redundant devices at their end.

-* Peers over Exchange peering must have at minimum 500 Mb of traffic and less than 2 Gb. For traffic exceeding 2 Gb Direct peering should be established.

-* Microsoft requires at minimum 2 Gb for direct peering. Each mutually agreed to peering location must support failover that ensures peering remains localized during a failover scenario.

-* To learn about steps to set up Direct peering with Microsoft, follow [Direct peering walkthrough](walkthrough-direct-all.md).

-* To learn about steps to set up Exchange peering with Microsoft, follow [Exchange peering walkthrough](walkthrough-exchange-all.md).

-description: Direct peering walkthrough.

-This section explains the steps you need to follow to set up and manage a Direct peering.

-> [!div class="mx-imgBorder"]

-> 

-The following steps must be followed in order to provision a Direct peering:

-1. Review Microsoft [peering policy](https://peering.azurewebsites.net/peering) to understand requirements for Direct peering.

-1. Once peering request is approved, connection state changes to ProvisioningStarted.

- 1. configure BGP session and then notify us.

-1. If successful, you will receive a notification that peering connection state is Active.

-Note that connection states are not to be confused with standard [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol) session states.

-The following steps must be followed in order to convert a legacy Direct peering to Azure resource:

-1. Follow the instructions in [Convert a legacy Direct peering to Azure resource](howto-legacy-direct-powershell.md)

-1. After you submit the conversion request, Microsoft will review the request and contact you if required.

-1. Once approved, you will see your Direct peering with a connection state as Active.

-Contact [Microsoft peering](mailto:peering@microsoft.com) team to deprovision Direct peering.

-When a Direct peering is set for deprovision, you will see the connection state as **PendingRemove**

-> If you run PowerShell cmdlet to delete the Direct peering when the ConnectionState is ProvisioningStarted or ProvisioningCompleted the operation will fail.

-* Learn about [Prerequisites to set up peering with Microsoft](prerequisites.md).

-description: Exchange peering walkthrough.

-This section explains the steps you need to follow to set up and manage an Exchange peering.

-> [!div class="mx-imgBorder"]

-> 

-1. Review Microsoft [peering policy](https://peering.azurewebsites.net/peering) to understand requirements for Exchange peering.

-1. Find Microsoft peering location and peering facility id in [PeeringDB](https://www.peeringdb.com/net/694)

-1. Request Exchange peering for a peering location using the instructions in [Create and modify an Exchange peering using PowerShell](howto-exchange-powershell.md) article for more details.

-1. After you submit a peering request, Microsoft will review the request and contact you if required.

-1. Once approved, connection state changes to Approved

-1. Configure BGP session at your end and notify Microsoft

-1. We will provision BGP session with DENY ALL policy and validate end-to-end.

-1. If successful, you will receive a notification that peering connection state is Active.

-Note that connection states are not to be confused with standard [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol) session states.

-1. Follow the instructions in [Convert a legacy Exchange peering to Azure resource](howto-legacy-exchange-powershell.md)

-1. After you submit the conversion request, Microsoft will review the request and contact you if required.

-1. Once approved, you will see your Exchange peering with connection state as Active.

-When an Exchange peering is set for deprovision, you will see the connection state as **PendingRemove**

-> If you run PowerShell cmdlet to delete the Exchange peering when the connection state is ProvisioningStarted or ProvisioningCompleted the operation will fail.

-* Learn about [Prerequisites to set up peering with Microsoft](prerequisites.md).

-description: Peering Service partner walkthrough.

-This section explains the steps a provider needs to follow to enable a Direct peering for Peering Service.

-Service Providers can expand their geographical reach by creating new Direct peering that support Peering Service. To accomplish this,

-1. Become a Peering Service partner if not already

-1. Follow the instructions to [Create or modify a Direct peering using the portal](howto-direct-portal.md). Ensure it meets high-availability requirement.

-1. Then, follow steps to [Enable Peering Service on a Direct peering using the portal](howto-peering-service-portal.md).

-If you have legacy Direct peering that you want to use to support Peering Service,

-1. Become a Peering Service partner if not already.

-1. Follow the instructions to [Convert a legacy Direct peering to Azure resource using the portal](howto-legacy-direct-portal.md). If required, order additional circuits to meet high-availability requirement.

-1. Then, follow steps to [Enable Peering Service on a Direct peering using the portal](howto-peering-service-portal.md).

-* Learn about [peering policy](https://peering.azurewebsites.net/peering).

-* To learn about steps to set up Direct peering with Microsoft, follow [Direct peering walkthrough](walkthrough-direct-all.md).

-* To learn about steps to set up Exchange peering with Microsoft, follow [Exchange peering walkthrough](walkthrough-exchange-all.md).

-To verify that the enrollment group has been created:

-1. Add the environment variable name `NVIDIA_VISIBLE_DEVICES` with the value `0`. The value represents a list of your modules on your device, with `0` being the beginning of the list. This value is how many devices you want assigned to a GPU. Since we only have one module here, we want the first one on our list to be GPU-enabled.

- sudo systemctl restart adu-agent

-| Shared access policies | Maximum number of shared access policies is 16. |

-In this article, learn about Azure Machine Learning compute instance image releases. Azure Machine Learning maintains host operating system images for [Azure ML compute instance](./concept-compute-instance.md) and [Data Science Virtual Machines](./data-science-virtual-machine/release-notes.md). Due to the rapidly evolving needs and package updates, we target to release new images every month.

-* **Kubernetes cluster** - used to deploy trained machine learning models to Azure Kubernetes Service. You can create an Azure Kubernetes Service (AKS) cluster from your Azure ML workspace, or attach an existing AKS cluster.

-Azure ML supports two types of environments: curated and custom.

-In custom environments, you're responsible for setting up your environment. Make sure to install the packages and any other dependencies that your training or scoring script needs on the compute. Azure ML allows you to create your own environment using

-Azure machine learning models consist of the binary file(s) that represent a machine learning model and any corresponding metadata. Models can be created from a local or remote file or directory. For remote locations `https`, `wasbs` and `azureml` locations are supported. The created model will be tracked in the workspace under the specified name and version. Azure ML supports three types of storage format for models:

- - For all job types, changed the schema for defining data inputs to the job in the job YAML. Instead of specifying the data path using either the `file` or `folder` fields, use the `path` field to specify either a local path, a URI to a cloud path containing the data, or a reference to an existing registered Azure ML data asset via `path: azureml:<data_name>:<data_version>`. Also specify the `type` field to clarify whether the data source is a single file (`uri_file`) or a folder (`uri_folder`). If `type` field is omitted, it defaults to `type: uri_folder`. For more information, see the section of any of the [job YAML references](reference-yaml-job-command.md) that discuss the schema for specifying input data.

- - Changed the expression syntax from `{ <expression> }` to `${{ <expression> }}`. For more information, see [Expression syntax for configuring Azure ML jobs](reference-yaml-core-syntax.md#expression-syntax-for-configuring-azure-ml-jobs-and-components)

- - Added the `az ml component` commands for managing Azure ML components

-You can navigate to Custom components in AzureML Studio as shown in the following image.

-When you convert a dataset to CSV, the csv is saved in your Azure ML workspace. You can use an Azure storage utility to open and use the file directly. You can also access the CSV in the designer by selecting the **Convert to CSV** component, then select the histogram icon under the **Outputs** tab in the right panel to view the output. You can download the CSV from the Results folder to a local directory.

-+ **Register dataset**: Select the icon and save the CSV file back to the Azure ML workspace as a separate dataset. You can find the dataset as a component in the component tree under the **My Datasets** section.

- # The script MUST define a class named AzureMLModel.

- Verify that the query works correctly outside of Azure ML by logging in to the database server directly and running the query.

-|Datastore information is invalid. Failed to get AzureML datastore '{datastore_name}' in workspace '{workspace_name}'.|

-1. **Choose whether you want a code-first experience or a no-code studio web experience**: Users who prefer a code-first experience can use the [AzureML SDKv2](how-to-configure-auto-train.md) or the [AzureML CLIv2](how-to-train-cli.md). Get started with [Tutorial: Train an object detection model with AutoML and Python](tutorial-auto-train-image-models.md). Users who prefer a limited/no-code experience can use the [web interface](how-to-use-automated-ml-for-ml-models.md) in Azure Machine Learning studio at [https://ml.azure.com](https://ml.azure.com/). Get started with [Tutorial: Create a classification model with automated ML in Azure Machine Learning](tutorial-first-experiment-automated-ml.md).

-1. **Specify the source of the labeled training data**: You can bring your data to AzureML in [many different ways](concept-data.md).

-Similar to classification, regression tasks are also a common supervised learning task. AzureML offers featurization specific to regression problems. Learn more about [featurization options](how-to-configure-auto-train.md#data-featurization). You can also find the list of algorithms supported by AutoML [here](how-to-configure-auto-train.md#supported-algorithms).

-Authoring AutoML models for vision tasks is supported via the Azure ML Python SDK. The resulting experimentation jobs, models, and outputs can be accessed from the Azure Machine Learning studio UI.

-You can configure featurization from the AutoML SDK via the [ForecastingJob](/python/api/azure-ai-ml/azure.ai.ml.automl.forecastingjob#azure-ai-ml-automl-forecastingjob-set-forecast-settings) class or from the [AzureML Studio web interface](how-to-use-automated-ml-for-ml-models.md#customize-featurization).

-* **Assets**: created using Azure ML commands or as part of a training/scoring run. Assets are versioned and can be registered in the Azure ML workspace. They include:

-This [Jupyter notebook](https://github.com/Azure/azureml-examples/blob/main/sdk/python/resources/workspace/workspace.ipynb) shows more ways to create an Azure ML workspace using SDK v2.

-* **Inference cluster** - used to deploy trained machine learning models to Azure Kubernetes Service. You can create an Azure Kubernetes Service (AKS) cluster from your Azure ML workspace, or attach an existing AKS cluster.

-Azure machine learning models consist of the binary file(s) that represent a machine learning model and any corresponding metadata. Models can be created from a local or remote file or directory. For remote locations `https`, `wasbs` and `azureml` locations are supported. The created model will be tracked in the workspace under the specified name and version. Azure ML supports three types of storage format for models:

-Azure ML supports two types of environments: curated and custom.

-In custom environments, you're responsible for setting up your environment and installing packages or any other dependencies that your training or scoring script needs on the compute. Azure ML allows you to create your own environment using

-### Create an Azure ML custom environment

-Now, it's time to develop the code of executing a step. You can use your preferred languages (python, R, etc.). The code must be able to be executed by a shell command. During the development, you may want to add a few inputs to control how this step is going to be executed. For example, for a training step, you may like to add learning rate, number of epochs as the inputs to control the training. These additional inputs plus the inputs and outputs required to connect with other steps are the interface of the component. The argument of a shell command is used to pass inputs and outputs to the code. The environment to execute the command and the code needs to be specified. The environment could be a curated AzureML environment, a docker image or a conda environment.

-You may opt out from diagnostic data being collected by setting the `hbi_workspace` parameter to `TRUE` while provisioning the workspace. This functionality is supported when using the AzureML Python SDK, the Azure CLI, REST APIs, or Azure Resource Manager templates.

-|Azure ML [Datastore](#datastore) | `azureml://datastores/<data_store_name>/paths/<folder1>/<folder2>/<folder3>/<file>.parquet` |

-An Azure ML job maps URIs to the compute target filesystem. This mapping means that in a command that consumes or produces a URI, that URI works like a file or a folder. A URI uses **identity-based authentication** to connect to storage services, with either your Azure Active Directory ID (default), or Managed Identity. Azure ML [Datastore](#datastore) URIs can apply either identity-based authentication, or **credential-based** (for example, Service Principal, SAS token, account key) without exposure of secrets.

-A URI can serve as either *input* or an *output* to an Azure ML job, and it can map to the compute target filesystem with one of four different *mode* options:

-|**Table**<br> Reference a data table | `mltable` | `TabularDataset` | You have a complex schema subject to frequent changes, or you need a subset of large tabular data.<br><br>AutoML with Tables. | In V1 APIs, the Azure ML back-end stored the data materialization blueprint. As a result, `TabularDataset` only worked if you had an Azure ML workspace. `mltable` stores the data materialization blueprint in *your* storage. This storage location means you can use it *disconnected to Azure ML* - for example, local, on-premises. In V2 APIs, you'll find it easier to transition from local to remote jobs. Read [Working with tables in Azure Machine Learning](how-to-mltable.md) for more information. |

-Azure ML uses its own *data runtime* for mounts/uploads/downloads, to map storage URIs to the compute target filesystem, or to materialize tabular data into pandas/spark with Azure ML tables (`mltable`). The Azure ML data runtime is designed for machine learning task *high speed and high efficiency*. Its key benefits include:

-> - Light weight; the Azure ML data runtime has *no* dependencies on other technologies - JVM, for example - so the runtime installs quickly on compute targets.

-An Azure ML datastore serves as a *reference* to an *existing* Azure storage account. The benefits of Azure ML datastore creation and use include:

-1. For credential-based access (service principal/SAS/key), Azure ML datastore secures connection information. This way, you won't need to place that information in your scripts.

-An Azure ML data asset resembles web browser bookmarks (favorites). Instead of remembering long storage paths (URIs) that point to your most frequently used data, you can create a data asset, and then access that asset with a friendly name.

-Data asset creation also creates a *reference* to the data source location, along with a copy of its metadata. Because the data remains in its existing location, you incur no extra storage cost, and you don't risk data source integrity. You can create Data assets from Azure ML datastores, Azure Storage, public URLs, or local files.

-Deploy locally to test your endpoints without deploying to the cloud. Azure Machine Learning creates a local Docker image that mimics the Azure ML image. Azure Machine Learning will build and run deployments for you locally, and cache the image for rapid iterations.

-You can create environments from clients like the AzureML Python SDK, Azure Machine Learning CLI, Environments page in Azure Machine Learning studio, and [VS Code extension](how-to-manage-resources-vscode.md#create-environment). Every client allows you to customize the base image, Dockerfile, and Python layer if needed.

-To determine whether to reuse a cached image or build a new one, AzureML computes a [hash value](https://en.wikipedia.org/wiki/Hash_table) from the environment definition and compares it to the hashes of existing environments. The hash is based on the environment definition's:

-Microsoft is responsible for patching the base images for known security vulnerabilities. Updates for supported images are released every two weeks, with a commitment of no unpatched vulnerabilities older than 30 days in the the latest version of the image. Patched images are released with a new immutable tag and the `:latest` tag is updated to the latest version of the patched image.

-* [Deploy a TensorFlow model using a custom container](how-to-deploy-custom-container.md)

-In such scenarios, you may be using different AzureML workspaces for development, testing and production. This configuration presents the following challenges for model training and deployment:

-Azure Machine Learning offers different methods to build a pipeline. For users who are familiar with DevOps practices, we recommend using [CLI](how-to-create-component-pipelines-cli.md). For data scientists who are familiar with python, we recommend writing pipeline using the [Azure ML SDK v1](v1/how-to-create-machine-learning-pipelines.md). For users who prefer to use UI, they could use the [designer to build pipeline by using registered components](how-to-create-component-pipelines-ui.md).

-+ [Define pipelines with the Azure ML CLI v2](./how-to-create-component-pipelines-cli.md)

-+ [Define pipelines with the Azure ML SDK v2](./how-to-create-component-pipeline-python.md)

-> * When deployed, Model's deployments have a Swagger generated automatically and the __Test__ feature can be used in Azure ML studio.

-> Azure Machine Learning generates Swagger for model's deployment in MLflow format with a signature available. This makes easier to test deployed endpoints using the Azure ML studio.

-> MLflow environments and Azure Machine Learning environments are different concepts. While the former opperates at the level of the model, the latter operates at the level of the workspace (for registered environments) or jobs/deployments (for annonymous environments). When you deploy MLflow models in Azure Machine Learning, the model's environment is built and used for deployment. Alternatively, you can override this behaviour with the [Azure ML CLI v2](concept-v2.md) and deploy MLflow models using a specific Azure Machine Learning environments.

-> - MLflow in R support is limited to tracking experiment's metrics, parameters and models on Azure Machine Learning jobs. Interactive training on RStudio, Posit (formerly RStudio Workbench) or Jupyter Notebooks with R kernels is not supported. Model management and registration is not supported using the MLflow R SDK. As an alternative, use Azure ML CLI or [Azure ML studio](https://ml.azure.com) for model registration and management. View the following [R example about using the MLflow tracking client with Azure Machine Learning](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/single-step/r).

-* [Deploying models trained in Azure Databricks to Azure Machine Learning with MLflow](https://github.com/Azure/azureml-examples/blob/main/sdk/python/using-mlflow/deploy/track_with_databricks_deploy_aml.ipynb): Demonstrates how to train models in Azure Databricks and deploy them in Azure ML. It also includes how to handle cases where you also want to track the experiments with the MLflow instance in Azure Databricks.

-Development with Azure Machine Learning often involves web-based development environments (Notebooks & Azure ML studio). When using web-based development environments, the potential threats are:

- * __Access token/cookies__: XSS attacks can also access local storage and browser cookies. Your Azure Active Directory (AAD) authentication token is stored in local storage. An XSS attack could use this token to make API calls on your behalf, and then send the data to an external system or API.

-## Azure ML studio notebooks

-## Azure ML compute instance

-Azure Machine Learning compute instance hosts __Jupyter__ and __Jupyter Lab__. When using either, cells in a notebook or code in can output HTML documents or fragments that contain malicious code. When the output is rendered, the code can be executed. The same threats also apply when using __RStudio__ and __Posit Workbench (formerly RStudio Workbench)__ hosted on a compute instance.

-| [Use Docker images managed by Azure Machine Learning](#scenario-use-docker-images-managed-by-azure-ml) | NA | <ul><li>Microsoft Container Registry</li><li>`viennaglobal.azurecr.io` global container registry</li></ul> | If the Azure Container Registry for your workspace is behind the VNet, configure the workspace to use a compute cluster to build images. For more information, see [How to secure a workspace in a virtual network](how-to-secure-workspace-vnet.md#enable-azure-container-registry-acr). |

-## Scenario: Use Docker images managed by Azure ML

-Now that you've learned how network traffic flows in a secured configuration, learn more about securing Azure ML in a virtual network by reading the [Virtual network isolation and privacy overview](how-to-network-security-overview.md) article.

-The Azure Machine Learning CLI v2 (CLI v2) is the latest extension for the [Azure CLI](/cli/azure/what-is-azure-cli). The CLI v2 provides commands in the format *az ml __\<noun\> \<verb\> \<options\>__* to create and maintain Azure ML assets and workflows. The assets or workflows themselves are defined using a YAML file. The YAML file defines the configuration of the asset or workflow ΓÇô what is it, where should it run, and so on.

-* On board to Azure ML without the need to learn a specific programming language

- The YAML file defines the configuration of the asset or workflow ΓÇô what is it, where should it run, and so on. Any custom logic/IP used, say data preparation, model training, model scoring can remain in script files, which are referred to in the YAML, but not part of the YAML itself. Azure ML supports script files in python, R, Java, Julia or C#. All you need to learn is YAML format and command lines to use Azure ML. You can stick with script files of your choice.

- Azure ML offers [endpoints](concept-endpoints.md) to streamline model deployments for both real-time and batch inference deployments. This functionality is available only via CLI v2 and SDK v2.

- Azure ML introduces [components](concept-component.md) for managing and reusing common logic across pipelines. This functionality is available only via CLI v2 and SDK v2.

-Azure ML Python SDK v2 is an updated Python SDK package, which allows users to:

-* Stitch together multiple tasks and production workflows using Azure ML pipelines

- Azure ML introduces [components](concept-component.md) for managing and reusing common logic across pipelines. This functionality is available only via CLI v2 and SDK v2.

- Azure ML offers [endpoints](concept-endpoints.md) to streamline model deployments for both real-time and batch inference deployments. This functionality is available only via CLI v2 and SDK v2.

-* You were a user of R SDK previously - Azure ML won't support an SDK in `R`. However, the CLI v2 has support for `R` scripts.

- * [Train models with the Azure ML Python SDK v2](how-to-train-model.md)

-Azure Machine Learning manages host OS VM images for Azure ML compute instance, Azure ML compute clusters, and Data Science Virtual Machines. The update frequency is monthly and includes the following:

-* The frequency is on a monthly interval for most images. For compute instance, the image release is aligned with the Azure ML SDK release cadence as it comes preinstalled in the environment.

-Next to the regular release cadence, hot fixes are applied in the case vulnerabilities are discovered. Hot fixes get rolled out within 72 hours for Azure ML compute and within a week for Compute Instance.

-By default, dependencies are layered on top of base images provided by Azure ML when building environments. You can also use your own base images when using environments in Azure Machine Learning. Once you install more dependencies on top of the Microsoft-provided images, or bring your own base images, vulnerability management becomes your responsibility.

-* Azure Machine Learning frequently publishes new versions of AzureML extension container images into Microsoft Container Registry. It's MicrosoftΓÇÖs responsibility to ensure new image versions are free from vulnerabilities. Vulnerabilities are fixed with [each release](https://github.com/Azure/AML-Kubernetes/blob/master/docs/release-notes.md).

-* Automated ML jobs run on environments that layer on top of Azure ML [base docker images](https://github.com/Azure/AzureML-Containers).

- * Designer jobs are compartmentalized into [Components](concept-designer.md#component). Each component has its own environment that layers on top of the Azure ML base docker images. For more information on components, see the [Component reference](./component-reference/component-reference.md).

-* [AzureML Python SDK Release Notes](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ml/azure-ai-ml/CHANGELOG.md)

-These sub resources are the main resources that are made in the AzureML workspace.

-* VMs: provide computing power for your AzureML workspace and are an integral part in deploying and training models.

-Add the following code to your notebook (or script) to set the AzureML workspace object.

-On the left-hand menu in [AzureML Studio](https://ml.azure.com) click on __Compute__ and then the __Inference clusters__ tab. Next, click on __+ New__ as discussed below:

-You can test the endpoint using [Postman](https://www.postman.com/), or you can use the AzureML SDK:

-* Learn more about [deploying models in AzureML](../v1/how-to-deploy-and-where.md)

-The DSVM is a customized VM image for Data Science but [Azure Machine Learning](../overview-what-is-azure-machine-learning.md) (AzureML) is an end-to-end platform that encompasses:

-### Comparison with AzureML Compute Instances

- * **R**: The Azure-related libraries in R are AzureML and RODBC.

- - `azureml_py38`: environment based on Python 3.8 with preinstalled [AzureML SDK](/python/api/overview/azure/ml/?view=azure-ml-py&preserve-view=true) containing also [AutoML](../concept-automated-ml.md) environment

- - `azureml_py38`: environment based on Python 3.8 with preinstalled [AzureML SDK](/python/api/overview/azure/ml/?view=azure-ml-py&preserve-view=true) containing also [AutoML](../concept-automated-ml.md) environment

-| Integration with [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/) (Python) | <span class='green-check'>&#9989;</span></br> (Python SDK, samples) | <span class='green-check'>&#9989;</span></br> (Python SDK,CLI, samples) | <span class='green-check'>&#9989;</span></br> (Python SDK,CLI, samples) | [Azure ML SDK](./dsvm-tools-data-science.md#azure-machine-learning-sdk-for-python) |

-| `AzureMonitor` | 443 | Used to log monitoring and metrics to Azure Monitor. |

-| `marketplace.visualstudio.com`<br>`vscode.blob.core.windows.net`<br>`*.gallerycdn.vsassets.io` | Required to download and install VS Code extensions. These hosts enable the remote connection to compute instances using the Azure ML extension for VS Code. For more information, see [Connect to an Azure Machine Learning compute instance in Visual Studio Code](./how-to-set-up-vs-code-remote.md) |

-To support logging of metrics and other monitoring information to Azure Monitor and Application Insights, allow outbound traffic to the following hosts:

-> Local data folders/files can be used when executing batch endpoints from the Azure ML CLI or Azure ML SDK for Python. However, that operation will result in the local data to be uploaded to the default Azure Machine Learning Data Store of the workspace you are working on.

- Use the Azure ML CLI, Azure ML SDK for Python, or Studio to get the location (region), workspace, and data asset name and version. You will need them later.

- Use the Azure ML CLI, Azure ML SDK for Python, or Studio to get the data store information.

-> To assign an identity to the compute used by a batch deployment, follow the instructions at [Set up authentication between Azure ML and other services](how-to-identity-based-service-authentication.md#compute-cluster). Configure the identity on the compute cluster associated with the deployment. Notice that all the jobs running on such compute are affected by this change. However, different deployments (even under the same deployment) can be configured to run under different clusters so you can administer the permissions accordingly depending on your requirements.

-> * Access data from a Azure ML Datastores URI as if it were a file system.

-> * Materialize Azure ML data assets into Pandas using `mltable` Python library.

-An Azure ML datastore is a *reference* to an *existing* storage account on Azure. The benefits of creating and using a datastore include:

-# AzureML workspace details:

-The Azure ML Datastore implementation of `fsspec` automatically handles credential/identity passthrough used by the Azure ML datastore. This means you don't need to expose account keys in your scripts or do additional sign-in procedures on a compute instance.

-You can also instantiate an Azure ML filesystem and do filesystem-like commands like `ls`, `glob`, `exists`, `open`, etc. The `open()` method will return a file-like object, which can be passed to any other library that expects to work with python files, or used by your own code as you would a normal python file object. These file-like objects respect the use of `with` contexts, for example:

-The Pandas `read_csv()` method doesn't support reading a folder of CSV files. You need to glob csv paths and concatenate them to a data frame using Pandas `concat()` method. The code below demonstrates how to achieve this concatenation with the Azure ML filesystem:

-|A long-form Azure ML datastore | `azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<wsname>/datastores/<name>/paths/<path>` |

-> `mltable` does user credential passthrough for paths on Azure Storage and Azure ML datastores. If you do not have permission to the data on the underlying storage then you will not be able to access the data.

-##### [Azure ML Datastore](#tab/datastore)

-##### [Azure ML Datastore](#tab/datastore)

-In this section, you'll learn how to access your Azure ML data assets into pandas.

-If you've previously created a Table asset in Azure ML (an `mltable`, or a V1 `TabularDataset`), you can load that into pandas using:

-> For large datasets we recommend that you use AzureML managed Spark, which provides the [PySpark Pandas API](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/https://docsupdatetracker.net/index.html).

-You may want to download the data to the local SSD of your host (local machine, cloud VM, Azure ML Compute Instance) and use the local filesystem. You can do this with the `azcopy` utility, which is pre-installed on an Azure ML compute instance. If you are **not** using an Azure ML compute instance or a Data Science Virtual Machine (DSVM), you may need to install `azcopy`. For more information please read [azcopy](../storage/common/storage-ref-azcopy.md).

-* Install and configure the Azure ML Python SDK (v2). For more information, see [Install and set up SDK (v2)](https://aka.ms/sdk-v2-install).

-* Install and configure the Azure ML Python SDK (v2). For more information, see [Install and set up SDK (v2)](https://aka.ms/sdk-v2-install).

- - If authentication is made using a user identity, then it's important to know *which* user is trying to access storage. For more information on authenticating a _user_, see [authentication for Azure Machine Learning](how-to-setup-authentication.md). For more information on service-level authentication, see [authentication between AzureML and other services](how-to-identity-based-service-authentication.md).

-> If you need to access data from outside Azure Machine Learning, such as using Azure Storage Explorer, *user* identity is probably what is used. Consult the documentation for the tool or service you are using for specific information. For more information on how Azure Machine Learning works with data, see [Setup authentication between AzureML and other services](how-to-identity-based-service-authentication.md).

-> While this article focuses on Azure Machine Learning, individual services that Azure ML relies on provide their own RBAC settings. For example, using the information in this article, you can configure who can submit scoring requests to a model deployed as a web service on Azure Kubernetes Service. But Azure Kubernetes Service provides its own set of Azure roles. For service specific RBAC information that may be useful with Azure Machine Learning, see the following links:

-You can combine the roles to grant different levels of access. For example, you can grant a workspace user both **AzureML Data Scientist** and **Azure ML Compute Operator** roles to permit the user to perform experiments while creating computes in a self-service manner.

-### User-assigned managed identity with Azure ML compute cluster

-Allows a data scientist to perform all MLflow AzureML supported operations **except**:

-description: Learn how Azure Machine Learning Kubernetes compute enable AzureML across different infrastructures in cloud and on-premises

-# Introduction to Kubernetes compute target in AzureML

-With AzureML CLI/Python SDK v2, AzureML introduced a new compute target - Kubernetes compute target. You can easily enable an existing **Azure Kubernetes Service** (AKS) cluster or **Azure Arc-enabled Kubernetes** (Arc Kubernetes) cluster to become a Kubernetes compute target in AzureML, and use it to train or deploy models.

-AzureML Kubernetes compute supports two kinds of Kubernetes cluster:

-With a simple cluster extension deployment on AKS or Arc Kubernetes cluster, Kubernetes cluster is seamlessly supported in AzureML to run training or inference workload. It's easy to enable and use an existing Kubernetes cluster for AzureML workload with the following simple steps:

-1. [Deploy the AzureML extension](how-to-deploy-kubernetes-extension.md).

-1. [Attach Kubernetes cluster to your Azure ML workspace](how-to-attach-kubernetes-to-workspace.md).

-**IT-operation team**. The IT-operation team is responsible for the first 3 steps above: prepare an AKS or Arc Kubernetes cluster, deploy Azure ML cluster extension, and attach Kubernetes cluster to Azure ML workspace. In addition to these essential compute setup steps, IT-operation team also uses familiar tools such as Azure CLI or kubectl to take care of the following tasks for the data-science team:

-**Data-science team**. Once the IT-operations team finishes compute setup and compute target(s) creation, the data-science team can discover a list of available compute targets and instance types in AzureML workspace. These compute resources can be used for training or inference workload. Data science specifies compute target name and instance type name using their preferred tools or APIs such as AzureML CLI v2, Python SDK v2, or Studio UI.

-With Arc Kubernetes cluster, you can build, train, and deploy models in any infrastructure on-premises and across multicloud using Kubernetes. This opens some new use patterns previously not possible in cloud setting environment. The following table provides a summary of the new use patterns enabled by AzureML Kubernetes compute:

-| Usage pattern | Location of data | Motivation | Infra setup & Azure ML implementation |

-| Bring your own AKS in Azure | Cloud | More security and controls.<br/>All private IP machine learning to prevent data exfiltration. | 1. AKS cluster behind an Azure VNet.<br/>2. Create private endpoints in the same VNet for AzureML workspace and its associated resources.<br/>3. Fully automated MLOps. |

-**Create and manage instance types for different ML workload scenarios**. Each ML workload uses different amounts of compute resources such as CPU/GPU and memory. AzureML implements instance type as Kubernetes custom resource definition (CRD) with properties of nodeSelector and resource request/limit. With a carefully curated list of instance types, IT-operations can target ML workload on specific node(s) and manage compute resource utilization efficiently.

-**Multiple AzureML workspaces share the same Kubernetes cluster**. You can attach Kubernetes cluster multiple times to the same AzureML workspace or different AzureML workspaces, creating multiple compute targets in one workspace or multiple workspaces. Since many customers organize data science projects around AzureML workspace, multiple data science projects can now share the same Kubernetes cluster. This significantly reduces ML infrastructure management overheads and IT cost saving.

-**Team/project workload isolation using Kubernetes namespace**. When you attach Kubernetes cluster to AzureML workspace, you can specify a Kubernetes namespace for the compute target. All workloads run by the compute target will be placed under the specified namespace.

-With AzureML CLI/Python SDK v1, you can deploy models on AKS using AksCompute target. Both KubernetesCompute target and AksCompute target support AKS integration, however they support it differently. The following table shows their key differences:

-With these key differences and overall AzureML evolution to use SDK/CLI v2, AzureML recommends you to use Kubernetes compute target to deploy models if you decide to use AKS for model deployment.

-All AzureML examples can be found in [https://github.com/Azure/azureml-examples.git](https://github.com/Azure/azureml-examples).

-For any AzureML example, you only need to update the compute target name to your Kubernetes compute target, then you're all done.

-# Attach a Kubernetes cluster to AzureML workspace

-Once AzureML extension is deployed on AKS or Arc Kubernetes cluster, you can attach the Kubernetes cluster to AzureML workspace and create compute targets for ML professionals to use.

-Attaching a Kubernetes cluster to AzureML workspace can flexibly support many different scenarios, such as the shared scenarios with multiple attachments, model training scripts accessing Azure resources, and the authentication configuration of the workspace. But you need to pay attention to the following prerequisites.

-> The namespace you plan to specify when attaching the cluster to AzureML workspace should be previously created in your cluster.

-Azure Machine Learning workspace defaults to having a system-assigned managed identity to access Azure ML resources. The steps are completed if the system assigned default setting is on.

-## How to attach a Kubernetes cluster to AzureML workspace

-We support two ways to attach a Kubernetes cluster to AzureML workspace, using Azure CLI or studio UI.

-az ml compute attach --resource-group <resource-group-name> --workspace-name <workspace-name> --type Kubernetes --name k8s-compute --resource-id "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerService/managedclusters/<cluster-name>" --identity-type SystemAssigned --namespace <Kubernetes namespace to run AzureML workloads> --no-wait

-> Compute attach won't create the Kubernetes namespace automatically or validate whether the kubernetes namespace existed. You need to verify that the specified namespace exists in your cluster, otherwise, any AzureML workloads submitted to this compute will fail.

- * **(Optional)** Enter Kubernetes namespace, which defaults to `default`. All machine learning workloads will be sent to the specified Kubernetes namespace in the cluster. Compute attach won't create the Kubernetes namespace automatically or validate whether the kubernetes namespace exists. You need to verify that the specified namespace exists in your cluster, otherwise, any AzureML workloads submitted to this compute will fail.

-> When working on a private link-enabled workspaces, batch endpoints can't be invoked from the UI in Azure ML studio. Please use the Azure ML CLI v2 instead for job creation.

-> When working on Azure ML studio, batch endpoints/deployments are always executed using the identity of the current user logged in.

-1. Use the Azure ML SDK for Python to log in using either interactive or device authentication:

-You can now define an input data object, which is required to start a training job, using the AzureML Python SDK as follows:

-AutoML uses AzureML Compute, which is a fully managed compute resource, to run the training job. In the following example, a compute cluster named `cpu-compute` is created:

-> Many models and hierarchical time series are currently only supported in AzureML v1. Support for AzureML v2 is forthcoming.

- * Use the following commands to install Azure ML Python SDK v2:

- * Install the Azure ML Python SDK v2:

-You can seamlessly integrate with the [Azure Machine Learning data labeling](how-to-create-text-labeling-projects.md) capability to label your text data or bring your existing labeled data. Automated ML provides the option to use distributed training on multi-GPU compute clusters for faster model training. The resulting model can be operationalized at scale by leveraging Azure MLΓÇÖs MLOps capabilities.

-You can also run your NLP experiments with distributed training on an Azure ML compute cluster.

-|**Default AutoML**|Recommended if the dataset has a small number of time series that have roughly similar historic behavior.|- Simple to configure from code/SDK or AzureML Studio <br><br> - AutoML has the chance to cross-learn across different time series since the regression models pool all series together in training. See the [model grouping](./concept-automl-forecasting-methods.md#model-grouping) section for more information.|- Regression models may be less accurate if the time series in the training data have divergent behavior <br> <br> - Time series models may take a long time to train if there are a large number of series in the training data. See the ["why is AutoML slow on my data"](#why-is-automl-slow-on-my-data) answer for more information.|

-|**AutoML with deep learning**|Recommended for datasets with more than 1000 observations and, potentially, numerous time series exhibiting complex patterns. When enabled, AutoML will sweep over temporal convolutional neural network (TCN) models during training. See the [enable deep learning](./how-to-auto-train-forecast.md#enable-deep-learning) section for more information.|- Simple to configure from code/SDK or AzureML Studio <br> <br> - Cross-learning opportunities since the TCN pools data over all series <br> <br> - Potentially higher accuracy due to the large capacity of DNN models. See the [forecasting models in AutoML](./concept-automl-forecasting-methods.md#forecasting-models-in-automl) section for more information.|- Training can take much longer due to the complexity of DNN models <br> <br> - Series with small amounts of history are unlikely to benefit from these models.|

-|**Many Models**|Recommended if you need to train and manage a large number of forecasting models in a scalable way. See the [forecasting at scale](./how-to-auto-train-forecast.md#forecasting-at-scale) section for more information.|- Scalable <br> <br> - Potentially higher accuracy when time series have divergent behavior from one another.|- No cross-learning across time series <br> <br> - You can't configure or launch Many Models jobs from AzureML Studio, only the code/SDK experience is currently available.|

-If you aren't familiar with Azure Machine Learning concepts, start with the ["What is AzureML"](overview-what-is-azure-machine-learning.md) article and the [workspaces](./concept-workspace.md) article.

-Today, you can manage autoscaling using either the Azure CLI, REST, ARM, or the browser-based Azure portal. Other Azure ML SDKs, such as the Python SDK, will add support over time.

-Get Azure ML and Azure Monitor clients:

-Batch endpoints allow you to deploy models to perform inference at scale. Because how inference should be executed varies from model's format, model's type and use case, batch endpoints require a scoring script (also known as batch driver script) to indicate the deployment how to use the model over the provided data. In this article you will learn how to use scoring scripts in different scenarios and their best practices.

-> MLflow models don't require a scoring script as it is autogenerated for you. For more details about how batch endpoints work with MLflow models, see the dedicated tutorial [Using MLflow models in batch deployments](how-to-mlflow-batch.md). Notice that this feature doesn't prevent you from writing an specific scoring script for MLflow models as explained at [Using MLflow models with a scoring script](how-to-mlflow-batch.md#customizing-mlflow-models-deployments-with-a-scoring-script).

-Use the `init()` method for any costly or common preparation. For example, use it to load the model into a global object. This function will be called once at the beginning of the process. You model's files will be available in an environment variable called `AZUREML_MODEL_DIR`. Use this variable to locate the files associated with the model. Notice that some models may be contained in a folder (in the following example, the model has several files in a folder named `model`). See [how you can find out what's the folder used by your model](#using-models-that-are-folders).

-Use the `run(mini_batch: List[str]) -> Union[List[Any], pandas.DataFrame]` method to perform the scoring of each mini-batch generated by the batch deployment. Such method will be called once per each `mini_batch` generated for your input data. Batch deployments read data in batches accordingly to how the deployment is configured.

-def run(mini_batch: List[str]) -> Union[List[Any], pandas.DataFrame]:

-The method receives a list of file paths as a parameter (`mini_batch`). You can use this list to either iterate over each file and process it one by one, or to read the entire batch and process it at once. The best option will depend on your compute memory and the throughput you need to achieve. For an example of how to read entire batches of data at once see [High throughput deployments](how-to-image-processing-batch.md#high-throughput-deployments).

-The `run()` method should return a Pandas `DataFrame` or an array/list. Each returned output element indicates one successful run of an input element in the input `mini_batch`. For file datasets, each row/element will represent a single file processed. For a tabular dataset, each row/element will represent a row in a processed file.

-The resulting DataFrame or array is appended to the output file indicated. There's no requirement on the cardinality of the results (1 file can generate 1 or many rows/elements in the output). All elements in the result DataFrame or array will be written to the output file as-is (considering the `output_action` isn't `summary_only`).

-Any library that your scoring script requires to run needs to be indicated in the environment where your batch deployment runs. As for scoring scripts, environments are indicated per deployment. Usually, you will indicate your requirements using a `conda.yml` dependencies file which may look as follows:

-By default, the batch deployment will write the model's predictions in a single file as indicated in the deployment. However, there are some cases where you need to write the predictions in multiple files. For instance, if the input data is partitioned, you typically would want to generate your output partitioned too. On those cases you can [Customize outputs in batch deployments](how-to-deploy-model-custom-output.md) to indicate:

-Batch deployments distribute work at the file level, which means that a folder containing 100 files with mini-batches of 10 files will generate 10 batches of 10 files each. Notice that this will happen regardless of the size of the files involved. If your files are too big to be processed in large mini-batches we suggest to either split the files in smaller files to achieve a higher level of parallelism or to decrease the number of files per mini-batch. At this moment, batch deployment can't account for skews in the file's size distribution.

-When running multiple workers on the same instance, take into account that memory will be shared across all the workers. Usually, increasing the number of workers per node should be accompanied by a decrease in the mini-batch size or by a change in the scoring strategy (if data size and compute SKU remains the same).

-For an example about how to achieve it see [High throughput deployments](how-to-image-processing-batch.md#high-throughput-deployments). This example processes an entire batch of files at a time.

-# Set up AutoML training with the Azure ML Python SDK v2

- * Create a compute instance, which already has installed the latest AzureML Python SDK and is pre-configured for ML workflows. See [Create and manage an Azure Machine Learning compute instance](how-to-create-manage-compute-instance.md) for more information.

- * Use the followings commands to install Azure ML Python SDK v2:

- * Install the Azure ML Python SDK v2:

- # Enter details of your AzureML workspace

-Automated ML jobs with the Python SDK v2 (or CLI v2) are currently only supported on Azure ML remote compute (cluster or compute instance).

-To leverage AutoML in your MLOps workflows, you can add AutoML Job steps to your [AzureML Pipelines](./how-to-create-component-pipeline-python.md). This allows you to automate your entire workflow by hooking up your data prep scripts to AutoML and then registering and validating the resulting best model.

-description: Learn to set up a development environment in Azure Machine Learning and Azure Databricks. Use the Azure ML SDKs for Databricks and Databricks with AutoML.

-+ With [automated machine learning](concept-automated-ml.md) capabilities using an Azure ML SDK.

-## Add the Azure ML SDK to Databricks

-To use automated ML, skip to [Add the Azure ML SDK with AutoML](#add-the-azure-ml-sdk-with-automl-to-databricks).

-## Add the Azure ML SDK with AutoML to Databricks

-If the cluster was created with Databricks Runtime 7.3 LTS (*not* ML), run the following command in the first cell of your notebook to install the AzureML SDK.

- #Enter details of your AzureML workspace

-* Azure allows you to place _locks_ on resources, so that they can't be deleted or are read only. __Do not apply resource locks to the resource group that contains your workspace__. Applying a lock to the resource group that contains your workspace will prevent scaling operations for Azure ML compute clusters. For more information on locking resources, see [Lock resources to prevent unexpected changes](../azure-resource-manager/management/lock-resources.md).

-This article uses the Python SDK for Azure ML to create and control an Azure Machine Learning pipeline. The article assumes that you'll be running the code snippets interactively in either a Python REPL environment or a Jupyter notebook.

-This article is based on the [image_classification_keras_minist_convnet.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/pipelines/2e_image_classification_keras_minist_convnet/image_classification_keras_minist_convnet.ipynb) notebook found in the `sdk/jobs/pipelines/2e_image_classification_keras_minist_convnet` directory of the [AzureML Examples](https://github.com/azure/azureml-examples) repository.

-If you're following along with the example in the [AzureML Examples repo](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/pipelines/2e_image_classification_keras_minist_convnet), the source files are already available in `prep/` folder. This folder contains two files to construct the component: `prep_component.py`, which defines the component and `conda.yaml`, which defines the run-time environment of the component.

-The source files of this component are under `train/` folder in the [AzureML Examples repo](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/pipelines/2e_image_classification_keras_minist_convnet). This folder contains three files to construct the component:

-After defining the training function successfully, you can use @command_component in Azure Machine Learning SDK v2 to wrap your function as a component, which can be used in AzureML pipelines.

-If you're following along with the example in the [AzureML Examples repo](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/pipelines/2e_image_classification_keras_minist_convnet), the source files are already available in `score/` folder. This folder contains three files to construct the component:

-You can check the logs and outputs of each component by right clicking the component, or select the component to open its detail pane. To learn more about how to debug your pipeline in UI, see [How to use studio UI to build and debug Azure ML pipelines](how-to-use-pipeline-ui.md).

-In this article, you learn how to create and run [machine learning pipelines](concept-ml-pipelines.md) by using the Azure CLI and components (for more, see [What is an Azure Machine Learning component?](concept-component.md)). You can create pipelines without using components, but components offer the greatest amount of flexibility and reuse. AzureML Pipelines may be defined in YAML and run from the CLI, authored in Python, or composed in AzureML Studio Designer with a drag-and-drop UI. This document focuses on the CLI.

-Let's create your first pipeline with component using an example. This section aims to give you an initial impression of what pipeline and component look like in AzureML with a concrete example.

- - Command, code & environment: the command, code and environment to run the component. Command is the shell command to execute the component. Code usually refers to a source code directory. Environment could be an AzureML environment(curated or customer created), docker image or conda environment.

-One common scenario is to read and write data in your pipeline. In AzureML, we use the same schema to [read and write data](how-to-read-write-data-v2.md) for all type of jobs (pipeline job, command job, and sweep job). Below are pipeline job examples of using data for common scenarios.

-|name|**Required**. Name of the component. Must be unique across the AzureML workspace. Must start with lowercase letter. Allow lowercase letters, numbers and underscore(_). Maximum length is 255 characters.|

-For the example in *3b_pipeline_with_data/componentA.yml*, componentA has one data input and one data output, which can be connected to other steps in the parent pipeline. All the files under `code` section in component YAML will be uploaded to AzureML when submitting the pipeline job. In this example, files under `./componentA_src` will be uploaded (line 16 in *componentA.yml*). You can see the uploaded source code in Studio UI: double select the ComponentA step and navigate to Snapshot tab, as shown in below screenshot. We can see it's a hello-world script just doing some simple printing, and write current datetime to the `componentA_output` path. The component takes input and output through command line argument, and it's handled in the *hello.py* using `argparse`.

-Environment defines the environment to execute the component. It could be an AzureML environment(curated or custom registered), docker image or conda environment. See examples below.

-In this article, you'll learn how to create and run [machine learning pipelines](concept-ml-pipelines.md) by using the Azure Machine Learning studio and [Components](concept-component.md). You can create pipelines without using components, but components offer better amount of flexibility and reuse. Azure ML Pipelines may be defined in YAML and [run from the CLI](how-to-create-component-pipelines-cli.md), [authored in Python](how-to-create-component-pipeline-python.md), or composed in Azure ML Studio Designer with a drag-and-drop UI. This document focuses on the AzureML studio designer UI.

-1. Register the components to AzureML workspace using following commands. Learn more about [ML components](concept-component.md).

-In this article, you'll learn how to create a data asset in Azure ML. An Azure ML data asset is similar to web browser bookmarks (favorites). Instead of remembering long storage paths (URIs) that point to your most frequently used data, you can create a data asset, and then access that asset with a friendly name.

-Data asset creation also creates a *reference* to the data source location, along with a copy of its metadata. Because the data remains in its existing location, you incur no extra storage cost, and don't risk data source integrity. You can create Data assets from Azure ML datastores, Azure Storage, public URLs, and local files.

-|**Table**<br> Reference a data table | `mltable` | `TabularDataset` | In V1 APIs, the Azure ML back-end stored the data materialization blueprint. This storage location meant that `TabularDataset` only worked if you had an Azure ML workspace. `mltable` stores the data materialization blueprint in *your* storage. This storage location means you can use it *disconnected to Azure ML* - for example, local, on-premises. In V2 APIs, you'll find it easier to transition from local to remote jobs. Read [Working with tables in Azure Machine Learning](how-to-mltable.md) for more information. | You have a complex schema subject to frequent changes, or you need a subset of large tabular data.<br><br>AutoML with Tables. |

-* No active AzureML runs or experiments

-# Enter details of your AML workspace

-You can create compute instance with managed identity from Azure ML Studio:

-azure ml compute create --file compute.yaml --resource-group my-resource-group --workspace-name my-workspace

-1. In the Run and Debug dropdown, select **Azure ML: Debug Local Endpoint** to start debugging your endpoint locally.

- :::image type="content" source="media/how-to-debug-managed-online-endpoints-visual-studio-code/configure-debug-profile.png" alt-text="Configure Azure ML Debug Local Environment debug profile":::

- 1. A configuration file containing your run configuration settings opens in the editor. If you're satisfied with the settings, select **Submit experiment**. Alternatively, you open the command palette (**View > Command Palette**) from the menu bar and enter the `Azure ML: Submit experiment` command into the text box.

- help=f'Defines how much time the AzureML compute target '

- help=f'Defines how much time the AzureML compute target '

-> You can also find the IP address from the run logs for the child run for this pipeline step. For more information on viewing this information, see [Monitor Azure ML experiment runs and metrics](how-to-log-view-metrics.md).

- # enter details of your AzureML workspace

-> The code snippets are pulled from the `/cli/batch-score-rest.sh` file in the [AzureML Example repository](https://github.com/Azure/azureml-examples).

-There are many ways to create an Azure Machine Learning batch endpoint, including the Azure CLI, Azure ML SDK for Python, and visually with the studio. The following example creates a batch endpoint and a batch deployment with the REST API.

-> For more information about data URI, see [Azure Machine Learning data reference URI](reference-yaml-core-syntax.md#azure-ml-data-reference-uri).

-description: Learn how to use a custom container to use open-source servers in Azure Machine Learning.

-# Deploy a TensorFlow model served with TensorFlow Serving using a custom container in an online endpoint

-Learn how to deploy a custom container as an online endpoint in Azure Machine Learning.

-You can find [various examples](https://github.com/Azure/azureml-examples/tree/main/cli/endpoints/online/custom-container) for TensorFlow Serving, TorchServe, Triton Inference Server, Plumber R package, and AzureML Inference Minimal image as below:

-|[minimal/multimodel](https://github.com/Azure/azureml-examples/blob/main/cli/endpoints/online/custom-container/minimal/multimodel)|[deploy-custom-container-minimal-multimodel](https://github.com/Azure/azureml-examples/blob/main/cli/deploy-custom-container-minimal-multimodel.sh)|Deploy multiple models to a single deployment by extending the AzureML Inference Minimal image.|

-|[minimal/single-model](https://github.com/Azure/azureml-examples/blob/main/cli/endpoints/online/custom-container/minimal/single-model)|[deploy-custom-container-minimal-single-model](https://github.com/Azure/azureml-examples/blob/main/cli/deploy-custom-container-minimal-single-model.sh)|Deploy a single model by extending the AzureML Inference Minimal image.|

-|[mlflow/multideployment-scikit](https://github.com/Azure/azureml-examples/blob/main/cli/endpoints/online/custom-container/mlflow/multideployment-scikit)|[deploy-custom-container-mlflow-multideployment-scikit](https://github.com/Azure/azureml-examples/blob/main/cli/deploy-custom-container-mlflow-multideployment-scikit.sh)|Deploy two MLFlow models with different Python requirements to two separate deployments behind a single endpoint using the AzureML Inference Minimal Image.|

-This article focuses on serving a TensorFlow model with TensorFlow (TF) Serving.

-# enter details of your AzureML workspace

-description: Learn about the AzureML extension, available configuration settings and different deployment scenarios, and verify and managed AzureML extension

-# Deploy AzureML extension on AKS or Arc Kubernetes cluster

-To enable your AKS or Arc Kubernetes cluster to run training jobs or inference workloads, you must first deploy the AzureML extension on an AKS or Arc Kubernetes cluster. The AzureML extension is built on the [cluster extension for AKS](../aks/cluster-extensions.md) and [cluster extension or Arc Kubernetes](../azure-arc/kubernetes/conceptual-extensions.md), and its lifecycle can be managed easily with Azure CLI [k8s-extension](/cli/azure/k8s-extension).

-> * Review AzureML extension config settings

-> * AzureML extension deployment scenarios

-> * Verify AzureML extension deployment

-> * Review AzureML extension components

-> * Manage AzureML extension

-## Review AzureML extension configuration settings

-You can use AzureML CLI command `k8s-extension create` to deploy AzureML extension. CLI `k8s-extension create` allows you to specify a set of configuration settings in `key=value` format using `--config` or `--config-protected` parameter. Following is the list of available configuration settings to be specified during AzureML extension deployment.

- |`enableTraining` |`True` or `False`, default `False`. **Must** be set to `True` for AzureML extension deployment with Machine Learning model training and batch scoring support. | **&check;**| N/A | **&check;** |

- | `enableInference` |`True` or `False`, default `False`. **Must** be set to `True` for AzureML extension deployment with Machine Learning inference support. |N/A| **&check;** | **&check;** |

- | `inferenceRouterHA` |`True` or `False`, default `True`. By default, AzureML extension will deploy three inference router replicas for high availability, which requires at least three worker nodes in a cluster. Set to `False` if your cluster has fewer than three worker nodes, in this case only one inference router service is deployed. | N/A| Optional | Optional |

- |`installNvidiaDevicePlugin` | `True` or `False`, default `False`. [NVIDIA Device Plugin](https://github.com/NVIDIA/k8s-device-plugin#nvidia-device-plugin-for-kubernetes) is required for ML workloads on NVIDIA GPU hardware. By default, AzureML extension deployment won't install NVIDIA Device Plugin regardless Kubernetes cluster has GPU hardware or not. User can specify this setting to `True`, to install it, but make sure to fulfill [Prerequisites](https://github.com/NVIDIA/k8s-device-plugin#prerequisites). | Optional |Optional |Optional |

- |`installPromOp`|`True` or `False`, default `True`. AzureML extension needs prometheus operator to manage prometheus. Set to `False` to reuse the existing prometheus operator. For more information about reusing the existing prometheus operator, refer to [reusing the prometheus operator](./how-to-troubleshoot-kubernetes-extension.md#prometheus-operator)| Optional| Optional | Optional |

- |`installVolcano`| `True` or `False`, default `True`. AzureML extension needs volcano scheduler to schedule the job. Set to `False` to reuse existing volcano scheduler. For more information about reusing the existing volcano scheduler, refer to [reusing volcano scheduler](./how-to-troubleshoot-kubernetes-extension.md#volcano-scheduler) | Optional| N/A | Optional |

- |`installDcgmExporter` |`True` or `False`, default `False`. Dcgm-exporter can expose GPU metrics for AzureML workloads, which can be monitored in Azure portal. Set `installDcgmExporter` to `True` to install dcgm-exporter. But if you want to utilize your own dcgm-exporter, refer to [DCGM exporter](./how-to-troubleshoot-kubernetes-extension.md#dcgm-exporter) |Optional |Optional |Optional |

- | `sslCertPemFile`, `sslKeyPemFile` |Path to TLS/SSL certificate and key file (PEM-encoded), required for AzureML extension deployment with inference HTTPS endpoint support, when ``allowInsecureConnections`` is set to False. **Note** PEM file with pass phrase protected isn't supported | N/A| Optional | Optional |

-As you can see from above configuration settings table, the combinations of different configuration settings allow you to deploy AzureML extension for different ML workload scenarios:

-If you plan to deploy AzureML extension for real-time inference workload and want to specify `enableInference=True`, pay attention to following configuration settings related to real-time inference workload:

- * To ensure high availability of `azureml-fe` routing service, AzureML extension deployment by default creates three replicas of `azureml-fe` for clusters having three nodes or more. If your cluster has **less than 3 nodes**, set `inferenceRouterHA=False`.

- * By default, AzureML extension deployment expects config settings for **HTTPS** support. For development or testing purposes, **HTTP** support is conveniently provided through config setting `allowInsecureConnections=True`.

-## AzureML extension deployment - CLI examples and Azure portal

-To deploy AzureML extension with CLI, use `az k8s-extension create` command passing in values for the mandatory parameters.

-We list four typical extension deployment scenarios for reference. To deploy extension for your production usage, carefully read the complete list of [configuration settings](#review-azureml-extension-configuration-settings).

- For AzureML extension deployment on AKS cluster, make sure to specify `managedClusters` value for `--cluster-type` parameter. Run the following Azure CLI command to deploy AzureML extension:

- For AzureML extension deployment on [Arc Kubernetes](../azure-arc/kubernetes/overview.md) cluster, you would need to specify `connectedClusters` value for `--cluster-type` parameter. Run the following Azure CLI command to deploy AzureML extension:

- For AzureML extension deployment on AKS, make sure to specify `managedClusters` value for `--cluster-type` parameter. Assuming your cluster has more than three nodes, and you'll use an Azure public load balancer and HTTPS for inference workload support. Run the following Azure CLI command to deploy AzureML extension:

- For AzureML extension deployment on [Arc Kubernetes](../azure-arc/kubernetes/overview.md) cluster, make sure to specify `connectedClusters` value for `--cluster-type` parameter. Assuming your cluster has more than three nodes, you'll use a NodePort service type and HTTPS for inference workload support, run following Azure CLI command to deploy AzureML extension:

-The UI experience to deploy extension is only available for **[Arc Kubernetes](../azure-arc/kubernetes/overview.md)**. If you have an AKS cluster without Azure Arc connection, you need to use CLI to deploy AzureML extension.

- :::image type="content" source="media/how-to-attach-kubernetes-to-workspace/deploy-extension-from-ui-extension-list.png" alt-text="Screenshot of selecting AzureML extension from Azure portal.":::

-1. Follow the prompts to deploy the extension. You can customize the installation by configuring the installation in the tab of **Basics**, **Configurations** and **Advanced**. For a detailed list of AzureML extension configuration settings, see [AzureML extension configuration settings](#review-azureml-extension-configuration-settings).

- :::image type="content" source="media/how-to-attach-kubernetes-to-workspace/deploy-extension-from-ui-settings.png" alt-text="Screenshot of configuring AzureML extension settings from Azure portal.":::

-1. After the deployment completes, you're able to see the AzureML extension in **Extension** page. If the extension installation succeeds, you can see **Installed** for the **Install status**.

- :::image type="content" source="media/how-to-attach-kubernetes-to-workspace/deploy-extension-from-ui-extension-detail.png" alt-text="Screenshot of installed AzureML extensions listing in Azure portal.":::

-### Verify AzureML extension deployment

-1. Run the following CLI command to check AzureML extension details:

-## Review AzureML extension component

-Upon AzureML extension deployment completes, you can use `kubectl get deployments -n azureml` to see list of resources created in the cluster. It usually consists a subset of following resources per configuration settings specified.

- > * By default, the kubernetes deployment resources are randomly deployed to 1 or more nodes of the cluster, and daemonset resources are deployed to ALL nodes. If you want to restrict the extension deployment to specific nodes, use `nodeSelector` configuration setting described in [configuration settings table](#review-azureml-extension-configuration-settings).

-### Manage AzureML extension

-Update, list, show and delete an AzureML extension.

-# [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-Use the Azure ML CLI v2 to create a model from a training job output. In the following example, a model named `$MODEL_NAME` is registered using the artifacts of a job with ID `$RUN_ID`. The path where the model is stored is `$MODEL_PATH`.

-# [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-> Customizing the scoring script for MLflow deployments is only available from the Azure CLI or SDK for Python. If you are creating a deployment using [Azure ML studio](https://ml.azure.com), please switch to the CLI or the SDK.

- # [Python (Azure ML SDK)](#tab/sdk)

- On [Azure ML studio portal](https://ml.azure.com), follow these steps:

- # [Python (Azure ML SDK)](#tab/sdk)

- > You can't create custom MLflow deployments in Online Endpoints using the Azure Machine Learning portal. Switch to [Azure ML CLI](?tabs=azure-cli) or the [Azure ML SDK for Python](?tabs=python).

- # [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- This functionality is not available in the MLflow SDK. Go to [Azure ML studio](https://ml.azure.com), navigate to the endpoint and retrieve the secret key from there.

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

- # [Python (Azure ML SDK)](#tab/sdk)

-# [Python (Azure ML SDK)](#tab/sdk)

-> - Azure ML CLI and Azure ML SDK for Python

-| Scenario | MLflow SDK | Azure ML CLI/SDK | Azure ML studio |

-If you are familiar with MLflow or your platform support MLflow natively (like Azure Databricks) and you wish to continue using the same set of methods, use the MLflow SDK. On the other hand, if you are more familiar with the [Azure ML CLI v2](concept-v2.md), you want to automate deployments using automation pipelines, or you want to keep deployments configuration in a git repository; we recommend you to use the [Azure ML CLI v2](concept-v2.md). If you want to quickly deploy and test models trained with MLflow, you can use [Azure Machine Learning studio](https://ml.azure.com) UI deployment.

-| Input type | MLflow built-in server | Azure ML Online Endpoints |

-# things such as dependencies and AzureML components.

- No extra step is required for the Azure ML CLI. The environment definition will be included in the deployment file.

- # enter details of your AzureML workspace

-You should already have a resource group in Azure with [Azure Machine Learning](overview-what-is-azure-machine-learning.md). To deploy your DevOps pipeline to AzureML, you'll need to create variables for your subscription ID, resource group, and machine learning workspace.

-1. Create a new variable for `AzureML_Workspace_Name` with the name of your Azure ML workspace (example: `docs-ws`).

-* Use the Azure CLI task to pass the values of your three variables and use papermill to run your Jupyter notebook and push output to AzureML.

- :::image type="content" source="media/how-to-devops-machine-learning/machine-learning-azurecli-output.png" alt-text="Screenshot of machine learning output to AzureML.":::

-You can unregister data assets and archive jobs, but these operations don't delete the data. To entirely remove the data, data assets and job data require deletion at the storage level. Storage level deletion happens in the portal, as described earlier. Azure ML Studio can handle individual deletion. Job deletion deletes the data of that job.

-Azure ML Studio can handle training artifact downloads from experimental jobs. Choose the relevant **Job**. Choose **Output + logs**, and navigate to the specific artifacts you wish to download. Choose **...** and **Download**, or select **Download all**.

-* Automated ML code generation is only available for experiments run on remote Azure ML compute targets. Code generation isn't supported for local runs.

-* All automated ML runs triggered through AzureML Studio, SDKv2 or CLIv2 will have code generation enabled.

-By default, each automated ML trained model generates its training code after training completes. Automated ML saves this code in the experiment's `outputs/generated_code` for that specific model. You can view them in the Azure ML studio UI on the **Outputs + logs** tab of the selected model.

-* **script_run_notebook.ipynb** Notebook with boiler-plate code to run the model's training code (script.py) in AzureML compute through Azure ML SDKv2.

-The `script.py` file contains the core logic needed to train a model with the previously used hyperparameters. While intended to be executed in the context of an Azure ML script run, with some modifications, the model's training code can also be run standalone in your own on-premises environment.

-The function `get_training_dataset()` loads the previously used dataset. It assumes that the script is run in an AzureML script run under the same workspace as the original experiment.

- # The following code is for when running this code as part of an AzureML script run.

-The `script_run_notebook.ipynb` notebook serves as an easy way to execute `script.py` on an Azure ML compute.

-print(returned_job.studio_url) # link to naviagate to submitted run in AzureML Studio

- |WORKSPACE | Name of Azure ML workspace |

-You'll use a `pipeline.yml` file to deploy your Azure ML pipeline. This is a machine learning pipeline and not a DevOps pipeline. You only need to make this update if you're using a name other than `cpu-cluster` for your computer cluster name.

-| Azure ML pipelines (code-generated) | [az ml pipeline get --path {PATH}](/cli/azure/ml(v1)/pipeline#az-ml(v1)-pipeline-get) | [az ml pipeline create --name {NAME} -y {PATH}](/cli/azure/ml(v1)/pipeline#az-ml(v1)-pipeline-create)

-> * __Registered datasets__ cannot be downloaded or moved. This includes datasets generated by Azure ML, such as intermediate pipeline datasets. However datasets that refer to a shared file location that both workspaces can access, or where the underlying data storage is replicated, can be registered on both workspaces. Use the [az ml dataset register](/cli/azure/ml(v1)/dataset#ml-az-ml-dataset-register) to register a dataset.

-description: Learn how to set up and configure authentication between Azure ML and other Azure services.

-# Set up authentication between Azure ML and other services

-* Azure Machine Learning uses Azure Container Registry (ACR) to store Docker images used to train and deploy models. If you allow Azure ML to automatically create ACR, it will enable the __admin account__.

-* The Azure ML compute cluster uses a __managed identity__ to retrieve connection information for datastores from Azure Key Vault and to pull Docker images from ACR. You can also configure identity-based access to datastores, which will instead use the managed identity of the compute cluster.

-> Cross tenant access to storage accounts is not supported. If cross tenant access is needed for your scenario, please reach out to the AzureML Data Support team alias at amldatasupport@microsoft.com for assistance with a custom code solution.

-When you disable the admin user for ACR, Azure ML uses a managed identity to build and pull Docker images. There are two workflows when configuring Azure ML to use an ACR with the admin user disabled:

-* Allow Azure ML to create the ACR instance and then disable the admin user afterwards.

-### Azure ML with auto-created ACR instance

-[Create ACR from Azure CLI](../container-registry/container-registry-get-started-azure-cli.md) without setting ```--admin-enabled``` argument, or from Azure portal without enabling admin user. Then, when creating Azure Machine Learning workspace, specify the Azure resource ID of the ACR. The following example demonstrates creating a new Azure ML workspace that uses an existing ACR:

- No extra step is required for the Azure ML CLI. The environment definition will be included in the deployment file.

- # [Azure ML CLI](#tab/cli)

- # [Azure ML SDK for Python](#tab/sdk)

- # Enter details of your AML workspace

-Here we describe logs of the AzureML inference HTTP server. You can get the log when you run the `azureml-inference-server-http` locally, or [get container logs](how-to-troubleshoot-online-endpoints.md#get-container-logs) if you're using online endpoints.

-> If you are using online endpoints, the log from the inference server starts with `Azure ML Inferencing HTTP server <version>`.

-Azure ML Inferencing HTTP server <version>

-Azure ML Inferencing HTTP server v0.8.0

-description: Debug or monitor your Machine Learning job as it runs on AzureML compute with your training application of choice.

-description: Learn about what is AzureML inference router, how autoscaling works, and how to configure and meet inference requests performance (# of requests per second and latency)

-# AzureML inference router and connectivity requirements

-AzureML inference router is a critical component for real-time inference with Kubernetes cluster. In this article, you can learn about:

- * What is AzureML inference router

-## What is AzureML inference router

-AzureML inference router is the front-end component (`azureml-fe`) which is deployed on AKS or Arc Kubernetes cluster at AzureML extension deployment time. It has following functions:

-As you can see from above diagram, by default 3 `azureml-fe` instances are created during AzureML extension deployment, one instance acts as coordinating role, and the other instances serve incoming inference requests. The coordinating instance has all information about model pods and makes decision about which model pod to serve incoming request, while the serving `azureml-fe` instances are responsible for routing the request to selected model pod and propagate the response back to the original user.

-AzureML inference router handles autoscaling for all model deployments on the Kubernetes cluster. Since all inference requests go through it, it has the necessary data to automatically scale the deployed model(s).

-> * **Do not enable Kubernetes Horizontal Pod Autoscaler (HPA) for model deployments**. Doing so would cause the two auto-scaling components to compete with each other. Azureml-fe is designed to auto-scale models deployed by AzureML, where HPA would have to guess or approximate model utilization from a generic metric like CPU usage or a custom metric configuration.

->* Increase the number of instances for `azureml-fe`. By default, AzureML creates 3 or 1 `azureml-fe` instances per cluster.

-> * This instance count depends on your configuration of `inferenceRouterHA` of the [AzureML entension](how-to-deploy-kubernetes-extension.md#review-azureml-extension-configuration-settings).

-For accessing Azure ML services behind a firewall, see [Configure inbound and outbound network traffic](how-to-access-azureml-behind-firewall.md).

-> * Swagger is enabled in deployed endpoints automatically and the __Test__ feature can be used in Azure ML studio.

-> When submitting jobs using Azure ML CLI v2, you can set the experiment name using the property `experiment_name` in the YAML definition of the job. You don't have to configure it on your training script. See [YAML: display name, experiment name, description, and tags](reference-yaml-job-command.md#yaml-display-name-experiment-name-description-and-tags) for details.

-> Azure ML SDK v1 logging can't log parameters. We recommend the use of MLflow for tracking experiments as it offers a superior set of features.

-The method above will list all the artifacts logged in the run, but they will remain stored in the artifacts store (Azure ML storage). To download any of them, use the method `download_artifact`:

-Log files are an essential resource for debugging the Azure ML workloads. After submitting a training job, drill down to a specific run to view its logs and outputs:

- [](./media/how-to-machine-learning-interpretability-aml/model-explanation-dashboard-aml-studio.png#lightbox)

-* **Local explanation for data index**: The explanation dashboard doesnΓÇÖt support relating local importance values to a row identifier from the original validation dataset if that dataset is greater than 5000 datapoints as the dashboard randomly downsamples the data. However, the dashboard shows raw dataset feature values for each datapoint passed into the dashboard under the Individual feature importance tab. Users can map local importances back to the original dataset through matching the raw dataset feature values. If the validation dataset size is less than 5000 samples, the `index` feature in AzureML studio will correspond to the index in the validation dataset.

-[Techniques for model interpretability in Azure ML](how-to-machine-learning-interpretability.md)

-description: Learn how to manage Azure ML environments using Python SDK and Azure CLI extension for Machine Learning.

-Azure Machine Learning environments define the execution environments for your jobs or deployments and encapsulate the dependencies for your code. Azure ML uses the environment specification to create the Docker container that your training or scoring code runs in on the specified compute target. You can define an environment from a conda specification, Docker image, or Docker build context.

-In this article, learn how to create and manage Azure ML environments using the SDK & CLI (v2).

-When using the Azure CLI, you need identifier parameters - a subscription, resource group, and workspace name. While you can specify these parameters for each command, you can also set defaults that will be used for all the commands. Use the following commands to set default values. Replace `<subscription ID>`, `<AzureML workspace name>`, and `<resource group>` with the values for your configuration:

-az configure --defaults workspace=<AzureML workspace name> group=<resource group>

-#Enter details of your AzureML workspace

-There are two types of environments in Azure ML: curated and custom environments. Curated environments are predefined environments containing popular ML frameworks and tooling. Custom environments are user-defined and can be created via `az ml environment create`.

-Curated environments are provided by Azure ML and are available in your workspace by default. Azure ML routinely updates these environments with the latest framework version releases and maintains them for bug fixes and security patches. They're backed by cached Docker images, which reduce job preparation cost and model deployment time.

-You can see the set of available curated environments in the Azure ML studio UI, or by using the CLI (v2) via `az ml environments list`.

-> Azure ML maintains a set of CPU and GPU Ubuntu Linux-based base images with common system dependencies. For example, the GPU images contain Miniconda, OpenMPI, CUDA, cuDNN, and NCCL. You can use these images for your environments, or use their corresponding Dockerfiles as reference when building your own custom images.

-The following example is a YAML specification file for an environment defined from a build context. The local path to the build context folder is specified in the `build.path` field, and the relative path to the Dockerfile within that build context folder is specified in the `build.dockerfile_path` field. If `build.dockerfile_path` is omitted in the YAML file, Azure ML will look for a Dockerfile named `Dockerfile` at the root of the build context.

-In the following example, the local path to the build context folder is specified in the `path' parameter. Azure ML will look for a Dockerfile named `Dockerfile` at the root of the build context.

-Azure ML will start building the image from the build context when the environment is created. You can monitor the status of the build and view the build logs in the studio UI.

-You must also specify a base Docker image for this environment. Azure ML will build the conda environment on top of the Docker image provided. If you install some Python dependencies in your Docker image, those packages won't exist in the execution environment thus causing runtime failures. By default, Azure ML will build a Conda environment with dependencies you specified, and will execute the job in that environment instead of using any Python libraries that you installed on the base image.

-The following example is a YAML specification file for an environment defined from a conda specification. Here the relative path to the conda file from the Azure ML environment YAML file is specified via the `conda_file` property. You can alternatively define the conda specification inline using the `conda_file` property, rather than defining it in a separate file.

-Azure ML will build the final Docker image from this environment specification when the environment is used in a job or deployment. You can also manually trigger a build of the environment in the studio UI.

-The SDK and CLI (v2) also allow you to manage the lifecycle of your Azure ML environment assets.

-To use an environment for a training job, specify the `environment` field of the job YAML configuration. You can either reference an existing registered Azure ML environment via `environment: azureml:<environment-name>:<environment-version>` or `environment: azureml:<environment-name>@latest` (to reference the latest version of an environment), or define an environment specification inline. If defining an environment inline, don't specify the `name` and `version` fields, as these environments are treated as "unregistered" environments and aren't tracked in your environment asset registry.

-> Content in notebooks and scripts can potentially read data from your sessions and access data without your organization in Azure. Only load files from trusted sources. For more information, see [Secure code best practices](concept-secure-code-best-practice.md#azure-ml-studio-notebooks).

-In Kubernetes clusters, instance types are represented in a custom resource definition (CRD) that is installed with the AzureML extension. Two elements in AzureML extension represent the instance types:

-> If you have [specified a nodeSelector when deploying the AzureML extension](./how-to-deploy-kubernetes-extension.md#review-azureml-extension-configuration-settings), the nodeSelector will be applied to all instance types. This means that:

-By default, a `defaultinstancetype` with the following definition is created when you attach a Kubernetes cluster to an AzureML workspace:

-The above example creates two instance types: `cpusmall` and `defaultinstancetype`. This `defaultinstancetype` definition overrides the `defaultinstancetype` definition created when Kubernetes cluster was attached to AzureML workspace.