+

+ ```html

+ </body>

+1. Expand **Settings** on the left, and for **Version Name**, enter a template version.

+ Title: Localization string IDs - Azure Active Directory B2C

+The **Localization** element enables you to support multiple locales or languages in the policy for the user journeys. This article provides the list of localization IDs that you can use in your policy. For more information about UI localization, see [Localization](localization.md).

+| `forgotpassword_link` | Forgot your password? | `All` |

+| `createaccount_intro` | Don't have an account? | `All` |

+| `button_signin` | Sign in | `All` |

+| `social_intro` | Sign in with your social account | `All` |

+| `remember_me` |Keep me signed in. | `All` |

+| `unknown_error` | We are having trouble signing you in. Please try again later. | `All` |

+| `divider_title` | OR | `All` |

+| `local_intro_email` | Sign in with your existing account | `< 2.0.0` |

+| `logonIdentifier_email` | Email Address | `< 2.0.0` |

+| `requiredField_email` | Please enter your email | `< 2.0.0` |

+| `invalid_email` | Please enter a valid email address | `< 2.0.0` |

+| `email_pattern` | ```^[a-zA-Z0-9.!#$%&''\*+/=?^\_\`{\|}~-]+@[a-zA-Z0-9-]+(?:\\.[a-zA-Z0-9-]+)\*$``` | `< 2.0.0` |

+| `local_intro_username` | Sign in with your user name | `< 2.0.0` |

+| `logonIdentifier_username` | Username | `< 2.0.0` |

+| `requiredField_username` | Please enter your user name | `< 2.0.0` |

+| `password` | Password | `< 2.0.0` |

+| `requiredField_password` | Please enter your password | `< 2.0.0` |

+| `createaccount_link` | Sign up now | `< 2.0.0` |

+| `cancel_message` | The user has forgotten their password | `< 2.0.0` |

+| `invalid_password` | The password you entered is not in the expected format. | `< 2.0.0` |

+| `createaccount_one_link` | Sign up now | `>= 2.0.0` |

+| `createaccount_two_links` | Sign up with {0} or {1} | `>= 2.0.0` |

+| `createaccount_three_links` | Sign up with {0}, {1}, or {2} | `>= 2.0.0` |

+| `local_intro_generic` | Sign in with your {0} | `>= 2.1.0` |

+| `requiredField_generic` | Please enter your {0} | `>= 2.1.0` |

+| `invalid_generic` | Please enter a valid {0} | `>= 2.1.1` |

+| `heading` | Sign in | `>= 2.1.1` |

+> * Placeholders like `{0}` are populated automatically with the `DisplayName` value of `ClaimType`.

+The following example shows the use of some user interface elements in the sign-up or sign-in page:

+The ID of the identity providers is configured in the user journey **ClaimsExchange** element. To localize the title of the identity provider, the **ElementType** is set to `ClaimsProvider`, while the **StringId** is set to the ID of the `ClaimsExchange`.

+| `UserMessageIfInvalidPassword` | Your password is incorrect. |

+| `UserMessageIfPasswordExpired`| Your password has expired.|

+| `UserMessageIfClaimsPrincipalDoesNotExist` | We can't seem to find your account. |

+| `UserMessageIfOldPasswordUsed` | Looks like you used an old password. |

+| `DefaultMessage` | Invalid username or password. |

+| `UserMessageIfUserAccountDisabled` | Your account has been locked. Contact your support person to unlock it, then try again. |

+| `UserMessageIfUserAccountLocked` | Your account is temporarily locked to prevent unauthorized use. Try again later. |

+| `AADRequestsThrottled` | There are too many requests at this moment. Please wait for some time and try again. |

+The following IDs are used for a content definition having an ID of `api.localaccountsignup` or any content definition that starts with `api.selfasserted`, such as `api.selfasserted.profileupdate` and `api.localaccountpasswordreset`, and [self-asserted technical profile](self-asserted-technical-profile.md).

+| `ver_sent` | Verification code has been sent to: |

+| `ver_but_default` | Default |

+| `cancel_message` | The user has canceled entering self-asserted information |

+| `preloader_alt` | Please wait |

+| `ver_but_send` | Send verification code |

+| `alert_yes` | Yes |

+| `error_fieldIncorrect` | One or more fields are filled out incorrectly. Please check your entries and try again. |

+| `year` | Year |

+| `verifying_blurb` | Please wait while we process your information. |

+| `button_cancel` | Cancel |

+| `ver_fail_no_retry` | You've made too many incorrect attempts. Please try again later. |

+| `month` | Month |

+| `ver_success_msg` | E-mail address verified. You can now continue. |

+| `months` | January, February, March, April, May, June, July, August, September, October, November, December |

+| `ver_fail_server` | We are having trouble verifying your email address. Please enter a valid email address and try again. |

+| `error_requiredFieldMissing` | A required field is missing. Please fill out all required fields and try again. |

+| `heading` | User Details |

+| `initial_intro` | Please provide the following details. |

+| `ver_but_resend` | Send new code |

+| `button_continue` | Create |

+| `error_passwordEntryMismatch` | The password entry fields do not match. Please enter the same password in both fields and try again. |

+| `ver_incorrect_format` | Incorrect format. |

+| `ver_but_edit` | Change e-mail |

+| `ver_but_verify` | Verify code |

+| `alert_no` | No |

+| `ver_info_msg` | Verification code has been sent to your inbox. Please copy it to the input box below. |

+| `day` | Day |

+| `ver_fail_throttled` | There have been too many requests to verify this email address. Please wait a while, then try again. |

+| `helplink_text` | What is this? |

+| `ver_fail_retry` | That code is incorrect. Please try again. |

+| `alert_title` | Cancel Entering Your Details |

+| `required_field` | This information is required. |

+| `alert_message` | Are you sure that you want to cancel entering your details? |

+| `ver_intro_msg` | Verification is necessary. Please click Send button. |

+| `ver_input` | Verification code |

+| `disclaimer_msg_intro` | By providing your phone number, you consent to receiving a one-time passcode sent by text message to help you sign into {insert your application name}. Standard message and data rates may apply. |

+| `disclaimer_link_1_text` | Privacy Statement |

+| `disclaimer_link_1_url` | {insert your privacy statement URL} |

+| `disclaimer_link_2_text` | Terms and Conditions |

+| `disclaimer_link_2_url` | {insert your terms and conditions URL} |

+| `UserMessageIfClaimsPrincipalAlreadyExists` | A user with the specified ID already exists. Please choose a different one. |

+| `UserMessageIfClaimNotVerified` | Claim not verified: {0} |

+| `UserMessageIfIncorrectPattern` | Incorrect pattern for: {0} |

+| `UserMessageIfMissingRequiredElement` | Missing required element: {0} |

+| `UserMessageIfValidationError` | Error in validation by: {0} |

+| `UserMessageIfInvalidInput` | {0} has invalid input. |

+| `ServiceThrottled` | There are too many requests at this moment. Please wait for some time and try again. |

+| `button_verify` | Call Me | `All` |

+| `country_code_label` | Country Code | `All` |

+| `cancel_message` | The user has canceled multi-factor authentication | `All` |

+| `text_button_send_second_code` | send a new code | `All` |

+| `code_pattern` | \\d{6} | `All` |

+| `intro_mixed` | We have the following number on record for you. We can send a code via SMS or phone to authenticate you. | `All` |

+| `intro_mixed_p` | We have the following numbers on record for you. Choose a number that we can phone or send a code via SMS to authenticate you. | `All` |

+| `button_verify_code` | Verify Code | `All` |

+| `requiredField_code` | Please enter the verification code you received | `All` |

+| `invalid_code` | Please enter the 6-digit code you received | `All` |

+| `button_cancel` | Cancel | `All` |

+| `local_number_input_placeholder_text` | Phone number | `All` |

+| `button_retry` | Retry | `All` |

+| `alternative_text` | I don't have my phone | `All` |

+| `intro_phone_p` | We have the following numbers on record for you. Choose a number that we can phone to authenticate you. | `All` |

+| `intro_phone` | We have the following number on record for you. We will phone to authenticate you. | `All` |

+| `enter_code_text_intro` | Enter your verification code below, or | `All` |

+| `intro_entry_phone` | Enter a number below that we can phone to authenticate you. | `All` |

+| `intro_entry_sms` | Enter a number below that we can send a code via SMS to authenticate you. | `All` |

+| `button_send_code` | Send Code | `All` |

+| `invalid_number` | Please enter a valid phone number | `All` |

+| `intro_sms` | We have the following number on record for you. We will send a code via SMS to authenticate you. | `All` |

+| `intro_entry_mixed` | Enter a number below that we can send a code via SMS or phone to authenticate you. | `All` |

+| `number_pattern` | `^\\+(?:[0-9][\\x20-]?){6,14}[0-9]$` | `All` |

+| `intro_sms_p` |We have the following numbers on record for you. Choose a number that we can send a code via SMS to authenticate you. | `All` |

+| `requiredField_countryCode` | Please select your country code | `All` |

+| `requiredField_number` | Please enter your phone number | `All` |

+| `country_code_input_placeholder_text` |Country or region | `All` |

+| `number_label` | Phone Number | `All` |

+| `error_tryagain` | The phone number you provided is busy or unavailable. Please check the number and try again. | `All` |

+| `error_sms_throttled` | You hit the limit on the number of text messages. Try again shortly. | `>= 1.2.3` |

+| `error_phone_throttled` | You hit the limit on the number of call attempts. Try again shortly. | `>= 1.2.3` |

+| `error_throttled` | You hit the limit on the number of verification attempts. Try again shortly. | `>= 1.2.3` |

+| `error_incorrect_code` | The verification code you have entered does not match our records. Please try again, or request a new code. | `All` |

+| `countryList` | See [the countries list](#phone-factor-authentication-page-example). | `All` |

+| `error_448` | The phone number you provided is unreachable. | `All` |

+| `error_449` | User has exceeded the number of retry attempts. | `All` |

+| `verification_code_input_placeholder_text` | Verification code | `All` |

+The following IDs are used for a [Verification display control](display-control-verification.md) with [page layout version](page-layout.md) 2.1.0 or higher.

+| `intro_msg` <sup>1</sup>| Verification is necessary. Please click Send button.|

+| `success_send_code_msg` | Verification code has been sent. Please copy it to the input box below.|

+| `failure_send_code_msg` | We are having trouble verifying your email address. Please enter a valid email address and try again.|

+| `success_verify_code_msg` | E-mail address verified. You can now continue.|

+| `failure_verify_code_msg` | We are having trouble verifying your email address. Please try again.|

+| `but_send_code` | Send verification code|

+| `but_verify_code` | Verify code|

+| `but_send_new_code` | Send new code|

+| `but_change_claims` | Change e-mail|

+| `UserMessageIfVerificationControlClaimsNotVerified` <sup>2</sup> | The claims for verification control have not been verified. |

+`.verificationInfoText div{display: block!important}`

+The following IDs are used for a [Verification display control](display-control-verification.md) with [page layout version](page-layout.md) 2.0.0.

+| `verification_control_but_change_claims` |Change |

+| `verification_control_fail_send_code` |Failed to send the code, please try again later. |

+| `verification_control_fail_verify_code` |Failed to verify the code, please try again later. |

+| `verification_control_but_send_code` |Send Code |

+| `verification_control_but_send_new_code` |Send New Code |

+| `verification_control_but_verify_code` |Verify Code |

+| `verification_control_code_sent`| Verification code has been sent. Please copy it to the input box below. |

+The following IDs are used for a [time-based one-time password (TOTP) display control](display-control-time-based-one-time-password.md) with [page layout version](page-layout.md) 2.1.9 and later.

+| `title_text` |Download the Microsoft Authenticator using the download links for iOS and Android or use any other authenticator app of your choice. |

+| `DN` |Once you've downloaded the Authenticator app, you can use any of the methods below to continue with enrollment. |

+| `DisplayName` |Once you've downloaded the Authenticator app, you can use any of the methods below to continue with enrollment. |

+| `title_text` |Scan the QR code |

+| `info_msg` |You can download the Microsoft Authenticator app or use any other authenticator app of your choice. |

+| `link_text` |Can't scan? Try this |

+| `title_text`| Enter the account details manually. |

+| `account_name` | Account Name: |

+| `display_prefix` | Secret |

+| `collapse_text` | Still having trouble? |

+| `DisplayName` | Enter the verification code from your authenticator appΓÇï.|

+| `DisplayName` | Enter your code. |

+| `button_continue` | Verify |

+The following IDs are used for [Restful service technical profile](restful-technical-profile.md) error messages:

+| `DefaultUserMessageIfRequestFailed` | Failed to establish connection to restful service end point. Restful service URL: {0} |

+| `UserMessageIfCircuitOpen` | {0} Restful Service URL: {1} |

+| `UserMessageIfDnsResolutionFailed` | Failed to resolve the hostname of the restful service endpoint. Restful service URL: {0} |

+| `UserMessageIfRequestTimeout` | Failed to establish connection to restful service end point within timeout limit {0} seconds. Restful service URL: {1} |

+The following IDs are used for an [Azure AD MFA technical profile](multi-factor-auth-technical-profile.md) error message:

+| `UserMessageIfCouldntSendSms` | Cannot Send SMS to the phone, please try another phone number. |

+| `UserMessageIfInvalidFormat` | Your phone number is not in a valid format, please correct it and try again.|

+| `UserMessageIfMaxAllowedCodeRetryReached` | Wrong code entered too many times, please try again later.|

+| `UserMessageIfServerError` | Cannot use MFA service, please try again later.|

+| `UserMessageIfThrottled` | Your request has been throttled, please try again later.|

+| `UserMessageIfWrongCodeEntered` |Wrong code entered, please try again.|

+The following IDs are used for [Azure AD SSPR technical profile](aad-sspr-technical-profile.md) error messages:

+|`UserMessageIfChallengeExpired` | The code has expired.|

+|`UserMessageIfInternalError` | The email service has encountered an internal error, please try again later.|

+|`UserMessageIfThrottled` | You have sent too many requests, please try again later.|

+|`UserMessageIfVerificationFailedNoRetry` | You have exceeded maximum number of verification attempts.|

+|`UserMessageIfVerificationFailedRetryAllowed` | The verification has failed, please try again.|

+## One-time password error messages

+The following IDs are used for a [one-time password technical profile](one-time-password-technical-profile.md) error messages

+| `UserMessageIfSessionDoesNotExist` | No | The message to display to the user if the code verification session has expired. It is either the code has expired or the code has never been generated for a given identifier. |

+| `UserMessageIfMaxRetryAttempted` | No | The message to display to the user if they've exceeded the maximum allowed verification attempts. |

+| `UserMessageIfMaxNumberOfCodeGenerated` | No | The message to display to the user if the code generation has exceeded the maximum allowed number of attempts. |

+| `UserMessageIfInvalidCode` | No | The message to display to the user if they've provided an invalid code. |

+| `UserMessageIfVerificationFailedRetryAllowed` | No | The message to display to the user if they've provided an invalid code, and user is allowed to provide the correct code. |

+| `UserMessageIfSessionConflict` | No | The message to display to the user if the code cannot be verified.|

+The following IDs are used for claims transformations error messages:

+| `UserMessageIfClaimsTransformationBooleanValueIsNotEqual` |[AssertBooleanClaimIsEqualToValue](boolean-transformations.md#assertbooleanclaimisequaltovalue) | Boolean claim value comparison failed for claim type "inputClaim".|

+| `DateTimeGreaterThan` |[AssertDateTimeIsGreaterThan](date-transformations.md#assertdatetimeisgreaterthan) | Claim value comparison failed: The provided left operand is greater than the right operand.|

+| `UserMessageIfClaimsTransformationStringsAreNotEqual` |[AssertStringClaimsAreEqual](string-transformations.md#assertstringclaimsareequal) | Claim value comparison failed using StringComparison "OrdinalIgnoreCase".|

+ </RelyingParty>

+ ```yaml

+ version: '3'

+2. After executing `docker-compose -f docker-compose.yml up`, the header-based application should have SSO enabled with Azure AD B2C. Open a browser and type in `http://localhost:9772/`.

+1. Find and copy the entire contents of the **UserJourneys** element that includes `Id=SignUpOrSignIn`.

+ ```xml

+The sign-in flow involves the following steps:

+1. Users go to the [My Apps portal](https://myapps.microsoft.com/) and select your app. The app opens the app sign-in URL.

+1. The app sign-in URL starts an authorization request and redirects users to the Azure AD B2C authorization endpoint.

+Upon successful sign-in, Azure AD returns a token to Azure AD B2C. Azure AD B2C validates and reads the token claims, and then returns a token to your application.

+## Step 2: Set up sign-in for multitenant Azure AD

+In your app, copy the URL of the sign-in endpoint. If you use the [web application sample](configure-authentication-sample-web-app.md), the sign-in URL is `https://localhost:5001/MicrosoftIdentity/Account/SignIn?`. This URL is where the Azure AD app gallery takes users to sign in to your app.

+Finally, add the multitenant app to the Azure AD app gallery. Follow the instructions in [Publish your app to the Azure AD app gallery](../active-directory/manage-apps/v2-howto-app-gallery-listing.md). To add your app to the app gallery, use the following steps:

+ | Application sign in URL|Provide the app sign-in URL as it's configured in [Step 3. Prepare your app](#step-3-prepare-your-app).|

+- Learn how to [Publish your Azure AD app to the Azure AD app gallery](../active-directory/manage-apps/v2-howto-app-gallery-listing.md).

+ Title: Register apps in Azure Active Directory B2C

+description: Learn how to register different apps types such as web app, web API, single-page apps, mobile and desktop apps, daemon apps, Microsoft Graph apps and SAML app in Azure Active Directory B2C

+# Register apps in Azure Active Directory B2C

+Before your [applications](application-types.md) can interact with Azure Active Directory B2C (Azure AD B2C), you must register them in a tenant that you manage.

+Azure AD B2C supports authentication for various modern application architectures. The interaction of every application type with Azure AD B2C is different. Hence, you need to specify the type of app that you want to register.

+## Prerequisites

+- If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.

+- If you haven't already created your own [Azure AD B2C Tenant](tutorial-create-tenant.md), create one now. You can use an existing Azure AD B2C tenant.

+## Select an app type to register

+You can register different app types in your Azure AD B2C Tenant. The how-to guides below show you how to register and configure the various app types:

+- [Single-page application (SPA)](tutorial-register-spa.md)

+- [Web application](tutorial-register-applications.md)

+- [Native client (for mobile and desktop)](add-native-application.md)

+- [Web API](add-web-api-application.md)

+- [Daemon apps](client-credentials-grant-flow.md)

+- [Microsoft Graph application](microsoft-graph-get-started.md)

+- [SAML application](saml-service-provider.md?tabs=windows&pivots=b2c-custom-policy)

+- [Publish app in Azure AD app gallery](publish-app-to-azure-ad-app-gallery.md)

+

+

+

+ }

+ }

+ #region constants

+ /// <summary>

+ /// The AAD authentication endpoint uri

+ /// </summary>

+ static readonly string AadAuthenticationEndpoint = "https://login.microsoftonline.com/common/oauth2/v2.0/authorize";

+ /// <summary>

+ /// The application ID of the connector in AAD

+ /// </summary>

+ static readonly string ConnectorAppId = "55747057-9b5d-4bd4-b387-abf52a8bd489";

+

+ /// <summary>

+ /// The AppIdUri of the registration service in AAD

+ /// </summary>

+ static readonly string RegistrationServiceAppIdUri = "https://proxy.cloudwebappproxy.net/registerapp/user_impersonation";

+ #endregion

+ #region private members

+ private string token;

+ private string tenantID;

+ #endregion

+ public void GetAuthenticationToken()

+ IPublicClientApplication clientApp = PublicClientApplicationBuilder

+ .Create(ConnectorAppId)

+ .WithDefaultRedirectUri() // will automatically use the default Uri for native app

+ .WithAuthority(AadAuthenticationEndpoint)

+ .Build();

+ AuthenticationResult authResult = null;

+ IAccount account = null;

+ IEnumerable<string> scopes = new string[] { RegistrationServiceAppIdUri };

+ try

+ {

+ authResult = await clientApp.AcquireTokenSilent(scopes, account).ExecuteAsync();

+ }

+ catch (MsalUiRequiredException ex)

+ {

+ authResult = await clientApp.AcquireTokenInteractive(scopes).ExecuteAsync();

+ }

+ if (authResult == null || string.IsNullOrEmpty(authResult.AccessToken) || string.IsNullOrEmpty(authResult.TenantId))

+ {

+ Trace.TraceError("Authentication result, token or tenant id returned are null");

+ throw new InvalidOperationException("Authentication result, token or tenant id returned are null");

+ }

+ token = authResult.AccessToken;

+ tenantID = authResult.TenantId;

+ }

+ ```

+ Add-Type -Path "..\MSAL\Microsoft.Identity.Client.dll"

+ # Define the resources and scopes you want to call

+ # Acquiring the token

+ $token = $authResult.AccessToken

+ $tenantId = $authResult.TenantId

+ Write-Output "Success: Authentication result returned."

+ Write-Output "Error: Authentication result, token or tenant id returned with null."

+* [Troubleshoot issues you're having with Application Proxy](application-proxy-troubleshoot.md)

+For example, in response to increasing MFA fatigue attacks, Microsoft recommended ways for customers to [defend users](https://techcommunity.microsoft.com/t5/microsoft-entra-azure-ad-blog/defend-your-users-from-mfa-fatigue-attacks/ba-p/2365677). One recommendation to prevent users from accidental multifactor authentication (MFA) approvals is to enable [number matching](how-to-mfa-number-match.md). As a result, default behavior for number matching will be explicitly **Enabled** for all Microsoft Authenticator users. You can learn more about new security features like number matching in our blog post [Advanced Microsoft Authenticator security features are now generally available!](https://techcommunity.microsoft.com/t5/microsoft-entra-azure-ad-blog/advanced-microsoft-authenticator-security-features-are-now/ba-p/2365673).

+ ```http

+ ```

+Administrators can exclude the entire Office 365 suite or specific Office 365 cloud apps from the Conditional Access policy.

+The following key applications are affected by the Office 365 cloud app:

+> If your organization uses proxy servers that intercept SSL traffic for scenarios like data loss prevention or Azure AD tenant restrictions, ensure that traffic to `https://devices.login.microsoftonline.com` is excluded from TLS break-and-inspect. Failure to exclude this URL may cause interference with client certificate authentication, cause issues with device registration, and device-based Conditional Access.

+All users in your tenant must register for multifactor authentication (MFA) in the form of the Azure AD Multi-Factor Authentication. Users have 14 days to register for Azure AD Multi-Factor Authentication by using the [Microsoft Authenticator app](../authentication/concept-authentication-authenticator-app.md) or any app supporting [OATH TOTP](../authentication/concept-authentication-oath-tokens.md). After the 14 days have passed, the user can't sign in until registration is completed. A user's 14-day period begins after their first successful interactive sign-in after enabling security defaults.

+Security defaults users are required to register for and use Azure AD Multi-Factor Authentication using the [Microsoft Authenticator app using notifications](../authentication/concept-authentication-authenticator-app.md). Users may use verification codes from the Microsoft Authenticator app but can only register using the notification option. Users can also use any third party application using [OATH TOTP](../authentication/concept-authentication-oath-tokens.md) to generate codes.

+ 1. If you select **(Preview) User-to-Group Affiliation**, reviewers will get the recommendation to Approve or Deny access for the users based on userΓÇÖs average distance in the organizationΓÇÖs reporting-structure. Users who are very distant from all the other users within the group are considered to have "low affiliation" and will get a deny recommendation in the group access reviews.

+ ```

+ * If your runbook is only performing queries or updates within a single catalog, then you do not need to assign it tenant-wide application permissions; instead you can assign the service principal to the catalog's **Catalog owner** or **Catalog reader** role.

+ * If your runbook is only performing queries for entitlement management, then it can use the **EntitlementManagement.Read.All** permission.

+ * If your runbook is making changes to entitlement management, for example to create assignments across multiple catalogs, then use the **EntitlementManagement.ReadWrite.All** permission.

+ * For other APIs, ensure that the necessary permission is added. For example, for identity protection, the **IdentityRiskyUser.Read.All** permission should be added.

+1. Select **Grant admin permissions** to give your app those permissions.

+Connect-MgGraph -clientId $ClientId -tenantId $TenantId -certificatethumbprint $Thumbprint

+ [String] $AccessPackageAssignmentId

+- [Create an Automation account using the Azure portal](../../automation/quickstarts/create-azure-automation-account-portal.md)

+## User-to-Group Affiliation (preview)

+Making the review experience easier and more accurate empowers IT admins and reviewers to make more informed decisions. This Machine Learning based recommendation opens the journey to automate access reviews, thereby enabling intelligent automation and reducing access rights attestation fatigue.

+User-to-Group Affiliation in an organizationΓÇÖs chart is defined as two or more users who share similar characteristics in an organization's reporting structure.

+This recommendation detects user affiliation with other users within the group, based on organization's reporting-structure similarity. The recommendation relies on a scoring mechanism which is calculated by computing the userΓÇÖs average distance with the remaining users in the group. Users who are very distant from all the other group members based on their organization's chart, are considered to have "low affiliation" within the group.

+If this decision helper is enabled by the creator of the access review, reviewers can receive User-to-Group Affiliation recommendations for group access reviews.

+> [!NOTE]

+> This feature is only available for users in your directory. A user should have a manager attribute and should be a part of an organizational hierarchy for the User-to-group Affiliation to work.

+The following image has an example of an organization's reporting structure in a cosmetics company:

+

+Based on the reporting structure in the example image, users who are statistically significant amount of distance away from other users within the group, would get a "Deny" recommendation by the system if the User-to-Group Affiliation recommendation was selected by the reviewer for group access reviews.

+For example, Phil who works within the Personal care division is in a group with Debby, Irwin, and Emily who all work within the Cosmetics division. The group is called *Fresh Skin*. If an Access Review for the group Fresh Skin is performed, based on the reporting structure and distance away from the other group members, Phil would be considered to have low affiliation. The system will create a **Deny** recommendation in the group access review.

+- [Review access to groups or applications](perform-access-review.md)

+Some tasks do update existing attributes; however, we donΓÇÖt currently share those specific details. As these tasks are updating attributes related to other Azure AD features, so you can find that info in those docs. For temporary access pass, we're writing to the appropriate attributes listed [here](/graph/api/resources/temporaryaccesspassauthenticationmethod).

+ ```powershell

+ ```

+ ```powershell

+ ```

+ ```powershell

+ Disable-ADSyncExportDeletionThreshold [[-AADCredential] <PSCredential>] [-WhatIf] [-Confirm]

+ ```

+ ```powershell

+ ```

+ ```yaml

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```powershell

+ ```

+ ```

+ ```

+ ```

+```powershell

+#Retrieve the service principal object ID.

+$app_name = "<Your App's display name>"

+$sp = Get-AzureADServicePrincipal -Filter "displayName eq '$app_name'"

+$sp.ObjectId

+ Get-MgServicePrincipal

+ ```

+ ```

+ ```

+ **Microsoft.Compute/virtualMachineScaleSets API version 2018-06-01**

+ ```json

+ {

+ "access_token":"eyJ0eXAiOi...",

+ "expires_in":"3599",

+ "expires_on":"1518503375",

+ "not_before":"1518499475",

+ "resource":"https://management.azure.com/",

+ "token_type":"Bearer",

+ "client_id":"1ef89848-e14b-465f-8780-bf541d325cd5"

+ }

+ ```

+ ```json

+ {

+ "access_token": "eyJ0eXAiOiJ...",

+ "refresh_token": "",

+ "expires_in": "3599",

+ "expires_on": "1504130527",

+ "not_before": "1504126627",

+ "resource": "https://management.azure.com",

+ "token_type": "Bearer"

+ }

+ ```

+>[Manage your storage access keys](../../storage/common/storage-account-create.md)

+ ```json

+ {

+ "access_token":"eyJ0eXAiOiJ...",

+ "refresh_token":"",

+ "expires_in":"3599",

+ "expires_on":"1504130527",

+ "not_before":"1504126627",

+ "resource":"https://management.azure.com",

+ "token_type":"Bearer"

+ }

+ ```

+>[Using shared access signatures (SAS)](../../storage/common/storage-sas-overview.md)

+ Title: User management permissions for Azure AD custom roles (preview) - Azure Active Directory

+description: User management permissions for Azure AD custom roles in the Azure portal, PowerShell, or Microsoft Graph API.

+# User management permissions for Azure AD custom roles (preview)

+> [!IMPORTANT]

+> User management permissions for Azure AD custom roles is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+User management permissions can be used in custom role definitions in Azure Active Directory (Azure AD) to grant fine-grained access such as the following:

+- Read or update basic properties of users

+- Read or update identity of users

+- Read or update job information of users

+- Update contact information of users

+- Update parental controls of users

+- Update settings of users

+- Read direct reports of users

+- Update extension properties of users

+- Read device information of users

+- Read or manage licenses of users

+- Update password policies of users

+- Read assignments and memberships of users

+This article lists the permissions you can use in your custom roles for different user management scenarios. For information about how to create custom roles, see [Create and assign a custom role](custom-create.md).

+## License requirements

+## Read or update basic properties of users

+The following permissions are available to read or update basic properties of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/standard/read | Read basic properties on users. |

+> | microsoft.directory/users/basic/update | Update basic properties on users. |

+## Read or update identity of users

+The following permissions are available to read or update identity of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/identities/read | Read identities of users. |

+> | microsoft.directory/users/identities/update | Update the identity properties of users, such as name and user principal name. |

+## Read or update job information of users

+The following permissions are available to read or update job information of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/manager/read | Read manager of users. |

+> | microsoft.directory/users/manager/update | Update manager for users. |

+> | microsoft.directory/users/jobInfo/update | Update the job info properties of users, such as job title, department, and company name. |

+## Update contact information of users

+The following permissions are available to update contact information of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/contactInfo/update | Update the contact info properties of users, such as address, phone, and email. |

+## Update parental controls of users

+The following permissions are available to update parental controls of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/parentalControls/update | Update parental controls of users. |

+## Update settings of users

+The following permissions are available to update settings of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/usageLocation/update | Update usage location of users. |

+## Read direct reports of users

+The following permissions are available to read direct reports of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/directReports/read | Read the direct reports for users. |

+## Update extension properties of users

+The following permissions are available to update extension properties of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/extensionProperties/update | Update extension properties of users. |

+## Read device information of users

+The following permissions are available to read device information of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/ownedDevices/read | Read owned devices of users |

+> | microsoft.directory/users/registeredDevices/read | Read registered devices of users |

+> | microsoft.directory/users/deviceForResourceAccount/read | Read deviceForResourceAccount of users. |

+## Read or manage licenses of users

+The following permissions are available to read or manage licenses of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/licenseDetails/read | Read license details of users. |

+> | microsoft.directory/users/assignLicense | Manage user licenses. |

+> | microsoft.directory/users/reprocessLicenseAssignment | Reprocess license assignments for users. |

+## Update password policies of users

+The following permissions are available to update password policies of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/passwordPolicies/update | Update password policies properties of users. |

+## Read assignments and memberships of users

+The following permissions are available to read assignments and memberships of users.

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/appRoleAssignments/read | Read application role assignments for users |

+> | microsoft.directory/users/scopedRoleMemberOf/read | Read user's membership of an Azure AD role, that is scoped to an administrative unit |

+> | microsoft.directory/users/memberOf/read | Read the group memberships of users |

+## Full list of permissions

+> [!div class="mx-tableFixed"]

+> | Permission | Description |

+> | - | -- |

+> | microsoft.directory/users/appRoleAssignments/read | Read application role assignments for users. |

+> | microsoft.directory/users/assignLicense | Manage user licenses. |

+> | microsoft.directory/users/basic/update | Update basic properties on users. |

+> | microsoft.directory/users/contactInfo/update | Update the contact info properties of users, such as address, phone, and email. |

+> | microsoft.directory/users/deviceForResourceAccount/read | Read deviceForResourceAccount of users. |

+> | microsoft.directory/users/directReports/read | Read the direct reports for users. |

+> | microsoft.directory/users/extensionProperties/update | Update extension properties of users. |

+> | microsoft.directory/users/identities/read | Read identities of users. |

+> | microsoft.directory/users/identities/update | Update the identity properties of users, such as name and user principal name. |

+> | microsoft.directory/users/jobInfo/update | Update the job info properties of users, such as job title, department, and company name. |

+> | microsoft.directory/users/licenseDetails/read | Read license details of users. |

+> | microsoft.directory/users/manager/read | Read manager of users. |

+> | microsoft.directory/users/manager/update | Update manager for users. |

+> | microsoft.directory/users/memberOf/read | Read the group memberships of users. |

+> | microsoft.directory/users/ownedDevices/read | Read owned devices of users. |

+> | microsoft.directory/users/parentalControls/update | Update parental controls of users. |

+> | microsoft.directory/users/passwordPolicies/update | Update password policies properties of users. |

+> | microsoft.directory/users/registeredDevices/read | Read registered devices of users. |

+> | microsoft.directory/users/reprocessLicenseAssignment | Reprocess license assignments for users. |

+> | microsoft.directory/users/scopedRoleMemberOf/read | Read user's membership of an Azure AD role, that is scoped to an administrative unit. |

+> | microsoft.directory/users/standard/read | Read basic properties on users. |

+> | microsoft.directory/users/usageLocation/update | Update usage location of users. |

+## Next steps

+- [Create and assign a custom role in Azure Active Directory](custom-create.md)

+- [List Azure AD role assignments](view-assignments.md)

+ ```https

+- [What is conditional access in Azure Active Directory?](../conditional-access/overview.md)

+> You may also choose to enable SAML-based single sign-on for Cerner Central, following the instructions provided in [Azure portal](https://portal.azure.com). Single sign-on can be configured independently of automatic provisioning, though these two features complement each other. For more information, see the [Cerner Central single sign-on tutorial](cernercentral-tutorial.md).

+ * You should see a success notification on the upper-right side of your portal.

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md).

+ > [!NOTE]

+ > These values are just patterns. You need to use the actual **Sign on URL**, **Identifier**, **Reply URL**, and **Logout URL** that is configured on the FortiGate.

+ 

+ 

+ 

+ | SP entity ID (`entity-id`) | Identifier (Entity ID) |

+| SP Single Sign-On URL (`single-sign-on-url`) | Reply URL (Assertion Consumer Service URL) |

+ ```console

+ edit azure

+ set cert <FortiGate VPN Server Certificate Name>

+ set entity-id < Identifier (Entity ID)Entity ID>

+ set single-sign-on-url < Reply URL Reply URL>

+ set single-logout-url <Logout URL>

+ set idp-entity-id <Azure AD Identifier>

+ set idp-single-sign-on-url <Azure Login URL>

+ set idp-single-logout-url <Azure Logout URL>

+ set idp-cert <Base64 SAML Certificate Name>

+ set user-name username

+ set group-name group

+ next

+ ```

+ ```console

+ edit FortiGateAccess

+ set member azure

+ config match

+ edit 1

+ set server-name azure

+ set group-name <Object Id>

+ next

+ end

+ next

+ ```

+ > When auto-assign licenses is not enabled, it means that only user data is synced.

+ > When auto-license assignment is enabled, you need to note the application instance and license type. Licenses are assigned on a first come, first serve basis until all the licenses are taken.

+ * You should see a success notification on the upper-right side of

+ Title: 'Tutorial: Configure Uber for automatic user provisioning with Azure Active Directory | Microsoft Docs'

+description: Learn how to automatically provision and de-provision user accounts from Azure AD to Uber.

+documentationcenter: ''

+writer: Thwimmer

+ms.assetid: f16047ee-8ed6-4f8f-86e4-d9bc2cbd9016

+ms.devlang: na

+# Tutorial: Configure Uber for automatic user provisioning

+This tutorial describes the steps you need to perform in both Uber and Azure Active Directory (Azure AD) to configure automatic user provisioning. When configured, Azure AD automatically provisions and de-provisions users to [Uber](https://www.uber.com/) using the Azure AD Provisioning service. For important details on what this service does, how it works, and frequently asked questions, see [Automate user provisioning and deprovisioning to SaaS applications with Azure Active Directory](../app-provisioning/user-provisioning.md).

+## Capabilities supported

+> [!div class="checklist"]

+> * Create users in Uber.

+> * Remove users in Uber when they do not require access anymore.

+> * Keep user attributes synchronized between Azure AD and Uber.

+## Prerequisites

+The scenario outlined in this tutorial assumes that you already have the following prerequisites:

+* [An Azure AD tenant](../develop/quickstart-create-new-tenant.md).

+* A user account in Azure AD with [permission](../roles/permissions-reference.md) to configure provisioning (for example, Application Administrator, Cloud Application administrator, Application Owner, or Global Administrator).

+* You must be onboarded to a [Uber for Business](https://business.uber.com/) organization and have Admin access to it.

+## Step 1. Plan your provisioning deployment

+1. Learn about [how the provisioning service works](../app-provisioning/user-provisioning.md).

+1. Determine who will be in [scope for provisioning](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. Determine what data to [map between Azure AD and Uber](../app-provisioning/customize-application-attributes.md).

+## Step 2. Configure Uber to support provisioning with Azure AD

+Before you start the setup, below are the requirements to enable SCIM provisioning end to end

+* You must be onboarded to a [Uber for Business](https://business.uber.com/) organization and have Admin access to it.

+* You must allow syncing via identity providers, you can find this by hovering your mouse above your profile photo in the top right corner and navigating to **Settings > Integrations section > toggle Allow**

+* Grab your `organization-id` and replace it in `https://api.uber.com/v1/scim/organizations/{organization-id}/v2` to create your **Tenant Url** .This Tenant Url is to be entered in the Provisioning tab of your Uber application in the Azure portal.

+ 

+## Step 3. Add Uber from the Azure AD application gallery

+Add Uber from the Azure AD application gallery to start managing provisioning to Uber. If you have previously setup Uber for SSO, you can use the same application. However it is recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).

+## Step 4. Define who will be in scope for provisioning

+The Azure AD provisioning service allows you to scope who will be provisioned based on assignment to the application and or based on attributes of the user. If you choose to scope who will be provisioned to your app based on assignment, you can use the following [steps](../manage-apps/assign-user-or-group-access-portal.md) to assign users to the application. If you choose to scope who will be provisioned based solely on attributes of the user, you can use a scoping filter as described [here](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* Start small. Test with a small set of users before rolling out to everyone. When scope for provisioning is set to assigned users, you can control this by assigning one or two users to the app. When scope is set to all users, you can specify an [attribute based scoping filter](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+* If you need additional roles, you can [update the application manifest](../develop/howto-add-app-roles-in-azure-ad-apps.md) to add new roles.

+## Step 5. Configure automatic user provisioning to Uber

+This section guides you through the steps to configure the Azure AD provisioning service to create, update, and disable users in Uber based on user assignments in Azure AD.

+### To configure automatic user provisioning for Uber in Azure AD:

+1. Sign in to the [Azure portal](https://portal.azure.com). Select **Enterprise Applications**, then select **All applications**.

+ 

+1. In the applications list, select **Uber**.

+ 

+1. Select the **Provisioning** tab.

+ 

+1. Set the **Provisioning Mode** to **Automatic**.

+ 

+1. In the **Admin Credentials** section, enter the **Tenant Url** and then click on Authorize, make sure that you enter your Uber account's Admin credentials. Click **Test Connection** to ensure Azure AD can connect to Uber. If the connection fails, ensure your Uber account has Admin permissions and try again.

+ 

+1. In the **Notification Email** field, enter the email address of a person who should receive the provisioning error notifications and select the **Send an email notification when a failure occurs** check box.

+ 

+1. Select **Save**.

+1. Under the **Mappings** section, select **Synchronize Azure Active Directory Users to Uber**.

+1. Review the user attributes that are synchronized from Azure AD to Uber in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in Uber for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you will need to ensure that the Uber API supports filtering users based on that attribute. Select the **Save** button to commit any changes.

+ |Attribute|Type|Supported for filtering|Required by Uber|

+ |||||

+ |userName|String|&check;|&check;

+ |active|Boolean||&check;

+ |name.givenName|String||&check;

+ |name.familyName|String||&check;

+ |externalId|String||&check;

+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:employeeNumber|String|||

+

+1. To configure scoping filters, refer to the following instructions provided in the [Scoping filter tutorial](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).

+1. To enable the Azure AD provisioning service for Uber, change the **Provisioning Status** to **On** in the **Settings** section.

+ 

+1. Define the users that you would like to provision to Uber by choosing the desired values in **Scope** in the **Settings** section.

+ 

+1. When you are ready to provision, click **Save**.

+ 

+This operation starts the initial synchronization cycle of all users defined in **Scope** in the **Settings** section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running.

+## Step 6. Monitor your deployment

+Once you've configured provisioning, use the following resources to monitor your deployment:

+* Use the [provisioning logs](../reports-monitoring/concept-provisioning-logs.md) to determine which users have been provisioned successfully or unsuccessfully

+* Check the [progress bar](../app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user.md) to see the status of the provisioning cycle and how close it is to completion

+* If the provisioning configuration seems to be in an unhealthy state, the application will go into quarantine. Learn more about quarantine states [here](../app-provisioning/application-provisioning-quarantine-status.md).

+## More resources

+* [Managing user account provisioning for Enterprise Apps](../app-provisioning/configure-automatic-user-provisioning-portal.md)

+* [What is application access and single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)

+## Next steps

+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)

+The API is protected through Azure Active Directory and uses OAuth2 bearer tokens. The app registration needs to have the API Permission for `Verifiable Credentials Service Admin` and then when acquiring the access token the app should use scope `6a8b4b39-c021-437c-b060-5a14a3fd65f3/full_access`. The access token must be for a user with the [global administrator](../../active-directory/roles/permissions-reference.md#global-administrator) or the [authentication policy administrator](../../active-directory/roles/permissions-reference.md#authentication-policy-administrator) role.

+ "manifest": "https://verifiedid.did.msidentity.com/v1.0/tenants/12345678-0000-0000-0000-000000000000/verifiableCredentials/contracts/MTIzNDU2NzgtMDAwMC0wMDAwLTAwMDAtMDAwMDAwMDAwMDAwdmVyaWZpZWRjcmVkZW50aWFsZXhwZXJ0/manifest",

+ ```

+- Premium SSD v1 and v2 are supported.

+|skuName | Azure Disks storage account type (alias: `storageAccountType`)| `Standard_LRS`, `Premium_LRS`, `StandardSSD_LRS`, `PremiumV2_LRS`, `UltraSSD_LRS`, `Premium_ZRS`, `StandardSSD_ZRS` | No | `StandardSSD_LRS`|

+* Ensure you install [KEDA](/azure/aks/integrations#available-add-ons) as an add-on and [GitOps & Dapr](/azure/aks/cluster-extensions?tabs=azure-cli#currently-available-extensions) as extensions.

+> [!NOTE]

+> To create and use your own VNet and route table with `kubelet` network plugin, you need to use [user-assigned control plane identity][bring-your-own-control-plane-managed-identity]. For system-assigned control plane identity, the identity ID cannot be retrieved before creating a cluster, which causes a delay during role assignment.

+> To create and use your own VNet and route table with `azure` network plugin, both system-assigned and user-assigned managed identities are supported. But user-assigned managed identity is more recommended for BYO scenarios.

+After creating a custom route table and associating it with a subnet in your virtual network, you can create a new AKS cluster specifying your route table with a user-assigned managed identity.

+az aks create -g myResourceGroup -n myManagedCluster --vnet-subnet-id mySubnetIDResourceID --enable-managed-identity --assign-identity controlPlaneIdentityResourceID

+[bring-your-own-control-plane-managed-identity]: ../aks/use-managed-identity.md#bring-your-own-control-plane-managed-identity

+[Prometheus](https://aka.ms/azureprometheus-promio) and [Grafana](https://aka.ms/azureprometheus-promio-grafana) are CNCF backed widely popular open source tools for kubernetes monitoring. AKS exposes many metrics in Prometheus format which makes Prometheus a popular choice for monitoring. [Container insights](../azure-monitor/containers/container-insights-overview.md) has native integration with AKS, collecting critical metrics and logs, alerting on identified issues, and providing visualization with workbooks. It also collects certain Prometheus metrics, and many native Azure Monitor Insights are built-up on top of Prometheus metrics. Container insights complements and completes E2E monitoring of AKS including log collection which Prometheus as stand-alone tool doesnΓÇÖt provide. Many customers use Prometheus integration and Azure Monitor together for E2E monitoring.

+[az-provider-register]: /cli/azure/provider#az-provider-register

+[az-feature-list]: /cli/azure/feature#az-feature-list

+| [Deprecated (legacy) portal retirement][devportal2023] | October 31, 2023 |

+ * In `backend-id`, substitute the name of your Service Fabric backend.

+ * The `sf-resolve-condition` is a condition for re-resolving a service location and resending a request. The number of retries was set when configuring the backend. For example:

+ ```

+ az role assignment create --role contributor --subscription $subscriptionId --assignee-object-id $assigneeObjectId --assignee-principal-type ServicePrincipal --scopes /subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Web/sites/

+|Migrate cannot be called on this ASE until the active upgrade has finished. |App Service Environments can't be migrated during platform upgrades. You can set your [upgrade preference](how-to-upgrade-preference.md) from the Azure portal. In some cases, an upgrade will be initiated when visiting the migration page if your App Service Environment isn't on the current build. |Wait until the upgrade finishes and then migrate. |

+> [Custom domain suffix](./how-to-custom-domain-suffix.md)

+Replace the *server-name* placeholder with a unique SQL Database name. The SQL Database name is used as part of the globally unique SQL Database endpoint. Also, replace *db-username* and *db-password* with a username and password of your choice.

+You'll be billed only for the resources pre-provisioned and utilized based on actual hourly consumption. Costs associated with Application Gateway are classified into two components: fixed costs and variable costs. Actual costs within each component will vary according to the SKU being utilized.

+This article describes the costs associated with each SKU and it's recommended that users utilize this document for planning and managing costs associated with the Azure Application Gateway.

+* WAF_v2 - Each compute unit can support approximately 10 concurrent requests per second for 70-30% mix of traffic with 70% requests less than 2 KB GET/POST and remaining higher. WAF performance isn't affected by response size currently.

+ The number of instances running at any point of time isn't considered as a factor for fixed costs for V2 SKUs. The fixed costs of running a Standard_V2 (or WAF_V2) would be same per hour regardless of the number of instances running within the same Azure region.

+Variable Costs = $0.008 * 8 (Instance Units) * 10 (capacity units) * 730 (Hours) = $467.2

+Variable Costs = $0.008 * ( 3 (Instance Units) * 10 (capacity units) + 10 (additional capacity units) ) * 730 (Hours) = $233.6

+Variable Costs = $0.008 * ( 3(Instance Units) * 10 (capacity units) + 7 (additional capacity units) ) * 730 (Hours) = $216.08

+LetΓÇÖs assume youΓÇÖve provisioned a WAF_V2 with autoscaling enabled and set the minimum instance count to 6 for the entire month. The request load has caused the WAF instance to scale out and utilize 65 Capacity units (scale out of 5 capacity units, while 60 units were reserved) for the entire month.

+Variable Costs = $0.0144 * 65 (capacity units) * 730 (Hours) = $683.28

+Variable Costs = $0.0144 * 1 (capacity units) * 730 (Hours) = $10.512

+LetΓÇÖs assume youΓÇÖve provisioned a WAF_V2 with autoscaling enabled and set the minimum instance count to 0 for the entire month. However, there's 0 traffic directed to the WAF instance for the entire month.

+Variable Costs = $0.0144 * 0 (capacity units) * 730 (Hours) = $0

+### Example 3 (c) ΓÇô WAF_V2 instance with manual scaling set to 1 instance

+LetΓÇÖs assume youΓÇÖve provisioned a WAF_V2 and set it to manual scaling with the minimum acceptable value of 1 instance for the entire month. However, there's 0 traffic directed to the WAF for the entire month.

+Variable Costs = $0.0144 * 1 (Instance count) * 10 (capacity units) * 730 (Hours) =

+### Example 5 ΓÇô Standard_V2 with Autoscaling, time-based calculations

+Variable Costs = $0.008 * 10 (capacity units) * 1 (Hours) + $0.008 * 20 (capacity

+### Example 6 ΓÇô WAF_V2 with DDoS Protection Standard Plan, and with manual scaling set to 2 instance

+LetΓÇÖs assume youΓÇÖve provisioned a WAF_V2 and set it to manual scaling with 2 instance for the entire month with 2 CUs. Let's also assume that you've enabled DDoS Protection Standard Plan. In this example, since you're paying the monthly fee for DDoS Protection Standard, there's no additional charges for WAF; and you're charged at the lower Standard_V2 rates.

+Monthly price estimates are based on 730 hours of usage per month.

+Fixed Price = $0.246 * 730 (Hours) = $179.58

+Variable Costs = $0.008 * 2 (capacity units) * 730 (Hours) = $11.68

+DDoS Protection Standard Cost = $2,944 * 1 (month) = $2,944

+Total Costs = $179.58 + $11.68 + $2,944 = $3,135.26

+Fixed Price = $0.448 * 360 (Hours) = $161.28

+Variable Costs = 60 * 1000 * $0.0035/GB = $210 (Large tier has no costs for the first 40 TB processed per month)

+### Example 3 ΓÇô WAF Application Gateway with DDoS Protection Standard Plan

+Let's assume you've provisioned a medium type WAF application Gateway, and you've enabled DDoS Protection Standard Plan. This medium WAF application gateway processes 40 TB in the duration that it is active. Your Application Gateway costs using the pricing method above would be calculated as follows:

+Monthly price estimates are based on 730 hours of usage per month.

+Fixed Price = $0.07 * 730 (Hours) = $51.1

+Variable Costs = 30 * 1000 * $0.007/GB = $210 (Medium tier has no cost for the first 10 TB processed per month)

+DDoS Protection Standard Costs = $2,944 * 1 (month) = $2,944

+Total Costs = $3,507.08

+## Azure DDoS Protection Standard Plan

+When Azure DDoS Protection Standard Plan is enabled on your application gateway with WAF you'll be billed at the lower non-WAF rates. Please see [Azure DDoS Protection pricing](https://azure.microsoft.com/pricing/details/ddos-protection/) for more details.

+* Capacity Units calculated = max (17.38, 10.96/2.22, 123.08k/2500) = 49.232

+ Title: Business card data extraction - Form Recognizer

+description: OCR and machine learning based business card scanning in Form Recognizer extracts key data from business cards.

+# Business card data extraction

+## How business card data extraction works

+Business cards are a great way of representing a business or a professional. The company logo, fonts and background images found in business cards help the company branding and differentiate it from others. Applying OCR and machine-learning based techniques to automate scanning of business cards is a common image processing scenario. Enterprise systems used by sales and marketing teams typically have business card data extraction capability integrated into them for the benefit of their users.

+## Form Recognizer Business Card model

+### Try business card data extraction

+ Title: Composed custom models - Form Recognizer

+description: Compose several custom models into a single model for easier data extraction from groups of distinct form types.

+ Title: Custom neural document model - Form Recognizer

+description: Use the custom neural document model to train a model to extract data from structured, semistructured, and unstructured documents.

+# Custom neural document model

+Custom neural document models or neural models are a deep learned model type that combines layout and language features to accurately extract labeled fields from documents. The base custom neural model is trained on various document types that makes it suitable to be trained for extracting fields from structured, semi-structured and unstructured documents. The table below lists common document types for each category:

+ Title: Custom template document model - Form Recognizer

+description: Use the custom template document model to train a model to extract data from structured or templated forms.

+# Custom template document model

+Custom template (formerly custom form) is an easy-to-train document model that accurately extracts labeled key-value pairs, selection marks, tables, regions, and signatures from documents. Template models use layout cues to extract values from documents and are suitable to extract fields from highly structured documents with defined visual templates.

+ Title: Custom document models - Form Recognizer

+description: Label and train customized models for your documents and compose multiple models into a single model identifier.

+# Custom document models

+## Custom document model types

+Custom document models can be one of two types, [**custom template**](concept-custom-template.md ) or custom form and [**custom neural**](concept-custom-neural.md) or custom document models. The labeling and training process for both models is identical, but the models differ as follows:

+### Try building a custom model

+## Custom model extraction summary

+ Title: Identity document (ID) processing ΓÇô Form Recognizer

+description: Automate identity document (ID) processing of driver licenses, passports, and more with Form Recognizer.

+<!-- markdownlint-disable MD033 -->

+# Identity document (ID) processing

+## What is identity document (ID) processing

+Identity document (ID) processing involves extraction of data from identity documents whether manually or using OCR based techniques. Examples of identity documents include passports, driver licenses, resident cards, and national identity cards like the social security card in the US. It is an important step in any business process that requires some proof of identity. Examples include customer verification in banks and other financial institutions, mortgage applications, medical visits, claim processing, hospitality industry, and more. Individuals provide some proof of their identity via driver licenses, passports, and other similar documents so that the business can efficiently verify them before providing services and benefits.

+## Form Recognizer Identity document (ID) model

+The Form Recognizer Identity document (ID) model combines Optical Character Recognition (OCR) with deep learning models to analyze and extract key information from identity documents: US Drivers Licenses (all 50 states and District of Columbia), international passport biographical pages, US state IDs, social security cards, and permanent resident cards and more. The API analyzes identity documents, extracts key information, and returns a structured JSON data representation.

+### Try Identity document (ID) extraction

+|ID document| <ul><li>English (United States)ΓÇöen-US (driver's license)</li><li>Biographical pages from international passports</br> (excluding visa and other travel documents)</li><li>English (United States)ΓÇöen-US (state ID)</li><li>English (United States)ΓÇöen-US (social security card)</li><li>English (United States)ΓÇöen-US (Residence permit card)</li></ul></br>|English (United States)ΓÇöen-US|

+Below are the fields extracted per document type. The Azure Form Recognizer ID model `prebuilt-idDocument` extracts the below fields in the `documents.*.fields`. It also extracts all the text in the documents, words, lines and styles which will be included in the JSON output in the different sections.

+ * `pages.*.words`

+ * `pages.*.lines`

+ * `paragraphs`

+ * `styles`

+ * `documents`

+ * `documents.*.fields`

+#### Document type - `idDocument.driverLicense` fields extracted:

+| Field | Type | Description | Example |

+|:|:--|:|:--|

+|`CountryRegion`|`countryRegion`|Country or region code|USA|

+|`Region`|`string`|State or province|Washington|

+|`DocumentNumber`|`string`|Driver license number|WDLABCD456DG|

+|`DocumentDiscriminator`|`string`|Driver license document discriminator|12645646464554646456464544|

+|`FirstName`|`string`|Given name and middle initial if applicable|LIAM R.|

+|`LastName`|`string`|Surname|TALBOT|

+|`Address`|`address`|Address|123 STREET ADDRESS YOUR CITY WA 99999-1234|

+|`DateOfBirth`|`date`|Date of birth|01/06/1958|

+|`DateOfExpiration`|`date`|Date of expiration|08/12/2020|

+|`DateOfIssue`|`date`|Date of issue|08/12/2012|

+|`EyeColor`|`string`|Eye color|BLU|

+|`HairColor`|`string`|Hair color|BRO|

+|`Height`|`string`|Height|5'11"|

+|`Weight`|`string`|Weight|185LB|

+|`Sex`|`string`|Sex|M|

+|`Endorsements`|`string`|Endorsements|L|

+|`Restrictions`|`string`|Restrictions|B|

+|`VehicleClassifications`|`string`|Vehicle classification|D|

+#### Document type - `idDocument.passport` fields extracted:

+| Field | Type | Description | Example |

+|:|:--|:|:--|

+|`DocumentNumber`|`string`|Passport number|340020013|

+|`FirstName`|`string`|Given name and middle initial if applicable|JENNIFER|

+|`MiddleName`|`string`|Name between given name and surname|REYES|

+|`LastName`|`string`|Surname|BROOKS|

+|`Aliases`|`array`|||

+|`Aliases.*`|`string`|Also known as|MAY LIN|

+|`DateOfBirth`|`date`|Date of birth|1980-01-01|

+|`DateOfExpiration`|`date`|Date of expiration|2019-05-05|

+|`DateOfIssue`|`date`|Date of issue|2014-05-06|

+|`Sex`|`string`|Sex|F|

+|`CountryRegion`|`countryRegion`|Issuing country or organization|USA|

+|`DocumentType`|`string`|Document type|P|

+|`Nationality`|`countryRegion`|Nationality|USA|

+|`PlaceOfBirth`|`string`|Place of birth|MASSACHUSETTS, U.S.A.|

+|`PlaceOfIssue`|`string`|Place of issue|LA PAZ|

+|`IssuingAuthority`|`string`|Issuing authority|United States Department of State|

+|`PersonalNumber`|`string`|Personal Id. No.|A234567893|

+|`MachineReadableZone`|`object`|Machine readable zone (MRZ)|P<USABROOKS<<JENNIFER<<<<<<<<<<<<<<<<<<<<<<< 3400200135USA8001014F1905054710000307<715816|

+|`MachineReadableZone.FirstName`|`string`|Given name and middle initial if applicable|JENNIFER|

+|`MachineReadableZone.LastName`|`string`|Surname|BROOKS|

+|`MachineReadableZone.DocumentNumber`|`string`|Passport number|340020013|

+|`MachineReadableZone.CountryRegion`|`countryRegion`|Issuing country or organization|USA|

+|`MachineReadableZone.Nationality`|`countryRegion`|Nationality|USA|

+|`MachineReadableZone.DateOfBirth`|`date`|Date of birth|1980-01-01|

+|`MachineReadableZone.DateOfExpiration`|`date`|Date of expiration|2019-05-05|

+|`MachineReadableZone.Sex`|`string`|Sex|F|

+#### Document type - `idDocument.nationalIdentityCard` fields extracted:

+| Field | Type | Description | Example |

+|:|:--|:|:--|

+|`CountryRegion`|`countryRegion`|Country or region code|USA|

+|`Region`|`string`|State or province|Washington|

+|`DocumentNumber`|`string`|National identity card number|WDLABCD456DG|

+|`DocumentDiscriminator`|`string`|National identity card document discriminator|12645646464554646456464544|

+|`FirstName`|`string`|Given name and middle initial if applicable|LIAM R.|

+|`LastName`|`string`|Surname|TALBOT|

+|`Address`|`address`|Address|123 STREET ADDRESS YOUR CITY WA 99999-1234|

+|`DateOfBirth`|`date`|Date of birth|01/06/1958|

+|`DateOfExpiration`|`date`|Date of expiration|08/12/2020|

+|`DateOfIssue`|`date`|Date of issue|08/12/2012|

+|`EyeColor`|`string`|Eye color|BLU|

+|`HairColor`|`string`|Hair color|BRO|

+|`Height`|`string`|Height|5'11"|

+|`Weight`|`string`|Weight|185LB|

+|`Sex`|`string`|Sex|M|

+#### Document type - `idDocument.residencePermit` fields extracted:

+| Field | Type | Description | Example |

+|:|:--|:|:--|

+|`CountryRegion`|`countryRegion`|Country or region code|USA|

+|`DocumentNumber`|`string`|Residence permit number|WDLABCD456DG|

+|`FirstName`|`string`|Given name and middle initial if applicable|LIAM R.|

+|`LastName`|`string`|Surname|TALBOT|

+|`DateOfBirth`|`date`|Date of birth|01/06/1958|

+|`DateOfExpiration`|`date`|Date of expiration|08/12/2020|

+|`DateOfIssue`|`date`|Date of issue|08/12/2012|

+|`Sex`|`string`|Sex|M|

+|`PlaceOfBirth`|`string`|Place of birth|Germany|

+|`Category`|`string`|Permit category|DV2|

+#### Document type - `idDocument.usSocialSecurityCard` fields extracted:

+| Field | Type | Description | Example |

+|:|:--|:|:--|

+|`DocumentNumber`|`string`|Social security card number|WDLABCD456DG|

+|`FirstName`|`string`|Given name and middle initial if applicable|LIAM R.|

+|`LastName`|`string`|Surname|TALBOT|

+|`DateOfIssue`|`date`|Date of issue|08/12/2012|

+* Try the prebuilt ID model in the [Form Recognizer Studio](https://formrecognizer.appliedai.azure.com/studio/prebuilt?formType=idDocument). Use the sample documents or bring your own documents.

+ Title: Invoice data extraction ΓÇô Form Recognizer

+description: Automate invoice data extraction with Form RecognizerΓÇÖs invoice model to extract accounts payable data including invoice line items.

+## What is automated invoice processing?

+Automated invoice processing is the process of extracting key accounts payable fields from including invoice line items from invoices and integrating it with your accounts payable (AP) workflows for reviews and payments. Historically, the accounts payable process has been very manual and time consuming. Accurate extraction of key data from invoices is typically the first and one of the most critical steps in the invoice automation process.

+## Form Recognizer Invoice model

+The machine learning based invoice model combines powerful Optical Character Recognition (OCR) capabilities with invoice understanding models to analyze and extract key fields and line items from sales invoices. Invoices can be of various formats and quality including phone-captured images, scanned documents, and digital PDFs. The API analyzes invoice text; extracts key information such as customer name, billing address, due date, and amount due; and returns a structured JSON data representation. The model currently supports both English and Spanish invoices.

+### Try invoice data extraction

+ Title: Document layout analysis - Form Recognizer

+description: Extract text, tables, selections, titles, section headings, page headers, page footers, and more with layout analysis model from Form Recognizer.

+# Document layout analysis

+## What is document layout analysis?

+Document structure and layout analysis is the process of analyzing a document to extract regions of interest and their inter-relationships. The goal is to extract text and structural elements from the page for building better semantic understanding models. For all extracted text, there are two types of roles that text plays in a document layout. Text, tables, and selection marks are examples of geometric roles. Titles, headings, and footers are examples of logical roles. For example. a reading system requires differentiating text regions from non-textual ones along with their reading order.

+The following illustration shows the typical components in an image of a sample page.

+## Form Recognizer Layout model

+The Form Recognizer Layout is an advanced machine-learning based document layout analysis model available in the Form Recognizer cloud API. In the version v2.1, the document layout model extracted text lines, words, tables, and selection marks.

+**Starting with v3.0 GA**, it extracts paragraphs and additional structure information like titles, section headings, page header, page footer, page number, and footnote from the document page. These are examples of logical roles described in the previous section. This capability is supported for PDF documents and images (JPG, PNG, BMP, TIFF).

+| **Model** | **Text** | **Selection Marks** | **Tables** | **Paragraphs** | **Logical roles** |

+**Supported logical roles for paragraphs**:

+## Try document layout analysis

+### Paragraph extraction ^🆕

+The new machine-learning based page object detection extracts logical roles like titles, section headings, page headers, page footers, and more. The Form Recognizer Layout model assigns certain text blocks in the `paragraphs` collection with their specialized role or type predicted by the model. They're best used with unstructured documents to help understand the layout of the extracted content for a richer semantic analysis. The following paragraph roles are supported:

+### Pages extraction

+### Text lines and words extraction

+The document layout model in Form Recognizer extracts print and handwritten style text as `lines` and `words`. The model outputs bounding `polygon` coordinates and `confidence` for the extracted words. The `styles` collection includes any handwritten style for lines if detected along with the spans pointing to the associated text. This feature applies to [supported handwritten languages](language-support.md).

+### Selection marks extraction

+The Layout model also extracts selection marks from documents. Extracted selection marks appear within the `pages` collection for each page. They include the bounding `polygon`, `confidence`, and selection `state` (`selected/unselected`). Any associated text if extracted is also included as the starting index (`offset`) and `length` that references the top level `content` property that contains the full text from the document.

+### Extract tables from documents and images

+Extracting tables is a key requirement for processing documents containing large volumes of data typically formatted as tables. The Layout model extracts tables in the `pageResults` section of the JSON output. Extracted table information includes the number of columns and rows, row span, and column span. Each cell with its bounding polygon is output along with information whether it's recognized as a `columnHeader` or not. The model supports extracting tables that are rotated. Each table cell contains the row and column index and bounding polygon coordinates. For the cell text, the model outputs the `span` information containing the starting index (`offset`). The model also outputs the `length` within the top-level content that contains the full text from the document.

+```

+### Handwritten style for text lines (Latin languages only)

+The response includes classifying whether each text line is of handwriting style or not, along with a confidence score. This feature is only supported for Latin languages. The following example shows an example JSON snippet.

+```json

+"styles": [

+{

+ "confidence": 0.95,

+ "spans": [

+ {

+ "offset": 509,

+ "length": 24

+ }

+ "isHandwritten": true

+ ]

+}

+### Extracts selected pages from documents

+ Title: Document processing models - Form Recognizer

+description: Document processing models for OCR, document layout, invoices, identity, custom models, and more to extract text, structure, and key-value pairs.

+# Document processing models

+|**Document analysis models**||

+| [Read OCR](#read-ocr) | Extract print and handwritten text including words, locations, and detected languages.|

+| [Layout analysis](#layout-analysis) | Extract text and document layout elements like tables, selection marks, titles, section headings, and more.|

+| [General document](#general-document) | Extract key-value pairs in addition to text and document structure information.|

+|**Prebuilt models**||

+| [W-2](#w-2) | Process W2 forms to extract employee, employer, wage, and other information. |

+| [Invoice](#invoice) | Automate invoice processing for English and Spanish invoices. |

+| [Receipt](#receipt) | Extract receipt data from English receipts.|

+| [Identity document (ID)](#identity-document-id) | Extract identity (ID) fields from US driver licenses and international passports. |

+| [Business card](#business-card) | Scan business cards to extract key fields and data into your applications. |

+|**Custom models**||

+| [Custom models](#custom-models) | Extract data from forms and documents specific to your business. Custom models are trained for your distinct data and use cases. |

+| [Composed models](#composed-models) | Combine several custom models into a single model to automate processing of diverse document types with a single composed model.

+### Read OCR

+### Layout analysis

+[:::image type="icon" source="media/studio/layout.png":::](https://formrecognizer.appliedai.azure.com/studio/layout)

+The Layout analysis model analyzes and extracts text, tables, selection marks, and other structure elements like titles, section headings, page headers, page footers, and more.

+***Sample document processed using the [Form Recognizer Studio](https://formrecognizer.appliedai.azure.com/studio/layout)***:

+>

+> [Learn more: layout model](concept-layout.md)

+The general document model is ideal for extracting common key-value pairs from forms and documents. ItΓÇÖs a pre-trained model and can be directly invoked via the REST API and the SDKs. You can use the general document model as an alternative to training a custom model.

+### W-2

+[:::image type="icon" source="media/studio/w2.png":::](https://formrecognizer.appliedai.azure.com/studio/prebuilt?formType=tax.us.w2)

+The W-2 form model extracts key information reported in each box on a W-2 form. The model supports standard and customized forms from 2018 to the present, including single and multiple forms on one page.

+***Sample W-2 document processed using [Form Recognizer Studio](https://formrecognizer.appliedai.azure.com/studio/prebuilt?formType=tax.us.w2)***:

+> [Learn more: W-2 model](concept-w2.md)

+The invoice model automates processing of invoices to extracts customer name, billing address, due date, and amount due, line items and other key data. Currently, the model supports English, Spanish, German, French, Italian, Portuguese, and Dutch invoices.

+Use the receipt model to scan sales receipts for merchant name, dates, line items, quantities, and totals from printed and handwritten receipts. The version v3.0 also supports single-page hotel receipt processing.

+### Identity document (ID)

+Use the Identity document (ID) model to process U.S. Driver's Licenses (all 50 states and District of Columbia) and biographical pages from international passports (excluding visa and other travel documents) to extract key fields.

+Use the business card model to scan and extract key information from business card images.

+### Custom models

+Custom document models analyze and extract data from forms and documents specific to your business. They are trained to recognize form fields within your distinct content and extract key-value pairs and table data. You only need five examples of the same form type to get started.

+Version v3.0 custom model supports signature detection in custom forms (template model) and cross-page tables in both template and neural models.

+#### Composed models

+A composed model is created by taking a collection of custom models and assigning them to a single model built from your form types. You can assign multiple custom models to a composed model called with a single model ID. You can assign up to 100 trained custom models to a single composed model.

+| **Model ID** | **Text extraction** | **Language detection** | **Selection Marks** | **Tables** | **Paragraphs** | **Structure** | **Key-Value pairs** | **Fields** |

+ Title: OCR for documents - Form Recognizer

+description: Extract print and handwritten text from scanned and digital documents with Form RecognizerΓÇÖs Read OCR model.

+# OCR for documents

+> [!NOTE]

+>

+> For general, in-the-wild images like labels, street signs, and posters, use the [Computer Vision v4.0 preview Read](../../cognitive-services/Computer-vision/concept-ocr.md) feature optimized for general, non-document images with a performance-enhanced synchronous API that makes it easier to embed OCR in your user experience scenarios.

+>

+## What is OCR for documents?

+Optical Character Recognition (OCR) for documents is optimized for large text-heavy documents in multiple file formats and global languages. It should include features like higher-resolution scanning of document images for better handling of smaller and dense text, paragraphs detection, handling fillable forms, and advanced forms and document scenarios like single character boxes and accurate extraction of key fields commonly found in invoices, receipts, and other prebuilt scenarios.

+## Form Recognizer Read model

+Form Recognizer v3.0ΓÇÖs Read Optical Character Recognition (OCR) model runs at a higher resolution than Computer Vision Read and extracts print and handwritten text from PDF documents and scanned images. It also includes preview support for extracting text from Microsoft Word, Excel, PowerPoint, and HTML documents. It detects paragraphs, text lines, words, locations, and languages, and is the underlying OCR engine for other Form Recognizer models like Layout, General Document, Invoice, Receipt, Identity (ID) document, and other prebuilt models, as well as custom models.

+## Try OCR in Form Recognizer

+### Microsoft Office and HTML text extraction (preview) ^🆕

+Use the parameter `api-version=2022-06-30-preview` when using the REST API or the corresponding SDKs of that API version to preview text extraction from Microsoft Word, Excel, PowerPoint, and HTML files. The following illustration shows extraction of the digital text as well as text from the images embedded in the Word document by running OCR on the images.

+The page units in the model output are computed as shown:

+ **File format** | **Computed page unit** | **Total pages** |

+| | | |

+|Word (preview) | Up to 3,000 characters = 1 page unit, Each embedded image = 1 page unit | Total pages of up to 3,000 characters each + Total embedded images |

+|Excel (preview) | Each worksheet = 1 page unit, Each embedded image = 1 page unit | Total worksheets + Total images

+|PowerPoint (preview)| Each slide = 1 page unit, Each embedded image = 1 page unit | Total slides + Total images

+|HTML (preview)| Up to 3,000 characters = 1 page unit, embedded or linked images not supported | Total pages of up to 3,000 characters each |

+### Paragraphs extraction ^🆕

+The Read OCR model in Form Recognizer extracts all identified blocks of text in the `paragraphs` collection as a top level object under `analyzeResults`. Each entry in this collection represents a text block and includes the extracted text as`content`and the bounding `polygon` coordinates. The `span` information points to the text fragment within the top-level `content` property that contains the full text from the document.

+The Read OCR model in Form Recognizer adds [language detection](language-support.md#detected-languages-read-api) as a new feature for text lines. Read will predict the detected primary language for each text line along with the `confidence` in the `languages` collection under `analyzeResult`.

+### Extracting pages from documents

+### Extract text lines and words

+The Read OCR model extracts print and handwritten style text as `lines` and `words`. The model outputs bounding `polygon` coordinates and `confidence` for the extracted words. The `styles` collection includes any handwritten style for lines if detected along with the spans pointing to the associated text. This feature applies to [supported handwritten languages](language-support.md).

+### Handwritten style for text lines (Latin languages only)

+The response includes classifying whether each text line is of handwriting style or not, along with a confidence score. This feature is only supported for Latin languages. The following example shows an example JSON snippet.

+```json

+"styles": [

+{

+ "confidence": 0.95,

+ "spans": [

+ {

+ "offset": 509,

+ "length": 24

+ }

+ "isHandwritten": true

+ ]

+}

+```

+ Title: Receipt data extraction - Form Recognizer

+description: Use machine learning powered receipt data extraction model to digitize receipts.

+# Receipt data extraction

+## What is receipt digitization

+Receipt digitization is the process of converting scanned receipts into digital form for downstream processing. OCR powered receipt data extraction helps to automate the conversion and save time and effort. The output from the receipt data extraction is used for accounts payable and receivables automation, sales data analytics, and other business scenarios.

+## Form Recognizer receipt model

+The Form Recognizer receipt model combines powerful Optical Character Recognition (OCR) capabilities with deep learning models to analyze and extract key information from sales receipts. Receipts can be of various formats and quality including printed and handwritten receipts. The API extracts key information such as merchant name, merchant phone number, transaction date, tax, and transaction total and returns structured JSON data.

+### Try receipt data extraction

+ Title: Automated W-2 form processing - Form Recognizer

+description: Use the Form Recognizer prebuilt W-2 model to automate extraction of W2 form data.

+# Automated W-2 form processing

+## Why is automated W-2 form processing important?

+Form W-2, also known as the Wage and Tax Statement, is sent by an employer to each employee and the Internal Revenue Service (IRS) at the end of the year. A W-2 form reports employees' annual wages and the amount of taxes withheld from their paychecks. The IRS also uses W-2 forms to track individuals' tax obligations. The Social Security Administration (SSA) uses the information on this and other forms to compute the Social Security benefits for all workers.

+## Form Recognizer W-2 form model

+### Try W-2 form data extraction

+```json

+ "targetResourceId": "/subscriptions/{targetSub}/resourceGroups/{targetRG}/providers/Microsoft.CognitiveServices/accounts/{targetService}",

+ "targetResourceRegion": "region",

+ "targetModelId": "target-model-name",

+ "targetModelLocation": "model path",

+ "accessToken": "access token",

+ "expirationDateTime": "timestamp"

+ "targetResourceId": "/subscriptions/{targetSub}/resourceGroups/{targetRG}/providers/Microsoft.CognitiveServices/accounts/{targetService}",

+ "targetResourceRegion": "region",

+ "targetModelId": "target-model-name",

+ "targetModelLocation": "model path",

+ "accessToken": "access token",

+ "expirationDateTime": "timestamp"

+**Request**

+```bash

+curl -i -X POST "{YOUR-ENDPOINT}formrecognizer/documentModels:authorizeCopy?api-version=2022-08-31"

+-H "Content-Type: application/json"

+-H "Ocp-Apim-Subscription-Key: {YOUR-KEY}"

+--data-ascii "{

+ 'modelId': '{modelId}',

+ 'description': '{description}'

+}"

+```

+**Successful response**

+```json

+{

+}

+```

+**Request**

+```bash

+curl -i -X POST "{YOUR-ENDPOINT}/formrecognizer/documentModels/{modelId}:copyTo?api-version=2022-08-31"

+```

+**Successful response**

+```http

+HTTP/1.1 202 Accepted

+Operation-Location: https://{source-resource}.cognitiveservices.azure.com/formrecognizer/operations/{operation-id}?api-version=2022-08-31

+```

+ Title: Intelligent document processing - Form Recognizer

+description: Machine-learning based OCR and document understanding service to automate extraction of text, table and structure, and key-value pairs from your forms and documents.

+# What is Intelligent Document Processing?

+Intelligent Document Processing (IDP) refers to capturing, transforming, and processing data from documents (e.g., PDF, or scanned documents including Microsoft Office and HTML documents). It typically uses advanced machine-learning based technologies like computer vision, Optical Character Recognition (OCR), document layout analysis, and Natural Language Processing (NLP) to extract meaningful information, process and integrate with other systems.

+IDP solutions can extract data from structured documents with pre-defined layouts like a tax form, unstructured or free-form documents like a contract, and semi-structured documents. They have a wide variety of benefits spanning knowledge mining, business process automation, and industry-specific applications. Examples include invoice processing, medical claims processing, and contracts workflow automation.

+## What is Azure Form Recognizer?

+Azure Form Recognizer is a cloud-based [Azure Applied AI Service](../../applied-ai-services/index.yml) for developers to build intelligent document processing solutions. Form Recognizer applies machine-learning based optical character recognition (OCR) and document understanding technologies to extract print and handwritten text, tables, structure, and key-value pairs from documents. You can also label and train custom models to automate data extraction from structured, semi-structured, and unstructured documents. To learn more about each model, *see* the Concepts articles:

+|**Document analysis models**| &#9679; [**Read OCR model**](concept-read.md)</br> &#9679; [**General document model**](concept-general-document.md)</br> &#9679; [**Layout analysis model**](concept-layout.md) </br> |

+| **Prebuilt models** | &#9679; [**W-2 form model**](concept-w2.md) </br>&#9679; [**Invoice model**](concept-invoice.md)</br>&#9679; [**Receipt model**](concept-receipt.md) </br>&#9679; [**Identity (ID) document model**](concept-id-document.md) </br>&#9679; [**Business card model**](concept-business-card.md) </br>

+|**A generic document** like a contract or letter.|You want to extract primarily text lines, words, locations, and detected languages.|</li></ul>The document is written or printed in a [supported language](language-support.md#read-layout-and-custom-form-template-model).| [**Read OCR model**](concept-read.md)|

+|**A document that includes structural information** like a report or study.|In addition to text, you need to extract structural information like tables, selection marks, paragraphs, titles, headings, and subheadings.|The document is written or printed in a [supported language](language-support.md#read-layout-and-custom-form-template-model)| [**Layout analysis model**](concept-layout.md)

+|**Identity document (ID)** like a passport or driver's license. |You want to extract key information such as first name, last name, and date of birth from US drivers' licenses or international passports. |Your ID document is a US driver's license or the biographical page from an international passport (not a visa).| [**Identity document (ID) model**](concept-id-document.md)|

+> * If you're still unsure which model to use, try the General Document model to extract key-value pairs.

+> * The General Document model is powered by the Read OCR engine to detect text lines, words, locations, and languages.

+> * General document also extracts the same data as the document layout model (pages, tables, styles).

+## Document processing models and development options

+>The following document understanding models and development options are supported by the Form Recognizer service v3.0.

+You can Use Form Recognizer to automate your document processing in applications and workflows, enhance data-driven strategies, and enrich document search capabilities. Use the links in the table to learn more about each model and browse the API references.

+|[**Read OCR model**](concept-read.md)|Extract text lines, words, detected languages, and handwritten style if detected.| <ul><li>Contract processing. </li><li>Financial or medical report processing.</li></ul>|<ul ><li>[**Form Recognizer Studio**](https://formrecognizer.appliedai.azure.com/studio/read)</li><li>[**REST API**](how-to-guides/use-prebuilt-read.md?pivots=programming-language-rest-api)</li><li>[**C# SDK**](how-to-guides/use-prebuilt-read.md?pivots=programming-language-csharp)</li><li>[**Python SDK**](how-to-guides/use-prebuilt-read.md?pivots=programming-language-python)</li><li>[**Java SDK**](how-to-guides/use-prebuilt-read.md?pivots=programming-language-java)</li><li>[**JavaScript**](how-to-guides/use-prebuilt-read.md?pivots=programming-language-javascript)</li></ul> |

+|[**Layout analysis model**](concept-layout.md) | Extract text, selection marks, and tables structures, along with their bounding box coordinates, from forms and documents.</br></br> Layout API has been updated to a prebuilt model. |<ul><li>Document indexing and retrieval by structure.</li><li>Preprocessing prior to OCR analysis.</li></ul> |<ul><li>[**Form Recognizer Studio**](https://formrecognizer.appliedai.azure.com/studio/layout)</li><li>[**REST API**](quickstarts/get-started-v3-sdk-rest-api.md)</li><li>[**C# SDK**](quickstarts/get-started-v3-sdk-rest-api.md#layout-model)</li><li>[**Python SDK**](quickstarts/get-started-v3-sdk-rest-api.md#layout-model)</li><li>[**Java SDK**](quickstarts/get-started-v3-sdk-rest-api.md#layout-model)</li><li>[**JavaScript**](quickstarts/get-started-v3-sdk-rest-api.md#layout-model)</li></ul>|

+|[**Identity document (ID) model (updated)**](concept-id-document.md) |Automated data processing and extraction of key information from US driver's licenses and international passports.</br></br>Prebuilt ID document API supports the **extraction of endorsements, restrictions, and vehicle classifications from US driver's licenses**. |<ul><li>Know your customer (KYC) financial services guidelines compliance.</li><li>Medical account management.</li><li>Identity checkpoints and gateways.</li><li>Hotel registration.</li></ul> |<ul><li> [**Form Recognizer Studio**](https://formrecognizer.appliedai.azure.com/studio/prebuilt?formType=idDocument)</li><li>[**REST API**](quickstarts/get-started-v3-sdk-rest-api.md)</li><li>[**C# SDK**](quickstarts/get-started-v3-sdk-rest-api.md#prebuilt-model)</li><li>[**Python SDK**](quickstarts/get-started-v3-sdk-rest-api.md#prebuilt-model)</li><li>[**Java SDK**](quickstarts/get-started-v3-sdk-rest-api.md#prebuilt-model)</li><li>[**JavaScript**](quickstarts/get-started-v3-sdk-rest-api.md#prebuilt-model)</li></ul>|

+Azure Form Recognizer is a cloud-based [Azure Applied AI Service](../../applied-ai-services/index.yml) for developers to build intelligent document processing solutions. Form Recognizer applies machine-learning based optical character recognition (OCR) and document understanding technologies to extract print and handwritten text, tables, structure, and key-value pairs from documents. You can also label and train custom models to automate data extraction from structured, semi-structured, and unstructured documents. To learn more about each model, *see* the Concepts articles:

+|**Document analysis model**| &#9679; [**Layout analysis model**](concept-layout.md) </br> |

+| **Prebuilt models** | &#9679; [**Invoice model**](concept-invoice.md)</br>&#9679; [**Receipt model**](concept-receipt.md) </br>&#9679; [**Identity document (ID) model**](concept-id-document.md) </br>&#9679; [**Business card model**](concept-business-card.md) </br>

+## Which document processing model should I use?

+|**A document that includes structural information** like a report or study.|In addition to text, you need to extract structural information like tables and selection marks.|The document is written or printed in a [supported language](language-support.md#read-layout-and-custom-form-template-model)| [**Layout analysis model**](concept-layout.md)

+|**Identity document (ID)** like a passport or driver's license. |You want to extract key information such as first name, last name, and date of birth from US drivers' licenses or international passports. |Your ID document is a US driver's license or the biographical page from an international passport (not a visa).| [**ID document model**](concept-id-document.md)|

+|[**Layout analysis**](concept-layout.md) | Extraction and analysis of text, selection marks, tables, and bounding box coordinates, from forms and documents. | <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#analyze-layout)</li><li>[**REST API**](quickstarts/get-started-v2-1-sdk-rest-api.md#try-it-layout-model)</li><li>[**Client-library SDK**](quickstarts/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?branch=main&tabs=layout#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+|[**Custom model**](concept-custom.md) | Extraction and analysis of data from forms and documents specific to distinct business data and use cases.| <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#train-a-custom-form-model)</li><li>[**REST API**](quickstarts/try-sdk-rest-api.md)</li><li>[**Client-library SDK**](how-to-guides/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?tabs=custom#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+|[**Invoice model**](concept-invoice.md) | Automated data processing and extraction of key information from sales invoices. | <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#analyze-using-a-prebuilt-model)</li><li>[**REST API**](quickstarts/get-started-v2-1-sdk-rest-api.md#try-it-prebuilt-model)</li><li>[**Client-library SDK**](quickstarts/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?tabs=invoice#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+|[**Receipt model**](concept-receipt.md) | Automated data processing and extraction of key information from sales receipts.| <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#analyze-using-a-prebuilt-model)</li><li>[**REST API**](quickstarts/get-started-v2-1-sdk-rest-api.md#try-it-prebuilt-model)</li><li>[**Client-library SDK**](how-to-guides/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?tabs=receipt#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+|[**Identity document (ID) model**](concept-id-document.md) | Automated data processing and extraction of key information from US driver's licenses and international passports.| <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#analyze-using-a-prebuilt-model)</li><li>[**REST API**](quickstarts/get-started-v2-1-sdk-rest-api.md#try-it-prebuilt-model)</li><li>[**Client-library SDK**](how-to-guides/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?tabs=id-document#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+|[**Business card model**](concept-business-card.md) | Automated data processing and extraction of key information from business cards.| <ul><li>[**Form Recognizer labeling tool**](quickstarts/try-sample-label-tool.md#analyze-using-a-prebuilt-model)</li><li>[**REST API**](quickstarts/get-started-v2-1-sdk-rest-api.md#try-it-prebuilt-model)</li><li>[**Client-library SDK**](how-to-guides/try-sdk-rest-api.md)</li><li>[**Form Recognizer Docker container**](containers/form-recognizer-container-install-run.md?tabs=business-card#run-the-container-with-the-docker-compose-up-command)</li></ul>|

+ As with all AI services, developers using the Form Recognizer service should be aware of Microsoft policies on customer data. See our [Data, privacy, and security for Form Recognizer](/legal/cognitive-services/form-recognizer/fr-data-privacy-security) page.

+A new prebuilt that extracts information from contracts such as parties, title, contract ID, execution date and more. Contracts is currently in preview, please request access [here](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUQTRDQUdHMTBWUDRBQ01QUVNWNlNYMVFDViQlQCN0PWcu_).

+* [**Read model now supports common Microsoft Office document types**](concept-read.md). Document types like Word (docx) and PowerPoint (ppt) are now supported with the Read API. See [Microsoft Office and HTML text extraction (preview)](concept-read.md#microsoft-office-and-html-text-extraction-preview-).

+3. Add the CocoaPod to your Podfile by adding `pod 'immersive-reader-sdk', :path => 'https://github.com/microsoft/immersive-reader-sdk/tree/master/iOS/immersive-reader-sdk'`. Your Podfile should look like the following, with your target's name replacing picture-to-immersive-reader-swift:

+ ```ruby

+ platform :ios, '9.0'

+ target 'picture-to-immersive-reader-swift' do

+ use_frameworks!

+ # Pods for picture-to-immersive-reader-swift

+ pod 'immersive-reader-sdk', :git => 'https://github.com/microsoft/immersive-reader-sdk.git'

+ end

+ ```

+Azure operations can be divided into two categories - control plane and data plane. You use the control plane to manage resources in your subscription. You use the data plane to use capabilities exposed by your instance of a resource type.

+Metrics Advisor requires at least a 'Reader' role to use its capabilities, but cannot perform edit/delete action to the resource itself.

+Within Metrics Advisor there're other fine-grained roles to enable permission control on specific entities, like data feeds, hooks, credentials etc. There are two types of roles:

+- **Administrator**: Has full permissions to manage a data feed, hook, credentials, etc. including modify and delete.

+- **Viewer**: Has access to a read-only view of the data feed, hook, credentials, etc.

+### Send anomaly notification through a Microsoft Teams channel

+This section will walk through the practice of sending anomaly notifications through a Microsoft Teams channel. This can help enable scenarios where team members are collaborating on analyzing anomalies that are detected by Metrics Advisor. The workflow is easy to configure and doesn't have a large number of prerequisites.

+ Title: Disaster recovery for Azure Automation

+description: This article details on disaster recovery strategy to handle service outage or zone failure for Azure Automation

+keywords: automation disaster recovery

+# Disaster recovery for Azure Automation

+**Applies to:** :heavy_check_mark: Linux VMs :heavy_check_mark: Windows VMs

+This article explains the disaster recovery strategy to handle a region-wide or zone-wide failure.

+You must have a disaster recovery strategy to handle a region-wide service outage or zone-wide failure to help reduce the impact and effects arising from unpredictable events on your business and customers. You are responsible to set up disaster recovery of Automation accounts, and its dependent resources such as Modules, Connections, Credentials, Certificates, Variables and Schedules. An important aspect of a disaster recovery plan is preparing to failover to the replica of the Automation account created in advance in the secondary region, if the Automation account in the primary region becomes unavailable. Ensure that your disaster recovery strategy considers your Automation account and the dependent resources.

+In addition to high availability offered by Availability zones, some regions are paired with another region to provide protection from regional or large geographical disasters. Irrespective of whether the primary region has a regional pair or not, the disaster recovery strategy for the Automation account remains the same. For more information about regional pairs, [learn more](../availability-zones/cross-region-replication-azure.md).

+## Enable disaster recovery

+Every Automation account that you [create](https://learn.microsoft.com/azure/automation/quickstarts/create-azure-automation-account-portal)

+requires a location that you must use for deployment. This would be the primary region for your Automation account and it includes Assets, runbooks created for the Automation account, job execution data, and logs. For disaster recovery, the replica Automation account must be already deployed and ready in the secondary region.

+- Begin by [creating a replica Automation account](https://learn.microsoft.com/azure/automation/quickstarts/create-azure-automation-account-portal#create-automation-account) in any alternate [region](https://azure.microsoft.com/global-infrastructure/services/?products=automation&regions=all).

+- Select the secondary region of your choice - paired region or any other region where Azure Automation is available.

+- Apart from creating a replica of the Automation account, replicate the dependent resources such as Runbooks, Modules, Connections, Credentials, Certificates, Variables, Schedules and permissions assigned for the Run As account and Managed Identities in the Automation account in primary region to the Automation account in secondary region. You can use the [PowerShell script](#script-to-migrate-automation-account-assets-from-one-region-to-another) to migrate assets of the Automation account from one region to another.

+- If you are using [ARM templates](../azure-resource-manager/management/overview.md) to define and deploy Automation runbooks, you can use these templates to deploy the same runbooks in any other Azure region where you create the replica Automation account. In case of a region-wide outage or zone-wide failure in the primary region, you can execute the runbooks replicated in the secondary region to continue business as usual. This ensures that the secondary region steps up to continue the work if the primary region has a disruption or failure.

+>[!NOTE]

+> Due to data residency requirements, jobs data and logs present in the primary region are not available in the secondary region.

+## Scenarios for cloud and hybrid jobs

+### Scenario: Execute Cloud jobs in secondary region

+For Cloud jobs, there would be a negligible downtime, provided a replica Automation account and all dependent resources and runbooks are already deployed and available in the secondary region. You can use the replica account for executing jobs as usual.

+### Scenario: Execute jobs on Hybrid Runbook Worker deployed in a region different from primary region of failure

+If the Windows or Linux Hybrid Runbook worker is deployed using the extension-based approach in a region *different* from the primary region of failure, follow these steps to continue executing the Hybrid jobs:

+1. [Delete](extension-based-hybrid-runbook-worker-install.md?tabs=windows#delete-a-hybrid-runbook-worker) the extension installed on Hybrid Runbook worker in the Automation account in the primary region.

+1. [Add](extension-based-hybrid-runbook-worker-install.md?tabs=windows#create-hybrid-worker-group) the same Hybrid Runbook worker to a Hybrid Worker group in the Automation account in the secondary region. The Hybrid worker extension is installed on the machine in the replica of the Automation account.

+1. Execute the jobs on the Hybrid Runbook worker created in Step 2.

+For Hybrid Runbook worker deployed using the agent-based approach, choose from below:

+#### [Windows Hybrid Runbook worker](#tab/win-hrw)

+If the Windows Hybrid Runbook worker is deployed using an agent-based approach in a region different from the primary region of failure, follow the steps to continue executing Hybrid jobs:

+1. [Uninstall](automation-windows-hrw-install.md#remove-windows-hybrid-runbook-worker) the agent from the Hybrid Runbook worker present in the Automation account in the primary region.

+1. [Re-install](automation-windows-hrw-install.md#installation-options) the agent on the same machine in the replica Automation account in the secondary region.

+1. You can now execute jobs on the Hybrid Runbook worker created in Step 2.

+#### [Linux Hybrid Runbook worker](#tab/linux-hrw)

+If the Linux Hybrid Runbook worker is deployed using agent-based approach in a region different from the primary region of failure, follow the below steps to continue executing Hybrid jobs:

+1. [Uninstall](automation-linux-hrw-install.md#remove-linux-hybrid-runbook-worker) the agent from the Hybrid Runbook worker present in Automation account in the primary region.

+1. [Re-install](automation-linux-hrw-install.md#install-a-linux-hybrid-runbook-worker) the agent on the same machine in the replica Automation account in the secondary region.

+1. You can now execute jobs on the Hybrid Runbook worker created in Step 2.

+### Scenario: Execute jobs on Hybrid Runbook Worker deployed in the primary region of failure

+If the Hybrid Runbook worker is deployed in the primary region, and there is a compute failure in that region, the machine will not be available for executing Automation jobs. You must provision a new virtual machine in an alternate region and register it as Hybrid Runbook Worker in Automation account in the secondary region.

+

+- See the installation steps in [how to deploy an extension-based Windows or Linux User Hybrid Runbook Worker](extension-based-hybrid-runbook-worker-install.md?tabs=windows#create-hybrid-worker-group).

+- See the installation steps in [how to deploy an agent-based Windows Hybrid Worker](automation-windows-hrw-install.md#installation-options).

+- See the installation steps in [how to deploy an agent-based Linux Hybrid Worker](automation-linux-hrw-install.md#install-a-linux-hybrid-runbook-worker).

+## Script to migrate Automation account assets from one region to another

+You can use these scripts for migration of Automation account assets from the account in primary region to the account in the secondary region. These scripts are used to migrate only Runbooks, Modules, Connections, Credentials, Certificates and Variables. The execution of these scripts does not affect the Automation account and its assets present in the primary region.

+### Prerequisites

+ 1. Ensure that the Automation account in the secondary region is created and available so that assets from primary region can be migrated to it. It is preferred if the destination automation account is one without any custom resources as it prevents potential resource class due to same name and loss of data.

+ 1. Ensure that the system assigned identities are enabled in the Automation account in the primary region.

+ 1. Ensure that the primary Automation account's Managed Identity has Contributor access with read and write permissions to the Automation account in secondary region. To enable, provide the necessary permissions in secondary Automation account's managed identities. [Learn more](../role-based-access-control/quickstart-assign-role-user-portal.md).

+ 1. Ensure that the script has access to the Automation account assets in primary region. Hence, it should be executed as a runbook in that Automation account for successful migration.

+ 1. If the primary Automation account is deployed using a Run as account, then it must be switched to Managed Identity before migration. [Learn more](migrate-run-as-accounts-managed-identity.md).

+ 1. Modules required are:

+

+ - Az.Accounts version 2.8.0

+ - Az.Resources version 6.0.0

+ - Az.Automation version 1.7.3

+ - Az.Storage version 4.6.0

+1. Ensure that both the source and destination Automation accounts should belong to the same Azure Active Directory tenant.

+### Create and execute the runbook

+You can use the[PowerShell script](https://github.com/azureautomation/Migrate-automation-account-assets-from-one-region-to-another) or [PowerShell workflow](https://github.com/azureautomation/Migrate-automation-account-assets-from-one-region-to-another-PwshWorkflow/tree/main) runbook or import from the Runbook gallery and execute it to enable migration of assets from one Automation account to another.

+Follow the steps to import and execute the runbook:

+#### [PowerShell script](#tab/ps-script)

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Go to Automation account that you want to migrate to another region.

+1. Under **Process Automation**, select **Runbooks**.

+1. Select **Browse gallery** and in the search, enter *Migrate Automation account assets from one region to another* and select **PowerShell script**.

+1. In the **Import a runbook** page, enter a name for the runbook.

+1. Select **Runtime version** as either 5.1 or 7.1 (preview)

+1. Enter the description and select **Import**.

+1. In the **Edit PowerShell Runbook** page, edit the required parameters and execute it.

+You can choose either of the options to edit and execute the script. You can provide the seven mandatory parameters as given in Option 1 **or** three mandatory parameters given in Option 2 to edit and execute the script.

+#### [PowerShell Workflow](#tab/ps-workflow)

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Go to Automation account that you want to migrate to another region.

+1. Under **Process Automation**, select **Runbooks**.

+1. Select **Browse gallery** and in the search, enter *Migrate Automation account assets from one region to another* and Select **PowerShell workflow**.

+1. In the **Import a runbook** page, enter a name for the runbook.

+1. Select **Runtime version** as 5.1

+1. Enter the description and select **Import**.

+You can input the parameters during execution of PowerShell Workflow runbook. You can provide the seven mandatory parameters as given in Option 1 **or** three mandatory parameters given in Option 2 to execute the script.

+The options are:

+#### [Option 1](#tab/option-one)

+**Name** | **Required** | **Description**

+-- | - | --

+SourceAutomationAccountName | True | Name of automation account in primary region from where assets need to be migrated. |

+DestinationAutomationAccountName | True | Name of automation account in secondary region to which assets need to be migrated. |

+SourceResourceGroup | True | Resource group name of the Automation account in the primary region. |

+DestinationResourceGroup | True | Resource group name of the Automation account in the secondary region. |

+SourceSubscriptionId | True | Subscription ID of the Automation account in primary region |

+DestinationSubscriptionId | True | Subscription ID of the Automation account in secondary region. |

+Type[] | True | Array consisting of all the types of assets that need to be migrated, possible values are Certificates, Connections, Credentials, Modules, Runbooks, and Variables. |

+#### [Option 2](#tab/option-two)

+**Name** | **Required** | **Description**

+-- | - | --

+SourceAutomationAccountResourceId | True | Resource ID of the Automation account in primary region from where assets need to be migrated. |

+DestinationAutomationAccountResourceId | True | Resource ID of the Automation account in secondary region to which assets need to be migrated. |

+Type[] | True | Array consisting of all the types of assets that need to be migrated, possible values are Certificates, Connections, Credentials, Modules, Runbooks, and Variables. |

+### Limitations

+- The script migrates only Custom PowerShell modules. Default modules and Python packages would not be migrated to replica Automation account.

+- The script does not migrate **Schedules** and **Managed identities** present in Automation account in primary region. These would have to be created manually in replica Automation account.

+- Jobs data and activity logs would not be migrated to the replica account.

+## Next steps

+- Learn more about [regions that support availability zones](../availability-zones/az-region.md).

+A new Automation account is always encrypted using Microsoft-managed keys. It's not possible to enable customer-managed keys at the time that the account is created. Customer-managed keys are stored in Azure Key Vault, and the key vault must be provisioned with access policies that grant key permissions to the managed identity that is associated with the Automation account. The managed identity is available only after the automation account is created.

+ - South Central US

+ - Sweden Central

+ - Qatar Central

+ - Central India

+ ```

+* Learn how to [configure Private Link for Azure Monitor](../../azure-monitor/logs/private-link-security.md).

+ Title: Configure passive geo-replication for Premium Azure Cache for Redis instances

+description: Learn how to use cross-region replication to provide disaster recovery on the Premium tier of Azure Cache for Redis.

+# Configure passive geo-replication for Premium Azure Cache for Redis instances

+In this article, you learn how to configure passive geo-replication on a pair of Azure Cache for Redis instances using the Azure portal.

+Passive geo-replication links together two Premium tier Azure Cache for Redis instances and creates an _active-passive_ data replication relationship. Active-passive means that there's a pair of caches, primary and secondary, that have their data synchronized. But you can only write to one side of the pair, the primary. The other side of the pair, the secondary cache, is read-only.

+Compare _active-passive_ to _active-active_, where you can write to either side of the pair, and it will synchronize with the other side.

+With passive geo-replication, the cache instances are typically located in different Azure regions, though that isn't required. One instance acts as the primary, and the other as the secondary. The primary handles read and write requests and propagates changes to the secondary.

+Failover is not automatic. For more information and information on how to use failover, see [Initiate a failover from geo-primary to geo-secondary (preview)](#initiate-a-failover-from-geo-primary-to-geo-secondary-preview).

+## Scope of availability

+|Tier | Basic, Standard | Premium |Enterprise, Enterprise Flash |

+|||||

+|Available | No | Yes | Yes |

+_Passive geo-replication_ is only available in the Premium tier of Azure Cache for Redis. The Enterprise and Enterprise Flash tiers also offer geo-replication, but those tiers use a more advanced version called _active geo-replication_.

+- The secondary linked cache is either the same cache size or a larger cache size than the primary linked cache. To use geo-failover, both caches must be the same size.

+> Data transfer between Azure regions is charged at standard [bandwidth rates](https://azure.microsoft.com/pricing/details/bandwidth/).

+- Caches with more than one replica can't be geo-replicated.

+- The secondary linked cache is read-only. You can read from it, but you can't write any data to it. If you choose to read from the Geo-Secondary instance when a full data sync is happening between the Geo-Primary and the Geo-Secondary, the Geo-Secondary instance throws errors on any Redis operation against it until the full data sync is complete. The errors state that a full data sync is in progress. Also, the errors are thrown when either Geo-Primary or Geo-Secondary is updated and on some reboot scenarios. Applications reading from Geo-Secondary should be built to fall back to the Geo-Primary whenever the Geo-Secondary is throwing such errors.

+- Failover is not automatic. You must start the failover from the primary to the secondary inked cache. For more information and information on how to use failover, see [Initiate a failover from geo-primary to geo-secondary (preview)](#initiate-a-failover-from-geo-primary-to-geo-secondary-preview).

+1. To link two caches together for geo-replication, fist select **Geo-replication** from the Resource menu of the cache that you intend to be the primary linked cache. Next, select **Add cache replication link** from the working pane.

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-menu.png" alt-text="Screenshot showing the cache's Geo-replication menu.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-select-link.png" alt-text="Screenshot showing compatible caches for linking with geo-replication.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-select-link-context-menu.png" alt-text="Screenshot showing the Geo-replication context menu.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-confirm-link.png" alt-text="Screenshot showing how to link caches for geo-replication.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-linking.png" alt-text="Screenshot showing the current Linking status.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-link-successful.png" alt-text="Screenshot showing cache linking status as Succeeded.":::

+## Geo-primary URLs (preview)

+Once the caches are linked, URLs are generated that always point to the geo-primary cache. If a failover is initiated from the geo-primary to the geo-secondary, the URL remains the same, and the underlying DNS record is updated automatically to point to the new geo-primary.

+Four URLs are shown:

+- **Geo-Primary URL** is a proxy URL with the format of `<cache-1-name>.geo.redis.cache.windows.net`. This URL always has the name of the first cache to be linked, but it always points to whichever cache is the current geo-primary.

+- **Linked cache Geo-Primary URL** is a proxy URL with the format of `<cache-2-name>.geo.redis.cache.windows.net`. This URL always has the name of the second cache to be linked, and it will also always point to whichever cache is the current geo-primary.

+- **Current Geo Primary Cache** is the direct address of the cache that is currently the geo-primary. The address is `redis.cache.windows.net` not `geo.redis.cache.windows.net`. The address listed in this field changes if a failover is initiated.

+- **Current Geo Secondary Cache** is the direct address of the cache that is currently the geo-secondary. The address is `redis.cache.windows.net` not `geo.redis.cache.windows.net`. The address listed in this field changes if a failover is initiated.

+The goal of the two geo-primary URLs is to make updating the cache address easier on the application side in the event of a failover. Changing the address of either linked cache from `redis.cache.windows.net` to `geo.redis.cache.windows.net` ensures that your application is always pointing to the geo-primary, even if a failover is triggered.

+The URLs for the current geo-primary and current geo-secondary cache are provided in case youΓÇÖd like to link directly to a cache resource without any automatic routing.

+## Initiate a failover from geo-primary to geo-secondary (preview)

+With one click, you can trigger a failover from the geo-primary to the geo-secondary.

+This causes the following steps to be taken:

+1. The geo-secondary cache is promoted to geo-primary.

+1. DNS records are updated to redirect the geo-primary URLs to the new geo-primary.

+1. The old geo-primary cache is demoted to secondary, and attempts to form a link to the new geo-primary cache.

+The geo-failover process takes a few minutes to complete.

+### Settings to check before initiating geo-failover

+When the failover is initiated, the geo-primary and geo-secondary caches will swap. If the new geo-primary is configured differently from the geo-secondary, it can create problems for your application.

+Be sure to check the following items:

+- If youΓÇÖre using a firewall in either cache, make sure that the firewall settings are similar so you have no connection issues.

+- Make sure both caches are using the same port and TLS/SSL settings

+- The geo-primary and geo-secondary caches have different access keys. In the event of a failover being triggered, make sure your application can update the access key it's using to match the new geo-primary.

+### Failover with minimal data loss

+Geo-failover events can introduce data inconsistencies during the transition, especially if the client maintains a connection to the old geo-primary during the failover process. It's possible to minimize data loss in a planned geo-failover event using the following tips:

+- Check the geo-replication data sync offset metric. The metric is emitted by the current geo-primary cache. This metric indicates how much data has yet to be replicated to the geo-primary. If possible, only initiate failover if the metric indicates fewer than 14 bytes remain to be written.

+- Run the `CLIENT PAUSE` command in the current geo-primary before initiating failover. Running `CLIENT PAUSE` blocks any new write requests and instead returns timeout failures to the Azure Cache for Redis client. The `CLIENT PAUSE` command requires providing a timeout period in milliseconds. Make sure a long enough timeout period is provided to allow the failover to occur. Setting this to around 30 minutes (1,800,000 milliseconds) is a good place to start. You can always lower this number as needed.

+There's no need to run the CLIENT UNPAUSE command as the new geo-primary does retain the client pause.

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-unlink.png" alt-text="Screenshot showing how to unlink caches.":::

+- Can I track the health of the geo-replication link?

+No, passive geo-replication is only available in the Premium tier. A more advanced version of geo-replication called, _active geo-replication_, is available in the Enterprise and Enterprise Flash tier.

+### Can I track the health of the geo-replication link?

+Yes, there are several metrics available to help track the status of the geo-replication. These metrics are available in the Azure portal.

+- **Geo Replication Healthy** shows the status of the geo-replication link. The link will show up as unhealthy if either the geo-primary or geo-secondary caches are down. This is typically due to standard patching operations, but it could also indicate a failure situation.

+- **Geo Replication Connectivity Lag** shows the time since the last successful data synchronization between geo-primary and geo-secondary.

+- **Geo Replication Data Sync Offset** shows the amount of data that has yet to be synchronized to the geo-secondary cache.

+- **Geo Replication Fully Sync Event Started** indicates that a full synchronization action has been initiated between the geo-primary and geo-secondary caches. This occurs if standard replication can't keep up with the number of new writes.

+- **Geo Replication Full Sync Event Finished** indicates that a full synchronization action has been completed.

+Yes, as long as the secondary linked cache is larger than the primary linked cache. However, you can't use the failover feature if the caches are different sizes.

+Replication is incremental, asynchronous, and continuous and the time taken isn't much different from the latency across regions. Under certain circumstances, the secondary cache can be required to do a full sync of the data from the primary. The replication time in this case depends on many factors like: load on the primary cache, available network bandwidth, and inter-region latency. We have found replication time for a full 53-GB geo-replicated pair can be anywhere between 5 to 10 minutes. You can track the amount of data that has yet to be replicated using the `Geo Replication Data Sync Offset` metric in Azure monitor.

+In this article, you learn how to move Azure Cache for Redis instances to a different Azure region. You might move your resources to another region for many reasons:

+The tier of Azure Cache for Redis you use determines the option that's best for you.

+| Cache Tier | Options |

+| | - |

+| Premium | Geo-replication, create a new cache, dual-write to two caches, export and import data via RDB file, or migrate programmatically |

+| Basic or Standard | Create a new cache, dual-write to two caches, or migrate programmatically |

+| Enterprise or Enterprise Flash | Create a new cache or export and import data with an RDB file, or migrate programmatically |

+## Passive geo-replication (Premium)

+### Prerequisites

+To move your cache instance to another region, you need to [create a second premium cache instance](quickstart-create-redis.md) in the desired region. Once both caches are running, you can set up geo-replication between the two cache instances.

+- The secondary linked cache is read-only. You can read from it, but you can't write any data to it.

+ - If you choose to read from the Geo-Secondary instance when a full data sync is happening between the Geo-Primary and the Geo-Secondary, such as when either Geo-Primary or Geo-Secondary is updated and on some reboot scenarios as well, the Geo-Secondary instance throws errors on any Redis operation against it until the full data sync between Geo-Primary and Geo-Secondary is complete.

+ - Applications reading from Geo-Secondary should be built to fall back to the Geo-Primary whenever the Geo-Secondary is throwing such errors.

+- Failover is not automatic. You must start the failover from the primary to the secondary inked cache. For more information and information on how to failover a client application, see [Initiate a failover from geo-primary to geo-secondary (preview)](cache-how-to-geo-replication.md#initiate-a-failover-from-geo-primary-to-geo-secondary-preview).

+1. To link two caches together for geo-replication, fist select **Geo-replication** from the Resource menu of the cache that you intend to be the primary linked cache. Next, select **Add cache replication link** from **Geo-replication** on the left.

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-menu.png" alt-text="Screenshot showing the cache's Geo-replication menu.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-select-link.png" alt-text="Screenshot showing compatible caches for linking with geo-replication.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-select-link-context-menu.png" alt-text="Screenshot showing the Geo-replication context menu.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-confirm-link.png" alt-text="Screenshot showing how to link caches for geo-replication.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-linking.png" alt-text="Screenshot showing the current Linking status.":::

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-link-successful.png" alt-text="Screenshot showing cache linking status as Succeeded.":::

+### Clean up source resources

+1. To remove the link between two caches and stop geo-replication, select **Unlink caches** from the **Geo-replication** on the left.

+ :::image type="content" source="media/cache-how-to-geo-replication/cache-geo-location-unlink.png" alt-text="Screenshot showing how to unlink caches.":::

+1. Delete the original instance.

+Finally, update your application to use the new instances.

+### Clean up source resources

+Once your new cache in the targeted region is running, delete the original instance.

+1. For more information on how to import and export data in Azure Cache for Redis. see [Import and Export data in Azure Cache for Redis](cache-how-to-import-export-data.md).

+1. Update your application to use the new cache instance.

+### Clean up source resources

+1. Continue reading data from the original instance until the new instance is sufficiently populated with data.

+1. Update the application code to reading and writing from the new instance only.

+### Clean up source resources

+Once your new cache in the targeted region is running, delete the original instance.

+You can create a custom migration process by programmatically reading data from an existing cache and writing them into Azure Cache for Redis. This [open-source tool](https://github.com/deepakverma/redis-copy) can be used to copy data from one Azure Cache for Redis instance to another instance in a different Azure Cache region. A [compiled version](https://github.com/deepakverma/redis-copy/releases/download/alpha/Release.zip) is available as well. You may also find the source code to be a useful guide for writing your own migration tool.

+> This tool isn't officially supported by Microsoft.

+### Clean up source resources

+zone_pivot_groups: python-mode-functions

+In this article, you use command-line tools to create a Python function that responds to HTTP requests. After testing the code locally, you deploy it to the serverless environment of Azure Functions.

+This article covers both Python programming models supported by Azure Functions. Use the selector at the top to choose your programming model.

+>[!NOTE]

+>The Python v2 programming model for Functions is currently in Preview. To learn more about the Python v2 programming model, see the [Developer Reference Guide](functions-reference-python.md).

+1. Run the `func init` command as follows to create a functions project in a folder named *LocalFunctionProj* with the specified runtime and the specified programming model version.

+ ```console

+ func init LocalFunctionProj --python -m V2

+ ```

+1. Go to the project folder.

+ ```console

+ cd LocalFunctionProj

+ ```

+ This folder contains various files for the project, including configuration files named *[local.settings.json]*(functions-develop-local.md#local-settings-file) and *[host.json]*(functions-host-json.md). Because *local.settings.json* can contain secrets downloaded from Azure, the file is excluded from source control by default in the *.gitignore* file.

+1. The file `function_app.py` can include all functions within your project. To start with, there's already an HTTP function stored in the file.

+```python

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="HttpTrigger1")

+@app.route(route="hello")

+def test_function(req: func.HttpRequest) -> func.HttpResponse:

+ return func.HttpResponse("HttpTrigger1 function processed a request!")%

+```

+`function_app.py` is the entry point to the function and where functions will be stored and/or referenced. This file will include configuration of triggers and bindings through decorators, and the function content itself.

+For more information, see [Azure Functions HTTP triggers and bindings](./functions-bindings-http-webhook.md?tabs=python).

+## Start the storage emulator

+Before running the function locally, you must start the local Azurite storage emulator. You can skip this step if the `AzureWebJobsStorage` setting in the local.settings.json file is set to the connection string for an Azure Storage account.

+Use the following command to start the Azurite storage emulator:

+```cmd

+azurite

+```

+For more information, see [Run Azurite](../storage/common/storage-use-azurite.md?tabs=npm#run-azurite)

+ The [`az login`](/cli/azure/reference-index#az-login) command signs you into your Azure account.

+ ::: zone pivot="python-mode-decorators"

+ In the current v2 programming model preview, choose a region from one of the following locations: France Central, West Central US, North Europe, China East, East US, or North Central US.

+ ::: zone-end

+ This command creates a function app running in your specified language runtime under the [Azure Functions Consumption Plan](consumption-plan.md), which is free for the amount of usage you incur here. The command also creates an associated Azure Application Insights instance in the same resource group, with which you can monitor your function app and view logs. For more information, see [Monitor Azure Functions](functions-monitoring.md). The instance incurs no costs until you activate it.

+## Update app settings

+To use the Python v2 model in your function app, you need to add a new application setting in Azure named `AzureWebJobsFeatureFlags` with a value of `EnableWorkerIndexing`. This setting is already in your local.settings.json file.

+Run the following command to add this setting to your new function app in Azure.

+# [Azure CLI](#tab/azure-cli)

+```azurecli

+az functionapp config appsettings set --name <FUNCTION_APP_NAME> --resource-group <RESOURCE_GROUP_NAME> --settings AzureWebJobsFeatureFlags=EnableWorkerIndexing

+```

+# [Azure PowerShell](#tab/azure-powershell)

+```azurepowershell

+Update-AzFunctionAppSetting -Name <FUNCTION_APP_NAME> -ResourceGroupName <RESOURCE_GROUP_NAME> -AppSetting @{"AzureWebJobsFeatureFlags" = "EnableWorkerIndexing"}

+```

+In the previous example, replace `<FUNCTION_APP_NAME>` and `<RESOURCE_GROUP_NAME>` with the name of your function app and resource group, respectively. This setting is already in your local.settings.json file.

+## Verify in Azure

+Having issues with this article?

+zone_pivot_groups: python-mode-functions

+This article covers both Python programming models supported by Azure Functions. Use the selector at the top to choose your programming model.

+>[!NOTE]

+>The Python v2 programming model for Functions is currently in Preview. To learn more about the v2 programming model, see the [Developer Reference Guide](functions-reference-python.md).

+2. Choose the directory location for your project workspace and choose **Select**. You should either create a new folder or choose an empty folder for the project workspace. Don't choose a project folder that is already part of a workspace.

+3. Provide the following information at the prompts:

+4. Visual Studio Code uses the provided information and generates an Azure Functions project with an HTTP trigger. You can view the local project files in the Explorer. For more information about the files that are created, see [Generated project files](functions-develop-vs-code.md?tabs=python#generated-project-files).

+3. Provide the following information at the prompts:

+ |Prompt|Selection|

+ |--|--|

+ |**Select a language**| Choose `Python (Programming Model V2)`.|

+ |**Select a Python interpreter to create a virtual environment**| Choose your preferred Python interpreter. If an option isn't shown, type in the full path to your Python binary.|

+ |**Select how you would like to open your project**| Choose `Add to workspace`.|

+4. Visual Studio Code uses the provided information and generates an Azure Functions project.

+5. Open the generated `function_app.py` project file, which contains your functions.

+6. Uncomment the `test_function` function, which is an HTTP triggered function.

+7. Replace the `app.route()` method call with the following code:

+ ```python

+ @app.route(route="hello", auth_level=func.AuthLevel.ANONYMOUS)

+ ```

+ This code enables your HTTP function endpoint to be called in Azure without having to provide an [Authorization keys](functions-bindings-http-webhook-trigger.md#authorization-keys). Local execution doesn't require authorization keys.

+ Your function code should now look like the following example:

+ ```python

+ @app.function_name(name="HttpTrigger1")

+ @app.route(route="hello", auth_level=func.AuthLevel.ANONYMOUS)

+ def test_function(req: func.HttpRequest) -> func.HttpResponse:

+ logging.info('Python HTTP trigger function processed a request.')

+ name = req.params.get('name')

+ if not name:

+ try:

+ req_body = req.get_json()

+ except ValueError:

+ pass

+ else:

+ name = req_body.get('name')

+ if name:

+ return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")

+ else:

+ return func.HttpResponse(

+ "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",

+ status_code=200

+ )

+ ```

+

+8. Open the local.settings.json project file and updated the `AzureWebJobsStorage` setting as in the following example:

+ ```json

+ "AzureWebJobsStorage": "UseDevelopmentStorage=true",

+ ```

+ This tells the local Functions host to use the storage emulator for the storage connection currently required by the v2 model. When you publish your project to Azure, you'll instead use the default storage account. If you're instead using an Azure Storage account, set your storage account connection string here.

+## Start the emulator

+1. In Visual Studio Code, press <kbd>F1</kbd> to open the command palette. In the command palette, search for and select `Azurite: Start`.

+1. Check the bottom bar and verify that Azurite emulation services are running. If so, you can now run your function locally.

+<! Go back to the shared include after preview

+-->

+## <a name="publish-the-project-to-azure"></a>Create the function app in Azure

+In this section, you create a function app and related resources in your Azure subscription.

+1. Choose the Azure icon in the Activity bar. Then in the **Resources** area, select the **+** icon and choose the **Create Function App in Azure** option.

+ 

+1. Provide the following information at the prompts:

+ |Prompt|Selection|

+ |--|--|

+ |**Select subscription**| Choose the subscription to use. You won't see this prompt when you have only one subscription visible under **Resources**. |

+ |**Enter a globally unique name for the function app**| Type a name that is valid in a URL path. The name you type is validated to make sure that it's unique in Azure Functions.|

+ |**Select a runtime stack**| Choose the language version on which you've been running locally. |

+ |**Select a location for new resources**| Choose a region for your function app.|

+ ::: zone pivot="python-mode-decorators"

+ In the current v2 programming model preview, choose a region from one of the following locations: France Central, West Central US, North Europe, China East, East US, or North Central US.

+ ::: zone-end

+ The extension shows the status of individual resources as they're being created in Azure in the **Azure: Activity Log** panel.

+ 

+1. When the creation is complete, the following Azure resources are created in your subscription. The resources are named based on your function app name:

+ [!INCLUDE [functions-vs-code-created-resources](../../includes/functions-vs-code-created-resources.md)]

+ A notification is displayed after your function app is created and the deployment package is applied.

+ [!INCLUDE [functions-vs-code-create-tip](../../includes/functions-vs-code-create-tip.md)]

+## Deploy the project to Azure

+## Update app settings

+To use the Python v2 model in your function app, you need to add a new application setting in Azure named `AzureWebJobsFeatureFlags` with a value of `EnableWorkerIndexing`. This setting is already in your local.settings.json file.

+1. In Visual Studio Code, press <kbd>F1</kbd> to open the command palette. In the command palette, search for and select `Azure Functions: Add New Setting...`.

+1. Choose your new function app, type `AzureWebJobsFeatureFlags` for the new app setting name, and press <kbd>Enter</kbd>.

+1. For the value, type `EnableWorkerIndexing` and press <kbd>Enter</kbd>.

+The setting added to your new function app, which enables it to run the v2 model in Azure.

+| customHeaders|none|Allows you to set custom headers in the HTTP response. The previous example adds the `X-Content-Type-Options` header to the response to avoid content type sniffing. This custom header applies to all HTTP triggered functions in the function app. |

+[Update your extensions]: ./functions-bindings-register.md

+ Title: Python V2 model Azure Functions triggers and bindings

+description: Provides examples of how to define Python triggers and bindings in Azure Functions using the preview v2 model

+ms.devlang: python

+# Python V2 model Azure Functions triggers and bindings (preview)

+The new Python v2 programming model in Azure Functions is intended to provide better alignment with Python development principles and with commonly used Python frameworks.

+The improved v2 programming model requires fewer files than the default model (v1), and specifically eliminates the need for a configuration file (`function.json`). Instead, triggers and bindings are represented in the `function_app.py` file as decorators. Moreover, functions can be logically organized with support for multiple functions to be stored in the same file. Functions within the same function application can also be stored in different files, and be referenced as blueprints.

+To learn more about using the new Python programming model for Azure Functions, see the [Azure Functions Python developer guide](./functions-reference-python.md). In addition to the documentation, [hints](https://aka.ms/functions-python-hints) are available in code editors that support type checking with .pyi files.

+This article contains example code snippets that define various triggers and bindings using the Python v2 programming model. To be able to run the code snippets below, ensure the following:

+- The function application is defined and named `app`.

+- Confirm that the parameters within the trigger reflect values that correspond with your storage account.

+- The name of the file the function is in must be `function_app.py`.

+To create your first function in the new v2 model, see one of these quickstart articles:

+## Blob trigger

+The following code snippet defines a function triggered from Azure Blob Storage:

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="BlobTrigger1")

+@app.blob_trigger(arg_name="myblob", path="samples-workitems/{name}",

+ connection="<STORAGE_CONNECTION_SETTING>")

+def test_function(myblob: func.InputStream):

+ logging.info(f"Python blob trigger function processed blob \n"

+ f"Name: {myblob.name}\n"

+ f"Blob Size: {myblob.length} bytes")

+```

+## Azure Cosmos DB trigger

+The following code snippet defines a function triggered from an Azure Cosmos DB (SQL API):

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="CosmosDBTrigger1")

+@app.cosmos_db_trigger(arg_name="documents", database_name="<DB_NAME>", collection_name="<COLLECTION_NAME>", connection_string_setting="<COSMOS_CONNECTION_SETTING>",

+ lease_collection_name="leases", create_lease_collection_if_not_exists="true")

+def test_function(documents: func.DocumentList) -> str:

+ if documents:

+ logging.info('Document id: %s', documents[0]['id'])

+```

+## Azure EventHub trigger

+The following code snippet defines a function triggered from an event hub instance:

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="EventHubTrigger1")

+@app.event_hub_message_trigger(arg_name="myhub", event_hub_name="samples-workitems",

+ connection="<EVENT_HUB_CONNECTION_SETTING>")

+def test_function(myhub: func.EventHubEvent):

+ logging.info('Python EventHub trigger processed an event: %s',

+ myhub.get_body().decode('utf-8'))

+```

+## HTTP trigger

+The following code snippet defines an HTTP triggered function:

+```python

+import azure.functions as func

+import logging

+app = func.FunctionApp(auth_level=func.AuthLevel.ANONYMOUS)

+@app.function_name(name="HttpTrigger1")

+@app.route(route="hello")

+def test_function(req: func.HttpRequest) -> func.HttpResponse:

+ logging.info('Python HTTP trigger function processed a request.')

+ name = req.params.get('name')

+ if not name:

+ try:

+ req_body = req.get_json()

+ except ValueError:

+ pass

+ else:

+ name = req_body.get('name')

+ if name:

+ return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")

+ else:

+ return func.HttpResponse(

+ "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",

+ status_code=200

+ )

+```

+## Azure Queue Storage trigger

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="QueueTrigger1")

+@app.queue_trigger(arg_name="msg", queue_name="python-queue-items",

+ connection="")

+def test_function(msg: func.QueueMessage):

+ logging.info('Python EventHub trigger processed an event: %s',

+ msg.get_body().decode('utf-8'))

+```

+## Azure Service Bus queue trigger

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="ServiceBusQueueTrigger1")

+@app.service_bus_queue_trigger(arg_name="msg", queue_name="myinputqueue", connection="")

+def test_function(msg: func.ServiceBusMessage):

+ logging.info('Python ServiceBus queue trigger processed message: %s',

+ msg.get_body().decode('utf-8'))

+```

+## Azure Service Bus topic trigger

+```python

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="ServiceBusTopicTrigger1")

+@app.service_bus_topic_trigger(arg_name="message", topic_name="mytopic", connection="", subscription_name="testsub")

+def test_function(message: func.ServiceBusMessage):

+ message_body = message.get_body().decode("utf-8")

+ logging.info("Python ServiceBus topic trigger processed message.")

+ logging.info("Message Body: " + message_body)

+```

+## Timer trigger

+```python

+import datetime

+import logging

+import azure.functions as func

+app = func.FunctionApp()

+@app.function_name(name="mytimer")

+@app.schedule(schedule="0 */5 * * * *", arg_name="mytimer", run_on_startup=True,

+ use_monitor=False)

+def test_function(mytimer: func.TimerRequest) -> None:

+ utc_timestamp = datetime.datetime.utcnow().replace(

+ tzinfo=datetime.timezone.utc).isoformat()

+ if mytimer.past_due:

+ logging.info('The timer is past due!')

+ logging.info('Python timer trigger function ran at %s', utc_timestamp)

+```

+## Next steps

+You can add a new function to an existing project by using one of the predefined Functions trigger templates. To add a new function trigger, select F1 to open the command palette, and then search for and run the command **Azure Functions: Create Function**. Follow the prompts to choose your trigger type and define the required attributes of the trigger. If your trigger requires an access key or connection string to connect to a service, get it ready before you create the function trigger.

+The results depend on the Python programming model. For more information, see the [Azure Functions Python developer guide](./functions-reference-python.md).

+**Python v1**: A new folder is created in the project. The folder contains a new function.json file and the new Python code file.

+**Python v2**: New function code is added either to the default function_app.py file or to another Python file you selected.

+description: Understand how to develop functions with Python

+zone_pivot_groups: python-mode-functions

+This article is an introduction to developing Azure Functions using Python. The content below assumes that you've already read the [Azure Functions developers guide](functions-reference.md).

+> [!IMPORTANT]

+> This article supports both the v1 and v2 programming model for Python in Azure Functions.

+> The v2 programming model is currently in preview.

+> While the v1 model uses a functions.json file to define functions, the new v2 model lets you instead use a decorator-based approach. This new approach results in a simpler file structure and a more code-centric approach. Choose the **v2** selector at the top of the article to learn about this new programming model. .

+As a Python developer, you may also be interested in one of the following articles:

+| <ul><li>[Python function using Visual Studio Code](./create-first-function-vs-code-python.md?pivots=python-mode-configuration)</li><li>[Python function with terminal/command prompt](./create-first-function-cli-python.md?pivots=python-mode-configuration)</li></ul> | <ul><li>[Developer guide](functions-reference.md)</li><li>[Hosting options](functions-scale.md)</li><li>[Performance&nbsp;considerations](functions-best-practices.md)</li></ul> | <ul><li>[Image classification with PyTorch](machine-learning-pytorch.md)</li><li>[Azure Automation sample](/samples/azure-samples/azure-functions-python-list-resource-groups/azure-functions-python-sample-list-resource-groups/)</li><li>[Machine learning with TensorFlow](functions-machine-learning-tensorflow.md)</li><li>[Browse Python samples](/samples/browse/?products=azure-functions&languages=python)</li></ul> |

+| Getting started | Concepts|

+|--|--|--|

+| <ul><li>[Python function using Visual Studio Code](./create-first-function-vs-code-python.md?pivots=python-mode-decorators)</li><li>[Python function with terminal/command prompt](./create-first-function-cli-python.md?pivots=python-mode-decorators)</li></ul> | <ul><li>[Developer guide](functions-reference.md)</li><li>[Hosting options](functions-scale.md)</li><li>[Performance&nbsp;considerations](functions-best-practices.md)</li></ul> |

+> While you can develop your Python based Azure Functions locally on Windows, Python is only supported on a Linux based hosting plan when running in Azure. See the list of supported [operating system/runtime](functions-scale.md#operating-systemruntime) combinations.

+Azure Functions expects a function to be a stateless method in your Python script that processes input and produces output. By default, the runtime expects the method to be implemented as a global method called `main()` in the `__init__.py` file. You can also [specify an alternate entry point](#alternate-entry-point).

+Data from triggers and bindings is bound to the function via method attributes using the `name` property defined in the *function.json* file. For example, the _function.json_ below describes a simple function triggered by an HTTP request named `req`:

+Based on this definition, the `__init__.py` file that contains the function code might look like the following example:

+```python

+def main(req):

+ user = req.params.get('user')

+ return f'Hello, {user}!'

+```

+You can also explicitly declare the attribute types and return type in the function using Python type annotations. This action helps you to use the IntelliSense and autocomplete features provided by many Python code editors.

+```python

+import azure.functions

+def main(req: azure.functions.HttpRequest) -> str:

+ user = req.params.get('user')

+ return f'Hello, {user}!'

+```

+Use the Python annotations included in the [azure.functions.*](/python/api/azure-functions/azure.functions) package to bind input and outputs to your methods.

+Azure Functions expects a function to be a stateless method in your Python script that processes input and produces output. By default, the runtime expects the method to be implemented as a global method in the `function_app.py` file.

+Triggers and bindings can be declared and used in a function in a decorator based approach. They're defined in the same file, `function_app.py`, as the functions. As an example, the below _function_app.py_ file represents a function trigger by an HTTP request.

+@app.function_name(name="HttpTrigger1")

+@app.route(route="req")

+You can also explicitly declare the attribute types and return type in the function using Python type annotations. This helps you use the IntelliSense and autocomplete features provided by many Python code editors.

+@app.function_name(name="HttpTrigger1")

+@app.route(route="req")

+At this time, only specific triggers and bindings are supported by the v2 programming model. Supported triggers and bindings are as follows.

+| Type | Trigger | Input Binding | Output Binding |

+| | | | |

+| HTTP | x | | |

+| Timer | x | | |

+| Azure Queue Storage | x | | x |

+| Azure Service Bus Topic | x | | x |

+| Azure Service Bus Queue | x | | x |

+| Azure Cosmos DB | x | x | x |

+| Azure Blob Storage | x | x | x |

+| Azure Event Grid | x | | x |

+To learn about known limitations with the v2 model and their workarounds, see [Troubleshoot Python errors in Azure Functions](./recover-python-functions.md?pivots=python-mode-decorators).

+You can change the default behavior of a function by optionally specifying the `scriptFile` and `entryPoint` properties in the *function.json* file. For example, the _function.json_ below tells the runtime to use the `customentry()` method in the _main.py_ file, as the entry point for your Azure Function.

+During preview, the entry point is only in the file `function_app.py`. However, functions within the project can be referenced in function_app.py using [blueprints](#blueprints) or by importing.

+The recommended folder structure for a Python Functions project looks like the following example:

+The main project folder (<project_root>) can contain the following files:

+* *local.settings.json*: Used to store app settings and connection strings when running locally. This file doesn't get published to Azure. To learn more, see [local.settings.file](functions-develop-local.md#local-settings-file).

+* *requirements.txt*: Contains the list of Python packages the system installs when publishing to Azure.

+* *host.json*: Contains configuration options that affect all functions in a function app instance. This file does get published to Azure. Not all options are supported when running locally. To learn more, see [host.json](functions-host-json.md).

+* *.vscode/*: (Optional) Contains store VSCode configuration. To learn more, see [VSCode setting](https://code.visualstudio.com/docs/getstarted/settings).

+* *.venv/*: (Optional) Contains a Python virtual environment used by local development.

+* *Dockerfile*: (Optional) Used when publishing your project in a [custom container](functions-create-function-linux-custom-image.md).

+* *tests/*: (Optional) Contains the test cases of your function app.

+* *.funcignore*: (Optional) Declares files that shouldn't get published to Azure. Usually, this file contains `.vscode/` to ignore your editor setting, `.venv/` to ignore local Python virtual environment, `tests/` to ignore test cases, and `local.settings.json` to prevent local app settings being published.

+Each function has its own code file and binding configuration file (function.json).

+The recommended folder structure for a Python Functions project looks like the following example:

+```

+ <project_root>/

+ | - .venv/

+ | - .vscode/

+ | - function_app.py

+ | - additional_functions.py

+ | - tests/

+ | | - test_my_function.py

+ | - .funcignore

+ | - host.json

+ | - local.settings.json

+ | - requirements.txt

+ | - Dockerfile

+```

+The main project folder (<project_root>) can contain the following files:

+* *.venv/*: (Optional) Contains a Python virtual environment used by local development.

+* *.vscode/*: (Optional) Contains store VSCode configuration. To learn more, see [VSCode setting](https://code.visualstudio.com/docs/getstarted/settings).

+* *function_app.py*: This is the default location for all functions and their related triggers and bindings.

+* *additional_functions.py*: (Optional) Any other Python files that contain functions (usually for logical grouping) that are referenced in `function_app.py` through blueprints.

+* *.funcignore*: (Optional) Declares files that shouldn't get published to Azure. Usually, this file contains `.vscode/` to ignore your editor setting, `.venv/` to ignore local Python virtual environment, `tests/` to ignore test cases, and `local.settings.json` to prevent local app settings being published.

+* *host.json*: Contains configuration options that affect all functions in a function app instance. This file does get published to Azure. Not all options are supported when running locally. To learn more, see [host.json](functions-host-json.md).

+* *local.settings.json*: Used to store app settings and connection strings when running locally. This file doesn't get published to Azure. To learn more, see [local.settings.file](functions-develop-local.md#local-settings-file).

+* *requirements.txt*: Contains the list of Python packages the system installs when publishing to Azure.

+* *Dockerfile*: (Optional) Used when publishing your project in a [custom container](functions-create-function-linux-custom-image.md).

+When you deploy your project to a function app in Azure, the entire contents of the main project (*<project_root>*) folder should be included in the package, but not the folder itself, which means `host.json` should be in the package root. We recommend that you maintain your tests in a folder along with other functions, in this example `tests/`. For more information, see [Unit Testing](#unit-testing).

+## Blueprints

+The v2 programming model introduces the concept of _blueprints_. A blueprint is a new class instantiated to register functions outside of the core function application. The functions registered in blueprint instances aren't indexed directly by function runtime. To get these blueprint functions indexed, the function app needs to register the functions from blueprint instances.

+Using blueprints provides the following benefits:

+* Lets you break-up the function app into modular components enabling you to define functions in multiple Python files and divide them into different components per file.

+* Provides extensible public function app interfaces to build and reuse your own APIs.

+The following example shows how to use blueprints:

+First, in an `http_blueprint.py` file HTTP triggered function is first defined and added to a blueprint object.

+```python

+import logging

+

+import azure.functions as func

+

+bp = func.Blueprint()

+@bp.route(route="default_template")

+def default_template(req: func.HttpRequest) -> func.HttpResponse:

+ logging.info('Python HTTP trigger function processed a request.')

+

+ name = req.params.get('name')

+ if not name:

+ try:

+ req_body = req.get_json()

+ except ValueError:

+ pass

+ else:

+ name = req_body.get('name')

+

+ if name:

+ return func.HttpResponse(

+ f"Hello, {name}. This HTTP triggered function "

+ f"executed successfully.")

+ else:

+ return func.HttpResponse(

+ "This HTTP triggered function executed successfully. "

+ "Pass a name in the query string or in the request body for a"

+ " personalized response.",

+ status_code=200

+ )

+```

+

+Next, in `function_app.py` the blueprint object is imported and its functions are registered to function app.

+```python

+import azure.functions as func

+from http_blueprint import bp

+

+app = func.FunctionApp()

+

+app.register_functions(bp)

+```

+You can import modules in your function code using both absolute and relative references. Based on the folder structure shown above, the following imports work from within the function file *<project_root>\my\_first\_function\\_\_init\_\_.py*:

+> The *shared_code/* folder needs to contain an \_\_init\_\_.py file to mark it as a Python package when using absolute import syntax.

+The following \_\_app\_\_ import and beyond top-level relative import are deprecated, since it isn't supported by static type checker and not supported by Python test frameworks:

+Inputs are divided into two categories in Azure Functions: trigger input and other input. Although they're different in the `function.json` file, usage is identical in Python code. Connection strings or secrets for trigger and input sources map to values in the `local.settings.json` file when running locally, and the application settings when running in Azure.

+For example, the following code demonstrates the difference between the two:

+When the function is invoked, the HTTP request is passed to the function as `req`. An entry will be retrieved from the Azure Blob Storage based on the _ID_ in the route URL and made available as `obj` in the function body. Here, the storage account specified is the connection string found in the AzureWebJobsStorage app setting, which is the same storage account used by the function app.

+Inputs are divided into two categories in Azure Functions: trigger input and other input. Although they're defined using different decorators, usage is similar in Python code. Connection strings or secrets for trigger and input sources map to values in the `local.settings.json` file when running locally, and the application settings when running in Azure.

+As an example, the following code demonstrates the difference between the two:

+```json

+// local.settings.json

+{

+ "IsEncrypted": false,

+ "Values": {

+ "FUNCTIONS_WORKER_RUNTIME": "python",

+ "AzureWebJobsStorage": "<azure-storage-connection-string>"

+ }

+}

+```

+```python

+# function_app.py

+import azure.functions as func

+import logging

+app = func.FunctionApp()

+@app.route(route="req")

+@app.read_blob(arg_name="obj", path="samples/{id}", connection="AzureWebJobsStorage")

+def main(req: func.HttpRequest,

+ obj: func.InputStream):

+ logging.info(f'Python HTTP triggered function processed: {obj.read()}')

+```

+When the function is invoked, the HTTP request is passed to the function as `req`. An entry will be retrieved from the Azure Blob Storage based on the _ID_ in the route URL and made available as `obj` in the function body. Here, the storage account specified is the connection string found in the AzureWebJobsStorage app setting, which is the same storage account used by the function app.

+At this time, only specific triggers and bindings are supported by the v2 programming model. Supported triggers and bindings are as follows.

+| Type | Trigger | Input Binding | Output Binding |

+| | | | |

+| HTTP | x | | |

+| Timer | x | | |

+| Azure Queue Storage | x | | x |

+| Azure Service Bus topic | x | | x |

+| Azure Service Bus queue | x | | x |

+| Azure Cosmos DB | x | x | x |

+| Azure Blob Storage | x | x | x |

+| Azure Event Grid | x | | x |

+To learn more about defining triggers and bindings in the v2 model, see this [documentation](https://github.com/Azure/azure-functions-python-library/blob/dev/docs/ProgModelSpec.pyi).

+Output can be expressed both in return value and output parameters. If there's only one output, we recommend using the return value. For multiple outputs, you'll have to use output parameters.

+To use the return value of a function as the value of an output binding, the `name` property of the binding should be set to `$return` in `function.json`.

+To produce multiple outputs, use the `set()` method provided by the [`azure.functions.Out`](/python/api/azure-functions/azure.functions.out) interface to assign a value to the binding. For example, the following function can push a message to a queue and also return an HTTP response.

+Output can be expressed both in return value and output parameters. If there's only one output, we recommend using the return value. For multiple outputs, you'll have to use output parameters.

+To produce multiple outputs, use the `set()` method provided by the [`azure.functions.Out`](/python/api/azure-functions/azure.functions.out) interface to assign a value to the binding. For example, the following function can push a message to a queue and also return an HTTP response.

+```python

+# function_app.py

+import azure.functions as func

+@app.write_blob(arg_name="msg", path="output-container/{name}",

+ connection="AzureWebJobsStorage")

+

+def test_function(req: func.HttpRequest,

+ msg: func.Out[str]) -> str:

+

+ message = req.params.get('body')

+ msg.set(message)

+ return message

+```

+The following example logs an info message when the function is invoked via an HTTP trigger.

+| **`critical(_message_)`** | Writes a message with level CRITICAL on the root logger. |

+| **`error(_message_)`** | Writes a message with level ERROR on the root logger. |

+| **`warning(_message_)`** | Writes a message with level WARNING on the root logger. |

+| **`info(_message_)`** | Writes a message with level INFO on the root logger. |

+| **`debug(_message_)`** | Writes a message with level DEBUG on the root logger. |

+By default, the Functions runtime collects logs and other telemetry data generated by your functions. This telemetry ends up as traces in Application Insights. Request and dependency telemetry for certain Azure services are also collected by default by [triggers and bindings](functions-triggers-bindings.md#supported-bindings). To collect custom request and custom dependency telemetry outside of bindings, you can use the [OpenCensus Python Extensions](https://github.com/census-ecosystem/opencensus-python-extensions-azure). This extension sends custom telemetry data to your Application Insights instance. You can find a list of supported extensions at the [OpenCensus repository](https://github.com/census-instrumentation/opencensus-python/tree/master/contrib).

+## HTTP trigger

+The HTTP trigger is defined in the function.json file. The `name` of the binding must match the named parameter in the function.

+In the previous examples, a binding name `req` is used. This parameter is an [HttpRequest] object, and an [HttpResponse] object is returned.

+From the [HttpRequest] object, you can get request headers, query parameters, route parameters, and the message body.

+The following example is from the [HTTP trigger template for Python](https://github.com/Azure/azure-functions-templates/tree/dev/Functions.Templates/Templates/HttpTrigger-Python).

+In this function, the value of the `name` query parameter is obtained from the `params` parameter of the [HttpRequest] object. The JSON-encoded message body is read using the `get_json` method.

+Likewise, you can set the `status_code` and `headers` for the response message in the returned [HttpResponse] object.

+The HTTP trigger is defined in the function.json file. The `name` of the binding must match the named parameter in the function.

+In the previous examples, a binding name `req` is used. This parameter is an [HttpRequest] object, and an [HttpResponse] object is returned.

+From the [HttpRequest] object, you can get request headers, query parameters, route parameters, and the message body.

+The following example is from the HTTP trigger template for Python v2 programming model. It's the sample code provided when you create a function from Core Tools or VS Code.

+```python

+@app.function_name(name="HttpTrigger1")

+@app.route(route="hello")

+def test_function(req: func.HttpRequest) -> func.HttpResponse:

+ logging.info('Python HTTP trigger function processed a request.')

+ name = req.params.get('name')

+ if not name:

+ try:

+ req_body = req.get_json()

+ except ValueError:

+ pass

+ else:

+ name = req_body.get('name')

+ if name:

+ return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")

+ else:

+ return func.HttpResponse(

+ "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",

+ status_code=200

+ )

+```

+In this function, the value of the `name` query parameter is obtained from the `params` parameter of the [HttpRequest] object. The JSON-encoded message body is read using the `get_json` method.

+Likewise, you can set the `status_code` and `headers` for the response message in the returned [HttpResponse] object.

+To pass in a name in this example, paste the URL provided when running the function, and append it with "?name={name}"

+First, the function.json file must be updated to include a `route` in the HTTP trigger, as shown in the following example:

+The host.json file must also be updated to include an HTTP `routePrefix`, as shown in the following example.

+Update the Python code file `init.py`, depending on the interface used by your framework. The following example shows either an ASGI handler approach or a WSGI wrapper approach for Flask:

+For a full example, see [Using Flask Framework with Azure Functions](/samples/azure-samples/flask-app-on-azure-functions/azure-functions-python-create-flask-app/).

+You can use ASGI and WSGI-compatible frameworks such as Flask and FastAPI with your HTTP-triggered Python functions, which is shown in the following example:

+# [ASGI](#tab/asgi)

+`AsgiFunctionApp` is the top-level function app class for constructing ASGI HTTP functions.

+```python

+# function_app.py

+import azure.functions as func

+from fastapi import FastAPI, Request, Response

+

+fast_app = FastAPI()

+

+@fast_app.get("/return_http_no_body")

+async def return_http_no_body():

+ return Response(content='', media_type="text/plain")

+

+app = func.AsgiFunctionApp(app=fast_app,

+ http_auth_level=func.AuthLevel.ANONYMOUS)

+```

+# [WSGI](#tab/wsgi)

+`WsgiFunctionApp` is top level function app class for constructing WSGI HTTP functions.

+```python

+# function_app.py

+import azure.functions as func

+from flask import Flask, request, Response, redirect, url_for

+

+flask_app = Flask(__name__)

+logger = logging.getLogger("my-function")

+@flask_app.get("/return_http")

+def return_http():

+ return Response('<h1>Hello WorldΓäó</h1>', mimetype='text/html')

+app = func.WsgiFunctionApp(app=flask_app.wsgi_app,

+ http_auth_level=func.AuthLevel.ANONYMOUS)

+```

+For scaling and performance best practices for Python function apps, see the [Python scale and performance article](python-scale-performance-reference.md).

+The [**Context**](/python/api/azure-functions/azure.functions.context) class has the following string attributes:

+`function_directory`

+The directory in which the function is running.

+`function_name`

+Name of the function.

+`invocation_id`

+ID of the current function invocation.

+`trace_context`

+Context for distributed tracing. For more information, see [`Trace Context`](https://www.w3.org/TR/trace-context/).

+`retry_context`

+Context for retries to the function. For more information, see [`retry-policies`](./functions-bindings-errors.md#retry-policies).

+It isn't guaranteed that the state of your app will be preserved for future executions. However, the Azure Functions runtime often reuses the same process for multiple executions of the same app. In order to cache the results of an expensive computation, declare it as a global variable.

+In Functions, [application settings](functions-app-settings.md), such as service connection strings, are exposed as environment variables during execution. There are two main ways to access these settings in your code.

+| **`os.environ["myAppSetting"]`** | Tries to get the application setting by key name, raising an error when unsuccessful. |

+| **`os.getenv("myAppSetting")`** | Tries to get the application setting by key name, returning null when unsuccessful. |

+In Functions, [application settings](functions-app-settings.md), such as service connection strings, are exposed as environment variables during execution. There are two main ways to access these settings in your code.

+| Method | Description |

+| | |

+| **`os.environ["myAppSetting"]`** | Tries to get the application setting by key name, raising an error when unsuccessful. |

+| **`os.getenv("myAppSetting")`** | Tries to get the application setting by key name, returning null when unsuccessful. |

+Both of these ways require you to declare `import os`.

+The following example uses `os.environ["myAppSetting"]` to get the [application setting](functions-how-to-use-azure-function-app-settings.md#settings), with the key named `myAppSetting`:

+```python

+import logging

+import os

+import azure.functions as func

+@app.function_name(name="HttpTrigger1")

+@app.route(route="req")

+def main(req: func.HttpRequest) -> func.HttpResponse:

+ # Get the setting named 'myAppSetting'

+ my_app_setting_value = os.environ["myAppSetting"]

+ logging.info(f'My app setting value:{my_app_setting_value}')

+For local development, application settings are [maintained in the local.settings.json file](functions-develop-local.md#local-settings-file).

+When using the new programming model, the following app setting needs to be enabled in the file `localsettings.json` as follows.

+```json

+"AzureWebJobsFeatureFlags": "EnableWorkerIndexing"

+When deploying the function, this setting won't be automatically created. You must explicitly create this setting in your function app in Azure for it to run using the v2 model.

+Multiple Python workers aren't supported in v2 at this time. This means that setting `FUNCTIONS_WORKER_PROCESS_COUNT` to greater than 1 isn't supported for the functions using the v2 model.

+## Python version

+Azure Functions supports the following Python versions:

+| Functions version | Python^* versions |

+| -- | -- |

+| 4.x | 3.9<br/> 3.8<br/>3.7 |

+| 3.x | 3.9<br/> 3.8<br/>3.7<br/>3.6 |

+| 2.x | 3.7<br/>3.6 |

+^*Official Python distributions

+To request a specific Python version when you create your function app in Azure, use the `--runtime-version` option of the [`az functionapp create`](/cli/azure/functionapp#az-functionapp-create) command. The Functions runtime version is set by the `--functions-version` option. The Python version is set when the function app is created and can't be changed.

+The runtime uses the available Python version, when you run it locally.

+### Changing Python version

+To set a Python function app to a specific language version, you need to specify the language and the version of the language in `LinuxFxVersion` field in site config. For example, to change Python app to use Python 3.8, set `linuxFxVersion` to `python|3.8`.

+To learn how to view and change the `linuxFxVersion` site setting, see [How to target Azure Functions runtime versions](set-runtime-version.md#manual-version-updates-on-linux).

+For more general information, see the [Azure Functions runtime support policy](./language-support-policy.md) and [Supported languages in Azure Functions](./supported-languages.md).

+When developing locally using the Azure Functions Core Tools or Visual Studio Code, add the names and versions of the required packages to the `requirements.txt` file and install them using `pip`.

+For example, the following requirements file and pip command can be used to install the `requests` package from PyPI.

+When you're ready to publish, make sure that all your publicly available dependencies are listed in the requirements.txt file. You can locate this file at the root of your project directory.

+Project files and folders that are excluded from publishing, including the virtual environment folder, you can find them in the root directory of your project.

+There are three build actions supported for publishing your Python project to Azure: remote build, local build, and builds using custom dependencies.

+You can also use Azure Pipelines to build your dependencies and publish using continuous delivery (CD). To learn more, see [Continuous delivery with Azure Pipelines](functions-how-to-azure-devops.md).

+When you use remote build, dependencies restored on the server and native dependencies match the production environment. This results in a smaller deployment package to upload. Use remote build when developing Python apps on Windows. If your project has custom dependencies, you can [use remote build with extra index URL](#remote-build-with-extra-index-url).

+Dependencies are obtained remotely based on the contents of the requirements.txt file. [Remote build](functions-deployment-technologies.md#remote-build) is the recommended build method. By default, the Azure Functions Core Tools requests a remote build when you use the following [`func azure functionapp publish`](functions-run-local.md#publish) command to publish your Python project to Azure.

+Remember to replace `<APP_NAME>` with the name of your function app in Azure.

+The [Azure Functions Extension for Visual Studio Code](./create-first-function-vs-code-csharp.md#publish-the-project-to-azure) also requests a remote build by default.

+Dependencies are obtained locally based on the contents of the requirements.txt file. You can prevent doing a remote build by using the following [`func azure functionapp publish`](functions-run-local.md#publish) command to publish with a local build.

+Remember to replace `<APP_NAME>` with the name of your function app in Azure.

+When you use the `--build local` option, project dependencies are read from the requirements.txt file, and those dependent packages are downloaded and installed locally. Project files and dependencies are deployed from your local computer to Azure. This results in a larger deployment package being uploaded to Azure. If for some reason, you can't get requirements.txt file by Core Tools, you must use the custom dependencies option for publishing.

+We don't recommend using local builds when developing locally on Windows.

+When your project has dependencies not found in the [Python Package Index](https://pypi.org/), there are two ways to build the project. The build method depends on how you build the project.

+When your packages are available from an accessible custom package index, use a remote build. Before publishing, make sure to [create an app setting](functions-how-to-use-azure-function-app-settings.md#settings) named `PIP_EXTRA_INDEX_URL`. The value for this setting is the URL of your custom package index. Using this setting tells the remote build to run `pip install` using the `--extra-index-url` option. To learn more, see the [Python pip install documentation](https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format).

+#### Install local packages

+If your project uses packages not publicly available to our tools, you can make them available to your app by putting them in the \_\_app\_\_/.python_packages directory. Before publishing, run the following command to install the dependencies locally:

+When using custom dependencies, you should use the `--no-build` publishing option, since you've already installed the dependencies into the project folder.

+Remember to replace `<APP_NAME>` with the name of your function app in Azure.

+Functions written in Python can be tested like other Python code using standard testing frameworks. For most bindings, it's possible to create a mock input object by creating an instance of an appropriate class from the `azure.functions` package. Since the [`azure.functions`](https://pypi.org/project/azure-functions/) package isn't immediately available, be sure to install it via your `requirements.txt` file as described in the [package management](#package-management) section above.

+Take *my_second_function* as an example, following is a mock test of an HTTP triggered function:

+First we need to create *<project_root>/my_second_function/function.json* file and define this function as an http trigger.

+Now, we can implement the *my_second_function* and the *shared_code.my_second_helper_function*.

+# Define an http trigger which accepts ?value=<int> query parameter

+We can start writing test cases for our http trigger.

+Inside your `.venv` Python virtual environment, install your favorite Python test framework, such as `pip install pytest`. Then run `pytest tests` to check the test result.

+First we need to create *<project_root>/function_app.py* file and implement *my_second_function* function as http trigger and the *shared_code.my_second_helper_function*.

+```python

+# <project_root>/function_app.py

+import azure.functions as func

+import logging

+# Use absolute import to resolve shared_code modules

+from shared_code import my_second_helper_function

+app = func.FunctionApp()

+# Define http trigger which accepts ?value=<int> query parameter

+# Double the value and return the result in HttpResponse

+@app.function_name(name="my_second_function")

+@app.route(route="hello")

+def main(req: func.HttpRequest) -> func.HttpResponse:

+ logging.info('Executing my_second_function.')

+ initial_value: int = int(req.params.get('value'))

+ doubled_value: int = my_second_helper_function.double(initial_value)

+ return func.HttpResponse(

+ body=f"{initial_value} * 2 = {doubled_value}",

+ status_code=200

+ )

+```

+```python

+# <project_root>/shared_code/__init__.py

+# Empty __init__.py file marks shared_code folder as a Python package

+```

+```python

+# <project_root>/shared_code/my_second_helper_function.py

+def double(value: int) -> int:

+ return value * 2

+```

+We can start writing test cases for our http trigger.

+```python

+# <project_root>/tests/test_my_second_function.py

+import unittest

+import azure.functions as func

+from function_app import main

+class TestFunction(unittest.TestCase):

+ def test_my_second_function(self):

+ # Construct a mock HTTP request.

+ req = func.HttpRequest(

+ method='GET',

+ body=None,

+ url='/api/my_second_function',

+ params={'value': '21'})

+ # Call the function.

+ func_call = main.build().get_user_function()

+ resp = func_call(req)

+ # Check the output.

+ self.assertEqual(

+ resp.get_body(),

+ b'21 * 2 = 42',

+ )

+```

+Inside your `.venv` Python virtual environment, install your favorite Python test framework, such as `pip install pytest`. Then run `pytest tests` to check the test result.

+The `tempfile.gettempdir()` method returns a temporary folder, which on Linux is `/tmp`. Your application can use this directory to store temporary files generated and used by your functions during execution.

+> Files written to the temporary directory aren't guaranteed to persist across invocations. During scale out, temporary files aren't shared between instances.

+The following example creates a named temporary file in the temporary directory (`/tmp`):

+We recommend that you maintain your tests in a folder separate from the project folder. This action keeps you from deploying test code with your app.

+There are a few libraries that come with the Python Functions runtime.

+The Python Standard Library contains a list of built-in Python modules that are shipped with each Python distribution. Most of these libraries help you access system functionality, like file I/O. On Windows systems, these libraries are installed with Python. On the Unix-based systems, they're provided by package collections.

+To view the full details of the list of these libraries, see the links below:

+### Azure Functions Python worker dependencies

+The Functions Python worker requires a specific set of libraries. You can also use these libraries in your functions, but they aren't a part of the Python standard. If your functions rely on any of these libraries, they may not be available to your code when running outside of Azure Functions. You can find a detailed list of dependencies in the **install\_requires** section in the [setup.py](https://github.com/Azure/azure-functions-python-worker/blob/dev/setup.py#L282) file.

+> If your function app's requirements.txt contains an `azure-functions-worker` entry, remove it. The functions worker is automatically managed by Azure Functions platform, and we regularly update it with new features and bug fixes. Manually installing an old version of worker in requirements.txt may cause unexpected issues.

+> If your package contains certain libraries that may collide with worker's dependencies (e.g. protobuf, tensorflow, grpcio), please configure [`PYTHON_ISOLATE_WORKER_DEPENDENCIES`](functions-app-settings.md#python_isolate_worker_dependencies-preview) to `1` in app settings to prevent your application from referring worker's dependencies. This feature is in preview.

+### Azure Functions Python library

+Every Python worker update includes a new version of [Azure Functions Python library (azure.functions)](https://github.com/Azure/azure-functions-python-library). This approach makes it easier to continuously update your Python function apps, because each update is backwards-compatible. A list of releases of this library can be found in [azure-functions PyPi](https://pypi.org/project/azure-functions/#history).

+The runtime library version is fixed by Azure, and it can't be overridden by requirements.txt. The `azure-functions` entry in requirements.txt is only for linting and customer awareness.

+Use the following code to track the actual version of the Python Functions library in your runtime:

+For a list of preinstalled system libraries in Python worker Docker images, see the links below:

+| **Application-level** | When imported into any function trigger, the extension applies to every function execution in the app. |

+| **Function-level** | Execution is limited to only the specific function trigger into which it's imported. |

+Review the information for a given extension to learn more about the scope in which the extension runs.

+Extensions implement a Python worker extension interface. This action lets the Python worker process call into the extension code during the function execution lifecycle. To learn more, see [Creating extensions](#creating-extensions).

+1. Add the extension package in the requirements.txt file for your project.

+ + Locally: add `"PYTHON_ENABLE_WORKER_EXTENSIONS": "1"` in the `Values` section of your [local.settings.json file](functions-develop-local.md#local-settings-file).

+ + Azure: add `PYTHON_ENABLE_WORKER_EXTENSIONS=1` to your [app settings](functions-how-to-use-azure-function-app-settings.md#settings).

+1. Configure the extension instance, if needed. Configuration requirements should be called-out in the extension's documentation.

+> Third-party Python worker extension libraries are not supported or warrantied by Microsoft. You must make sure that any extensions you use in your function app is trustworthy, and you bear the full risk of using a malicious or poorly written extension.

+Third-parties should provide specific documentation on how to install and consume their specific extension in your function app. For a basic example of how to consume an extension, see [Consuming your extension](develop-python-worker-extensions.md#consume-your-extension-locally).

+# [Application-level](#tab/application-level)

+# [Function-level](#tab/function-level)

+Extensions are created by third-party library developers who have created functionality that can be integrated into Azure Functions. An extension developer design, implements, and releases Python packages that contain custom logic designed specifically to be run in the context of function execution. These extensions can be published either to the PyPI registry or to GitHub repositories.

+| **`init`** | Called after the extension is imported. |

+| **`configure`** | Called from function code when needed to configure the extension. |

+| **`post_function_load_app_level`** | Called right after the function is loaded. The function name and function directory are passed to the extension. Keep in mind that the function directory is read-only, and any attempt to write to local file in this directory fails. |

+| **`pre_invocation_app_level`** | Called right before the function is triggered. The function context and function invocation arguments are passed to the extension. You can usually pass other attributes in the context object for the function code to consume. |

+| **`post_invocation_app_level`** | Called right after the function execution completes. The function context, function invocation arguments, and the invocation return object are passed to the extension. This implementation is a good place to validate whether execution of the lifecycle hooks succeeded. |

+| **`__init__`** | This method is the constructor of the extension. It's called when an extension instance is initialized in a specific function. When implementing this abstract method, you may want to accept a `filename` parameter and pass it to the parent's method `super().__init__(filename)` for proper extension registration. |

+| **`post_function_load`** | Called right after the function is loaded. The function name and function directory are passed to the extension. Keep in mind that the function directory is read-only, and any attempt to write to local file in this directory fails. |

+| **`pre_invocation`** | Called right before the function is triggered. The function context and function invocation arguments are passed to the extension. You can usually pass other attributes in the context object for the function code to consume. |

+| **`post_invocation`** | Called right after the function execution completes. The function context, function invocation arguments, and the invocation return object are passed to the extension. This implementation is a good place to validate whether execution of the lifecycle hooks succeeded. |

+To improve throughput, Functions let your out-of-process Python language worker share memory with the Functions host process. When your function app is hitting bottlenecks, you can enable shared memory by adding an application setting named [FUNCTIONS_WORKER_SHARED_MEMORY_DATA_TRANSFER_ENABLED](functions-app-settings.md#functions_worker_shared_memory_data_transfer_enabled) with a value of `1`. With shared memory enabled, you can then use the [DOCKER_SHM_SIZE](functions-app-settings.md#docker_shm_size) setting to set the shared memory to something like `268435456`, which is equivalent to 256 MB.

+For example, you might enable shared memory to reduce bottlenecks when using Blob storage bindings to transfer payloads larger than 1 MB.

+This functionality is available only for function apps running in Premium and Dedicated (App Service) plans. To learn more, see [Shared memory](https://github.com/Azure/azure-functions-python-worker/wiki/Shared-Memory).

+## Known issues and FAQ

+The following is a list of troubleshooting guides for common issues:

+* [Can't import 'cygrpc'](recover-python-functions.md#troubleshoot-cannot-import-cygrpc).

+Following is a list of troubleshooting guides for known issues with the v2 programming model:

+* [Couldn't load file or assembly](recover-python-functions.md#troubleshoot-could-not-load-file-or-assembly)

+* [Unable to resolve the Azure Storage connection named Storage](recover-python-functions.md#troubleshoot-unable-to-resolve-the-azure-storage-connection).

+All known issues and feature requests are tracked using [GitHub issues](https://github.com/Azure/azure-functions-python-worker/issues) list. If you run into a problem and can't find the issue in GitHub, open a new issue and include a detailed description of the problem.

+* [Blob storage bindings](functions-bindings-storage-blob.md)

+* [HTTP and Webhook bindings](functions-bindings-http-webhook.md)

+* [Queue storage bindings](functions-bindings-storage-queue.md)

+[HttpResponse]: /python/api/azure-functions/azure.functions.httpresponse

+By default, Azure Functions automatically monitors the load on your application and creates more host instances for Python as needed. Azure Functions uses built-in thresholds for different trigger types to decide when to add instances, such as the age of messages and queue size for QueueTrigger. These thresholds aren't user configurable. For more information, see [Event-driven scaling in Azure Functions](event-driven-scaling.md).

+The default configurations are suitable for most of Azure Functions applications. However, you can improve the performance of your applications' throughput by employing configurations based on your workload profile. The first step is to understand the type of workload that you're running.

+Here's an example of a function with HTTP trigger that uses [aiohttp](https://pypi.org/project/aiohttp/) http client:

+A function without the `async` keyword is run automatically in a ThreadPoolExecutor thread pool:

+In order to achieve the full benefit of running functions asynchronously, the I/O operation/library that is used in your code needs to have async implemented as well. Using synchronous I/O operations in functions that are defined as asynchronous **may hurt** the overall performance. If the libraries you're using don't have async version implemented, you may still benefit from running your code asynchronously by [managing event loop](#managing-event-loop) in your app.

+Here are a few examples of client libraries that have implemented async patterns:

+When you define `async` in front of a function signature, Python will mark the function as a coroutine. When calling the coroutine, it can be scheduled as a task into an event loop. When you call `await` in an async function, it registers a continuation into the event loop, which allows the event loop to process the next task during the wait time.

+In our Python Worker, the worker shares the event loop with the customer's `async` function and it's capable for handling multiple requests concurrently. We strongly encourage our customers to make use of asyncio compatible libraries, such as [aiohttp](https://pypi.org/project/aiohttp/) and [pyzmq](https://pypi.org/project/pyzmq/). Following these recommendations increases your function's throughput compared to those libraries when implemented synchronously.

+By default, every Functions host instance has a single language worker process. You can increase the number of worker processes per host (up to 10) by using the [`FUNCTIONS_WORKER_PROCESS_COUNT`](functions-app-settings.md#functions_worker_process_count) application setting. Azure Functions then tries to evenly distribute simultaneous function invocations across these workers.

+For CPU bound apps, you should set the number of language workers to be the same as or higher than the number of cores that are available per function app. To learn more, see [Available instance SKUs](functions-premium-plan.md#available-instance-skus).

+The `FUNCTIONS_WORKER_PROCESS_COUNT` applies to each host that Functions creates when scaling out your application to meet demand.

+> [!NOTE]

+> Multiple Python workers are not supported in V2 at this time. This means that enabling intelligent concurrency and setting `FUNCTIONS_WORKER_PROCESS_COUNT` greater than 1 is not supported for functions developed using the V2 model.

+As mentioned in the async [section](#understanding-async-in-python-worker), the Python language worker treats functions and [coroutines](https://docs.python.org/3/library/asyncio-task.html#coroutines) differently. A coroutine is run within the same event loop that the language worker runs on. On the other hand, a function invocation is run within a [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor), which is maintained by the language worker as a thread.

+For I/O-bound apps, you should see substantial gains by increasing the number of threads working on each invocation. the recommendation is to start with the Python default - the number of cores + 4 and then tweak based on the throughput values you're seeing.

+For mix workloads apps, you should balance both `FUNCTIONS_WORKER_PROCESS_COUNT` and `PYTHON_THREADPOOL_THREAD_COUNT` configurations to maximize the throughput. To understand what your function apps spend the most time on, we recommend profiling them and set the values according to the behavior they present. Also refer to this [section](#use-multiple-language-worker-processes) to learn about FUNCTIONS_WORKER_PROCESS_COUNT application settings.

+There are many useful Python official documents discussing the [Coroutines and Tasks](https://docs.python.org/3/library/asyncio-task.html) and [Event Loop](https://docs.python.org/3.8/library/asyncio-eventloop.html) by using the built-in **asyncio** library.

+For more processing units especially in CPU-bound operation, you might be able to get this by upgrading to premium plan with higher specifications. With higher processing units, you can adjust the number of worker processes count according to the number of cores available and achieve higher degree of parallelism.

+zone_pivot_groups: python-mode-functions

+This article provides information to help you troubleshoot errors with your Python functions in Azure Functions. This article supports both the v1 and v2 programming models. Choose your desired model from the selector at the top of the article. The v2 model is currently in preview. For more information on Python programming models, see the [Python developer guide](./functions-reference-python.md).

+The following is a list of troubleshooting sections for common issues in Python functions:

+* [Troubleshoot errors with Protocol Buffers](#troubleshoot-errors-with-protocol-buffers)

+* [ModuleNotFoundError and ImportError](#troubleshoot-modulenotfounderror)

+* [Cannot import 'cygrpc'](#troubleshoot-cannot-import-cygrpc)

+* [Python exited with code 137](#troubleshoot-python-exited-with-code-137)

+* [Python exited with code 139](#troubleshoot-python-exited-with-code-139)

+* [Troubleshoot errors with Protocol Buffers](#troubleshoot-errors-with-protocol-buffers)

+* [Multiple Python workers not supported](#multiple-python-workers-not-supported)

+* [Could not load file or assembly](#troubleshoot-could-not-load-file-or-assembly)

+* [Unable to resolve the Azure Storage connection named Storage](#troubleshoot-unable-to-resolve-the-azure-storage-connection)

+* [Issues with deployment](#issue-with-deployment)

+* If the function app has `WEBSITE_RUN_FROM_PACKAGE` and it's set to `1`, navigate to `https://<app-name>.scm.azurewebsites.net/api/vfs/data/SitePackages` and download the file from the latest `href` URL.

+* If none of these suggestions resolve the issue, navigate to `https://<app-name>.scm.azurewebsites.net/DebugConsole` and view the content under `/home/site/wwwroot`.

+Python functions run only on Linux in Azure: Functions runtime v2.x runs on Debian Stretch and the v3.x runtime on Debian Buster. The artifact is expected to contain the correct Linux binaries. When you use the `--build local` flag in Core Tools, third-party, or outdated tools it may cause older binaries to be used.

+If you've verified that the package is resolved correctly with the proper Linux wheels, there may be a conflict with other packages. In certain packages, the PyPi documentations may clarify the incompatible modules. For example in [`azure 4.0.0`](https://pypi.org/project/azure/4.0.0/), there's a statement as follows:

+The `Module Not Found` error may not occur when you're using Windows or macOS for local development. However, the package fails to import on Azure Functions, which uses Linux at runtime. This issue is likely to be caused by using `pip freeze` to export virtual environment into requirements.txt from your Windows or macOS machine during project initialization.

+# [Visual Studio Code](#tab/vscode)

+# [Azure Functions Core Tools](#tab/coretools)

+# [Manual publishing](#tab/manual)

+If you're manually publishing your package into the `https://<app-name>.scm.azurewebsites.net/api/zipdeploy` endpoint, make sure that both `SCM_DO_BUILD_DURING_DEPLOYMENT` and `ENABLE_ORYX_BUILD` are set to `true`. To learn more, see [how to work with application settings](functions-how-to-use-azure-function-app-settings.md#settings).

+Browse the latest package version in `https://pypi.org/project/<package-name>` and check the **Classifiers:** section. The package should be `OS Independent`, or compatible with `POSIX` or `POSIX :: Linux` in **Operating System**. Also, the Programming Language should contain: `Python :: 3`, `Python :: 3.6`, `Python :: 3.7`, `Python :: 3.8`, or `Python :: 3.9`.

+Sometimes, the package may have been integrated into [Python Standard Library](https://docs.python.org/3/library/) (such as `pathlib`). If so, since we provide a certain Python distribution in Azure Functions (Python 3.6, Python 3.7, Python 3.8, and Python 3.9), the package in your requirements.txt should be removed.

+However, if you're facing an issue that it hasn't been fixed and you're on a deadline. I encourage you to do some research and find a similar package for your project. Usually, the Python community will provide you with a wide variety of similar libraries that you can use.

+- [The Python interpreter isn't supported by Azure Functions Python Worker](#the-python-interpreter-isnt-supported-by-azure-functions-python-worker)

+#### The Python interpreter isn't supported by Azure Functions Python Worker

+Check if your Python interpreter matches our expected version by `py --version` in Windows or `python3 --version` in Unix-like systems. Ensure the return result is Python 3.6.x, Python 3.7.x, Python 3.8.x, or Python 3.9.x.

+If your Python interpreter version doesn't meet the requirements for Functions, instead download the Python 3.6, 3.7, 3.8, or 3.9 interpreter from [Python Software Foundation](https://www.python.org/downloads).

+Visit the tutorial section in [memory profiling on Python functions](python-memory-profiler-reference.md#memory-profiling-process) to analyze the memory bottleneck in your function app.

+This error occurs when a Python function app is forced to terminate by the operating system with a SIGSEGV signal. This signal indicates a memory segmentation violation, which can be caused by unexpectedly reading from or writing into a restricted memory region. In the following sections, we provide a list of common root causes.

+If your function app is using the Python pickel library to load Python object from .pkl file, it's possible that the .pkl contains malformed bytes string, or invalid address reference in it. To recover from this issue, try commenting out the pickle.load() function.

+## Multiple Python workers not supported

+Multiple Python workers aren't supported in the v2 programming model at this time. This means that enabling intelligent concurrency by setting `FUNCTIONS_WORKER_PROCESS_COUNT` greater than 1 isn't supported for functions developed using the V2 model.

+## Troubleshoot could not load file or assembly

+If you're facing this error, it may be the case that you are using the V2 programming model. This error is due to a known issue that will be resolved in an upcoming release.

+This specific error may ready:

+> `DurableTask.Netherite.AzureFunctions: Could not load file or assembly 'Microsoft.Azure.WebJobs.Extensions.DurableTask, Version=2.0.0.0, Culture=neutral, PublicKeyToken=014045d636e89289'.`

+> `The system cannot find the file specified.`

+The reason this error may be occurring is because of an issue with how the extension bundle was cached. To detect if this is the issue, you can run the command with `--verbose` to see more details.

+> `func host start --verbose`

+Upon running the command, if you notice that `Loading startup extension <>` is not followed by `Loaded extension <>` for each extension, it is likely that you are facing a caching issue.

+To resolve this issue,

+1. Find the path of `.azure-functions-core-tools` by running

+```console

+func GetExtensionBundlePath

+```

+2. Delete the directory `.azure-functions-core-tools`

+# [bash](#tab/bash)

+```bash

+rm -r <insert path>/.azure-functions-core-tools

+```

+# [PowerShell](#tab/powershell)

+```powershell

+Remove-Item <insert path>/.azure-functions-core-tools

+```

+# [Cmd](#tab/cmd)

+```cmd

+rmdir <insert path>/.azure-functions-core-tools

+```

+## Troubleshoot unable to resolve the Azure Storage connection

+You may see this error in your local output as the following message:

+> `Microsoft.Azure.WebJobs.Extensions.DurableTask: Unable to resolve the Azure Storage connection named 'Storage'.`

+> `Value cannot be null. (Parameter 'provider')`

+This error is a result of how extensions are loaded from the bundle locally. To resolve this error, you can do one of the following:

+* Use a storage emulator such as [Azurite](../storage/common/storage-use-azurite.md). This is a good option when you aren't planning to use a storage account in your function application.

+* Create a storage account and add a connection string to the `AzureWebJobsStorage` environment variable in `localsettings.json`. Use this option when you are using a storage account trigger or binding with your application, or if you have an existing storage account. To get started, see [Create a storage account](../storage/common/storage-account-create.md).

+## Issue with Deployment

+In the [Azure portal](https://portal.azure.com), navigate to **Settings** > **Configuration** and make sure that the `AzureWebJobsFeatureFlags` application setting has a value of `EnableWorkerIndexing`. If it is not found, add this setting to the function app.

+**Resolution**: To allow ITSM connections make sure ActionGroup network tag is allowed on your network.

+> * For information about the pricing of the ITSM action, see the [pricing page](https://azure.microsoft.com/pricing/details/monitor/) for action groups.

+> * The short description field in the alert rule definition is limited to 40 characters when you send it by using the ITSM action.

+>

+> * In case you have policies for inbound traffic for your ServiceNow instances, add ActionGroup service tag to allowList.

+ ```xml

+ <?xml version="1.0" encoding="utf-8"?>

+ ```

+* [Configure sampling](sampling.md) to help reduce telemetry traffic and data storage costs.

+ <PropertyGroup>

+ <TargetFramework>netstandard2.0</TargetFramework>

+ </PropertyGroup>

+ <ItemGroup>

+ <PackageReference Include="Microsoft.ApplicationInsights" Version="2.15.0" /> <!-- Ensure youΓÇÖre using the latest version -->

+ </ItemGroup>

+ :::image type="content" source="media/availability-azure-functions/function-proj.png" alt-text=" Screenshot of function.proj in App Service Editor." lightbox="media/availability-azure-functions/function-proj.png":::

+ using (var httpClient = new HttpClient())

+ {

+ // TODO: Replace with your business logic

+ await httpClient.GetStringAsync("https://www.bing.com/");

+ }

+ if (telemetryClient == null)

+ {

+ // Initializing a telemetry configuration for Application Insights based on connection string

+

+ var telemetryConfiguration = new TelemetryConfiguration();

+ telemetryConfiguration.ConnectionString = Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING");

+ telemetryConfiguration.TelemetryChannel = new InMemoryChannel();

+ telemetryClient = new TelemetryClient(telemetryConfiguration);

+ }

+

+ string testName = executionContext.FunctionName;

+ string location = Environment.GetEnvironmentVariable("REGION_NAME");

+ var availability = new AvailabilityTelemetry

+ {

+ Name = testName,

+

+ RunLocation = location,

+

+ Success = false,

+ };

+

+ availability.Context.Operation.ParentId = Activity.Current.SpanId.ToString();

+ availability.Context.Operation.Id = Activity.Current.RootId;

+ var stopwatch = new Stopwatch();

+ stopwatch.Start();

+

+ try

+ {

+ using (var activity = new Activity("AvailabilityContext"))

+ {

+ activity.Start();

+ availability.Id = Activity.Current.SpanId.ToString();

+ // Run business logic

+ await RunAvailabilityTestAsync(log);

+ }

+ availability.Success = true;

+ }

+

+ catch (Exception ex)

+ {

+ availability.Message = ex.Message;

+ throw;

+ }

+

+ finally

+ {

+ stopwatch.Stop();

+ availability.Duration = stopwatch.Elapsed;

+ availability.Timestamp = DateTimeOffset.UtcNow;

+ telemetryClient.TrackAvailability(availability);

+ telemetryClient.Flush();

+ }

+> [Standard test](availability-standard-tests.md)

+Container insights collects stdout, stderr, and environmental variables from container workloads deployed to managed Kubernetes clusters from the containerized agent. You can configure agent data collection settings by creating a custom Kubernetes ConfigMap to control this experience.

+This article demonstrates how to create ConfigMaps and configure data collection based on your requirements.

+A template ConfigMap file is provided so that you can easily edit it with your customizations without having to create it from scratch. Before you start, review the Kubernetes documentation about [ConfigMaps](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/). Familiarize yourself with how to create, configure, and deploy ConfigMaps. You need to know how to filter stderr and stdout per namespace or across the entire cluster. You also need to know how to filter environment variables for any container running across all pods/nodes in the cluster.

+>The minimum agent version supported to collect stdout, stderr, and environmental variables from container workloads is **ciprod06142019** or later. To verify your agent version, on the **Node** tab, select a node. On the **Properties** pane, note the value of the **Agent Image Tag** property. For more information about the agent versions and what's included in each release, see [Agent release notes](https://github.com/microsoft/Docker-Provider/tree/ci_feature_prod).

+The following table describes the settings you can configure to control data collection.

+| `schema-version` | String (case sensitive) | v1 | This schema version is used by the agent<br> when parsing this ConfigMap.<br> Currently supported schema-version is v1.<br> Modifying this value isn't supported and will be<br> rejected when the ConfigMap is evaluated. |

+| `config-version` | String | | Supports the ability to keep track of this config file's version in your source control system/repository.<br> Maximum allowed characters are 10, and all other characters are truncated. |

+| `[log_collection_settings.stdout] enabled =` | Boolean | True or false | Controls if stdout container log collection is enabled. When set to `true` and no namespaces are excluded for stdout log collection<br> (`log_collection_settings.stdout.exclude_namespaces` setting), stdout logs will be collected from all containers across all pods/nodes in the cluster. If not specified in the ConfigMap,<br> the default value is `enabled = true`. |

+| `[log_collection_settings.stdout] exclude_namespaces =` | String | Comma-separated array | Array of Kubernetes namespaces for which stdout logs won't be collected. This setting is effective only if<br> `log_collection_settings.stdout.enabled`<br> is set to `true`.<br> If not specified in the ConfigMap, the default value is<br> `exclude_namespaces = ["kube-system","gatekeeper-system"]`. |

+| `[log_collection_settings.stderr] enabled =` | Boolean | True or false | Controls if stderr container log collection is enabled.<br> When set to `true` and no namespaces are excluded for stdout log collection<br> (`log_collection_settings.stderr.exclude_namespaces` setting), stderr logs will be collected from all containers across all pods/nodes in the cluster.<br> If not specified in the ConfigMap, the default value is<br> `enabled = true`. |

+| `[log_collection_settings.stderr] exclude_namespaces =` | String | Comma-separated array | Array of Kubernetes namespaces for which stderr logs won't be collected.<br> This setting is effective only if<br> `log_collection_settings.stdout.enabled` is set to `true`.<br> If not specified in the ConfigMap, the default value is<br> `exclude_namespaces = ["kube-system","gatekeeper-system"]`. |

+| `[log_collection_settings.env_var] enabled =` | Boolean | True or false | This setting controls environment variable collection<br> across all pods/nodes in the cluster<br> and defaults to `enabled = true` when not specified<br> in the ConfigMap.<br> If collection of environment variables is globally enabled, you can disable it for a specific container<br> by setting the environment variable<br> `AZMON_COLLECT_ENV` to `False` either with a Dockerfile setting or in the [configuration file for the Pod](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) under the `env:` section.<br> If collection of environment variables is globally disabled, you can't enable collection for a specific container. The only override that can be applied at the container level is to disable collection when it's already enabled globally. |

+| `[log_collection_settings.enrich_container_logs] enabled =` | Boolean | True or false | This setting controls container log enrichment to populate the `Name` and `Image` property values<br> for every log record written to the **ContainerLog** table for all container logs in the cluster.<br> It defaults to `enabled = false` when not specified in the ConfigMap. |

+| `[log_collection_settings.collect_all_kube_events] enabled =` | Boolean | True or false | This setting allows the collection of Kube events of all types.<br> By default, the Kube events with type **Normal** aren't collected. When this setting is set to `true`, the **Normal** events are no longer filtered, and all events are collected.<br> It defaults to `enabled = false` when not specified in the ConfigMap. |

+The following table describes the settings you can configure to control metric collection.

+| `[metric_collection_settings.collect_kube_system_pv_metrics] enabled =` | Boolean | True or false | This setting allows persistent volume (PV) usage metrics to be collected in the kube-system namespace. By default, usage metrics for persistent volumes with persistent volume claims in the kube-system namespace aren't collected. When this setting is set to `true`, PV usage metrics for all namespaces are collected. By default, this setting is set to `false`. |

+ConfigMap is a global list and there can be only one ConfigMap applied to the agent. You can't have another ConfigMap overruling the collections.

+To configure and deploy your ConfigMap configuration file to your cluster:

+1. Download the [template ConfigMap YAML file](https://aka.ms/container-azm-ms-agentconfig) and save it as *container-azm-ms-agentconfig.yaml*.

+1. Edit the ConfigMap YAML file with your customizations to collect stdout, stderr, and environmental variables:

+ - To exclude specific namespaces for stdout log collection, configure the key/value by using the following example:

+ `[log_collection_settings.stdout] enabled = true exclude_namespaces = ["my-namespace-1", "my-namespace-2"]`.

+ - To disable environment variable collection for a specific container, set the key/value `[log_collection_settings.env_var] enabled = true` to enable variable collection globally. Then follow the steps [here](container-insights-manage-agent.md#how-to-disable-environment-variable-collection-on-a-container) to complete configuration for the specific container.

+ - To disable stderr log collection cluster-wide, configure the key/value by using the following example: `[log_collection_settings.stderr] enabled = false`.

+1. Create a ConfigMap by running the following kubectl command: `kubectl apply -f <configmap_yaml_file.yaml>`.

+ Example: `kubectl apply -f container-azm-ms-agentconfig.yaml`

+The configuration change can take a few minutes to finish before taking effect. Then all Azure Monitor Agent pods in the cluster will restart. The restart is a rolling restart for all Azure Monitor Agent pods, so not all of them restart at the same time. When the restarts are finished, a message similar to this example includes the following result: `configmap "container-azm-ms-agentconfig" created`.

+To verify the configuration was successfully applied to a cluster, use the following command to review the logs from an agent pod: `kubectl logs ama-logs-fdf58 -n kube-system`. If there are configuration errors from the Azure Monitor Agent pods, the output will show errors similar to the following example:

+Errors related to applying configuration changes are also available for review. The following options are available to perform more troubleshooting of configuration changes:

+- From an agent pod log by using the same `kubectl logs` command.

+- From live logs. Live logs show errors similar to the following example:

+- From the **KubeMonAgentEvents** table in your Log Analytics workspace. Data is sent every hour with error severity for configuration errors. If there are no errors, the entry in the table will have data with severity info, which reports no errors. The **Tags** property contains more information about the pod and container ID on which the error occurred and also the first occurrence, last occurrence, and count in the last hour.

+After you correct the errors in the ConfigMap, save the YAML file and apply the updated ConfigMap by running the following command: `kubectl apply -f <configmap_yaml_file.yaml`.

+## Apply updated ConfigMap

+If you've already deployed a ConfigMap on clusters and you want to update it with a newer configuration, you can edit the ConfigMap file you've previously used. Then you can apply it by using the same command as before: `kubectl apply -f <configmap_yaml_file.yaml`.

+The configuration change can take a few minutes to finish before taking effect. Then all Azure Monitor Agent pods in the cluster will restart. The restart is a rolling restart for all Azure Monitor Agent pods, so not all of them restart at the same time. When the restarts are finished, a message similar to this example includes the following result: `configmap "container-azm-ms-agentconfig" updated`.

+## Verify schema version

+Supported config schema versions are available as pod annotation (schema-versions) on the Azure Monitor Agent pod. You can see them with the following kubectl command: `kubectl describe pod ama-logs-fdf58 -n=kube-system`.

+Output similar to the following example appears with the annotation schema-versions:

+- Container insights doesn't include a predefined set of alerts. Review the [Create performance alerts with Container insights](./container-insights-log-alerts.md) to learn how to create recommended alerts for high CPU and memory utilization to support your DevOps or operational processes and procedures.

+- With monitoring enabled to collect health and resource utilization of your Azure Kubernetes Service or hybrid cluster and workloads running on them, learn [how to use](container-insights-analyze.md) Container insights.

+- View [log query examples](container-insights-log-query.md) to see predefined queries and examples to evaluate or customize for alerting, visualizing, or analyzing your clusters.

+This article describes how to set up Container insights to monitor a managed Kubernetes cluster hosted on an [Azure Kubernetes Service (AKS)](../../aks/index.yml) cluster.

+You can enable monitoring for an AKS cluster when it's created by using any of the following methods:

+- **Azure CLI**: Follow the steps in [Create AKS cluster](../../aks/learn/quick-kubernetes-deploy-cli.md).

+- **Azure Policy**: Follow the steps in [Enable AKS monitoring add-on by using Azure Policy](container-insights-enable-aks-policy.md).

+- **Terraform**: If you're [deploying a new AKS cluster by using Terraform](/azure/developer/terraform/create-k8s-cluster-with-tf-and-aks), you specify the arguments required in the profile [to create a Log Analytics workspace](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/log_analytics_workspace) if you don't choose to specify an existing one. To add Container insights to the workspace, see [azurerm_log_analytics_solution](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/log_analytics_solution). Complete the profile by including the [addon_profile](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster) and specifying **oms_agent**.

+> Azure CLI version 2.39.0 or higher is required for managed identity authentication.

+Use the following command to enable monitoring of your AKS cluster by using a default Log Analytics workspace for the resource group. If a default workspace doesn't already exist in the cluster's region, one will be created with a name in the format *DefaultWorkspace-\<GUID>-\<Region>*.

+The output will resemble the following example:

+The output will resemble the following example:

+To enable monitoring by using Terraform:

+1. Add the **oms_agent** add-on profile to the existing [azurerm_kubernetes_cluster resource](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/kubernetes_cluster).

+1. Add the [azurerm_log_analytics_solution](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/log_analytics_solution) by following the steps in the Terraform documentation.

+1. Enable collection of custom metrics by using the guidance at [Enable custom metrics](container-insights-custom-metrics.md).

+To enable monitoring of your AKS cluster in the Azure portal from Azure Monitor:

+1. Select **Containers** from the list.

+1. On the **Monitor - containers** page, select **Unmonitored clusters**.

+1. From the list of unmonitored clusters, find the cluster in the list and select **Enable**.

+1. On the **Configure Container insights** page, select **Configure**.

+ :::image type="content" source="media/container-insights-enable-aks/container-insights-configure.png" lightbox="media/container-insights-enable-aks/container-insights-configure.png" alt-text="Screenshot that shows the configuration screen for an AKS cluster.":::

+1. On the **Configure Container insights** page, fill in the following information:

+ | Option | Description |

+ |:|:|

+ | Log Analytics workspace | Select a [Log Analytics workspace](../logs/log-analytics-workspace-overview.md) from the dropdown list or select **Create new** to create a default Log Analytics workspace. The Log Analytics workspace must be in the same subscription as the AKS container. |

+ | Enable Prometheus metrics | Select this option to collect Prometheus metrics for the cluster in [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md). |

+ | Azure Monitor workspace | If you select **Enable Prometheus metrics**, you must select an [Azure Monitor workspace](../essentials/azure-monitor-workspace-overview.md). The Azure Monitor workspace must be in the same subscription as the AKS container and the Log Analytics workspace. |

+ | Grafana workspace | To use the collected Prometheus metrics with dashboards in [Azure-managed Grafana](../../managed-grafan#link-a-grafana-workspace) to the Azure Monitor workspace if it isn't already. |

+

+1. Select **Use managed identity** if you want to use [managed identity authentication with Azure Monitor Agent](container-insights-onboard.md#authentication).

+>The template must be deployed in the same resource group as the cluster.

+You'll either download template and parameter files or create your own depending on the authentication mode you're using.

+To enable [managed identity authentication (preview)](container-insights-onboard.md#authentication):

+1. Download the template in the [GitHub content file](https://aka.ms/aks-enable-monitoring-msi-onboarding-template-file) and save it as **existingClusterOnboarding.json**.

+1. Download the parameter file in the [GitHub content file](https://aka.ms/aks-enable-monitoring-msi-onboarding-template-parameter-file) and save it as **existingClusterParam.json**.

+1. Edit the values in the parameter file:

+ - `aksResourceId`: Use the values on the **AKS Overview** page for the AKS cluster.

+ - `aksResourceLocation`: Use the values on the **AKS Overview** page for the AKS cluster.

+ - `workspaceResourceId`: Use the resource ID of your Log Analytics workspace.

+ - `resourceTagValues`: Match the existing tag values specified for the existing Container insights extension data collection rule (DCR) of the cluster and the name of the DCR. The name will be *MSCI-\<clusterName\>-\<clusterRegion\>* and this resource created in an AKS clusters resource group. If this is the first time onboarding, you can set the arbitrary tag values.

+To enable [managed identity authentication (preview)](container-insights-onboard.md#authentication):

+1. Save the following JSON as **existingClusterParam.json**.

+1. Download the parameter file in the [GitHub content file](https://aka.ms/aks-enable-monitoring-msi-onboarding-template-parameter-file) and save as **existingClusterParam.json**.

+1. Edit the values in the parameter file:

+ - `aksResourceId`: Use the values on the **AKS Overview** page for the AKS cluster.

+ - `aksResourceLocation`: Use the values on the **AKS Overview** page for the AKS cluster.

+ - `workspaceResourceId`: Use the resource ID of your Log Analytics workspace.

+ - `resourceTagValues`: Use the existing tag values specified for the AKS cluster.

+### Deploy the template

+Deploy the template with the parameter file by using any valid method for deploying Resource Manager templates. For examples of different methods, see [Deploy the sample templates](../resource-manager-samples.md#deploy-the-sample-templates).

+#### Deploy with Azure PowerShell

+The configuration change can take a few minutes to complete. When it's finished, a message similar to the following example includes this result:

+#### Deploy with Azure CLI

+The configuration change can take a few minutes to complete. When it's finished, a message similar to the following example includes this result:

+The output should resemble the following example, which indicates that it was deployed properly:

+If there are Windows Server nodes on the cluster, run the following command to verify that the agent is deployed successfully:

+The output should resemble the following example, which indicates that it was deployed properly:

+The output should resemble the following example, which indicates that it was deployed properly:

+Use the `aks show` command to find out whether the solution is enabled or not, what the Log Analytics workspace resource ID is, and summary information about the cluster.

+After a few minutes, the command completes and returns JSON-formatted information about the solution. The results of the command should show the monitoring add-on profile and resemble the following example output:

+This section explains two methods for migrating to managed identity authentication.

+### Existing clusters with a service principal

+AKS clusters with a service principal must first disable monitoring and then upgrade to managed identity. Only Azure public cloud, Azure China cloud, and Azure Government cloud are currently supported for this migration.

+1. Get the configured Log Analytics workspace resource ID:

+ ```cli

+ az aks show -g <resource-group-name> -n <cluster-name> | grep -i "logAnalyticsWorkspaceResourceID"

+ ```

+1. Disable monitoring with the following command:

+ ```cli

+ az aks disable-addons -a monitoring -g <resource-group-name> -n <cluster-name>

+ ```

+1. Upgrade cluster to system managed identity with the following command:

+ ```cli

+ az aks update -g <resource-group-name> -n <cluster-name> --enable-managed-identity

+ ```

+1. Enable the monitoring add-on with the managed identity authentication option by using the Log Analytics workspace resource ID obtained in step 1:

+ ```cli

+ az aks enable-addons -a monitoring --enable-msi-auth-for-monitoring -g <resource-group-name> -n <cluster-name> --workspace-resource-id <workspace-resource-id>

+ ```

+### Existing clusters with system or user-assigned identity

+AKS clusters with system-assigned identity must first disable monitoring and then upgrade to managed identity. Only Azure public cloud, Azure China cloud, and Azure Government cloud are currently supported for clusters with system identity. For clusters with user-assigned identity, only Azure public cloud is supported.

+1. Get the configured Log Analytics workspace resource ID:

+ ```cli

+ az aks show -g <resource-group-name> -n <cluster-name> | grep -i "logAnalyticsWorkspaceResourceID"

+ ```

+1. Disable monitoring with the following command:

+ ```cli

+ az aks disable-addons -a monitoring -g <resource-group-name> -n <cluster-name>

+ ```

+1. Enable the monitoring add-on with the managed identity authentication option by using the Log Analytics workspace resource ID obtained in step 1:

+ ```cli

+ az aks enable-addons -a monitoring --enable-msi-auth-for-monitoring -g <resource-group-name> -n <cluster-name> --workspace-resource-id <workspace-resource-id>

+ ```

+To enable network isolation by connecting your cluster to the Log Analytics workspace by using [Azure Private Link](../logs/private-link-security.md), your cluster must be using managed identity authentication with Azure Monitor Agent.

+1. Follow the steps in [Enable network isolation for the Azure Monitor agent](../agents/azure-monitor-agent-data-collection-endpoint.md) to create a data collection endpoint and add it to your Azure Monitor private link service.

+1. Create an association between the cluster and the data collection endpoint by using the following API call. For information on this call, see [Data collection rule associations - Create](/rest/api/monitor/data-collection-rule-associations/create). The DCR association name must beΓÇ»**configurationAccessEndpoint**, and `resourceUri` is the resource ID of the AKS cluster.

+ The following snippet is an example of this API call:

+1. Enable monitoring with the managed identity authentication option by using the steps in [Migrate to managed identity authentication](#migrate-to-managed-identity-authentication).

+- Enabling managed identity authentication (preview) isn't currently supported by using Terraform or Azure Policy.

+- When you enable managed identity authentication (preview), a data collection rule is created with the name *MSCI-\<cluster-name\>-\<cluster-region\>*. Currently, this name can't be modified.

+* If you experience issues while you attempt to onboard the solution, review the [Troubleshooting guide](container-insights-troubleshoot.md).

+ Title: View live data with Container insights

+# View Kubernetes logs, events, and pod metrics in real time

+Container insights includes the Live Data feature. You can use this advanced diagnostic feature for direct access to your Azure Kubernetes Service (AKS) container logs (stdout/stderror), events, and pod metrics. It exposes direct access to `kubectl logs -c`, `kubectl get` events, and `kubectl top pods`. A console pane shows the logs, events, and metrics generated by the container engine to help with troubleshooting issues in real time.

+This article provides an overview of this feature and helps you understand how to use it.

+For help with setting up or troubleshooting the Live Data feature, see the [Setup guide](container-insights-livedata-setup.md). This feature directly accesses the Kubernetes API. For more information about the authentication model, see [The Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/).

+To view the live logs for pods, deployments, and replica sets with or without Container insights from the AKS resource view:

+1. Select **Workloads** in the **Kubernetes resources** section of the menu.

+1. Select a pod, deployment, or replica set from the respective tab.

+1. Select **Live Logs** from the resource's menu.

+1. Select a pod to start collecting the live data.

+ [](./media/container-insights-livedata-overview/live-data-deployment.png#lightbox)

+You can view real-time log data as it's generated by the container engine on the **Nodes**, **Controllers**, or **Containers** view. To view log data:

+1. On the AKS cluster dashboard, under **Monitoring** on the left side, select **Insights**.

+1. Select the **Nodes**, **Controllers**, or **Containers** tab.

+1. Select an object from the performance grid. In the **Properties** pane on the right side, select **View live data**. If the AKS cluster is configured with single sign-on by using Azure Active Directory (Azure AD), you're prompted to authenticate on first use during that browser session. Select your account and finish authentication with Azure.

+ >To view the data from your Log Analytics workspace, select **View in analytics** in the **Properties** pane. The log search results potentially show **Nodes**, **Daemon Sets**, **Replica Sets**, **Jobs**, **Cron Jobs**, **Pods**, and **Containers**. These logs might no longer exist. Attempting to search logs for a container that isn't available in `kubectl` will also fail here. To learn more about viewing historical logs, events, and metrics, see [How to query logs from Container insights](container-insights-log-query.md).

+After successful authentication, the Live Data console pane appears below the performance data grid. You can view log data here in a continuous stream. If the fetch status indicator shows a green check mark at the far right, it means data can be retrieved, and it begins streaming to your console.

+

+You can view real-time event data as it's generated by the container engine on the **Nodes**, **Controllers**, **Containers**, or **Deployments** view when a container, pod, node, ReplicaSet, DaemonSet, job, CronJob, or Deployment is selected. To view events:

+1. On the AKS cluster dashboard, under **Monitoring** on the left side, select **Insights**.

+1. Select the **Nodes**, **Controllers**, **Containers**, or **Deployments** tab.

+1. Select an object from the performance grid. In the **Properties** pane on the right side, select **View live data**. If the AKS cluster is configured with single sign-on by using Azure AD, you're prompted to authenticate on first use during that browser session. Select your account and finish authentication with Azure.

+ >To view the data from your Log Analytics workspace, select **View in analytics** in the **Properties** pane. The log search results potentially show **Nodes**, **Daemon Sets**, **Replica Sets**, **Jobs**, **Cron Jobs**, **Pods**, and **Containers**. These logs might no longer exist. Attempting to search logs for a container that isn't available in `kubectl` will also fail here. To learn more about viewing historical logs, events, and metrics, see [How to query logs from Container insights](container-insights-log-query.md).

+After successful authentication, the Live Data console pane appears below the performance data grid. If the fetch status indicator shows a green check mark at the far right, it means data can be retrieved, and it begins streaming to your console.

+If the object you selected was a container, select the **Events** option in the pane. If you selected a node, pod, or controller, viewing events is automatically selected.

+

+While you view events, you can also limit the results by using the **Filter** pill found to the right of the search bar. Depending on the resource you select, the pill lists a pod, namespace, or cluster to choose from.

+You can view real-time metric data as it's generated by the container engine from the **Nodes** or **Controllers** view only when a **Pod** is selected. To view metrics:

+1. On the AKS cluster dashboard, under **Monitoring** on the left side, select **Insights**.

+1. Select either the **Nodes** or **Controllers** tab.

+1. Select a **Pod** object from the performance grid. In the **Properties** pane on the right side, select **View live data**. If the AKS cluster is configured with single sign-on by using Azure AD, you're prompted to authenticate on first use during that browser session. Select your account and finish authentication with Azure.

+ >To view the data from your Log Analytics workspace, select the **View in analytics** option in the **Properties** pane. The log search results potentially show **Nodes**, **Daemon Sets**, **Replica Sets**, **Jobs**, **Cron Jobs**, **Pods**, and **Containers**. These logs might no longer exist. Attempting to search logs for a container that isn't available in `kubectl` will also fail here. To learn more about viewing historical logs, events, and metrics, see [How to query logs from Container insights](container-insights-log-query.md).

+After successful authentication, the Live Data console pane appears below the performance data grid. Metric data is retrieved and begins streaming to your console for presentation in the two charts. The pane title shows the name of the pod the container is grouped with.

+

+## Use live data views

+The Live Data feature includes search functionality. In the **Search** box, you can filter results by entering a keyword or term. Any matching results are highlighted to allow quick review. While you view the events, you can also limit the results by using the **Filter** feature to the right of the search bar. Depending on what resource you've selected, you can choose from a pod, namespace, or cluster.

+

+

+### Scroll lock and pause

+To suspend autoscroll and control the behavior of the pane so that you can manually scroll through the new data read, select the **Scroll** option. To re-enable autoscroll, select **Scroll** again. You can also pause retrieval of log or event data by selecting the **Pause** option. When you're ready to resume, select **Play**.

+

+

+Suspend or pause autoscroll for only a short period of time while you're troubleshooting an issue. These requests might affect the availability and throttling of the Kubernetes API on your cluster.

+>No data is stored permanently during the operation of this feature. All information captured during the session is deleted when you close your browser or navigate away from it. Data only remains present for visualization inside the five-minute window of the metrics feature. Any metrics older than five minutes are also deleted. The Live Data buffer queries within reasonable memory usage limits.

+- To see predefined queries and examples to create alerts and visualizations or perform further analysis of your clusters, see [How to query logs from Container insights](container-insights-log-query.md).

+Metric alerts in Azure Monitor proactively identify issues related to system resources of your Azure resources, including monitored Kubernetes clusters. Container insights provides preconfigured alert rules so that you don't have to create your own. This article describes the different types of alert rules you can create and how to enable and configure them.

+| [Prometheus rules](#prometheus-alert-rules) | Alert rules that use metrics stored in [Azure Monitor managed service for Prometheus (preview)](../essentials/prometheus-metrics-overview.md). There are two sets of Prometheus alert rules that you can choose to enable.<br><br>- *Community alerts* are handpicked alert rules from the Prometheus community. Use this set of alert rules if you don't have any other alert rules enabled.<br>- *Recommended alerts* are the equivalent of the custom metric alert rules. Use this set if you're migrating from custom metrics to Prometheus metrics and want to retain identical functionality.

+| [Metric rules](#metric-alert-rules) | Alert rules that use [custom metrics collected for your Kubernetes cluster](container-insights-custom-metrics.md). Use these alert rules if you're not ready to move to Prometheus metrics yet or if you want to manage your alert rules in the Azure portal. |

+[Prometheus alert rules](../alerts/alerts-types.md#prometheus-alerts-preview) use metric data from your Kubernetes cluster sent to [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md).

+Your cluster must be configured to send metrics to [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md). For more information, see [Collect Prometheus metrics from Kubernetes cluster with Container insights](container-insights-prometheus-metrics-addon.md).

+The only method currently available for creating Prometheus alert rules is an Azure Resource Manager template (ARM template).

+1. Download the template that includes the set of alert rules you want to enable. For a list of the rules for each, see [Alert rule details](#alert-rule-details).

+1. Deploy the template by using any standard methods for installing ARM templates. For guidance, see [ARM template samples for Azure Monitor](../resource-manager-samples.md#deploy-the-sample-templates).

+> [!NOTE]

+> Although you can create the Prometheus alert in a resource group different from the target resource, use the same resource group as your target resource.

+ To edit the query and threshold or configure an action group for your alert rules, edit the appropriate values in the ARM template and redeploy it by using any deployment method.

+Perform the following steps to configure your ConfigMap configuration file to override the default utilization thresholds. These steps only apply to the following alertable metrics:

+- cpuExceededPercentage

+- cpuThresholdViolated

+- memoryRssExceededPercentage

+- memoryRssThresholdViolated

+- memoryWorkingSetExceededPercentage

+- memoryWorkingSetThresholdViolated

+- pvUsageExceededPercentage

+- pvUsageThresholdViolated

+> Download the new ConfigMap from [this GitHub content](https://raw.githubusercontent.com/microsoft/Docker-Provider/ci_prod/kubernetes/container-azm-ms-agentconfig.yaml).

+ - **Example:** Use the following ConfigMap configuration to modify the `cpuExceededPercentage` threshold to 90%:

+ - **Example:** Use the following ConfigMap configuration to modify the `pvUsageExceededPercentage` threshold to 80%:

+1. Run the following kubectl command: `kubectl apply -f <configmap_yaml_file.yaml>`.

+The configuration change can take a few minutes to finish before it takes effect. Then all omsagent pods in the cluster will restart. The restart is a rolling restart for all omsagent pods, so they don't all restart at the same time. When the restarts are finished, a message similar to the following example includes the result: `configmap "container-azm-ms-agentconfig" created`.

+## Metric alert rules

+[Metric alert rules](../alerts/alerts-types.md#metric-alerts) use [custom metric data from your Kubernetes cluster](container-insights-custom-metrics.md).

+ - You might need to enable collection of custom metrics for your cluster. See [Metrics collected by Container insights](container-insights-custom-metrics.md).

+ - See the supported regions for custom metrics at [Supported regions](../essentials/metrics-custom-overview.md#supported-regions).

+1. On the **Insights** menu for your cluster, select **Recommended alerts**.

+ :::image type="content" source="media/container-insights-metric-alerts/command-bar-recommended-alerts.png" lightbox="media/container-insights-metric-alerts/command-bar-recommended-alerts.png" alt-text="Screenshot that shows recommended alerts option in Container insights.":::

+1. Toggle the **Status** for each alert rule to enable. The alert rule is created and the rule name updates to include a link to the new alert resource.

+ :::image type="content" source="media/container-insights-metric-alerts/recommended-alerts-pane-enable.png" lightbox="media/container-insights-metric-alerts/recommended-alerts-pane-enable.png" alt-text="Screenshot that shows a list of recommended alerts and options for enabling each.":::

+1. Alert rules aren't associated with an [action group](../alerts/action-groups.md) to notify users that an alert has been triggered. Select **No action group assigned** to open the **Action Groups** page. Specify an existing action group or create an action group by selecting **Create action group**.

+ :::image type="content" source="media/container-insights-metric-alerts/select-action-group.png" lightbox="media/container-insights-metric-alerts/select-action-group.png" alt-text="Screenshot that shows selecting an action group.":::

+To edit the threshold for a rule or configure an [action group](../alerts/action-groups.md) for your Azure Kubernetes Service (AKS) cluster.

+2. Select the **Rule Name** to open the alert rule.

+3. See [Create an alert rule](../alerts/alerts-create-new-alert-rule.md?tabs=metric) for information on the alert rule settings.

+1. Change the status for the alert rule to **Disabled**.

+For custom metrics, a separate ARM template is provided for each alert rule.

+1. Create and use a [parameters file](../../azure-resource-manager/templates/parameter-files.md) as a JSON to set the values required to create the alert rule.

+1. Deploy the template by using any standard methods for installing ARM templates. For guidance, see [ARM template samples for Azure Monitor](../resource-manager-samples.md).

+To disable custom alert rules, use the same ARM template to create the rule, but change the `isEnabled` value in the parameters file to `false`.

+The following sections present information on the alert rules provided by Container insights.

+These handpicked alerts come from the Prometheus community. Source code for these mixin alerts can be found in [GitHub](https://aka.ms/azureprometheus-mixins):

+| Average Persistent Volume Usage % | Average Persistent Volume Usage % | Calculates average persistent volume usage per pod. | 80% |

+> The recommended alert rules in the Azure portal also include a log alert rule called *Daily Data Cap Breach*. This rule alerts when the total data ingestion to your Log Analytics workspace exceeds the [designated quota](../logs/daily-cap.md). This alert rule isn't included with the Prometheus alert rules.

+>

+> You can create this rule on your own by creating a [log alert rule](../alerts/alerts-types.md#log-alerts) that uses the query `_LogOperation | where Operation == "Data collection Status" | where Detail contains "OverQuota"`.

+Common properties across all these alert rules include:

+- All alert rules are evaluated once per minute, and they look back at the last five minutes of data.

+- Alerts rules don't have an action group assigned to them by default. To add an [action group](../alerts/action-groups.md) to the alert, either select an existing action group or create a new action group while you edit the alert rule.

+- You can modify the threshold for alert rules by directly editing the template and redeploying it. Refer to the guidance provided in each alert rule before you modify its threshold.

+- The `completedJobsCount` metric is only sent when there are jobs that are completed greater than six hours ago.

+- The `containerRestartCount` metric is only sent when there are containers restarting.

+- The `oomKilledContainerCount` metric is only sent when there are OOM killed containers.

+- The `cpuExceededPercentage`, `memoryRssExceededPercentage`, and `memoryWorkingSetExceededPercentage` metrics are sent when the CPU, memory RSS, and memory working set values exceed the configured threshold. The default threshold is 95%. The `cpuThresholdViolated`, `memoryRssThresholdViolated`, and `memoryWorkingSetThresholdViolated` metrics are equal to 0 if the usage percentage is below the threshold and are equal to 1 if the usage percentage is above the threshold. These thresholds are exclusive of the alert condition threshold specified for the corresponding alert rule.

+- The `pvUsageExceededPercentage` metric is sent when the persistent volume usage percentage exceeds the configured threshold. The default threshold is 60%. The `pvUsageThresholdViolated` metric is equal to 0 when the persistent volume usage percentage is below the threshold and is equal to 1 if the usage is above the threshold. This threshold is exclusive of the alert condition threshold specified for the corresponding alert rule.

+- The `pvUsageExceededPercentage` metric is sent when the persistent volume usage percentage exceeds the configured threshold. The default threshold is 60%. The `pvUsageThresholdViolated` metric is equal to 0 when the persistent volume usage percentage is below the threshold and is equal to 1 if the usage is above the threshold. This threshold is exclusive of the alert condition threshold specified for the corresponding alert rule.

+**Prometheus only**

+- If you want to collect `pvUsageExceededPercentage` and analyze it from [metrics explorer](../essentials/metrics-getting-started.md), configure the threshold to a value lower than your alerting threshold. The configuration related to the collection settings for persistent volume utilization thresholds can be overridden in the ConfigMaps file under the section `alertable_metrics_configuration_settings.pv_utilization_thresholds`. For details related to configuring your ConfigMap configuration file, see [Configure alertable metrics ConfigMaps](#configure-alertable-metrics-in-configmaps). Collection of persistent volume metrics with claims in the `kube-system` namespace are excluded by default. To enable collection in this namespace, use the section `[metric_collection_settings.collect_kube_system_pv_metrics]` in the ConfigMap file. For more information, see [Metric collection settings](./container-insights-agent-config.md#metric-collection-settings).

+- The `cpuExceededPercentage`, `memoryRssExceededPercentage`, and `memoryWorkingSetExceededPercentage` metrics are sent when the CPU, memory RSS, and Memory Working set values exceed the configured threshold. The default threshold is 95%. The `cpuThresholdViolated`, `memoryRssThresholdViolated`, and `memoryWorkingSetThresholdViolated` metrics are equal to 0 if the usage percentage is below the threshold and are equal to 1 if the usage percentage is above the threshold. These thresholds are exclusive of the alert condition threshold specified for the corresponding alert rule. If you want to collect these metrics and analyze them from [metrics explorer](../essentials/metrics-getting-started.md), configure the threshold to a value lower than your alerting threshold. The configuration related to the collection settings for their container resource utilization thresholds can be overridden in the ConfigMaps file under the section `[alertable_metrics_configuration_settings.container_resource_utilization_thresholds]`. For details related to configuring your ConfigMap configuration file, see the section [Configure alertable metrics ConfigMaps](#configure-alertable-metrics-in-configmaps).

+View fired alerts for your cluster from **Alerts** in the **Monitor** menu in the Azure portal with other fired alerts in your subscription. You can also select **View in alerts** on the **Recommended alerts** pane to view alerts from custom metrics.

+> [!NOTE]

+> Currently, Prometheus alerts won't be displayed when you select **Alerts** from your AKS cluster because the alert rule doesn't use the cluster as its target.

+- Read about the [different alert rule types in Azure Monitor](../alerts/alerts-types.md).

+- Read about [alerting rule groups in Azure Monitor managed service for Prometheus](../essentials/prometheus-rule-groups.md).

+This article provides an overview of the requirements and options that are available for configuring Container insights to monitor the performance of workloads that are deployed to Kubernetes environments. You can enable Container insights for a new deployment or for one or more existing deployments of Kubernetes by using several supported methods.

+- [Azure Kubernetes Service (AKS)](../../aks/index.yml)

+The versions of Kubernetes and support policy are the same as those versions [supported in AKS](../../aks/supported-kubernetes-versions.md).

+- Windows doesn't have a Memory RSS metric. As a result, it isn't available for Windows nodes and containers. The [Working Set](/windows/win32/memory/working-set) metric is available.

+> Container insights support for the Windows Server 2022 operating system is in preview.

+You can let the onboarding experience create a Log Analytics workspace in the default resource group of the AKS cluster subscription. If you already have a workspace, you'll probably want to use that one. For more information, see [Designing your Azure Monitor Logs deployment](../logs/design-logs-deployment.md).

+ You can attach an AKS cluster to a Log Analytics workspace in a different Azure subscription in the same Azure Active Directory tenant. Currently, you can't do it with the Azure portal, but you can use the Azure CLI or an Azure Resource Manager template.

+If you're going to configure the cluster to [collect Prometheus metrics](container-insights-prometheus-metrics-addon.md) with [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md), you must have an Azure Monitor workspace where Prometheus metrics are stored. You can let the onboarding experience create an Azure Monitor workspace in the default resource group of the AKS cluster subscription or use an existing Azure Monitor workspace.

+- You must be a member of the [Log Analytics contributor](../logs/manage-access.md#azure-rbac) role.

+- You must be a member of the [*Owner* group](../../role-based-access-control/built-in-roles.md#owner) on any AKS cluster resources.

+To view data after container monitoring is enabled, you require the following permissions:

+- You must be a member of the [Log Analytics reader](../logs/manage-access.md#azure-rbac) role if you aren't already a member of the [Log Analytics contributor](../logs/manage-access.md#azure-rbac) role.

+The containerized Linux agent (replicaset pod) makes API calls to all the Windows nodes on Kubelet secure port (10250) within the cluster to collect node and container performance-related metrics. Kubelet secure port (:10250) should be opened in the cluster's virtual network for both inbound and outbound for Windows node and container performance-related metrics collection to work.

+If you have a Kubernetes cluster with Windows nodes, review and configure the network security group and network policies to make sure the Kubelet secure port (:10250) is opened for both inbound and outbound in the cluster's virtual network.

+For information on the firewall requirements for the AKS cluster, see [Network firewall requirements](#network-firewall-requirements).

+Container insights now supports authentication by using managed identity (in preview). This secure and simplified authentication model has a monitoring agent that uses the cluster's managed identity to send data to Azure Monitor. It replaces the existing legacy certificate-based local authentication and removes the requirement of adding a *Monitoring Metrics Publisher* role to the cluster.

+> Container insights preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available." They're excluded from the service-level agreements and limited warranty. Container insights previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see [Frequently asked questions about Azure Kubernetes Service](../../aks/faq.md).

+This section reviews the agents used by Container insights.

+### Azure Monitor agent

+When Container insights uses managed identity authentication (in preview), it relies on a containerized Azure Monitor agent for Linux. This specialized agent collects performance and event data from all nodes in the cluster. The agent is automatically deployed and registered with the specified Log Analytics workspace during deployment.

+### Log Analytics agent

+When Container insights doesn't use managed identity authentication, it relies on a containerized Log Analytics agent for Linux. This specialized agent collects performance and event data from all nodes in the cluster. The agent is automatically deployed and registered with the specified Log Analytics workspace during deployment.

+The agent version is *microsoft/oms:ciprod04202018* or later. It's represented by a date in the following format: *mmddyyyy*. When a new version of the agent is released, it's automatically upgraded on your managed Kubernetes clusters that are hosted on AKS. To track which versions are released, see [Agent release announcements](https://github.com/microsoft/docker-provider/tree/ci_feature_prod).

+With the general availability of Windows Server support for AKS, an AKS cluster with Windows Server nodes has a preview agent installed as a daemonset pod on each individual Windows Server node to collect logs and forward them to Log Analytics. For performance metrics, a Linux node that's automatically deployed in the cluster as part of the standard deployment collects and forwards the data to Azure Monitor for all Windows nodes in the cluster.

+> If you've already deployed an AKS cluster and enabled monitoring by using either the Azure CLI or a Resource Manager template, you can't use `kubectl` to upgrade, delete, redeploy, or deploy the agent. The template needs to be deployed in the same resource group as the cluster.

+The following table lists the extra firewall configuration required for managed identity authentication.

+The following table lists the proxy and firewall configuration information for Azure China 21Vianet.

+The following table lists the extra firewall configuration required for managed identity authentication.

+The following table lists the proxy and firewall configuration information for Azure US Government.

+The following table lists the extra firewall configuration required for managed identity authentication.

+After you've enabled monitoring, you can begin analyzing the performance of your Kubernetes clusters that are hosted on AKS, Azure Stack, or another environment.

+To learn how to use Container insights, see [View Kubernetes cluster performance](container-insights-analyze.md).

+This is not a complete list of partners. The number keeps expanding and maintaining this list is no longer scalable. As such, we are not accepting new requests to be added to this list. Any GitHub changes opened will be closed without action. We suggest you use your favorite search engine to locate additional appropropriate partners.

+

+

+Microsoft Azure integration monitoring from New Relic gives you an overview of your ecosystem ΓÇô cloud migrations, digital transformations, and cloud native applications using New Relic Observability Platform.

+

+**NewRelic Azure monitoring helps you to:**

+* Monitor the entire software stack with Full-stack monitoring.

+* Reduce friction between engineers and ITOps teams by identifying, triaging, and delegating application and infrastructure issues quickly.

+* Identify service dependencies through cross-application tracing using New Relic APM.

+

+Refer to [New Relic Azure integration](https://newrelic.com/instant-observability/?category=azure&search=azure) for more information.

+

+

+## Zenduty

+

+Zenduty is a novel collaborative incident management platform that provides end-to-end incident alerting, on-call management, and response orchestration, which gives teams greater control and automation over the incident management lifecycle. Zenduty is ideal for always-on services, helping teams orchestrate incident response for creating better user experiences and brand value and centralizing all incoming alerts through predefined notification rules to ensure that the right people are notified at the right time.

+Zenduty provides your NOC, SRE, and application engineers with detailed context around the Azure Monitor alert along with playbooks and a complete incident command framework to triage, remediate, and resolve incidents with speed.

+For more information, see the [Zenduty documentation](https://docs.zenduty.com/docs/microsoftazure).

+| [InstallDependencyAgent-Windows.exe](https://aka.ms/dependencyagentwindows) | Windows | 9.10.15.22060 | 39427C875E08BF13E1FD3B78E28C96666B722DA675FAA94D8014D8F1A42AE724 |

+| [InstallDependencyAgent-Linux64.bin](https://aka.ms/dependencyagentlinux) | Linux | 9.10.15.22060 | 5B99CDEA77C6328BDEF448EAC9A6DEF03CE5A732C5F7C98A4D4F4FFB6220EF58 |

+* Sweden Central

+* UAE North

+* [items](#items)

+- [Run ARM template TTK Tests](https://marketplace.visualstudio.com/items?itemName=Sam-Cogan.ARMTTKExtensionXPlatform)

+ |**Existing account ID**|Select an existing Azure Video Indexer account from the dropdown.|

+| **Language** | **Code** | **Transcription** | **LID**\* | **MLID**\* | **Translation** | **Customization** (language model) |

+|::|:--:|:--:|:-:|:-:|:-:|::|

+| Afrikaans | `af-ZA` | | | | | Γ£ö |

+| Arabic (Israel) | `ar-IL` | Γ£ö | | | | Γ£ö |

+| Arabic (Jordan) | `ar-JO` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic (Kuwait) | `ar-KW` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic (Lebanon) | `ar-LB` | Γ£ö | | | Γ£ö | Γ£ö |

+| Arabic (Oman) | `ar-OM` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic (Palestinian Authority) | `ar-PS` | Γ£ö | | | Γ£ö | Γ£ö |

+| Arabic (Qatar) | `ar-QA` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic (Saudi Arabia) | `ar-SA` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic (United Arab Emirates) | `ar-AE` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic Egypt | `ar-EG` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic Modern Standard (Bahrain) | `ar-BH` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Arabic Syrian Arab Republic | `ar-SY` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Bangla | `bn-BD` | | | | Γ£ö | |

+| Bosnian | `bs-Latn` | | | | Γ£ö | |

+| Bulgarian | `bg-BG` | | | | Γ£ö | |

+| Catalan | `ca-ES` | | | | Γ£ö | |

+| Chinese (Cantonese Traditional) | `zh-HK` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Chinese (Simplified) | `zh-Hans` | Γ£ö | Γ£ö\*| | Γ£ö | Γ£ö |

+| Chinese (Traditional) | `zh-Hant` | | | | Γ£ö | |

+| Croatian | `hr-HR` | | | | Γ£ö | |

+| English United States | `en-US` | Γ£ö | Γ£ö\* | Γ£ö\* | Γ£ö | Γ£ö |

+| Estonian | `et-EE` | | | | Γ£ö | |

+| Fijian | `en-FJ` | | | | Γ£ö | |

+| Filipino | `fil-PH` | | | | Γ£ö | |

+| French | `fr-FR` | Γ£ö | Γ£ö\* | Γ£ö\* | Γ£ö | Γ£ö |

+| German | `de-DE` | Γ£ö | Γ£ö \*| Γ£ö \*| Γ£ö | Γ£ö |

+| Greek | `el-GR` | | | | Γ£ö | |

+| Haitian | `fr-HT` | | | | Γ£ö | |

+| Hungarian | `hu-HU` | | | | Γ£ö | |

+| Indonesian | `id-ID` | | | | Γ£ö | |

+| Italian | `it-IT` | Γ£ö | Γ£ö\* | Γ£ö | Γ£ö | Γ£ö |

+| Japanese | `ja-JP` | Γ£ö | Γ£ö\* | Γ£ö | Γ£ö | Γ£ö |

+| Kiswahili | `sw-KE` | | | | Γ£ö | |

+| Latvian | `lv-LV` | | | | Γ£ö | |

+| Lithuanian | `lt-LT` | | | | Γ£ö | |

+| Malagasy | `mg-MG` | | | | Γ£ö | |

+| Malay | `ms-MY` | | | | Γ£ö | |

+| Maltese | `mt-MT` | | | | Γ£ö | |

+| Persian | `fa-IR` | Γ£ö | | | Γ£ö | Γ£ö |

+| Portuguese | `pt-BR` | Γ£ö | Γ£ö\* | Γ£ö | Γ£ö | Γ£ö |

+| Romanian | `ro-RO` | | | | Γ£ö | |

+| Russian | `ru-RU` | Γ£ö | Γ£ö\* | Γ£ö | Γ£ö | Γ£ö |

+| Samoan | `en-WS` | | | | Γ£ö | |

+| Serbian (Cyrillic) | `sr-Cyrl-RS` | | | | Γ£ö | |

+| Serbian (Latin) | `sr-Latn-RS` | | | | Γ£ö | |

+| Slovak | `sk-SK` | | | | Γ£ö | |

+| Slovenian | `sl-SI` | | | | Γ£ö | |

+| Spanish | `es-ES` | Γ£ö | Γ£ö\* | Γ£ö\* | Γ£ö | Γ£ö |

+| Spanish (Mexico) | `es-MX` | Γ£ö | | | Γ£ö | Γ£ö |

+| Swedish | `sv-SE` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Tamil | `ta-IN` | | | | Γ£ö | |

+| Thai | `th-TH` | Γ£ö | Γ£ö | Γ£ö | Γ£ö | Γ£ö |

+| Tongan | `to-TO` | | | | Γ£ö | |

+| Urdu | `ur-PK` | | | | Γ£ö | |

+\*By default, languages marked by * are supported by LID or/and MLID auto-detection. When [uploading a video](https://api-portal.videoindexer.ai/api-details#api=Operations&operation=Upload-Video) with API, you can specify to use other supported languages (see the table above) to auto-detect one or more languages by language identification (LID) or multi-language identification (MLID) by using `customLanguages` parameter. The `customLanguages` parameter allows up to 10 languages to be identified by language identification (LID) or multi-language identification (MLID).

+> [!NOTE]

+> To change the default languages, set the `customLanguages` parameter. Setting the parameter, will replace the default languages supported by language identification (LID) and by multi-language identification (MLID).

+You should have at least contributor rights on both **source** and **target** subscriptions.

+>VNet and VNet gateway can't' be moved from one subscription to another. Additionally, moving your subscriptions has no impact on the management and workloads, like the vCenter, NSX, and workload virtual machines.

+## Prepare and move

+ :::image type="content" source="media/move-subscriptions/source-subscription-id.png" alt-text="Screenshot that shows the overview details of the selected private cloud."lightbox="media/move-subscriptions/source-subscription-id.png":::

+ :::image type="content" source="media/move-subscriptions/private-cloud-overview-subscription-id.png" alt-text="Screenshot showing the private cloud details."lightbox="media/move-subscriptions/private-cloud-overview-subscription-id.png":::

+ :::image type="content" source="media/move-subscriptions/move-resources-subscription-target.png" alt-text="Screenshot of the target resource."lightbox="media/move-subscriptions/move-resources-subscription-target.png":::

+1. Confirm the validation of the resources you selected to move. During the validation, youΓÇÖll see *Pending validation* under **Validation status**.

+ :::image type="content" source="media/move-subscriptions/pending-move-resources-subscription-target.png" alt-text="Screenshot showing the resource being moved."lightbox="media/move-subscriptions/pending-move-resources-subscription-target.png":::

+ :::image type="content" source="media/move-subscriptions/move-resources-succeeded.png" alt-text=" Screenshot showing the validation status of Succeeded."lightbox="media/move-subscriptions/move-resources-succeeded.png":::

+ :::image type="content" source="media/move-subscriptions/review-move-resources-subscription-target.png" alt-text="Screenshot showing the summary of the selected resource being moved."lightbox="media/move-subscriptions/review-move-resources-subscription-target.png":::

+A notification appears once the resource move is complete.

+In this how-to article, you'll do the following tasks:

+After you're finished, follow the recommended [Next steps](#next-steps) at the end of this article to continue with this getting started guide.

+Identify the [region](https://azure.microsoft.com/global-infrastructure/services/?products=azure-vmware) you want Azure VMware Solution deployed.

+>The name must not exceed 40 characters. If the name exceeds this limit, you won't be able to create public IP addresses for use with the private cloud.

+## Request a host quota

+Azure VMware Solution requires a /22 CIDR network, for example, `10.0.0.0/22`. This address space is carved into smaller network segments (subnets) and used for Azure VMware Solution management segments, including: vCenter Server, VMware HCX, NSX-T Data Center, and vMotion functionality. The diagram highlights Azure VMware Solution management IP address segments.

+Like with any VMware vSphere environment, the VMs must connect to a network segment. As the production deployment of Azure VMware Solution expands, there's often a combination of L2 extended segments from on-premises and local NSX-T network segments.

+For the initial deployment, identify a single network segment (IP network), for example, `10.0.4.0/24`. This network segment is used primarily for testing purposes during the initial deployment. The address block shouldn't overlap with any network segments on-premises or within Azure and shouldn't be within the /22 network segment already defined.

+Azure VMware Solution requires an Azure Virtual Network and an ExpressRoute circuit. Define whether you want to use an *existing* OR *new* ExpressRoute virtual network gateway. If you decide to use a *new* virtual network gateway, you'll create it after creating your private cloud. It's acceptable to use an existing ExpressRoute virtual network gateway. For planning purposes, make a note of which ExpressRoute virtual network gateway you'll use.

+VMware HCX is an application mobility platform that simplifies application migration, workload rebalancing, and business continuity across data centers and clouds. You can migrate your VMware vSphere workloads to Azure VMware Solution and other connected sites through various migration types.

+VMware HCX Connector deploys a subset of virtual appliances (automated) that require multiple IP segments. When you create your network profiles, you use the IP segments. Identify the following listed items for the VMware HCX deployment, which supports a pilot or small product use case. Depending on the needs of your migration, modify as necessary.

+- **Uplink network:** When deploying VMware HCX on-premises, you'll need to identify an Uplink network for VMware HCX. Use the same network you plan to use for the Management network.

+- **Replication network:** When deploying VMware HCX on-premises, you'll need to define a replication network. Use the same network you're using for your Management and Uplink networks. If the on-premises cluster hosts use a dedicated Replication VMkernel network, reserve **two** IP addresses in this network segment and use the Replication VMkernel network for the replication network.

+The capabilities include:

+For a complete list of features, see [Azure Application Gateway features](../application-gateway/features.md).

+The diagram shows how Application Gateway is used to protect Azure IaaS virtual machines (VMs), Azure Virtual Machine Scale Sets, or on-premises servers. Application Gateway treats Azure VMware Solution VMs as on-premises servers.

+- An Azure account with an active subscription.

+2. Provide the basic details as in the following figure; then select **Next: Frontends>**.

+ :::image type="content" source="media/application-gateway/create-app-gateway.png" alt-text="Screenshot showing Create application gateway page in Azure portal."lightbox="media/application-gateway/create-app-gateway.png":::

+6. On the **Listener** tab, provide the details for the listener. If HTTPS is selected, you must provide a certificate, either from a PFX file or an existing Azure Key Vault certificate.

+9. If you want to configure path-based rules, select **Add multiple targets to create a path-based rule**.

+10. Add a path-based rule and select **Add**. Repeat to add more path-based rules.

+11. When you have finished adding path-based rules, select **Add** again; then select **Next: Tags>**.

+Now you'll configure Application Gateway with Azure VMware Solution VMs as backend pools for the following use cases:

+This procedure shows you how to define backend address pools using VMs running on an Azure VMware Solution private cloud on an existing application gateway.

+1. In your private cloud, create two different pools of VMs. One represents Contoso and the second Fabrikam.

+ :::image type="content" source="media/application-gateway/app-gateway-multi-backend-pool.png" alt-text="Screenshot showing summary of a web server's details in VMware vSphere Client."lightbox="media/application-gateway/app-gateway-multi-backend-pool.png":::

+ We've used Windows Server 2016 with the Internet Information Services (IIS) role installed. Once the VMs are installed, run the following PowerShell commands to configure IIS on each of the VMs.

+1. In your private cloud, create a virtual machine pool to represent the web farm.

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool.png" alt-text="Screenshot of page in VMware vSphere Client showing summary of another VM."lightbox="media/application-gateway/app-gateway-url-route-backend-pool.png":::

+ Windows Server 2016 with IIS role installed has been used to illustrate this tutorial. Once the VMs are installed, run the following PowerShell commands to configure IIS for each VM tutorial.

+2. Add three new backend pools in an existing application gateway instance.

+ 1. Select **Add** and enter the details of the first pool, **contoso-web**.

+ 1. Add one VM as the target.

+ 1. Select **Add**.

+ 1. Repeat this process for **contoso-images** and **contoso-video**, adding one unique VM as the target.

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool-04.png" alt-text="Screenshot of Add HTTP setting page showing HTTP settings configuration."lightbox="media/application-gateway/app-gateway-url-route-backend-pool-04.png":::

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool-07.png" alt-text="Screenshot of Add a routing rule page to configure routing rules to a backend target."lightbox="media/application-gateway/app-gateway-url-route-backend-pool-07.png":::

+6. Test the configuration. Access the application gateway on the Azure portal and copy the public IP address in the **Overview** section.

+ 1. Open a new browser window and enter the URL `http://<app-gw-ip-address>:8080`.

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool-08.png" alt-text="Screenshot of browser page showing successful test of the configuration."lightbox="media/application-gateway/app-gateway-url-route-backend-pool-08.png":::

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool-09.png" alt-text="Screenshot of another successful test with the new URL."lightbox="media/application-gateway/app-gateway-url-route-backend-pool-09.png":::

+ :::image type="content" source="media/application-gateway/app-gateway-url-route-backend-pool-10.png" alt-text="Screenshot of successful test with the final URL."lightbox="media/application-gateway/app-gateway-url-route-backend-pool-10.png":::

+> [Explore more Azure Web PubSub samples](https://github.com/Azure/azure-webpubsub/tree/main/samples)

+# Back up a SharePoint farm to Azure with Data Protection Manager

+This article describes how to back up and restore SharePoint data using System Center Data Protection Manager (DPM). The backup operation of SharePoint to Azure with DPM is similar to SharePoint backup to DPM locally.

+System Center Data Protection Manager (DPM) enables you back up a SharePoint farm to Microsoft Azure, which gives an experience similar to back up of other data sources. Azure Backup provides flexibility in the backup schedule to create daily, weekly, monthly, or yearly backup points, and gives you retention policy options for various backup points. DPM provides the capability to store local disk copies for quick recovery-time objectives (RTO) and to store copies to Azure for economical, long-term retention.

+In this article, you'll learn about:

+> [!div class="checklist"]

+> - SharePoint supported scenarios

+> - Prerequisites

+> - Configure backup

+> - Monitor operations

+> - Restore SharePoint data

+> - Restore a SharePoint database from Azure using DPM

+> - Switch the Front-End Web Server

+## SharePoint supported scenarios

+For information on the supported SharePoint versions and the DPM versions required to back them up, see [What can DPM back up?](/system-center/dpm/dpm-protection-matrix#applications-backup).

+## Prerequisites

+Before you proceed to back up a SharePoint farm to Azure, ensure that you've met all the [prerequisites for using Microsoft Azure Backup](backup-azure-dpm-introduction.md#prerequisites-and-limitations) to protect workloads. The tasks in prerequisites also include: create a backup vault, download vault credentials, install Azure Backup agent, and register DPM/Azure Backup Server with the vault.

+For other prerequisites and limitations, see [Back up SharePoint with DPM](/system-center/dpm/back-up-sharepoint#prerequisites-and-limitations).

+To back up the SharePoint farm, configure protection for SharePoint using *ConfigureSharePoint.exe*, and then create a protection group in DPM. See the DPM documentation to learn [how to configure backup](/system-center/dpm/back-up-sharepoint#configure-backup).

+## Monitor operations

+To monitor the backup job, see [Monitoring DPM backup](/system-center/dpm/back-up-sharepoint#monitoring).

+## Restore a SharePoint database from Azure using DPM

+To recover a SharePoint content database, follow these steps:

+1. Browse through various recovery points (as shown previously), and select the recovery point that you want to restore.

+ 

+ > Because the SharePoint farm is protected for long-term retention in Azure, no catalog information (metadata) is available on the DPM server. So, whenever a point-in-time SharePoint content database needs to be recovered, you need to catalog the SharePoint farm again.

+ 

+ 

+ Once the cataloging is finished and the status changes to *Success*, select **Close**.

+ 

+4. On the DPM **Recovery** tab, select the *SharePoint object* to get the content database structure, right-click the item, and then select **Recover**.

+ 

+5. To recover a SharePoint content database from disk, see [this section](#restore-sharepoint-data).

+## Switch the Front-End Web Server

+If you've more than one front-end web server, and want to switch the server that DPM uses to protect the farm, see [Switching the Front-End Web Server](/system-center/dpm/back-up-sharepoint#switching-the-front-end-web-server).

+1. In your development environment, open the service definition file (CSDEF), add a **Certificates** section within the **WebRole** section, and include the following information about the certificate (and intermediate certificates):

+ ```xml

+ ```xml

+ ```xml

+ ```xml

+5. To deploy the application to Azure, use the **Publish-AzureServiceProject** cmdlet. For example:

+[The output of the Publish-AzureService command]: ./media/cloud-services-nodejs-chat-app-socketio/socketio-9.png

+| [Audio only](#audio-data-for-training-or-testing) | Yes (visual inspection) | 5+ audio files | Yes (Preview for `en-US`) | 1-20 hours of audio |

+### Audio data for training or testing

+> [!NOTE]

+> Audio only data for training is available in preview for the `en-US` locale. For other locales, to train with audio data you must also provide [human-labeled transcripts](#audio--human-labeled-transcript-data-for-training-or-testing).

+WORKDIR /opt/openenclave/share/openenclave/samples/attestation

+> [Communication between microservices](communicate-between-microservices.md)

+> [Communication between microservices](communicate-between-microservices.md)

+> [Communication between microservices](communicate-between-microservices.md)

+> * Create a Container Apps environment for your container apps

+> * Create an Azure Blob Storage state store for the container app

+> * Deploy two apps that produce and consume messages and persist them in the state store

+> * Verify the interaction between the two microservices.

+With Azure Container Apps, you get a [fully managed version of the Dapr APIs](./dapr-overview.md) when building microservices. When you use Dapr in Azure Container Apps, you can enable sidecars to run next to your microservices that provide a rich set of capabilities. Available Dapr APIs include [Service to Service calls](https://docs.dapr.io/developing-applications/building-blocks/service-invocation/), [Pub/Sub](https://docs.dapr.io/developing-applications/building-blocks/pubsub/), [Event Bindings](https://docs.dapr.io/developing-applications/building-blocks/bindings/), [State Stores](https://docs.dapr.io/developing-applications/building-blocks/state-management/), and [Actors](https://docs.dapr.io/developing-applications/building-blocks/actors/).

+In this tutorial, you deploy the same applications from the Dapr [Hello World](https://github.com/dapr/quickstarts/tree/master/tutorials/hello-kubernetes) quickstart.

+Open a text editor and create a config file named *statestore.yaml* with the properties that you sourced from the previous steps. This file helps enable your Dapr app to access your state store. The following example shows how your *statestore.yaml* file should look when configured for your Azure Blob Storage account:

+> [Communication between microservices](communicate-between-microservices.md)

+To request an increase in quota amounts for your container app, [submit a support ticket](https://azure.microsoft.com/support/create-ticket/).

+| Feature | Scope | Default | Is Configurable<sup>1</sup> | Remarks |

+| Environments | Region | 5 | Yes | |

+| Container Apps | Environment | 20 | Yes | |

+| Revisions | Container app | 100 | No | |

+| Replicas | Revision | 30 | Yes | |

+| Cores | Replica | 2 | No | Maximum number of cores that can be requested by a revision replica. |

+| Cores | Environment | 20 | Yes | Maximum number of cores an environment can accommodate. Calculated by the sum of cores requested by each active replica of all revisions in an environment. |

+<sup>1</sup> The **Is Configurable** column denotes that a feature maximum may be increased through a [support request](https://azure.microsoft.com/support/create-ticket/).

+After the 10 seconds is over, the burst capacity has been used up. If the workload continues to exceed the provisioned 100 RU/s, any requests that are consumed beyond the provisioned 100 RU/s would now be rate limited (429). The maximum amount of burst capacity a physical partition can accumulate at any point in time is equal to 300 seconds * the provisioned RU/s of the physical partition.

+To get started using burst capacity, enroll in the preview by submitting a request for the **Azure Cosmos DB Burst Capacity** feature via the [**Preview Features** page](../azure-resource-manager/management/preview-features.md) in your Azure Subscription overview page. You can also select the **Register for preview** button in the eligibility check page to open the **Preview Features** page.

+- Ensure that you have at least one Azure Cosmos DB account in the subscription. This account may be an existing account or a new one you've created to try out the preview feature. If you have no accounts in the subscription when the Azure Cosmos DB team receives your request, it will be declined, as there are no accounts to apply the feature to.

+- Verify that your Azure Cosmos DB account(s) meet all the [preview eligibility criteria](#limitations-preview-eligibility-criteria).

+## Limitations (preview eligibility criteria)

+- Your Azure Cosmos DB account is using provisioned throughput (manual or autoscale). Burst capacity doesn't apply to serverless accounts.

+- Your Azure Cosmos DB account is using API for NoSQL, Cassandra, Gremlin, MongoDB, or Table.

+- See the FAQ on [burst capacity.](burst-capacity-faq.yml)

+- Learn more about [provisioned throughput.](set-throughput.md)

+- Learn more about [request units.](request-units.md)

+- Trying to decide between provisioned throughput and serverless? See [choose between provisioned throughput and serverless.](throughput-serverless.md)

+- Want to learn the best practices? See [best practices for scaling provisioned throughput.](scaling-provisioned-throughput-best-practices.md)

+Unlike Azure Cosmos DB, Apache Cassandra doesn't natively provide precisely defined consistency guarantees. Instead, Apache Cassandra provides a write consistency level and a read consistency level, to enable the high availability, consistency, and latency tradeoffs. When using Azure Cosmos DB for Cassandra:

+- The write consistency level of Apache Cassandra is mapped to the default consistency level configured on your Azure Cosmos DB account. Consistency for a write operation (CL) can't be changed on a per-request basis.

+- Azure Cosmos DB will dynamically map the read consistency level specified by the Cassandra client driver. The consistency level will be mapped to one of the Azure Cosmos DB consistency levels configured dynamically on a read request.

+Apache Cassandra database is a multi-master system by default, and doesn't provide an out-of-box option for single-region writes with multi-region replication for reads. However, Azure Cosmos DB provides turnkey ability to have either single region, or [multi-region](../how-to-multi-master.md) write configurations. One of the advantages of being able to choose a single region write configuration across multiple regions is the avoidance of cross-region conflict scenarios, and the option of maintaining strong consistency across multiple regions.

+With single-region writes, you can maintain strong consistency, while still maintaining a level of high availability across regions with [service-managed failover](../high-availability.md#region-outages). In this configuration, you can still exploit data locality to reduce read latency by downgrading to eventual consistency on a per request basis. In addition to these capabilities, the Azure Cosmos DB platform also offers the option of [zone redundancy](/azure/architecture/reliability/architect) when selecting a region. Thus, unlike native Apache Cassandra, Azure Cosmos DB allows you to navigate the CAP Theorem [trade-off spectrum](../consistency-levels.md#rto) with more granularity.

+The Azure Cosmos DB platform provides a set of five well-defined, business use-case oriented consistency settings with respect to replication. The tradeoffs to these consistency settings are defined by the [CAP](https://en.wikipedia.org/wiki/CAP_theorem) and [PACLC](https://en.wikipedia.org/wiki/PACELC_theorem) theorems. As this approach differs significantly from Apache Cassandra, we would recommend that you take time to review and understand [Azure Cosmos DB consistency](../consistency-levels.md). Alternatively, you can review this short [video guide to understanding consistency settings](https://aka.ms/docs.consistency-levels) in the Azure Cosmos DB platform. The following table illustrates the possible mappings between Apache Cassandra and Azure Cosmos DB consistency levels when using API for Cassandra. This table shows configurations for single region, multi-region reads with single-region writes, and multi-region writes.

+### Mappings

+> [!NOTE]

+> These are not exact mappings. Rather, we have provided the closest analogues to Apache Cassandra, and disambiguated any qualitative differences in the rightmost column. As mentioned above, we recommend reviewing Azure Cosmos DB's [consistency settings](../consistency-levels.md).

+### `ALL`, `EACH_QUOROM`, `QUOROM`, `LOCAL_QUORUM`, or `THREE` write consistency in Apache Cassandra

+| Apache read consistency | Reading from | Closest Azure Cosmos DB consistency level to Apache Cassandra read/write settings |

+| | | |

+| `ALL` | Local region | `Strong` |

+| `EACH_QUOROM` | Local region | `Strong` |

+| `QUOROM` | Local region | `Strong` |

+| `LOCAL_QUORUM` | Local region | `Strong` |

+| `LOCAL_ONE` | Local region | `Eventual` |

+| `ONE` | Local region | `Eventual` |

+| `TWO` | Local region | `Strong` |

+| `THREE` | Local region | `Strong` |

+Unlike Apache and DSE Cassandra, Azure Cosmos DB durably commits a quorum write by default. At least three out of four (3/4) nodes commit the write to disk, and NOT just an in-memory commit log.

+### `ONE`, `LOCAL_ONE`, or `ANY` write consistency in Apache Cassandra

+| Apache read consistency | Reading from | Closest Azure Cosmos DB consistency level to Apache Cassandra read/write settings |

+| | | |

+| `ALL` | Local region | `Strong` |

+| `EACH_QUOROM` | Local region | `Eventual` |

+| `QUOROM` | Local region | `Eventual` |

+| `LOCAL_QUORUM` | Local region | `Eventual` |

+| `LOCAL_ONE` | Local region | `Eventual` |

+| `ONE` | Local region | `Eventual` |

+| `TWO` | Local region | `Eventual` |

+| `THREE` | Local region | `Eventual` |

+Azure Cosmos DB API for Cassandra always durably commits a quorum write by default, hence all read consistencies can be made use of.

+### `TWO` write consistency in Apache Cassandra

+| Apache read consistency | Reading from | Closest Azure Cosmos DB consistency level to Apache Cassandra read/write settings |

+| | | |

+| `ALL` | Local region | `Strong` |

+| `EACH_QUOROM` | Local region | `Strong` |

+| `QUOROM` | Local region | `Strong` |

+| `LOCAL_QUORUM` | Local region | `Strong` |

+| `LOCAL_ONE` | Local region | `Eventual` |

+| `ONE` | Local region | `Eventual` |

+| `TWO` | Local region | `Eventual` |

+| `THREE` | Local region | `Strong` |

+Azure Cosmos DB has no notion of write consistency to only two nodes, hence we treat this consistency similar to quorum for most cases. For read consistency `TWO`, this consistency is equivalent to write with `QUOROM` and read from `ONE`.

+### `Serial`, or `Local_Serial` write consistency in Apache Cassandra

+| Apache read consistency | Reading from | Closest Azure Cosmos DB consistency level to Apache Cassandra read/write settings |

+| | | |

+| `ALL` | Local region | `Strong` |

+| `EACH_QUOROM` | Local region | `Strong` |

+| `QUOROM` | Local region | `Strong` |

+| `LOCAL_QUORUM` | Local region | `Strong` |

+| `LOCAL_ONE` | Local region | `Eventual` |

+| `ONE` | Local region | `Eventual` |

+| `TWO` | Local region | `Strong` |

+| `THREE` | Local region | `Strong` |

+Serial only applies to lightweight transactions. Azure Cosmos DB follows a [durably committed algorithm](https://www.microsoft.com/research/publication/revisiting-paxos-algorithm/) by default, and hence `Serial` consistency is similar to quorum.

+### Other regions for single-region write

+Azure Cosmos DB facilitates five consistency settings, including strong, across multiple regions where single-region writes is configured. This facilitation occurs as long as regions are within 2,000 miles of each other.

+Azure Cosmos DB doesn't have an applicable mapping to Apache Cassandra as all nodes/regions are writes and a strong consistency guarantee isn't possible across all regions.

+### Other regions for multi-region write

+Azure Cosmos DB facilitates only four consistency settings; `eventual`, `consistent prefix`, `session`, and `bounded staleness` across multiple regions where multi-region write is configured.

+Apache Cassandra would only provide eventual consistency for reads across other regions regardless of settings.

+### Dynamic overrides supported

+| Azure Cosmos DB account setting | Override value in client request | Override effect |

+| | | |

+| `Strong` | `All` | No effect (remain as `strong`) |

+| `Strong` | `Quorum` | No effect (remain as `strong`) |

+| `Strong` | `LocalQuorum` | No effect (remain as `strong`) |

+| `Strong` | `Two` | No effect (remain as `strong`) |

+| `Strong` | `Three` | No effect (remain as `strong`) |

+| `Strong` | `Serial` | No effect (remain as `strong`) |

+| `Strong` | `LocalSerial` | No effect (remain as `strong`) |

+| `Strong` | `One` | Consistency changes to `Eventual` |

+| `Strong` | `LocalOne` | Consistency changes to `Eventual` |

+| `Strong` | `Any` | Not allowed (error) |

+| `Strong` | `EachQuorum` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `All` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `Quorum` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `LocalQuorum` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `Two` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `Three` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `Serial` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `LocalSerial` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `One` | Consistency changes to `Eventual` |

+| `Bounded staleness`, `session`, or `consistent prefix` | `LocalOne` | Consistency changes to `Eventual` |

+| `Bounded staleness`, `session`, or `consistent prefix` | `Any` | Not allowed (error) |

+| `Bounded staleness`, `session`, or `consistent prefix` | `EachQuorum` | Not allowed (error) |

+### Metrics

+If your Azure Cosmos DB account is configured with a consistency level other than the strong consistency, review the *Probabilistically Bounded Staleness* (PBS) metric. The metric captures the probability that your clients may get strong and consistent reads for your workloads. This metric is exposed in the Azure portal. To find more information about the PBS metric, see [Monitor Probabilistically Bounded Staleness (PBS) metric](../how-to-manage-consistency.md#monitor-probabilistically-bounded-staleness-pbs-metric).

+Probabilistically bounded staleness shows how eventual is your eventual consistency. This metric provides an insight into how often you can get a stronger consistency than the consistency level that you've currently configured on your Azure Cosmos DB account. In other words, you can see the probability (measured in milliseconds) of getting consistent reads for a combination of write and read regions.

+## Global strong consistency for write requests in Apache Cassandra

+Apache Cassandra, the setting of `EACH_QUORUM` or `QUORUM` gives a strong consistency. When a write request is sent to a region, `EACH_QUORUM` persists the data in a quorum number of nodes in each data center. This persistence requires every data center to be available for the write operation to succeed. `QUORUM` is slightly less restrictive, with a `QUORUM` number of nodes across all the data centers needed to persist the data prior to acknowledging the write to be successful.

+The following graphic illustrates a global strong consistency setting in Apache Cassandra between two regions 1 and 2. After data is written to region 1, the write needs to be persisted in a quorum number of nodes in both region 1, and region 2 before an acknowledgment is received by the application.

+## Global strong consistency for write requests in Azure Cosmos DB for Apache Cassandra

+In Azure Cosmos DB consistency is set at the account level. With `Strong` consistency in Azure Cosmos DB for Cassandra, data is replicated synchronously to the read regions for the account. The further apart the regions for the Azure Cosmos DB account are, the higher the latency of the consistent write operations.

+How the number of regions affects your read or write request:

+- Two regions: With strong consistency, quorum `(N/2 + 1) = 2`. So, if the read region goes down, the account can no longer accept writes with strong consistency since a quorum number of regions isn't available for the write to be replicated to.

+- Three or more regions: for `N = 3`, `quorum = 2`. If one of the read regions is down, the write region can still replicate the writes to a total of two regions that meet the quorum requirement. Similarly, with four regions, `quorum = 4/2 + 1 = 3`. Even with one read region being down, quorum can be met.

+> If a globally strong consistency is required for all write operations, the consistency for Azure Cosmos DB for Cassandra account must be set to Strong. The consistency level for write operations cannot be overridden to a lower consistency level on a per request basis in Azure Cosmos DB.

+## Weaker consistency for write requests in Apache Cassandra

+A consistency level of `ANY`, `ONE`, `TWO`, `THREE`, `LOCAL_QUORUM`, `Serial` or `Local_Serial`? Consider a write request with `LOCAL_QUORUM` with an `RF` of `4` in a six-node datacenter. `Quorum = 4/2 + 1 = 3`.

+## Weaker consistency for write requests in Azure Cosmos DB for Apache Cassandra

+When a write request is sent with any of the consistency levels lower than `Strong`, a success response is returned as soon as the local region persists the write in at least three out of four replicas.

+## Global strong consistency for read requests in Apache Cassandra

+With a consistency of `EACH_QUORUM`, a consistent read can be achieved in Apache Cassandra. In, a multi-region setup for `EACH_QUORUM` if the quorum number of nodes isn't met in each region, then the read will be unsuccessful.

+## Global strong consistency for read requests in Azure Cosmos DB for Apache Cassandra

+The read request is served from two replicas in the specified region. Since the write already took care of persisting to a quorum number of regions (and all regions if every region was available), simply reading from two replicas in the specified region provides Strong consistency. This strong consistency requires `EACH_QUORUM` to be specified in the driver when issuing the read against a region for the Cosmos DB account along with Strong Consistency as the default consistency level for the account.

+## Local strong consistency in Apache Cassandra

+A read request with a consistency level of `TWO`, `THREE`, or `LOCAL_QUORUM` will give us strong consistency reading from local region. With a consistency level of `LOCAL_QUORUM`, you need a response from two nodes in the specified datacenter for a successful read.

+## Local strong consistency in Azure Cosmos DB for Apache Cassandra

+In Azure Cosmos DB for Cassandra, having a consistency level of `TWO`, `THREE` or `LOCAL_QUORUM` will give a local strong consistency for a read request. Since the write path guarantees replicating to a minimum of three out of four replicas, a read from two replicas in the specified region will guarantee a quorum read of the data in that region.

+## Eventual consistency in Apache Cassandra

+A consistency level of `LOCAL_ONE`, `One` and `ANY with LOCAL_ONE` will result in eventual consistency. This consistency is used in cases where the focus is on latency.

+## Eventual consistency in Azure Cosmos DB for Apache Cassandra?

+A consistency level of `LOCAL_ONE`, `ONE` or `Any` will give you eventual consistency. With eventual consistency, a read is served from just one of the replicas in the specified region.

+## Override consistency level for read operations in Azure Cosmos DB for Cassandra

+Previously, the consistency level for read requests could only be overridden to a lower consistency than the default set on the account. For instance, with the default consistency of Strong, read requests could be issued with Strong by default and overridden on a per request basis (if needed) to a consistency level weaker than Strong. However, read requests couldn't be issued with an overridden consistency level higher than the accountΓÇÖs default. An account with Eventual consistency couldn't receive read requests with a consistency level higher than Eventual (which in the Apache Cassandra drivers translate to `TWO`, `THREE`, `LOCAL_QUORUM` or `QUORUM`).

+Azure Cosmos DB for Cassandra now facilitates overriding the consistency on read requests to a value higher than the accountΓÇÖs default consistency. For instance, with the default consistency on the Cosmos DB account set to Eventual (Apache Cassandra equivalent of `One` or `ANY`), read requests can be overridden on a per request basis to `LOCAL_QUORUM`. This override ensures that a quorum number of replicas within the specified region are consulted prior to returning the result set, as required by `LOCAL_QUORUM`.

+This option also prevents the need to set a default consistency that is higher than `Eventual`, when it's only needed for read requests.

+- [Global distribution overview](../distribute-data-globally.md)

+- [Consistency Level overview](../consistency-levels.md)

+- [High availability](../high-availability.md)

+Azure Cosmos DB offers multiple database APIs, which include NoSQL, MongoDB, PostgreSQL Cassandra, Gremlin, and Table. By using these APIs, you can model real world data using documents, key-value, graph, and column-family data models. These APIs allow your applications to treat Azure Cosmos DB as if it were various other databases technologies, without the overhead of management, and scaling approaches. Azure Cosmos DB helps you to use the ecosystems, tools, and skills you already have for data modeling and querying with its various APIs.

+API for MongoDB, PostgreSQL, Cassandra, Gremlin, and Table implement the wire protocol of open-source database engines. These APIs are best suited if the following conditions are true:

+* If you have existing MongoDB, PostgreSQL Cassandra, or Gremlin applications

+> [!NOTE]

+> This decision tree will be updated soon to include API for PostgreSQL.

+## API for PostgreSQL

+Azure Cosmos DB for PostgreSQL is a managed service for running PostgreSQL at any scale, with the [Citus open source](https://github.com/citusdata/citus) superpower of distributed tables. It stores data either on a single node, or distributed in a multi-node configuration.

+Azure Cosmos DB for PostgreSQL is built on native PostgreSQL--rather than a PostgreSQL fork--and lets you choose any major database versions supported by the PostgreSQL community. It's ideal for starting on a single-node database with rich indexing, geospatial capabilities, and JSONB support. Later, if your performance needs grow, you can add nodes to the cluster with zero downtime.

+If youΓÇÖre looking for a managed open source relational database with high performance and geo-replication, Azure Cosmos DB for PostgreSQL is the recommended choice. To learn more, see the [Azure Cosmos DB for PostgreSQL introduction](postgresql/introduction.md).

+* [Get started with Azure Cosmos DB for PostgreSQL](postgresql/quickstart-create-portal.md)

+| Allowed characters for ID value | Service-side all Unicode characters except for '/' and '\\' are allowed. <br/>**WARNING: But for best interoperability we STRONGLY RECOMMEND to only use alpha-numerical ASCII characters in the ID value only**. <br/>There are several known limitations in some versions of the Cosmos DB SDK, as well as connectors (ADF, Spark, Kafka etc.) and http-drivers/libraries etc. that can prevent successful processing when the ID value contains non-alphanumerical ASCII characters. So, to increase interoperability, please encode the ID value - [for example via Base64 + custom encoding of special charatcers allowed in Base64](https://github.com/Azure/azure-cosmos-dotnet-v3/blob/78fc16c35c521b4f9a7aeef11db4df79c2545dee/Microsoft.Azure.Cosmos.Encryption/src/EncryptionProcessor.cs#L475-L489). - if you have to support non-alphanumerical ASCII characters in your service/application. |

+- [High-level TLA+ specifications for the five consistency levels offered by Azure Cosmos DB](https://github.com/tlaplus/azure-cosmos-tla)

+|Regional outage ΓÇô availability | Availability loss | Availability loss | No availability loss for read region failure, temporary for write region failure | No availability loss for read region failure, temporary for write region failure | No read availability loss, temporary write availability loss in the affected region |

+To get started using partition merge, navigate to the **Features** page in your Azure Cosmos DB account. Select and enable the **Partition merge (preview)** feature.

+Before enabling the feature, verify that your Azure Cosmos DB account(s) meet all the [preview eligibility criteria](#preview-eligibility-criteria). Once you've enabled the feature, it will take 15-20 minutes to take effect.

+> [!CAUTION]

+> When merge is enabled on an account, only requests from .NET SDK version >= 3.27.0 will be allowed on the account, regardless of whether merges are ongoing or not. Requests from other SDKs (older .NET SDK, Java, JavaScript, Python, Go) or unsupported connectors (Azure Data Factory, Azure Search, Azure Cosmos DB Spark connector, Azure Functions, Azure Stream Analytics, and others) will be blocked and fail. Ensure you have upgraded to a supported SDK version before enabling the feature. After the feature is enabled or disabled, it may take 15-20 minutes to fully propagate to the account. If you plan to disable the feature after you've completed using it, it may take 15-20 minutes before requests from SDKs and connectors that are not supported for merge are allowed.

+- **Condition 1**: The current RU/s per physical partition is <3000 RU/s

+- **Condition 2**: The current average storage in GB per physical partition is <20 GB

+Condition 1 often occurs when you've previously scaled up the RU/s (often for a data ingestion) and now want to scale down in steady state.

+To determine the current RU/s per physical partition, from your Cosmos account, navigate to **Metrics**. Select the metric **Physical Partition Throughput** and filter to your database and container. Apply splitting by **PhysicalPartitionId**.

+For containers using autoscale, this metric will show the max RU/s currently provisioned on each physical partition. For containers using manual throughput, this metric will show the manual RU/s on each physical partition.

+In the below example, we have an autoscale container provisioned with 5000 RU/s (scales between 500 - 5000 RU/s). It has five physical partitions and each physical partition has 1000 RU/s.

+Next, find the total number of physical partitions. This metric is the distinct number of **PhysicalPartitionIds** in the **PhysicalPartitionThroughput** chart we saw in Criteria 1. In our example, we have five physical partitions.

+Finally, calculate: Total storage in GB / number of physical partitions. In our example, we have an average of (74 GB / five physical partitions) = 14.8 GB per physical partition.

+$parameters = @{

+ Name = "Az.CosmosDB"

+ AllowPrerelease = $true

+ Force = $true

+}

+Install-Module @parameters

+```

+```azurepowershell

+$parameters = @{

+ ResourceGroupName = "<resource-group-name>"

+ AccountName = "<cosmos-account-name>"

+ DatabaseName = "<cosmos-database-name>"

+ Name = "<cosmos-container-name>"

+ WhatIf = $true

+}

+Invoke-AzCosmosDBSqlContainerMerge @parameters

+```

+```azurepowershell

+$parameters = @{

+ ResourceGroupName = "<resource-group-name>"

+ AccountName = "<cosmos-account-name>"

+ DatabaseName = "<cosmos-database-name>"

+ Name = "<cosmos-container-name>"

+ WhatIf = $true

+}

+Invoke-AzCosmosDBMongoDBCollectionMerge @parameters

+```

+```azurecli

+```

+```azurecli

+- Your Azure Cosmos DB account uses API for NoSQL or MongoDB with version >=3.6.

+- Your Azure Cosmos DB account is using provisioned throughput (manual or autoscale). Merge doesn't apply to serverless accounts.

+ - Currently, merge isn't supported for shared throughput databases. You may enroll an account that has both shared throughput databases and containers with dedicated throughput (manual or autoscale).

+ - However, only the containers with dedicated throughput will be able to be merged.

+- Your Azure Cosmos DB account is a single-write region account (merge isn't currently supported for multi-region write accounts).

+- Your Azure Cosmos DB account doesn't use any of the following features:

+ - [Point-in-time restore](continuous-backup-restore-introduction.md)

+ - [Customer-managed keys](how-to-setup-cmk.md)

+ - [Analytical store](analytical-store-introduction.md)

+- Your Azure Cosmos DB account uses bounded staleness, session, consistent prefix, or eventual consistency (merge isn't currently supported for strong consistency).

+- If you're using API for NoSQL, your application must use the Azure Cosmos DB .NET V3 SDK, version 3.27.0 or higher. When merge preview enabled on your account, all requests sent from non .NET SDKs or older .NET SDK versions won't be accepted.

+ - There are no SDK or driver requirements to use the feature with API for MongoDB.

+- Your Azure Cosmos DB account doesn't use any currently unsupported connectors:

+ - Azure Data Factory

+ - Azure Stream Analytics

+ - Logic Apps

+ - Azure Functions

+ - Azure Search

+ - Azure Cosmos DB Spark connector

+ - Azure Cosmos DB data migration tool

+ - Any third party library or tool that has a dependency on an Azure Cosmos DB SDK that isn't .NET V3 SDK v3.27.0 or higher

+- Merge is only available for API for NoSQL and MongoDB accounts. For API for MongoDB accounts, the MongoDB account version must be 3.6 or greater.

+- Merge is only available for single-region write accounts. Multi-region write account support isn't available.

+- Accounts using merge functionality can't also use these features (if these features are added to a merge enabled account, resources in the account will no longer be able to be merged):

+ - [Point-in-time restore](continuous-backup-restore-introduction.md)

+ - [Customer-managed keys](how-to-setup-cmk.md)

+ - [Analytical store](analytical-store-introduction.md)

+- Containers using merge functionality must have their throughput provisioned at the container level. Database-shared throughput support isn't available.

+- Merge is only available for accounts using bounded staleness, session, consistent prefix, or eventual consistency. It isn't currently supported for strong consistency.

+- After a container has been merged, it isn't possible to read the change feed with start time. Support for this feature is planned for the future.

+Accounts with the merge feature enabled are supported only when you use the latest version of the .NET v3 SDK. When the feature is enabled on your account (regardless of whether you run the merge), you must only use the supported SDK using the account. Requests sent from other SDKs or earlier versions won't be accepted. As long as you're using the supported SDK, your application can continue to run while a merge is ongoing.

+> You should ensure that your application has been updated to use a compatible SDK version prior to enrolling in the preview. If you're using the legacy .NET V2 SDK, follow the [.NET SDK v3 migration guide](sql/migrate-dotnet-v3.md).

+- Azure Data Factory <sup>1</sup>

+- Azure Stream Analytics <sup>1</sup>

+- Logic Apps <sup>1</sup>

+- Azure Functions <sup>1</sup>

+- Azure Search <sup>1</sup>

+- Azure Cosmos DB Spark connector <sup>1</sup>

+- Azure Cosmos DB data migration tool

+- Any third party library or tool that has a dependency on an Azure Cosmos DB SDK that isn't .NET V3 SDK v3.27.0 or higher

+<sup>1</sup> Support for these connectors is planned for the future.

+- Learn more about [using Azure CLI with Azure Cosmos DB.](/cli/azure/azure-cli-reference-for-cosmos-db)

+- Learn more about [using Azure PowerShell with Azure Cosmos DB.](/powershell/module/az.cosmosdb/)

+- Learn more about [partitioning in Azure Cosmos DB.](partitioning-overview.md)

+|<input type="checkbox"/> | Retry Logic | A transient error is an error that has an underlying cause that soon resolves itself. Applications that connect to your database should be built to expect these transient errors. To handle them, implement retry logic in your code instead of surfacing them to users as application errors. The SDK has built-in logic to handle these transient failures on retryable requests like read or query operations. For accounts configured with a single write region, the SDK won't retry on writes for transient failures as writes aren't idempotent. For accounts configured with multiple write regions, there are [some scenarios](troubleshoot-sdk-availability.md#transient-connectivity-issues-on-tcp-protocol) where the SDK will automatically retry writes on other regions. The SDK does allow users to configure retry logic for throttles. For details on which errors to retry on [visit](conceptual-resilient-sdk-applications.md#should-my-application-retry-on-errors) |

+This tutorial walks through how to use the Jupyter Notebooks feature of Azure Cosmos DB to import sample retail data to an Azure Cosmos DB for NoSQL account. You'll see how to use the Azure Cosmos DB magic commands to run queries, analyze the data, and visualize the results.

+- [Azure Cosmos DB for NoSQL account](create-cosmosdb-resources-portal.md#create-an-azure-cosmos-db-account) (configured with serverless throughput).

+- [Import notebooks from GitHub into an Azure Cosmos DB for NoSQL account](tutorial-import-notebooks.md)

+ Title: 'Tutorial: Deploy an ASP.NET web application using Azure Cosmos DB for NoSQL, managed identity, and Azure Kubernetes Service via Bicep'

+description: Learn how to deploy an ASP.NET MVC web application with Azure Cosmos DB for NoSQL, managed identity, and Azure Kubernetes Service by using Bicep.

+# Tutorial: Deploy an ASP.NET web application by using Azure Cosmos DB for NoSQL, managed identity, and AKS via Bicep

+[Azure Cosmos DB](../introduction.md) is a fully managed distributed database platform for modern application development with NoSQL or relational databases.

+[AKS](../../aks/intro-kubernetes.md) is a managed Kubernetes service that helps you quickly deploy and manage clusters.

+> - This article requires the latest version of the Azure CLI. For more information, see [Install the Azure CLI](/cli/azure/install-azure-cli). If you're using Azure Cloud Shell, the latest version is already installed.

+> - This article also requires the latest version of the Bicep CLI within the Azure CLI. For more information, see [Install Bicep tools](../../azure-resource-manager/bicep/install.md#azure-cli).

+> - If you're running the commands in this tutorial locally instead of in Azure Cloud Shell, ensure that you use an administrator account.

+## Prerequisites

+The following tools are required to compile the ASP.NET web application and create its container image:

+This tutorial uses an [infrastructure as code (IaC)](/devops/deliver/what-is-infrastructure-as-code) approach to deploy the resources to Azure. You'll use [Bicep](../../azure-resource-manager/bicep/overview.md), which is a new declarative language that offers the same capabilities as [Azure Resource Manager templates](../../azure-resource-manager/templates/overview.md). However, Bicep includes a syntax that's more concise and easier to use.

+The Bicep modules will deploy the following Azure resources within the targeted subscription scope:

+- A [resource group](../../azure-resource-manager/management/overview.md#resource-groups) to organize the resources

+- A [managed identity](../../active-directory/managed-identities-azure-resources/overview.md) for authentication

+- A [container registry](../../container-registry/container-registry-intro.md) for storing container images

+- An [AKS](../../aks/intro-kubernetes.md) cluster

+- A [virtual network](../../virtual-network/network-overview.md) for configuring AKS

+- An [Azure Cosmos DB for NoSQL account](../introduction.md), along with a database, a container, and the [SQL role](/cli/azure/cosmosdb/sql/role)

+- A [key vault](../../key-vault/general/overview.md) to store secure keys

+- (Optional) A [Log Analytics workspace](../../azure-monitor/logs/log-analytics-overview.md)

+This tutorial uses the following security best practices with Azure Cosmos DB:

+- Implement access control by using [role-based access control (RBAC)](../../role-based-access-control/overview.md) and a [managed identity](../../active-directory/managed-identities-azure-resources/overview.md). These features eliminate the need for developers to manage secrets, credentials, certificates, and keys for secure communication between services.

+- Limit Azure Cosmos DB access to the AKS subnet by [configuring a virtual network service endpoint](../how-to-configure-vnet-service-endpoint.md).

+- Set `disableLocalAuth = true` in the `databaseAccount` resource to [enforce RBAC as the only authentication method](../how-to-setup-rbac.md#disable-local-auth).

+> The steps in this tutorial use [Azure Cosmos DB for NoSQL](./quickstart-dotnet.md). However, you can apply the same concepts to [Azure Cosmos DB for MongoDB](../mongodb/introduction.md).

+Download or [clone](https://docs.github.com/repositories/creating-and-managing-repositories/cloning-a-repository) the Bicep modules from the *Bicep* folder of the [azure-samples/cosmos-aks-samples](https://github.com/Azure-Samples/cosmos-aks-samples/tree/main/Bicep) GitHub repository:

+Use [az login](/cli/azure/authenticate-azure-cli) to connect to your default Azure subscription:

+Optionally, use [az account set](/cli/azure/account#az-account-set) with the name or ID of a specific subscription to set the active subscription if you have multiple subscriptions:

+Create a *param.json* file by using the JSON in the following example. Replace the `{resource group name}`, `{Azure Cosmos DB account name}`, and `{Azure Container Registry instance name}` placeholders with your own values.

+> All resource names that you use in the following code should comply with the [naming rules and restrictions for Azure resources](../../azure-resource-manager/management/resource-name-rules.md). Also ensure that the placeholder values are replaced consistently and match the values in *param.json*.

+Set shell variables by using the following commands. Replace the `{deployment name}` and `{location}` placeholders with your own values.

+deploymentName='{deployment name}' # Name of the deployment

+Within the *Bicep* folder, use [az deployment sub create](/cli/azure/deployment/sub#az-deployment-sub-create) to deploy the template to the current subscription scope:

+The deployment could take 20 to 30 minutes. After provisioning is completed, the console will output JSON with `Succeeded` as the provisioning state:

+You can also see the deployment status in the resource group:

+> When you create an AKS cluster, a second resource group is automatically created to store the AKS resources. For more information, see [Why are two resource groups created with AKS?](../../aks/faq.md#why-are-two-resource-groups-created-with-aks).

+Use the following commands to link your Azure Container Registry instance with AKS. Replace the `{Azure Container Registry instance name}` and `{resource group name}` placeholders with your own values.

+Run [az aks update](/cli/azure/aks#az-aks-update) to attach the existing Azure Container Registry resource with the AKS cluster:

+To manage a Kubernetes cluster, you use [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/), the Kubernetes command-line client. If you use Azure Cloud Shell, `kubectl` is already installed. To install `kubectl` locally, use [az aks install-cli](/cli/azure/aks#az-aks-install-cli):

+To configure `kubectl` to connect to your Kubernetes cluster, use [az aks get-credentials](/cli/azure/aks#az-aks-get-credentials). This command downloads credentials and configures the Kubernetes CLI to use them.

+Azure Active Directory (Azure AD) pod-managed identities use AKS primitives to associate managed identities for Azure resources and identities in Azure AD with pods. You'll use these identities to grant access to the Azure Key Vault Provider for Secrets Store CSI Driver.

+Use the following command to find the values of the tenant ID (`homeTenantId`):

+Use the following YAML template to create a *secretproviderclass.yml* file. Replace the `{Tenant Id}` and `{resource group name}` placeholders with your own values. Also ensure that the value for `{resource group name}` matches the value in *param.json*.

+# This is a SecretProviderClass example that uses aad-pod-identity to access the key vault

+ keyvaultName: "{resource group name}kv" # Replace resource group name. Bicep generates the key vault name.

+ tenantId: "{Tenant Id}" # The tenant ID of your account. Use the 'homeTenantId' attribute value from the 'az account show' command output.

+Use [kubectl apply](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply) to install the Secrets Store CSI Driver by using the YAML:

+Download or clone the web application's source code from the *Application* folder of the [azure-samples/cosmos-aks-samples](https://github.com/Azure-Samples/cosmos-aks-samples/tree/main/Application) GitHub repository:

+Open the *Application* folder in Visual Studio Code. Run the application by using either the F5 key or the **Debug: Start Debugging** command.

+1. To create a container image from the **Explorer** tab in Visual Studio Code, right-click **Dockerfile**, and then select **Build Image**.

+1. In the prompt that asks for the name and version to tag the image, enter the name **todo:latest**.

+1. Use the **Docker** pane to push the built image to Azure Container Registry. You'll find the built image under the **Images** node. Open the **todo** node, right-click **latest**, and then select **Push**.

+1. In the prompts, select your Azure subscription, Azure Container Registry resource, and image tags. The image tag format should be `{acrname}.azurecr.io/todo:latest`.

+1. Wait for Visual Studio Code to push the container image to Azure Container Registry.

+## Prepare the deployment YAML

+Use the following YAML template to create an *akstododeploy.yml* file. Replace the `{ACR name}`, `{Image name}`, `{Version}`, and `{resource group name}` placeholders with your own values.

+ image: "{ACR name}/{Image name}:{Version}" # Update per your environment; for example, myacrname.azurecr.io/todo:latest. Do not add https:// in ACR Name.

+ value: "{resource group name}kv" # Replace resource group name. Key Vault name is generated by Bicep.

+## Apply the deployment YAML

+Use `kubectl apply` again to deploy the application pods and expose the pods via a load balancer:

+When the application runs, a Kubernetes service exposes the application's front end to the internet. This process can take a few minutes to complete.

+Use [kubectl get](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get) to view the external IP that the load balancer exposes:

+To access the application, open the IP address that you received as output in a browser.

+To avoid Azure charges, clean up unneeded resources when you no longer need the cluster. Use [az group delete](/cli/azure/group#az-group-delete) and [az deployment sub delete](/cli/azure/deployment/sub#az-deployment-sub-delete) to delete the resource group and subscription deployment, respectively:

+- Learn how to [develop a web application with Azure Cosmos DB](./tutorial-dotnet-web-app.md).

+- Learn how to [query Azure Cosmos DB for NoSQL](./tutorial-query.md).

+- Learn how to [upgrade your cluster](../../aks/tutorial-kubernetes-upgrade-cluster.md).

+- Learn how to [scale your cluster](../../aks/tutorial-kubernetes-scale.md).

+- Learn how to [enable continuous deployment](../../aks/deployment-center-launcher.md).

+ Title: |

+ Tutorial: Import Jupyter notebooks from GitHub into Azure Cosmos DB for NoSQL (preview)

+description: |

+ Learn how to connect to GitHub and import the notebooks from a GitHub repository to your Azure Cosmos DB for NoSQL account.

+# Tutorial: Import Jupyter notebooks from GitHub into Azure Cosmos DB for NoSQL (preview)

+> [!IMPORTANT]

+> The Jupyter Notebooks feature of Azure Cosmos DB is currently in a preview state and is progressively rolling out to all customers over time.

+This tutorial walks through how to import Jupyter notebooks from a GitHub repository and run them in an Azure Cosmos DB for NoSQL account. After importing the notebooks, you can run, edit them, and persist your changes back to the same GitHub repository.

+## Prerequisites

+- [Azure Cosmos DB for NoSQL account](create-cosmosdb-resources-portal.md#create-an-azure-cosmos-db-account) (configured with serverless throughput).

+## Create a copy of a GitHub repository

+1. Navigate to the [azure-samples/cosmos-db-nosql-notebooks](https://github.com/azure-samples/cosmos-db-nosql-notebooks/generate) template repository.

+1. Create a new copy of the template repository in your own GitHub account or organization.

+## Pull notebooks from GitHub

+Instead of creating new notebooks each time you start a workspace, you can import existing notebooks from GitHub. In this section, you'll connect to an existing GitHub repository with sample notebooks.

+1. Navigate to your Azure Cosmos DB account and open the **Data Explorer.**

+1. Select **Connect to GitHub**.

+ :::image type="content" source="media/tutorial-import-notebooks/connect-github-option.png" lightbox="media/tutorial-import-notebooks/connect-github-option.png" alt-text="Screenshot of the Data Explorer with the 'Connect to GitHub' option highlighted.":::

+1. In the **Connect to GitHub** dialog, select the access option appropriate to your GitHub repository and then select **Authorize access**.

+ :::image type="content" source="media/tutorial-import-notebooks/authorize-access.png" alt-text="Screenshot of the 'Connect to GitHub' dialog with options for various levels of access.":::

+1. Complete the GitHub third-party authorization workflow granting access to the organization\[s\] required to access your GitHub repository. For more information, see [Authorizing GitHub Apps](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/authorizing-github-apps).

+1. In the **Manage GitHub settings** dialog, select the GitHub repository you created earlier.

+ :::image type="content" source="media/tutorial-import-notebooks/select-pinned-repositories.png" alt-text="Screenshot of the 'Manage GitHub settings' dialog with a list of unpinned and pinned repositories.":::

+1. Back in the Data Explorer, locate the new tree of nodes for your pinned repository and open the **website-metrics-python.ipynb** file.

+ :::image type="content" source="media/tutorial-import-notebooks/open-notebook-pinned-repositories.png" alt-text="Screenshot of the pinned repositories in the Data Explorer.":::

+1. In the editor for the notebook, locate the following cell.

+ ```python

+ import pandas as pd

+ pd.options.display.html.table_schema = True

+ pd.options.display.max_rows = None

+

+ df_cosmos.groupby("Item").size()

+ ```

+1. The cell currently outputs the number of unique items. Replace the final line of the cell with a new line to output the number of unique actions in the dataset.

+ ```python

+ df_cosmos.groupby("Action").size()

+ ```

+1. Run all the cells sequentially to see the new dataset. The new dataset should only include three potential values for the **Action** column. Optionally, you can select a data visualization for the results.

+ :::image type="content" source="media/tutorial-import-notebooks/updated-visualization.png" alt-text="Screenshot of the Pandas dataframe visualization for the data.":::

+## Push notebook changes to GitHub

+> [!TIP]

+> Currently, temporary workspaces will be de-allocated if left idle for 20 minutes. The maximum amount of usage time per day is 60 minutes. These limits are subject to change in the future.

+To save your work permanently, save your notebooks back to the GitHub repository. In this section, you'll persist your changes from the temporary workspace to GitHub as a new commit.

+1. Select **Save** to create a commit for your change to the notebook.

+ :::image type="content" source="media/tutorial-import-notebooks/save-option.png" alt-text="Screenshot of the 'Save' option in the Data Explorer menu.":::

+1. In the **Save** dialog, add a descriptive commit message.

+ :::image type="content" source="media/tutorial-import-notebooks/commit-message-dialog.png" alt-text="Screenshot of the 'Save' dialog with an example of a commit message.":::

+1. Navigate to the GitHub repository you created using your browser. The new commit should now be visible in the online repository.

+ :::image type="content" source="media/tutorial-import-notebooks/updated-github-repository.png" alt-text="Screenshot of the updated notebook on the GitHub website.":::

+## Next steps

+- [Learn about the Jupyter Notebooks feature in Azure Cosmos DB](../notebooks-overview.md)

+- [Create your first notebook in an Azure Cosmos DB for NoSQL account](tutorial-create-notebook.md)

+- [Review the FAQ on Jupyter Notebook support](../notebooks-faq.yml)

+- [Import notebooks from GitHub into an Azure Cosmos DB for NoSQL account](nosql/tutorial-import-notebooks.md)

+- Inspect shards and placements with [useful diagnostic queries](howto-useful-diagnostic-queries.md).

+> | [PostGIS](https://www.postgis.net/) | Spatial and geographic objects for PostgreSQL. | 2.5.5 | 3.0.5 | 3.0.5 | 3.1.5 | 3.3.1 |

+ Title: Understand reservations discount for Azure SQL Edge

+description: Learn how a reservation discount is applied to Azure SQL Edge.

+# How a reservation discount is applied to Azure SQL Edge

+After you buy Azure SQL Edge reserved capacity, the reservation discount is automatically applied to SQL Edge deployed to edge devices that match the attributes and quantity of the reservation. A reservation applies to the future use of Azure SQL Edge deployments. You're charged for software, storage, and networking at the normal rates.

+## How reservation discount is applied

+A reservation discount is "_use-it-or-lose-it_". So, if you don't have matching resources for any month, then you lose a reservation quantity for that month. You can't carry forward unused reserved months.

+When you shut down a resource, the reservation discount automatically applies to another matching resource in the specified scope. If no matching resources are found in the specified scope, then the reserved months are _lost_.

+Stopped resources are billed and continue to use reservation hours. Deallocate or delete resources or scale-in other resources to use your available reservation months with other workloads.

+## Discount applied to deployed devices

+The reserved capacity discount is applied to deployed devices monthly. The reservation that you buy is matched to the usage emitted by the deployed device. For devices that don't run the full month, the reservation is automatically applied to other deployed devices matching the reservation attributes. The discount can apply to deployed devices that are running concurrently. If you don't have deployed devices that run for the full month that match the reservation attributes, you don't get the full benefit of the reservation discount for that month.

+If your number of devices deployed exceeds your reservation quantity, then you're charged the non-discounted cost for the number beyond the reservation quantity.

+To understand and view the application of your Azure Reservations in billing usage reports, see [Understand Azure reservation usage](understand-reserved-instance-usage-ea.md).

+## Need help? Contact us

+If you have questions or need help, [create a support request](https://go.microsoft.com/fwlink/?linkid=2083458).

+## Next steps

+To learn more about Azure Reservations, see the following articles:

+- [What are Azure Reservations?](save-compute-costs-reservations.md)

+- [Manage Azure Reservations](manage-reserved-vm-instance.md)

+- [Understand reservation usage for your Pay-As-You-Go subscription](understand-reserved-instance-usage.md)

+- [Understand reservation usage for your Enterprise enrollment](understand-reserved-instance-usage-ea.md)

+- [Understand reservation usage for CSP subscriptions](/partner-center/azure-reservations)

+- [Azure SQL Edge](prepay-sql-edge.md)

+ Title: Prepay for Azure SQL Edge reservations

+description: Learn how you can prepay for Azure SQL Edge to save money over your pay-as-you-go costs.

+# Prepay for Azure SQL Edge reservations

+When you prepay for your SQL Edge reserved capacity, you can save money over your pay-as-you-go costs. With reserved capacity, you make a commitment for SQL Edge device use for a period of one or three years to get a significant discount on usage costs. The discounts only apply to SQL Edge deployed devices and not on other software or other container usage. The reservation discount is applied automatically to the deployed devices in the selected reservation scope. Because of this automatic application, you don't need to assign a reservation to a specific deployed device to get the discounts.

+You can buy SQL Edge reserved capacity from the [Azure portal](https://portal.azure.com/). Pay for the reservation [up front or with monthly payments](prepare-buy-reservation.md). To buy reserved capacity:

+- You must be in the Owner role for at least one Enterprise or individual subscription with pay-as-you-go rates.

+- For Enterprise subscriptions, **Add Reserved Instances** must be enabled in the [EA portal](https://ea.azure.com/). Or, if that setting is disabled, you must be an EA Admin on the subscription.

+- For the Cloud Solution Provider (CSP) program, only admin agents or sales agents can buy SQL Edge reserved capacity.

+## Buy a software plan

+1. Sign in to the [Azure portal](https://portal.azure.com/).

+2. Select **All services** > **Reservations**.

+3. Select **Add** and then in the **Purchase Reservations** pane, select **Azure SQL Edge**.

+4. Select a scope. The reservation's scope can cover one subscription or multiple subscriptions (shared scope):

+ - **Shared** - The reservation discount is applied to SQL Edge devices running in any subscriptions within your billing context. For enterprise customers, the shared scope is the enrollment and includes all subscriptions within the enrollment. For Pay-As-You-Go customers, the shared scope is all Pay-As-You-Go subscriptions created by the account administrator.

+ - **Management Group** - The reservation discount is applied to the matching resource in the list of subscriptions that are a part of both the management group and billing scope.

+ - **Single subscription** - The reservation discount is applied to the SQL Edge devices in this subscription.

+ - **Single resource group** - The reservation discount is applied to the SQL Edge devices in the selected subscription and the selected resource group within that subscription.

+5. Select a **Subscription**. The subscription used to pay for the capacity reservation. The subscription payment method is charged the upfront costs for the reservation.

+ - For an enterprise subscription, the charges are deducted from the enrollment's Azure Prepayment (previously called monetary commitment) balance or charged as overage.

+ - For an individual subscription with pay-as-you-go pricing, the charges are billed to the subscription's credit card or invoice payment method.

+6. Select a **Region**. The Azure region that's covered by the capacity reservation.

+7. Select the **Billing frequency**. It indicates how often the account is billed for the reservation. Options include _Monthly_ or _Upfront_.

+8. Select a **Term**. One year or three years.

+9. Add the product to the cart.

+10. Choose a quantity, which is the number of prepaid SQL Edge deployments that can get the billing discount.

+11. Review your selections and purchase.

+## Cancel, exchange, or refund reservations

+You can cancel, exchange, or refund reservations with certain limitations. For more information, see [Self-service exchanges and refunds for Azure Reservations](exchange-and-refund-azure-reservations.md).

+## Need help? Contact us.

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+To learn how to manage a reservation, see [Manage Azure reservations](manage-reserved-vm-instance.md).

+To learn more, see the following articles:

+- [What are Azure Reservations?](save-compute-costs-reservations.md)

+- [Manage Reservations in Azure](manage-reserved-vm-instance.md)

+- [Understand reservation usage for your Pay-As-You-Go subscription](understand-reserved-instance-usage.md)

+- [Understand reservation usage for your Enterprise enrollment](understand-reserved-instance-usage-ea.md)

+- [Azure SQL Edge](discount-sql-edge.md)

+# Using data flows in pipelines

+## Access Azure Synapse database templates in pipelines

+You can use an [Azure Synapse database template](../synapse-analytics/database-designer/overview-database-templates.md) when crating a pipeline. When creating a new dataflow, in the source or sink settings, select **Workspace DB**. The database dropdown will list the databases created through the database template. The Workspace DB option is only available for new data flows, it's not available when you use an existing pipeline from the Synapse studio gallery.

+- [Tutorial: Incrementally load data from a source data store to a destination data store](tutorial-incremental-copy-overview.md)

+The Until activity provides the same functionality that a do-until looping structure provides in programming languages. It executes a set of activities in a loop until the condition associated with the activity evaluates to true. If an inner activity fails, the Until activity does not stop. You can specify a timeout value for the until activity.

+When using data flows in Azure Synapse workspaces, you will have an additional option to sink your data directly into a database type that is inside your Synapse workspace. This will alleviate the need to add linked services or datasets for those databases. The databases created through the [Azure Synapse database templates](../synapse-analytics/database-designer/overview-database-templates.md) are also accessible when you select Workspace DB.

+In Azure Synapse workspaces, an additional option is present in data flow source transformations called ```Workspace DB```. This will allow you to directly pick a workspace database of any available type as your source data without requiring additional linked services or datasets. The databases created through the [Azure Synapse database templates](../synapse-analytics/database-designer/overview-database-templates.md) are also accessible when you select Workspace DB.

+ ```

+ ```powershell

+ ```json

+ ```

+ Set-Location 'C:\ADFv2Tutorial'

+ ```

+1. Create a JSON file named *SqlServerDataset.json* in the *C:\ADFv2Tutorial* folder, with the following code:

+>[Copy data in bulk](tutorial-bulk-copy.md)

+ Both **start** and **end** datetime values must be in [ISO format](https://en.wikipedia.org/wiki/ISO_8601), such as `2014-10-14T16:32:41Z`. The **end** time is optional. If you do not specify value for the **end** property, it is calculated as "**start + 48 hours.**" To run the pipeline indefinitely, specify **9999-09-09** as the value for the **end** property. See [JSON Scripting Reference](/previous-versions/azure/dn835050(v=azure.100)) for details about JSON properties.

+When using the reader and writer modules, it's good practice to use a Web service parameter for each property of these reader/writer modules. These web parameters enable you to configure the values during runtime. For example, you could create an experiment with a reader module that uses an Azure SQL Database instance: `XXX.database.windows.net`. After the web service has been deployed, you want to enable the consumers of the web service to specify another logical SQL Server instance called `YYY.database.windows.net`. You can use a Web service parameter to allow this value to be configured.

+```json

+```json

+```json

+```json

+```json

+```json

+```json

+```json

+## See also

+[azure-machine-learning]: https://azure.microsoft.com/services/machine-learning/

+Azure Data Share enables organizations to securely share data with multiple customers and partners. Data providers are always in control of the data that they've shared and Azure Data Share makes it simple to manage and monitor what data was shared, when and by whom.

+In today's world, data is viewed as a key strategic asset that many organizations need to simply and securely share with their customers and partners. There are many ways that customers do this today, including through FTP, e-mail, APIs to name a few. Organizations can easily lose track of who they've shared their data with. Sharing data through FTP or through standing up their own API infrastructure is often expensive to provision and administer. There's management overhead associated with using these methods of sharing on a large scale. In addition to accountability, many organizations would like to be able to control, manage, and monitor all of their data sharing in a simple way that stays up to date, so they can derive timely insights.

+Using Data Share, a data provider can share data and manage their shares all in one place. They can stay in control of how their data is handled by specifying terms of use for their data share. The data consumer must accept these terms before being able to receive the data. Data providers can specify the frequency at which their data consumers receive updates. Access to new updates can be revoked at any time by the data provider.

+Azure Data Share can be used in many different industries. For example, a retailer may want to share recent point of sales data with their suppliers. Using Azure Data Share, a retailer can set up a data share containing point of sales data for all of their suppliers and share sales on an hourly or daily basis.

+Azure Data Share currently offers [snapshot-based sharing](#snapshot-based-sharing) and [in-place sharing](#in-place-sharing).

+

+### Snapshot-based sharing

+### In-place sharing

+ 

+In this tutorial, you'll learn how to set up a new Azure Data Share and start sharing your data with customers and partners outside of your Azure organization.

+* If the source Azure data store is in a different Azure subscription than the one you'll use to create Data Share resource, register the [Microsoft.DataShare resource provider](concepts-roles-permissions.md#resource-provider-registration) in the subscription where the Azure data store is located.

+ 1. Select **+Add client IP**. Client IP address is subject to change. This process might need to be repeated the next time you're sharing SQL data from Azure portal. You can also add an IP range.

+* * An Azure Synapse Analytics (workspace) dedicated SQL pool with tables that you want to share. Sharing of view isn't currently supported. Sharing from serverless SQL pool isn't currently supported.

+ The *<share_acc_name>* is the name of your Data Share resource. If you haven't created a Data Share resource as yet, you can come back to this pre-requisite later.

+ 1. Select **+Add client IP**. Client IP address is subject to change. This process might need to be repeated the next time you're sharing SQL data from Azure portal. You can also add an IP range.

+1. Select **Review + create**, then **Create** to create your data share account. Creating a new data share account typically takes about 2 minutes or less.

+ :::image type="content" source="./media/share-receive-data.png" alt-text="Screenshot of the Azure Data Share overview page in the Azure portal.":::

+1. Select **Create**.

+ :::image type="content" source="./media/enter-share-details.png " alt-text="Screenshot of the share creation page in Azure Data Share, showing the share name, type, description, and terms of used filled out.":::

+1. To add Datasets to your share, select **Add Datasets**.

+ :::image type="content" source="./media/datasets.png" alt-text="Screenshot of the datasets page in share creation, the add datasets button is highlighted.":::

+1. Select the dataset type that you would like to add. You'll see a different list of dataset types depending on the share type (snapshot or in-place) you've selected in the previous step. If sharing from an Azure SQL Database or Azure Synapse Analytics (formerly Azure SQL DW), you'll be prompted for authentication method to list tables. Select Azure Active Directory authentication, and check the checkbox **Allow Data Share to run the above 'create user' script on my behalf**.

+ :::image type="content" source="./media/add-datasets.png" alt-text="Screenshot showing the available dataset types.":::

+1. Navigate to the object you would like to share and select 'Add Datasets'.

+ :::image type="content" source="./media/select-datasets.png" alt-text="Screenshot of the select datasets page, showing a folder selected.":::

+1. In the Recipients tab, enter in the email addresses of your Data Consumer by selecting '+ Add Recipient'.

+ :::image type="content" source="./media/add-recipient.png" alt-text="Screenshot of the recipients page, showing a recipient added.":::

+1. If you have selected snapshot share type, you can configure snapshot schedule to provide updates of your data to your data consumer.

+ :::image type="content" source="./media/enable-snapshots.png" alt-text="Screenshot of the settings page, showing the snapshot toggle enabled.":::

+1. Select a start time and recurrence interval.

+ --account-name ContosoMarketplaceAccount --target-email "jacob@fabrikam.com"

+1. If you don't already have data you would like to share, you can follow these steps to create a storage account. If you already have storage, you may skip to step 2.

+ 1. Run the [Set-AzStorageBlobContent](/powershell/module/az.storage/new-azstoragecontainer) command to upload a file. The following example uploads _textfile.csv_ from the _D:\testFiles_ folder on local memory, to the container you created.

+ New-AzDataShare -ResourceGroupName <String> -AccountName <String> -Name <String> -Description <String> -TermsOfUse <String>

+ Execute the script https://aka.ms/DevCenter/Install-DevCenterCli.ps1 directly in PowerShell to install:

+ iex "& { $(irm https://aka.ms/DevCenter/Install-DevCenterCli.ps1 ) }"

+ az extension add --source https://fidalgosetup.blob.core.windows.net/cli-extensions/devcenter-0.1.0-py3-none-any.whl

+ Execute the script https://aka.ms/DevCenter/Install-DevCenterCli.ps1 directly in PowerShell to install:

+ iex "& { $(irm https://aka.ms/DevCenter/Install-DevCenterCli.ps1 ) }"

+ az extension add --source https://fidalgosetup.blob.core.windows.net/cli-extensions/devcenter-0.1.0-py3-none-any.whl

+> [!IMPORTANT]

+> - You can't enter the Azure DNS IP address of 168.63.129.16 as the destination IP address for a rule. Attempting to add this IP address will output the error: **Exception while making add request for rule**.

+> - Do not use the private resolver's inbound endpoint IP address as a forwarding destination for zones that are not linked to the virtual network where the private resolver is provisioned.

+In this article, you'll learn how to add data partitions to an existing Microsoft Energy Data Services instance. The concept of "data partitions" is picked from [OSDU&trade;](https://osduforum.org/) where single deployment can contain multiple partitions.

+You can create maximum five data partitions in one MEDS instance. Currently, in line with the data partition capabilities that are available in OSDU&trade;, you can only create data partitions but can't delete or rename data existing data partitions.

+ The page shows a table of all data partitions in your MEDS instance with the status of the data partition next to it. Clicking the "Create" option on the top opens a right-pane for next steps.

+ Each data partition name needs to be 1-10 characters long and be a combination of lowercase letters, numbers and hyphens only. The data partition name will be prepended with the name of the MEDS instance. Choose a name for your data partition and hit create. As soon as you hit create, the deployment of the underlying data partition resources such as Azure Cosmos DB and Azure Storage accounts is started.

+The data-partition deployment triggered in the previous process might fail in some cases due to various issues. These issues include quota limits reached, ARM template deployment transient issues, data seeding failures, and failure in connecting to underlying AKS clusters.

+<hr width=100%>

+## October 20, 2022

+### Support for Private Links

+Azure Private Link on Microsoft Energy Data Services provides private access to the service. This means traffic between your private network and Microsoft Energy Data Services travels over the Microsoft backbone network therefore limiting any exposure over the internet. By using Azure Private Link, you can connect to a Microsoft Energy Data Services instance from your virtual network via a private endpoint, which is a set of private IP addresses in a subnet within the virtual network. You can then limit access to your Microsoft Energy Data Services instance over these private IP addresses. [Create a private endpoint for Microsoft Energy Data Services](how-to-set-up-private-links.md).

+### Encryption at Rest using Customer Managed Keys

+Microsoft Energy Data Services Preview supports customer managed encryption keys (CMK). All data in Microsoft Energy Data Services is encrypted with Microsoft-managed keys by default. In addition to Microsoft-managed key, you can use your own encryption key to protect the data in Microsoft Energy Data Services. When you specify a customer-managed key, that key is used to protect and control access to the Microsoft-managed key that encrypts your data. [Data security and encryption in Microsoft Energy Data Services](how-to-manage-data-security-and-encryption.md).

+<hr width=100%>

+- New data partitions can be [created dynamically](how-to-add-more-data-partitions.md) as needed post provisioning of the platform (up to five). Earlier, data partitions could only be created when provisioning a new instance.

+> [!NOTE]

+> The Event Hubs service natively converts some of the EventHubs specific [AmqpMessage properties](http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-properties) to [KafkaΓÇÖs record headers](https://kafka.apache.org/32/javadoc/org/apache/kafka/common/header/Headers.html) as **strings**. Kafka message header is a list of &lt;key, value&gt; pairs where key is string and value is always a byte array. For these supported properties, the byte array will have an UTF8encoded string.

+>

+> Here is the list of immutable properties that Event Hubs support in this conversion today. If you set values for user properties with the names in this list, you donΓÇÖt need to deserialize at the Kafka consumer side.

+>

+> - message-id

+> - user-id

+> - to

+> - reply-to

+> - content-type

+> - content-encoding

+> - creation-time

+ --network-plugin azure \

+> To create and use your own VNet and route table with `kubelet` network plugin, you need to use [user-assigned control plane identity][bring-your-own-control-plane-managed-identity]. For system-assigned control plane identity, we cannot get the identity ID before creating cluster, which causes delay for role assignment to take effect.

+> To create and use your own VNet and route table with `azure` network plugin, both system-assigned and user-assigned managed identities are supported.

+<!-- LINKS - Internal -->

+[bring-your-own-control-plane-managed-identity]: ../aks/use-managed-identity.md#bring-your-own-control-plane-managed-identity

+ condition = contains(["Standard_AzureFrontDoor", "Premium_AzureFrontDoor"], var.front_door_sku_name)

+ Title: 'Quickstart: Create an Azure Front Door (classic) using Terraform'

+- A policy or initiative is newly assigned to a scope. It takes around five minutes for the assignment

+A new policy or initiative assignment takes about five minutes to be applied. New or updated

+- microsoft.Kusto/clusters/databases (Azure Data Explorer databases)

+- microsoft.MobileNetwork/mobileNetworks (Mobile networks)

+- microsoft.MobileNetwork/mobileNetworks/simPolicies (SIM policies)

+- microsoft.MobileNetwork/mobileNetworks/sites (Mobile network sites)

+- microsoft.Network/customIpPrefixes (Custom IP prefixes)

+- microsoft.Network/dnsForwardingRulesets (DNS forwarding rulesets)

+- microsoft.Network/firewallPolicies (Firewall policies)

+- microsoft.NotificationHubs/namespaces (Notification Hub namespaces)

+- microsoft.Orbital/contactProfiles (Contact profiles)

+- microsoft.Orbital/GroundStations (Ground stations)

+- microsoft.Orbital/l2Connections (L2 connections)

+- microsoft.Peering/peeringServices (Peering services)

+| Qatar | Qatar Central | HDInsight.QatarCentral |

+| &nbsp; | Germany North| HDInsight.GermanyNorth |

+ - Qatar Central

+- If devices get disconnected from IoT Hub, devices should try to reconnect directly to the same IoT Hub for at least 15 minutes (If scenario permits 30 minutes or more), before attempting to go back to DPS.

+The secret will be returned with `\n` in place of newline:

+The `\n` above is a `\` and `n` character, not the newline character. Quotes `"` are included in the string.

+The secret will be returned with `\n` in place of newline:

+The `\n` above is a `\` and `n` character, not the newline character. Quotes `"` are included in the string.

+ - The virtual machine images that the labs can use to create VMs. For example, you might have a set of labs that need access to the [Data Science VM for Linux](https://azuremarketplace.microsoft.com/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) Azure Marketplace image.

+ - The virtual machine images that the labs can use to create VMs. For example, you might have a set of labs that need access to the [Data Science VM for Linux](https://azuremarketplace.microsoft.com/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) Azure Marketplace image.

+| VM image | [Data Science Virtual Machine for Linux (Ubuntu)](https://azuremarketplace.microsoft.com/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux). This image provides deep learning frameworks and tools for machine learning and data science. To view the full list of installed tools on this image, see [What's included on the DSVM?](../machine-learning/data-science-virtual-machine/overview.md#whats-included-on-the-dsvm). |

+| Virtual machine image | Choose **[Data Science Virtual Machine ΓÇô Windows Server 2019](https://azuremarketplace.microsoft.com/marketplace/apps/microsoft-dsvm.dsvm-win-2019)** or **[Data Science Virtual Machine ΓÇô Ubuntu](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux)** depending on your OS needs. |

+Before creating the lab plan, you'll need to set up the server to run the [Network License Manager](https://www.mathworks.com/help/install/administer-network-licenses.html) software. These instructions are only applicable for institutions that choose the networking licensing option for MATLAB, which allows users to share a pool of license keys. You'll also need to save the license file and file installation key for later. For detailed instructions on how to download a license file, see the first step in [Install License Manager on License Server](https://www.mathworks.com/help/install/ug/install-license-manager-on-license-server.html).

+For detailed instructions on how to install a licensing server, see [Install License Manager on License Server](https://www.mathworks.com/help/install/ug/install-license-manager-on-license-server.html). To enable borrowing, see [Borrow License](https://www.mathworks.com/help/install/license/borrow-licenses.html).

+class types setup RStudio differently. Each article describes how to use the [Data Science Virtual Machine for Linux (Ubuntu)](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) marketplace image, which has many [data science related tools](../machine-learning/data-science-virtual-machine/tools-included.md), including RStudio, pre-installed.

+class types set up RStudio differently. Each article describes how to use the [Data Science Virtual Machine for Linux (Ubuntu)](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) marketplace image, which has many [data science related tools](../machine-learning/data-science-virtual-machine/tools-included.md), including RStudio, pre-installed.

+> Linux labs only support automatic shut down when users disconnect and when VMs are started but users don't connect. Support also varies depending on [specific distributions and versions of Linux](../virtual-machines/extensions/diagnostics-linux.md#supported-linux-distributions). Shutdown settings are not supported by the [Data Science Virtual Machine - Ubuntu](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) image.

+> Some marketplace images already have a graphical desktop environment and remote desktop server installed. For example, the [Data Science Virtual Machine for Linux (Ubuntu)](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) already has [XFCE and X2Go Server installed and configured to accept client connections](../machine-learning/data-science-virtual-machine/dsvm-ubuntu-intro.md#x2go).

+> Prior to the [August 2022 Update](lab-services-whats-new.md), Linux labs only support automatic shut down when users disconnect and when VMs are started but users don't connect. Support also varies depending on [specific distributions and versions of Linux](../virtual-machines/extensions/diagnostics-linux.md#supported-linux-distributions). Shutdown settings are not supported by the [Data Science Virtual Machine - Ubuntu](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) image.

+> Only [specific distributions and versions of Linux](../virtual-machines/extensions/diagnostics-linux.md#supported-linux-distributions) are supported. Shutdown settings are not supported by the [Data Science Virtual Machine - Ubuntu](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) image.

+To manually download and install OneDrive, see the [OneDrive](https://onedrive.live.com/about/download/) or [OneDrive for Business](https://www.microsoft.com/microsoft-365/onedrive/onedrive-for-business) download pages.

+As described in the following table, the *compute* GPU size is intended for compute-intensive applications. For example, the [Deep Learning in Natural Language Processing class type](./class-type-deep-learning-natural-language-processing.md) uses the **Small GPU (Compute)** size. The compute GPU is suitable for this type of class, because students use deep learning frameworks and tools that are provided by the [Data Science Virtual Machine image](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) to train deep learning models with large sets of data.

+As described in the following table, the *compute* GPU size is intended for compute-intensive applications. For example, the [Deep Learning in Natural Language Processing class type](./class-type-deep-learning-natural-language-processing.md) uses the **Small GPU (Compute)** size. The compute GPU is suitable for this type of class, because students use deep learning frameworks and tools that are provided by the [Data Science Virtual Machine image](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=Data%20science%20Virtual%20machine&page=1&filters=microsoft%3Blinux) to train deep learning models with large sets of data.

+ > [!NOTE]

+ >

+ > The read-only **Runtime stack** property is automatically set at creation.

+ >

+ > If you get a validation error during this step, open and review the error details.

+ > For example, if your selected region reaches a quota for resources that you're

+ > trying to create, you might have to try a different region.

+| **Pfx** | `pfx` | Yes | <*encoded-pfx-file-content*> | The base64-encoded content from a Personal Information Exchange (PFX) file <p><p>To convert the PFX file into base64-encoded format, you can use PowerShell 7 by following these steps: <p>1. Save the certificate content into a variable: <p> `$pfx_cert = [System.IO.File]::ReadAllBytes('c:\certificate.pfx')` <p>2. Convert the certificate content by using the `ToBase64String()` function and save that content to a text file: <p> `[System.Convert]::ToBase64String($pfx_cert) | Out-File 'pfx-encoded-bytes.txt'` <p><p>**Troubleshooting**: If you use the `cert mmc/PowerShell` command, you might get this error: <p><p>`Could not load the certificate private key. Please check the authentication certificate password is correct and try again.` <p><p>To resolve this error, try converting the PFX file to a PEM file and back again by using the `openssl` command: <p><p>`openssl pkcs12 -in certificate.pfx -out certificate.pem` <br>`openssl pkcs12 -in certificate.pem -export -out certificate2.pfx` <p><p>Afterwards, when you get the base64-encoded string for the certificate's newly converted PFX file, the string now works in Azure Logic Apps. |

+## 2022-10-10

+### Azure Machine Learning CLI (v2) v2.10.0

+- The CLI is depending on GA version of azure-ai-ml.

+- Dropped support for Python 3.6.

+- `az ml registry`

+ - New command group added to manage ML asset registries.

+- `az ml job`

+ - Added `az ml job show-services` command.

+ - Added model sweeping and hyperparameter tuning to AutoML NLP jobs.

+- `az ml schedule`

+ - Added `month_days` property in recurrence schedule.

+- `az ml compute`

+ - Added custom setup scripts support for compute instances.

+## 2022-09-22

+### Azure Machine Learning CLI (v2) v2.8.0

+- `az ml job`

+ - Added spark job support.

+ - Added shm_size and docker_args to job.

+- `az ml compute`

+ - Compute instance supports managed identity.

+ - Added idle shutdown time support for compute instance.

+- `az ml online-deployment`

+ - Added support for data collection for eventhub and data storage.

+ - Added syntax validation for scoring script.

+- `az ml batch-deployment`

+ - Added syntax validation for scoring script.

+## 2022-08-10

+### Azure Machine Learning CLI (v2) v2.7.0

+- `az ml component`

+ - Added AutoML component.

+- `az ml dataset`

+ - Deprecated command group (Use `az ml data` instead).

+## 2022-07-16

+### Azure Machine Learning CLI (v2) v2.6.0

+- Added MoonCake cloud support.

+- `az ml job`

+ - Allow Git repo URLs to be used as code.

+ - AutoML jobs use the same input schema as other job types.

+ - Pipeline jobs now supports registry assets.

+- `az ml component`

+ - Allow Git repo URLs to be used as code.

+- `az ml online-endpoint`

+ - MIR now supports registry assets.

+## 2022-10-25

+### Azure Machine Learning SDK for Python v1.47.0

+ + **azureml-automl-dnn-nlp**

+ + Runtime changes for AutoML NLP to account for fixed training parameters, as part of the newly introduced model sweeping and hyperparameter tuning.

+ + **azureml-mlflow**

+ + AZUREML_ARTIFACTS_DEFAULT_TIMEOUT can be used to control the timeout for artifact upload

+ + **azureml-train-automl-runtime**

+ + Many Models and Hierarchical Time Series training now enforces check on timeout parameters to detect conflict before submitting the experiment for run. This will prevent experiment failure during the run by raising exception before submitting experiment.

+ + Customers can now control the step size while using rolling forecast in Many Models inference.

+ + ManyModels inference with unpartitioned tabular data now supports forecast_quantiles.

+* You must have an endpoint already created. If you don't, follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `heart-classifier-batch`.

+* You must have a compute created where to deploy the deployment. If you don't, follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.

+ > This example assumes you have an endpoint created with the name `heart-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).

+ Title: "Image processing with batch deployments"

+# Image processing with batch deployments

+* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `imagenet-classifier-batch`.

+* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.

+ > This example assumes you have an endpoint created with the name `imagenet-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).

+# Use MLflow models in batch deployments

+You can follow along this sample in the following notebooks. In the cloned repository, open the notebook: [mlflow-for-batch-tabular.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/batch/mlflow-for-batch-tabular.ipynb).

+ results = pd.DataFrame(columns=['file', 'predictions'])

+ results = pd.concat([results, df])

+ return results

+ Title: "Text processing with batch deployments"

+# Text processing with batch deployments

+* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `text-summarization-batch`.

+* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.

+ > This example assumes you have an endpoint created with the name `text-summarization-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).

+> When working on a private link-enabled workspaces, batch endpoints can be created and managed using Azure Machine Learning studio. However, they can't be invoked from the UI in studio. Please use the Azure ML CLI v2 instead for job creation. For more details about how to use it see [Invoke the batch endpoint to start a batch scoring job](how-to-use-batch-endpoint.md#invoke-the-batch-endpoint-to-start-a-batch-scoring-job).

+ Title: "Troubleshooting batch endpoints"

+description: Learn how to troubleshoot and diagnostic errors with batch endpoints jobs

+# Troubleshooting batch endpoints

+Learn how to troubleshoot and solve, or work around, common errors you may come across when using [batch endpoints](how-to-use-batch-endpoint.md) for batch scoring.

+## Understanding logs of a batch scoring job

+### Get logs

+After you invoke a batch endpoint using the Azure CLI or REST, the batch scoring job will run asynchronously. There are two options to get the logs for a batch scoring job.

+Option 1: Stream logs to local console

+You can run the following command to stream system-generated logs to your console. Only logs in the `azureml-logs` folder will be streamed.

+```azurecli

+az ml job stream -name <job_name>

+```

+Option 2: View logs in studio

+To get the link to the run in studio, run:

+```azurecli

+az ml job show --name <job_name> --query interaction_endpoints.Studio.endpoint -o tsv

+```

+1. Open the job in studio using the value returned by the above command.

+1. Choose __batchscoring__

+1. Open the __Outputs + logs__ tab

+1. Choose the log(s) you wish to review

+### Understand log structure

+There are two top-level log folders, `azureml-logs` and `logs`.

+The file `~/azureml-logs/70_driver_log.txt` contains information from the controller that launches the scoring script.

+Because of the distributed nature of batch scoring jobs, there are logs from several different sources. However, two combined files are created that provide high-level information:

+- `~/logs/job_progress_overview.txt`: This file provides high-level information about the number of mini-batches (also known as tasks) created so far and the number of mini-batches processed so far. As the mini-batches end, the log records the results of the job. If the job failed, it will show the error message and where to start the troubleshooting.

+- `~/logs/sys/master_role.txt`: This file provides the principal node (also known as the orchestrator) view of the running job. This log provides information on task creation, progress monitoring, the job result.

+For a concise understanding of errors in your script there is:

+- `~/logs/user/error.txt`: This file will try to summarize the errors in your script.

+For more information on errors in your script, there is:

+- `~/logs/user/error/`: This file contains full stack traces of exceptions thrown while loading and running the entry script.

+When you need a full understanding of how each node executed the score script, look at the individual process logs for each node. The process logs can be found in the `sys/node` folder, grouped by worker nodes:

+- `~/logs/sys/node/<ip_address>/<process_name>.txt`: This file provides detailed info about each mini-batch as it's picked up or completed by a worker. For each mini-batch, this file includes:

+ - The IP address and the PID of the worker process.

+ - The total number of items, the number of successfully processed items, and the number of failed items.

+ - The start time, duration, process time, and run method time.

+You can also view the results of periodic checks of the resource usage for each node. The log files and setup files are in this folder:

+- `~/logs/perf`: Set `--resource_monitor_interval` to change the checking interval in seconds. The default interval is `600`, which is approximately 10 minutes. To stop the monitoring, set the value to `0`. Each `<ip_address>` folder includes:

+ - `os/`: Information about all running processes in the node. One check runs an operating system command and saves the result to a file. On Linux, the command is `ps`.

+ - `%Y%m%d%H`: The sub folder name is the time to hour.

+ - `processes_%M`: The file ends with the minute of the checking time.

+ - `node_disk_usage.csv`: Detailed disk usage of the node.

+ - `node_resource_usage.csv`: Resource usage overview of the node.

+ - `processes_resource_usage.csv`: Resource usage overview of each process.

+### How to log in scoring script

+You can use Python logging in your scoring script. Logs are stored in `logs/user/stdout/<node_id>/processNNN.stdout.txt`.

+```python

+import argparse

+import logging

+# Get logging_level

+arg_parser = argparse.ArgumentParser(description="Argument parser.")

+arg_parser.add_argument("--logging_level", type=str, help="logging level")

+args, unknown_args = arg_parser.parse_known_args()

+print(args.logging_level)

+# Initialize Python logger

+logger = logging.getLogger(__name__)

+logger.setLevel(args.logging_level.upper())

+logger.info("Info log statement")

+logger.debug("Debug log statement")

+```

+## Common issues

+The following section contains common problems and solutions you may see during batch endpoint development and consumption.

+### No module named 'azureml'

+__Reason__: Azure Machine Learning Batch Deployments require the package `azureml-core` to be installed.

+__Solution__: Add `azureml-core` to your conda dependencies file.

+### Output already exists

+__Reason__: Azure Machine Learning Batch Deployment can't overwrite the `predictions.csv` file generated by the output.

+__Solution__: If you are indicated an output location for the predictions, ensure the path leads to a non-existing file.

+### The run() function in the entry script had timeout for [number] times

+__Message logged__: `No progress update in [number] seconds. No progress update in this check. Wait [number] seconds since last update.`

+__Reason__: Batch Deployments can be configured with a `timeout` value that indicates the amount of time the deployment shall wait for a single batch to be processed. If the execution of the batch takes more than such value, the task is aborted. Tasks that are aborted can be retried up to a maximum of times that can also be configured. If the `timeout` occurs on each retry, then the deployment job fails. These properties can be configured for each deployment.

+__Solution__: Increase the `timemout` value of the deployment by updating the deployment. These properties are configured in the parameter `retry_settings`. By default, a `timeout=30` and `retries=3` is configured. When deciding the value of the `timeout`, take into consideration the number of files being processed on each batch and the size of each of those files. You can also decrease them to account for more mini-batches of smaller size and hence quicker to execute.

+### Dataset initialization failed

+__Message logged__: Dataset initialization failed: UserErrorException: Message: Cannot mount Dataset(id='xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', name='None', version=None). Source of the dataset is either not accessible or does not contain any data.

+__Reason__: The compute cluster where the deployment is running can't mount the storage where the data asset is located. The managed identity of the compute don't have permissions to perform the mount.

+__Solutions__: Ensure the identity associated with the compute cluster where your deployment is running has at least has at least [Storage Blob Data Reader](../../role-based-access-control/built-in-roles.md#storage-blob-data-reader) access to the storage account. Only storage account owners can [change your access level via the Azure portal](../../storage/blobs/assign-azure-role-data-access.md).

+### Data set node [code] references parameter dataset_param which doesn't have a specified value or a default value

+__Message logged__: Data set node [code] references parameter dataset_param which doesn't have a specified value or a default value.

+__Reason__: The input data asset provided to the batch endpoint isn't supported.

+__Solution__: Ensure you are providing a data input that is supported for batch endpoints.

+### User program failed with Exception: Run failed, please check logs for details

+__Message logged__: User program failed with Exception: Run failed, please check logs for details. You can check logs/readme.txt for the layout of logs.

+__Reason__: There was an error while running the `init()` or `run()` function of the scoring script.

+__Solution__: Go to __Outputs + Logs__ and open the file at `logs > user > error > 10.0.0.X > process000.txt`. You will see the error message generated by the `init()` or `run()` method.

+### There is no succeeded mini batch item returned from run()

+__Message logged__: There is no succeeded mini batch item returned from run(). Please check 'response: run()' in https://aka.ms/batch-inference-documentation.

+__Reason__: The batch endpoint failed to provide data in the expected format to the `run()` method. This may be due to corrupted files being read or incompatibility of the input data with the signature of the model (MLflow).

+__Solution__: To understand what may be happening, go to __Outputs + Logs__ and open the file at `logs > user > stdout > 10.0.0.X > process000.stdout.txt`. Look for error entries like `Error processing input file`. You should find there details about why the input file can't be correctly read.

+ Title: 'Use batch endpoints for batch scoring'

+description: In this article, learn how to create a batch endpoint to continuously batch score large data.

+#Customer intent: As an ML engineer or data scientist, I want to create an endpoint to host my models for batch scoring, so that I can use the same endpoint continuously for different large datasets on-demand or on-schedule.

+# Use batch endpoints for batch scoring

+Batch endpoints provide a convenient way to run inference over large volumes of data. They simplify the process of hosting your models for batch scoring, so you can focus on machine learning, not infrastructure. For more information, see [What are Azure Machine Learning endpoints?](../concept-endpoints.md).

+Use batch endpoints when:

+> [!div class="checklist"]

+> * You have expensive models that requires a longer time to run inference.

+> * You need to perform inference over large amounts of data, distributed in multiple files.

+> * You don't have low latency requirements.

+> * You can take advantage of parallelization.

+In this article, you will learn how to use batch endpoints to do batch scoring.

+## Prerequisites

+### About this example

+On this example, we are going to deploy a model to solve the classic MNIST ("Modified National Institute of Standards and Technology") digit recognition problem to perform batch inferencing over large amounts of data (image files). In the first section of this tutorial, we are going to create a batch deployment with a model created using Torch. Such deployment will become our default one in the endpoint. On the second half, [we are going to see how we can create a second deployment](#adding-deployments-to-an-endpoint) using a model created with TensorFlow (Keras), test it out, and then switch the endpoint to start using the new deployment as default.

+### Clone the example repository

+### Create compute

+Batch endpoints run on compute clusters. They support both [Azure Machine Learning Compute clusters (AmlCompute)](../how-to-create-attach-compute-cluster.md) or [Kubernetes clusters](../how-to-attach-kubernetes-anywhere.md). Clusters are a shared resource so one cluster can host one or many batch deployments (along with other workloads if desired).

+Run the following code to create an Azure Machine Learning compute cluster. The following examples in this article use the compute created here named `batch-cluster`. Adjust as needed and reference your compute using `azureml:<your-compute-name>`.

+# [Azure ML CLI](#tab/cli)

+# [Azure ML SDK for Python](#tab/sdk)

+```python

+compute_name = "batch-cluster"

+compute_cluster = AmlCompute(name=compute_name, description="amlcompute", min_instances=0, max_instances=5)

+ml_client.begin_create_or_update(compute_cluster)

+```

+# [studio](#tab/studio)

+*Create a compute cluster as explained in the following tutorial [Create an Azure Machine Learning compute cluster](../how-to-create-attach-compute-cluster.md?tabs=azure-studio).*

+> [!NOTE]

+> You are not charged for compute at this point as the cluster will remain at 0 nodes until a batch endpoint is invoked and a batch scoring job is submitted. Learn more about [manage and optimize cost for AmlCompute](../how-to-manage-optimize-cost.md#use-azure-machine-learning-compute-cluster-amlcompute).

+### Registering the model

+Batch Deployments can only deploy models registered in the workspace. You can skip this step if the model you are trying to deploy is already registered. In this case, we are registering a Torch model for the popular digit recognition problem (MNIST).

+> [!TIP]

+> Models are associated with the deployment rather than with the endpoint. This means that a single endpoint can serve different models or different model versions under the same endpoint as long as they are deployed in different deployments.

+

+# [Azure ML CLI](#tab/cli)

+```azurecli

+MODEL_NAME='mnist'

+az ml model create --name $MODEL_NAME --type "custom_model" --path "./mnist/model/"

+```

+# [Azure ML SDK for Python](#tab/sdk)

+```python

+model_name = 'mnist'

+model = ml_client.models.create_or_update(

+ Model(name=model_name, path='./mnist/model/', type=AssetTypes.CUSTOM_MODEL)

+)

+```

+# [studio](#tab/studio)

+1. Navigate to the __Models__ tab on the side menu.

+1. Click on __Register__ > __From local files__.

+1. In the wizard, leave the option *Model type* as __Unspecified type__.

+1. Click on __Browse__ > __Browse folder__ > Select the folder `./mnist/model/` > __Next__.

+1. Configure the name of the model: `mnist`. You can leave the rest of the fields as they are.

+1. Click on __Register__.

+## Create a batch endpoint

+A batch endpoint is an HTTPS endpoint that clients can call to trigger a batch scoring job. A batch scoring job is a job that scores multiple inputs (for more, see [What are batch endpoints?](../concept-endpoints.md#what-are-batch-endpoints)). A batch deployment is a set of compute resources hosting the model that does the actual batch scoring. One batch endpoint can have multiple batch deployments.

+> [!TIP]

+> One of the batch deployments will serve as the default deployment for the endpoint. The default deployment will be used to do the actual batch scoring when the endpoint is invoked. Learn more about [batch endpoints and batch deployment](../concept-endpoints.md#what-are-batch-endpoints).

+### Steps

+1. Decide on the name of the endpoint. The name of the endpoint will end-up in the URI associated with your endpoint. Because of that, __batch endpoint names need to be unique within an Azure region__. For example, there can be only one batch endpoint with the name `mybatchendpoint` in `westus2`.

+ # [Azure ML CLI](#tab/cli)

+

+ In this case, let's place the name of the endpoint in a variable so we can easily reference it later.

+

+ ```azurecli

+ ENDPOINT_NAME="mnist-batch"

+ ```

+

+ # [Azure ML SDK for Python](#tab/sdk)

+

+ In this case, let's place the name of the endpoint in a variable so we can easily reference it later.

+ ```python

+ endpoint_name="mnist-batch"

+ ```

+

+ # [studio](#tab/studio)

+

+ *You will configure the name of the endpoint later in the creation wizard.*

+

+1. Configure your batch endpoint

+ # [Azure ML CLI](#tab/cli)

+ The following YAML file defines a batch endpoint, which you can include in the CLI command for [batch endpoint creation](#create-a-batch-endpoint). In the repository, this file is located at `/cli/endpoints/batch/batch-endpoint.yml`.

+ :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/mnist-endpoint.yml":::

+ The following table describes the key properties of the endpoint. For the full batch endpoint YAML schema, see [CLI (v2) batch endpoint YAML schema](../reference-yaml-endpoint-batch.md).

+

+ | Key | Description |

+ | | -- |

+ | `name` | The name of the batch endpoint. Needs to be unique at the Azure region level.|

+ | `description` | The description of the batch endpoint. This property is optional. |

+ | `auth_mode` | The authentication method for the batch endpoint. Currently only Azure Active Directory token-based authentication (`aad_token`) is supported. |

+ | `defaults.deployment_name` | The name of the deployment that will serve as the default deployment for the endpoint. |

+

+ # [Azure ML SDK for Python](#tab/sdk)

+

+ ```python

+ # create a batch endpoint

+ endpoint = BatchEndpoint(

+ name=endpoint_name,

+ description="A batch endpoint for scoring images from the MNIST dataset.",

+ )

+ ```

+

+ | Key | Description |

+ | | -- |

+ | `name` | The name of the batch endpoint. Needs to be unique at the Azure region level.|

+ | `description` | The description of the batch endpoint. This property is optional. |

+ | `auth_mode` | The authentication method for the batch endpoint. Currently only Azure Active Directory token-based authentication (`aad_token`) is supported. |

+ | `defaults.deployment_name` | The name of the deployment that will serve as the default deployment for the endpoint. |

+

+ # [studio](#tab/studio)