+## Update certificate user IDs using Microsoft Graph queries

+PATCH the user object certificateUserIds value for a given userId

+#### Request body:

+```http

+PATCH https://graph.microsoft.us/v1.0/users/{id}

+Content-Type: application/json

+{

+ "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users(authorizationInfo,department)/$entity",

+ "department": "Accounting",

+ "authorizationInfo": {

+ "certificateUserIds": [

+ "X509:<PN>123456789098765@mil"

+ ]

+ }

+}

+```

+> App passwords don't work with Conditional Access based multi-factor authentication policies and modern authentication. App passwords only work with legacy authentication protocols such as IMAP and SMTP.

+ Title: LDAP Authentication and Azure Multi-Factor Authentication Server - Azure Active Directory

+By default, the Azure Multi-Factor Authentication Server is configured to import or synchronize users from Active Directory. However, it can be configured to bind to different LDAP directories, such as an ADAM directory, or specific Active Directory domain controller. When connected to a directory via LDAP, the Azure Multi-Factor Authentication Server can act as an LDAP proxy to perform authentications. Azure Multi-Factor Authentication Server can also use LDAP bind as a RADIUS target to pre-authenticate IIS users, or for primary authentication in the Azure Multi-Factor Authentication user portal.

+> In September 2022, Microsoft announced deprecation of Azure Multi-Factor Authentication Server. Beginning September 30, 2024, Azure Multi-Factor Authentication Server deployments will no longer service multifactor authentication (MFA) requests, which could cause authentications to fail for your organization. To ensure uninterrupted authentication services and to remain in a supported state, organizations should [migrate their usersΓÇÖ authentication data](how-to-migrate-mfa-server-to-azure-mfa-user-authentication.md) to the cloud-based Azure Multi-Factor Authentication service by using the latest Migration Utility included in the most recent [Azure Multi-Factor Authentication Server update](https://www.microsoft.com/download/details.aspx?id=55849). For more information, see [Azure Multi-Factor Authentication Server Migration](how-to-migrate-mfa-server-to-azure-mfa.md).

+> To get started with cloud-based MFA, see [Tutorial: Secure user sign-in events with Azure Multi-Factor Authentication](tutorial-enable-azure-mfa.md).

+7. Check the **Require Azure Multi-Factor Authentication user match** box if all users have been or will be imported into the Server and subject to two-step verification. If a significant number of users haven't yet been imported into the Server and/or are exempt from two-step verification, leave the box unchecked. See the MFA Server help file for additional information on this feature.

+Repeat these steps to add more LDAP clients.

+When the **Use LDAP unique identifier attribute for matching usernames** radio button is selected, the Azure Multi-Factor Authentication Server attempts to resolve each username to a unique identifier in the LDAP directory. An LDAP search is performed on the Username attributes defined in the Directory Integration > Attributes tab. When a user authenticates, the username is resolved to the unique identifier in the LDAP directory. The unique identifier is used for matching the user in the Azure Multi-Factor Authentication data file. This allows for case-insensitive comparisons, and long and short username formats.

+* Configure your appliance, server, or application to authenticate via LDAP to the Azure Multi-Factor Authentication Server as though it were your LDAP directory. Use the same settings that you normally use to connect directly to your LDAP directory, but use the Azure Multi-Factor Authentication Server for the server name or IP address.

+* Configure the LDAP timeout to 30-60 seconds to provide enough time to validate the user's credentials with the LDAP directory, perform the second-step verification, receive their response, and respond to the LDAP access request.

+ Title: IIS Authentication and Azure Multi-Factor Authentication Server - Azure Active Directory

+Use the IIS Authentication section of the Azure Multi-Factor Authentication (MFA) Server to enable and configure IIS authentication for integration with Microsoft IIS web applications. The Azure Multi-Factor Authentication Server installs a plug-in that can filter requests being made to the IIS web server to add Azure Multi-Factor Authentication. The IIS plug-in provides support for Form-Based Authentication and Integrated Windows HTTP Authentication. Trusted IPs can also be configured to exempt internal IP addresses from two-factor authentication.

+> In September 2022, Microsoft announced deprecation of Azure Multi-Factor Authentication Server. Beginning September 30, 2024, Azure Multi-Factor Authentication Server deployments will no longer service multifactor authentication (MFA) requests, which could cause authentications to fail for your organization. To ensure uninterrupted authentication services and to remain in a supported state, organizations should [migrate their usersΓÇÖ authentication data](how-to-migrate-mfa-server-to-azure-mfa-user-authentication.md) to the cloud-based Azure Multi-Factor Authentication service by using the latest Migration Utility included in the most recent [Azure Multi-Factor Authentication Server update](https://www.microsoft.com/download/details.aspx?id=55849). For more information, see [Azure Multi-Factor Authentication Server Migration](how-to-migrate-mfa-server-to-azure-mfa.md).

+> To get started with cloud-based MFA, see [Tutorial: Secure user sign-in events with Azure Multi-Factor Authentication](tutorial-enable-azure-mfa.md).

+>>

+5. Check the **Require Multi-Factor Authentication user match** box if all users have been or will be imported into the Server and subject to multi-factor authentication. If a significant number of users haven't yet been imported into the Server and/or will be exempt from multi-factor authentication, leave the box unchecked.

+6. If the page variables can't be detected automatically, click **Specify Manually** in the Auto-Configure Form-Based Website dialog box.

+10. Check the **Require Azure Multi-Factor Authentication user match** box if all users have been or will be imported into the Server and subject to multi-factor authentication. If a significant number of users haven't yet been imported into the Server and/or will be exempt from multi-factor authentication, leave the box unchecked.

+To secure an IIS web application that uses Integrated Windows HTTP authentication, install the Azure Multi-Factor Authentication Server on the IIS web server, then configure the Server with the following steps:

+5. Adjust the Idle timeout and Maximum session times if the default isn't sufficient.

+6. Check the **Require Multi-Factor Authentication user match** box if all users have been or will be imported into the Server and subject to multi-factor authentication. If a significant number of users haven't yet been imported into the Server and/or will be exempt from multi-factor authentication, leave the box unchecked.

+1. If running on IIS 6, click the **ISAPI** tab. Select the website that the web application is running under (for example, Default Web Site) to enable the Azure Multi-Factor Authentication ISAPI filter plug-in for that site.

+The Trusted IPs allows users to bypass Azure Multi-Factor Authentication for website requests originating from specific IP addresses or subnets. For example, you may want to exempt users from Azure Multi-Factor Authentication while logging in from the office. In that case, you can specify the office subnet as a Trusted IPs entry. To configure Trusted IPs, use the following procedure:

+For Chrome support in **Windows 10 Creators Update (version 1703)** or later, install the [Windows Accounts](https://chrome.google.com/webstore/detail/windows-accounts/ppnbnpeolgkicgegkbkbjmhlideopiji) or [Office](https://chrome.google.com/webstore/detail/office/ndjpnladcallmjemlbaebfadecfhkepb) extensions. These extensions are required when a Conditional Access policy requires device-specific details.

+- Learn about permission and consent ( [v1.0](../azuread-dev/v1-permissions-consent.md), [v2.0](permissions-consent-overview.md)).

+Two of the most commonly referenced app registration settings are:

+# Consent experience for applications in Azure Active Directory

+In this article, you'll learn about the Azure Active Directory (Azure AD) application consent user experience. You'll then be able to intelligently manage applications for your organization and/or develop applications with a more seamless consent experience.

+| 5 | Publisher name and verification | The blue "verified" badge means that the app publisher has verified their identity using a Microsoft Partner Network account and has completed the verification process. If the app is publisher verified, the publisher name is displayed. If the app isn't publisher verified, "Unverified" is displayed instead of a publisher name. For more information, read about [Publisher Verification](publisher-verification-overview.md). Selecting the publisher name displays more app info as available, such as the publisher name, publisher domain, date created, certification details, and reply URLs. |

+| 8 | Permissions | This list contains the permissions being requested by the client application. Users should always evaluate the types of permissions being requested to understand what data the client application will be authorized to access on their behalf if they accept. As an application developer it's best to request access, to the permissions with the least privilege. |

+## Common scenarios and consent experiences

+The following section describes the common scenarios and the expected consent experience for each of them.

+### App requires a permission that the user has the right to grant

+In this consent scenario, the user accesses an app that requires a permission set that is within the user's scope of authority. The user is directed to the user consent flow.

+Admins will see an additional control on the traditional consent prompt that will allow to give consent on behalf of the entire tenant. The control will be defaulted to off, so only when admins explicitly check the box will consent be granted on behalf of the entire tenant. The check box will only show for the Global Admin role, so Cloud Admin and App Admin won't see this checkbox.

+### App requires a permission that the user has no right to grant

+In this consent scenario, the user accesses an app that requires at least one permission that is outside the user's scope of authority.

+Non-admin users will be blocked from granting consent to the application, and they'll be told to ask their admin for access to the app. If admin consent workflow is enabled in the user's tenant, non-admin users are able to submit a request for admin approval from the consent prompt. For more information on admin consent workflow, see [Admin consent workflow](../manage-apps/admin-consent-workflow-overview.md).

+### User is directed to the admin consent flow

+In this consent scenario, the user navigates to or is directed to the admin consent flow.

+Non-admin users will be blocked from granting consent to the application, and they'll be told to ask their admin for access to the app.

+### Admin consent through the Azure portal

+In this scenario, an administrator consents to all of the permissions that an application requests, which can include delegated permissions on behalf of all users in the tenant. The Administrator grants consent through the **API permissions** page of the application registration in the [Azure portal](https://portal.azure.com).

+ :::image type="content" source="./media/consent-framework/grant-consent.png" alt-text="Screenshot of explicit admin consent through the Azure portal." lightbox="./media/consent-framework/grant-consent.png":::

+All users in that tenant won't see the consent dialog unless the application requires new permissions. To learn which administrator roles can consent to delegated permissions, see [Administrator role permissions in Azure AD](../roles/permissions-reference.md).

+ > [!IMPORTANT]

+ > Granting explicit consent using the **Grant permissions** button is currently required for single-page applications (SPA) that use MSAL.js. Otherwise, the application fails when the access token is requested.

+## Common Issues

+This section outlines the common issues with the consent experience and possible troubleshooting tips.

+- 403 error

+ - Is this a [delegated scenario](permissions-consent-overview.md)? What permissions does a user have?

+ - Are necessary permissions added to use the endpoint?

+ - Check the [token](https://jwt.ms/) to see if it has necessary claims to call the endpoint.

+ - What permissions have been consented to? Who consented?

+- User is unable to consent

+ - Check if tenant admin has disabled user consent for your organization

+ - Confirm if the permissions you requesting are admin-restricted permissions.

+- User is still blocked even after admin has consented

+ - Check if [static permissions](consent-types-developer.md) are configured to be a superset of permissions requested dynamically.

+ - Check if user assignment is required for the app.

+## Troubleshoot known errors

+For troubleshooting steps, see [Unexpected error when performing consent to an application](../manage-apps/application-sign-in-unexpected-user-consent-error.md).

+ Title: Microsoft identity platform developers' guide to requesting permissions through consent

+description: Learn how developers can request for permissions through consent in the Microsoft identity platform endpoint.

+# Requesting permissions through consent

+Applications in the Microsoft identity platform rely on consent in order to gain access to necessary resources or APIs. Different types of consent are better for different application scenarios. Choosing the best approach to consent for your app will help it be more successful with users and organizations.

+In this article, you'll learn about the different types of consent and how to request permissions for your application through consent.

+## Static user consent

+In the static user consent scenario, you must specify all the permissions it needs in the app's configuration in the Azure portal. If the user (or administrator, as appropriate) hasn't granted consent for this app, then Microsoft identity platform will prompt the user to provide consent at this time.

+Static permissions also enable administrators to consent on behalf of all users in the organization.

+While relying on static consent and a single permissions list keeps the code nice and simple, it also means that your app will request all of the permissions it might ever need up front. This can discourage users and admins from approving your app's access request.

+## Incremental and dynamic user consent

+With the Microsoft identity platform endpoint, you can ignore the static permissions defined in the application registration information in the Azure portal. Instead, you can request permissions incrementally. You can ask for a bare minimum set of permissions upfront and request more over time as the customer uses additional application features. To do so, you can specify the scopes your application needs at any time by including the new scopes in the `scope` parameter when [requesting an access token](#requesting-individual-user-consent) - without the need to pre-define them in the application registration information. If the user hasn't yet consented to new scopes added to the request, they'll be prompted to consent only to the new permissions. Incremental, or dynamic consent, only applies to delegated permissions and not to application permissions.

+Allowing an application to request permissions dynamically through the `scope` parameter gives developers full control over your user's experience. You can also front load your consent experience and ask for all permissions in one initial authorization request. If your application requires a large number of permissions, you can gather those permissions from the user incrementally as they try to use certain features of the application over time.

+> [!IMPORTANT]

+> Dynamic consent can be convenient, but presents a big challenge for permissions that require admin consent. The admin consent experience in the **App registrations** and **Enterprise applications** blades in the portal doesn't know about those dynamic permissions at consent time. We recommend that a developer list all the admin privileged permissions that are needed by the application in the portal. This enables tenant admins to consent on behalf of all their users in the portal, once. Users won't need to go through the consent experience for those permissions on sign in. The alternative is to use dynamic consent for those permissions. To grant admin consent, an individual admin signs in to the app, triggers a consent prompt for the appropriate permissions, and selects **consent for my entire org** in the consent dialogue.

+## Requesting individual user consent

+In an [OpenID Connect or OAuth 2.0](active-directory-v2-protocols.md) authorization request, an application can request the permissions it needs by using the `scope` query parameter. For example, when a user signs in to an app, the application sends a request like the following example. (Line breaks are added for legibility).

+```HTTP

+GET https://login.microsoftonline.com/common/oauth2/v2.0/authorize?

+client_id=6731de76-14a6-49ae-97bc-6eba6914391e

+&response_type=code

+&redirect_uri=http%3A%2F%2Flocalhost%2Fmyapp%2F

+&response_mode=query

+&scope=

+https%3A%2F%2Fgraph.microsoft.com%2Fcalendars.read%20

+https%3A%2F%2Fgraph.microsoft.com%2Fmail.send

+&state=12345

+```

+The `scope` parameter is a space-separated list of delegated permissions that the application is requesting. Each permission is indicated by appending the permission value to the resource's identifier (the application ID URI). In the request example, the application needs permission to read the user's calendar and send mail as the user.

+After the user enters their credentials, the Microsoft identity platform checks for a matching record of *user consent*. If the user hasn't consented to any of the requested permissions in the past, and if the administrator hasn't consented to these permissions on behalf of the entire organization, the Microsoft identity platform asks the user to grant the requested permissions.

+In the following example, the `offline_access` ("Maintain access to data you have given it access to") permission and `User.Read` ("Sign you in and read your profile") permission are automatically included in the initial consent to an application. These permissions are required for proper application functionality. The `offline_access` permission gives the application access to refresh tokens that are critical for native apps and web apps. The `User.Read` permission gives access to the `sub` claim. It allows the client or application to correctly identify the user over time and access rudimentary user information.

+When the user approves the permission request, consent is recorded. The user doesn't have to consent again when they later sign in to the application.

+## Requesting consent for an entire tenant through admin consent

+Requesting consent for an entire tenant requires admin consent. Admin consent done on behalf of an organization requires the static permissions registered for the app. Set those permissions in the app registration portal if you need an admin to give consent on behalf of the entire organization.

+### Admin Consent for Delegated Permissions

+When your application requests [delegated permissions that require admin consent](scopes-oidc.md#admin-restricted-permissions), the user receives an error message that says they're unauthorized to consent to your app's permissions. The user is required to ask their admin for access to the app. If the admin grants consent for the entire tenant, the organization's users don't see a consent page for the application unless the previously granted permissions are revoked or the application requests for a new permission incrementally.

+Administrators using the same application will see the admin consent prompt. The admin consent prompt provides a checkbox that allows them to grant the application access to the requested data on behalf of the users for the entire tenant. For more information on the user and admin consent experience, see [Application consent experience](application-consent-experience.md).

+Examples of delegated permissions for Microsoft Graph that require admin consent are:

+- Read all user's full profiles by using User.Read.All

+- Write data to an organization's directory by using Directory.ReadWrite.All

+- Read all groups in an organization's directory by using Groups.Read.All

+To view the full list of Microsoft graph permissions, see [Microsoft graph permissions reference](/graph/permissions-reference).

+You can also configure permissions on your own resources to require admin consent. For more information on how to add scopes that require admin consent, see [Add a scope that requires admin consent](quickstart-configure-app-expose-web-apis.md#add-a-scope-requiring-admin-consent).

+Some organizations may change the default user consent policy for the tenant. When your application requests access to permissions they're evaluated against these policies. The user may need to request admin consent even when not required by default. To learn how administrators manage consent policies for applications, see [Manage app consent policies](../manage-apps/manage-app-consent-policies.md).

+>[!NOTE]

+>In requests to the authorization, token or consent endpoints for the Microsoft Identity platform, if the resource identifier is omitted in the scope parameter, the resource is assumed to be Microsoft Graph. For example, scope=User.Read is equivalent to `https://graph.microsoft.com/User.Read`.

+### Admin Consent for Application permissions

+Application permissions always require admin consent. Application permissions don't have a user context and the consent grant isn't done on behalf of any specific user. Instead, the client application is granted permissions directly, these types of permissions are used only by daemon services and other non-interactive applications that run in the background. Administrators need to configure the permissions upfront and [grant admin consent](../manage-apps/grant-admin-consent.md) through the Azure portal.

+### Admin consent for Multi-tenant applications

+In case the application requesting the permission is a multi-tenant application, its application registration only exists in the tenant where it was created, therefore permissions can't be configured in the local tenant. If the application requests permissions that require admin consent, the administrator needs to consent on behalf of the users. To consent to these permissions, the administrators need to log in to the application themselves, so the admin consent sign-in experience is triggered. To learn how to set up the admin consent experience for multi-tenant applications, see [Enable multi-tenant log-ins](howto-convert-app-to-be-multi-tenant.md#understand-user-and-admin-consent-and-make-appropriate-code-changes)

+An administrator can grant consent for an application with the following options.

+### Recommended: Sign the user into your app

+Typically, when you build an application that requires admin consent, the application needs a page or view in which the admin can approve the app's permissions. This page can be:

+- Part of the app's sign-up flow.

+- Part of the app's settings.

+- A dedicated "connect" flow.

+In many cases, it makes sense for the application to show the "connect" view only after a user has signed in with a work Microsoft account or school Microsoft account.

+When you sign the user into your app, you can identify the organization to which the admin belongs before you ask them to approve the necessary permissions. Although this step isn't strictly necessary, it can help you create a more intuitive experience for your organizational users.

+To sign the user in, follow the [Microsoft identity platform protocol tutorials](active-directory-v2-protocols.md).

+### Request the permissions in the app registration portal

+In the app registration portal, applications can list the permissions they require, including both delegated permissions and application permissions. This setup allows the use of the `.default` scope and the Azure portal's **Grant admin consent** option.

+In general, the permissions should be statically defined for a given application. They should be a superset of the permissions that the application will request dynamically or incrementally.

+> [!NOTE]

+>Application permissions can be requested only through the use of [`.default`](scopes-oidc.md#the-default-scope). So if your application needs application permissions, make sure they're listed in the app registration portal.

+To configure the list of statically requested permissions for an application:

+1. Go to your application in the <a href="https://go.microsoft.com/fwlink/?linkid=2083908" target="_blank">Azure portal - App registrations</a> quickstart experience.

+1. Select an application, or [create an app](quickstart-register-app.md) if you haven't already.

+1. On the application's **Overview** page, under **Manage**, select **API Permissions** > **Add a permission**.

+1. Select **Microsoft Graph** from the list of available APIs. Then add the permissions that your application requires.

+1. Select **Add Permissions**.

+### Successful response

+If the admin approves the permissions for your app, the successful response looks like this:

+```HTTP

+GET http://localhost/myapp/permissions?tenant=a8990e1f-ff32-408a-9f8e-78d3b9139b95&state=state=12345&admin_consent=True

+```

+| Parameter | Description |

+| | |

+| `tenant` | The directory tenant that granted your application the permissions it requested, in GUID format. |

+| `state` | A value included in the request that also will be returned in the token response. It can be a string of any content you want. The state is used to encode information about the user's state in the application before the authentication request occurred, such as the page or view they were on. |

+| `admin_consent` | Will be set to `True`. |

+After you've received a successful response from the admin consent endpoint, your application has gained the permissions it requested. Next, you can request a token for the resource you want.

+#### Error response

+If the admin doesn't approve the permissions for your app, the failed response looks like this:

+```HTTP

+GET http://localhost/myapp/permissions?error=permission_denied&error_description=The+admin+canceled+the+request

+```

+| Parameter | Description |

+| | |

+| `error` | An error code string that can be used to classify types of errors that occur. It can also be used to react to errors. |

+| `error_description` | A specific error message that can help a developer identify the root cause of an error. |

+## Using permissions after consent

+After the user consents to permissions for your app, your application can acquire access tokens that represent the app's permission to access a resource in some capacity. An access token can be used only for a single resource. But encoded inside the access token is every permission that your application has been granted for that resource. To acquire an access token, your application can make a request to the Microsoft identity platform token endpoint, like this:

+```HTTP

+POST common/oauth2/v2.0/token HTTP/1.1

+Host: https://login.microsoftonline.com

+Content-Type: application/json

+{

+ "grant_type": "authorization_code",

+ "client_id": "6731de76-14a6-49ae-97bc-6eba6914391e",

+ "scope": "https://microsoft.graph.com/Mail.Read https://microsoft.graph.com/mail.send",

+ "code": "AwABAAAAvPM1KaPlrEqdFSBzjqfTGBCmLdgfSTLEMPGYuNHSUYBrq...",

+ "redirect_uri": "https://localhost/myapp",

+ "client_secret": "zc53fwe80980293klaj9823" // NOTE: Only required for web apps

+}

+```

+You can use the resulting access token in HTTP requests to the resource. It reliably indicates to the resource that your application has the proper permission to do a specific task.

+For more information about the OAuth 2.0 protocol and how to get access tokens, see the [Microsoft identity platform endpoint protocol reference](active-directory-v2-protocols.md).

+## Next steps

+- [Consent experience](application-consent-experience.md)

+- [ID tokens](id-tokens.md)

+- [Access tokens](access-tokens.md)

+ Title: Microsoft identity platform delegated access scenario

+description: Learn about delegated access in the Microsoft identity platform endpoint.

+# Understanding delegated access

+When a user signs into an app and uses it to access some other resource, like Microsoft Graph, the app will first need to ask for permission to access this resource on the userΓÇÖs behalf. This common scenario is called delegated access.

+> [!VIDEO https://learn-video.azurefd.net/vod/player?show=one-dev-minute&ep=how-do-delegated-permissions-work]

+## Why should I use delegated access?

+People frequently use different applications to access their data from cloud services. For example, someone might want to use a favorite PDF reader application to view files stored in their OneDrive. Another example can be a companyΓÇÖs line-of-business application that might retrieve shared information about their coworkers so they can easily choose reviewers for a request. In such cases, the client application, the PDF reader, or the companyΓÇÖs request approval tool needs to be authorized to access this data on behalf of the user who signed into the application.

+Use delegated access whenever you want to let a signed-in user work with their own resources or resources they can access. Whether itΓÇÖs an admin setting up policies for their entire organization or a user deleting an email in their inbox, all scenarios involving user actions should use delegated access.

+In contrast, delegated access is usually a poor choice for scenarios that must run without a signed-in user, like automation. It may also be a poor choice for scenarios that involve accessing many usersΓÇÖ resources, like data loss prevention or backups. Consider using [application-only access](permissions-consent-overview.md) for these types of operations.

+## Requesting scopes as a client app

+Your app will need to ask the user to grant a specific scope, or set of scopes, for the resource app you want to access. Scopes may also be referred to as delegated permissions. These scopes describe which resources and operations your app wants to perform on the userΓÇÖs behalf. For example, if you want your app to show the user a list of recently received mail messages and chat messages, you might ask the user to consent to the Microsoft Graph `Mail.Read` and `Chat.Read` scopes.

+Once your app has requested a scope, a user or admin will need to grant the requested access. Consumer users with Microsoft Accounts, like Outlook.com or Xbox Live accounts, can always grant scopes for themselves. Organizational users with Azure AD accounts may or may not be able to grant scopes, depending on their organizationΓÇÖs settings. If an organizational user can't consent to scopes directly, they'll need to ask their organizationΓÇÖs administrator to consent for them.

+Always follow the principle of least privilege: you should never request scopes that your app doesnΓÇÖt need. This principle helps limit the security risk if your app is compromised and makes it easier for administrators to grant your app access. For example, if your app only needs to list the chats a user belongs to but doesnΓÇÖt need to show the chat messages themselves, you should request the more limited Microsoft Graph `Chat.ReadBasic` scope instead of `Chat.Read`. For more information about openID scopes, see [OpenID scopes](scopes-oidc.md).

+## Designing and publishing scopes for a resource service

+If youΓÇÖre building an API and want to allow delegated access on behalf of users, youΓÇÖll need to create scopes that other apps can request. These scopes should describe the actions or resources available to the client. You should consider developer scenarios when designing your scopes.

+## How does delegated access work?

+The most important thing to remember about delegated access is that both your client app and the signed-in user need to be properly authorized. Granting a scope isn't enough. If either the client app doesnΓÇÖt have the right scope, or the user doesnΓÇÖt have sufficient rights to read or modify the resource, then the call will fail.

+- **Client app authorization** - Client apps are authorized by granting scopes. When a client app is granted a scope by a user or admin to access some resource, that grant will be recorded in Azure AD. All delegated access tokens that are requested by the client to access the resource on behalf of the relevant user will then contain those scopesΓÇÖ claim values in the `scp` claim. The resource app checks this claim to determine whether the client app has been granted the correct scope for the call.

+- **User authorization** - Users are authorized by the resource youΓÇÖre calling. Resource apps may use one or more systems for user authorization, such as [role-based access control](custom-rbac-for-developers.md), ownership/membership relationships, access control lists, or other checks. For example, Azure AD checks that a user has been assigned to an app management or general admin role before allowing them to delete an organizationΓÇÖs applications, but also allows all users to delete applications that they own. Similarly, SharePoint Online service checks that a user has appropriate owner or reader rights over a file before allowing that user to open it.

+## Delegated access example ΓÇô OneDrive via Microsoft Graph

+Consider the following example:

+Alice wants to use a client app to open a file protected by a resource API, Microsoft Graph. For user authorization, the OneDrive service will check whether the file is stored in AliceΓÇÖs drive. If itΓÇÖs stored in another userΓÇÖs drive, then OneDrive will deny AliceΓÇÖs request as unauthorized, since Alice doesn't have the right to read other usersΓÇÖ drives.

+For client app authorization, OneDrive will check whether the client making the call has been granted the `Files.Read` scope on behalf of the signed-in user. In this case, the signed-in user is Alice. If `Files.Read` hasnΓÇÖt been granted to the app for Alice, then OneDrive will also fail the request.

+| GET /drives/{id}/files/{id} | Client app granted `Files.Read` scope for Alice | Client app not granted `Files.Read` scope for Alice |

+| -- | -- | -- |

+| The document is in AliceΓÇÖs OneDrive. | 200 ΓÇô Access granted. | 403 - Unauthorized. Alice (or her admin) hasnΓÇÖt allowed this client to read her files. |

+| The document is in another userΓÇÖs OneDrive*. | 403 - Unauthorized. Alice doesnΓÇÖt have rights to read this file. Even though the client has been granted `Files.Read` it should be denied when acting on AliceΓÇÖs behalf. | 403 ΓÇô Unauthorized. Alice doesnΓÇÖt have rights to read this file, and the client isnΓÇÖt allowed to read files she has access to either. |

+The example given is simplified to illustrate delegated authorization. The production OneDrive service supports many other access scenarios, such as shared files.

+## Next steps

+- [Open connect scopes](scopes-oidc.md)

+- [RBAC roles](custom-rbac-for-developers.md)

+- [Microsoft Graph permissions reference](/graph/permissions-reference)

+Another approach is to use Azure Active Directory (Azure AD) groups and group claims as shown in the [active-directory-aspnetcore-webapp-openidconnect-v2](https://aka.ms/groupssample) code sample on GitHub. Azure AD groups and application roles aren't mutually exclusive; they can be used together to provide even finer-grained access control.

+You define app roles by using the [Azure portal](https://portal.azure.com) during the [app registration process](quickstart-register-app.md). App roles are defined on an application registration representing a service, app or API. When a user signs in to the application, Azure AD emits a `roles` claim for each role that the user or service principal has been granted. This can be used to implement claim-based authorization. App roles can be assigned [to a user or a group of users](../manage-apps/add-application-portal-assign-users.md). App roles can also be assigned to the service principal for another application, or [to the service principal for a managed identity](../managed-identities-azure-resources/how-to-assign-app-role-managed-identity-powershell.md).

+App roles are declared using App roles UI in the Azure portal:

+If you're implementing app role business logic in an app-calling-API scenario, you have two app registrations. One app registration is for the app, and a second app registration is for the API. In this case, define the app roles and assign them to the user or group in the app registration of the API. When the user authenticates with the app and requests an ID token to call the API, a roles claim is included in the ID token. Your next step is to add code to your web API to check for those roles when the API is called.

+- `IIdentityLogger` is the logging implementation used by MSAL.NET to produce logs for debugging or health check purposes. Logs are only sent if logging is enabled.

+- `PiiLoggingEnabled` enables you to log personal and organizational data (PII) if set to true. By default, this parameter is set to false, so that your application doesn't log personal data.

+### IIdentityLogger Interface

+```CSharp

+namespace Microsoft.IdentityModel.Abstractions

+ public interface IIdentityLogger

+ {

+ //

+ // Summary:

+ // Checks to see if logging is enabled at given eventLogLevel.

+ //

+ // Parameters:

+ // eventLogLevel:

+ // Log level of a message.

+ bool IsEnabled(EventLogLevel eventLogLevel);

+ //

+ // Summary:

+ // Writes a log entry.

+ //

+ // Parameters:

+ // entry:

+ // Defines a structured message to be logged at the provided Microsoft.IdentityModel.Abstractions.LogEntry.EventLogLevel.

+ void Log(LogEntry entry);

+ }

+> [!NOTE]

+> Partner libraries (`Microsoft.Identity.Web`, `Microsoft.IdentityModel`) provide implementations of this interface already for various environments (in particular ASP.NET Core)

+### IIdentityLogger Implementation

+The following code snippets are examples of such an implementation. If you use the .NET core configuration, environment variable driven logs levels can be provided for free, in addition to the configuration file based log levels.

+#### Log level from configuration file

+It's highly recommended to configure your code to use a configuration file in your environment to set the log level as it will enable your code to change the MSAL logging level without needing to rebuild or restart the application. This is critical for diagnostic purposes, enabling us to quickly gather the required logs from the application that is currently deployed and in production. Verbose logging can be costly so it's best to use the *Information* level by default and enable verbose logging when an issue is encountered. [See JSON configuration provider](https://docs.microsoft.com/aspnet/core/fundamentals/configuration#json-configuration-provider) for an example on how to load data from a configuration file without restarting the application.

+#### Log Level as Environment Variable

+Another option we recommended is to configure your code to use an environment variable on the machine to set the log level as it will enable your code to change the MSAL logging level without needing to rebuild the application. This is critical for diagnostic purposes, enabling us to quickly gather the required logs from the application that is currently deployed and in production.

+See [EventLogLevel](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/blob/dev/src/Microsoft.IdentityModel.Abstractions/EventLogLevel.cs) for details on the available log levels.

+Example:

+```CSharp

+ class MyIdentityLogger : IIdentityLogger

+ {

+ public EventLogLevel MinLogLevel { get; }

+ public TestIdentityLogger()

+ {

+ //Try to pull the log level from an environment variable

+ var msalEnvLogLevel = Environment.GetEnvironmentVariable("MSAL_LOG_LEVEL");

+ if (Enum.TryParse(msalEnvLogLevel, out EventLogLevel msalLogLevel))

+ {

+ MinLogLevel = msalLogLevel;

+ }

+ else

+ {

+ //Recommended default log level

+ MinLogLevel = EventLogLevel.Informational;

+ }

+ }

+ public bool IsEnabled(EventLogLevel eventLogLevel)

+ {

+ return eventLogLevel <= MinLogLevel;

+ }

+ public void Log(LogEntry entry)

+ {

+ //Log Message here:

+ Console.WriteLine(entry.message);

+ }

+ }

+```

+Using `MyIdentityLogger`:

+```CSharp

+ MyIdentityLogger myLogger = new MyIdentityLogger(logLevel);

+ var app = ConfidentialClientApplicationBuilder

+ .Create(TestConstants.ClientId)

+ .WithClientSecret("secret")

+ .WithExperimentalFeatures() //Currently an experimental feature, will be removed soon

+ .WithLogging(myLogger, piiLogging)

+ .Build();

+```

+description: Learn the foundational concepts and scenarios around consent and permissions in the Microsoft identity platform

+As an application developer, you must identify how your application will access data. The application can use delegated access, acting on behalf of a signed-in user, or app-only access, acting only as the application's own identity.

+In this access scenario, a user has signed into a client application. The client application accesses the resource on behalf of the user. Delegated access requires delegated permissions. Both the client and the user must be authorized separately to make the request. For more information about the delegated access scenario, see [delegated access scenario](delegated-access-primer.md).

+For the client app, the correct delegated permissions must be granted. Delegated permissions can also be referred to as scopes. Scopes are permissions for a given resource that represent what a client application can access on behalf of the user.For more information about scopes, see [scopes and permissions](v2-permissions-and-consent.md#scopes-and-permissions).

+For the user, the authorization relies on the privileges that the user has been granted for them to access the resource. For example, the user could be authorized to access directory resources by [Azure Active Directory (Azure AD) role-based access control (RBAC)](../roles/custom-overview.md) or to access mail and calendar resources by Exchange Online RBAC. For more information on RBAC for applications, see [RBAC for applications](custom-rbac-for-developers.md).

+### App-only access (Access without a user)

+App-only access uses app roles instead of delegated scopes. When granted through consent, app roles may also be called applications permissions. For app-only access, the client app must be granted appropriate app roles of the resource app it's calling in order to access the requested data. For more information about assigning app roles to client applications, see [Assigning app roles to applications](howto-add-app-roles-in-azure-ad-apps.md#assign-app-roles-to-applications).

+**Delegated permissions** are used in the delegated access scenario. They're permissions that allow the application to act on a user's behalf. The application will never be able to access anything the signed in user themselves couldn't access.

+**Application permissions**, sometimes called app roles are used in the app-only access scenario, without a signed-in user present. The application will be able to access any data that the permission is associated with. For example, an application granted the Files.Read.All application permission will be able to read any file in the tenant. Only an administrator or owner of the service principal can consent to application permissions.

+There are other ways in which applications can be granted authorization for app-only access. For example, an application can be assigned an Azure AD RBAC role.

+### Comparison of delegated and application permissions

+| <!-- No header--> | Delegated permissions | Application permissions |

+|--|--|--|

+| Types of apps | Web / Mobile / single-page app (SPA) | Web / Daemon |

+| Access context | Get access on behalf of a user | Get access without a user |

+| Who can consent | - Users can consent for their data <br> - Admins can consent for all users | Only admin can consent |

+| Other names | - Scopes <br> - OAuth2 permission scopes | - App roles <br> - App-only permissions |

+| Result of consent (specific to Microsoft Graph) | [oAuth2PermissionGrant](/graph/api/resources/oauth2permissiongrant) | [appRoleAssignment](/graph/api/resources/approleassignment) |

+One way that applications are granted permissions is through consent. Consent is a process where users or admins authorize an application to access a protected resource. For example, when a user attempts to sign into an application for the first time, the application can request permission to see the user's profile and read the contents of the user's mailbox. The user sees the list of permissions the app is requesting through a consent prompt. Other scenarios where users may see a consent prompt include:

+- When previously granted consent is revoked.

+- When the application is coded to specifically prompt for consent during every sign-in.

+- When the application uses incremental or dynamic consent to ask for some permissions upfront and more permission later as needed.

+User consent happens when a user attempts to sign into an application. The user provides their sign-in credentials. These credentials are checked to determine whether consent has already been granted. If no previous record of user or admin consent for the required permissions exists, the user is shown a consent prompt, and asked to grant the application the requested permissions. In many cases, an admin may be required to grant consent on behalf of the user.

+Depending on the permissions they require, some applications might require an administrator to be the one who grants consent. For example, application permissions and many high-privilege delegated permissions can only be consented to by an administrator. Administrators can grant consent for themselves or for the entire organization. For more information about user and admin consent, see [user and admin consent overview](../manage-apps/consent-and-permissions-overview.md).

+- [Delegated access scenario](delegated-access-primer.md)

+- [OpenID connect scopes](scopes-oidc.md)

+The pattern for acquiring tokens for APIs with [MSAL.js](https://github.com/AzureAD/microsoft-authentication-library-for-js) is to first attempt a silent token request by using the `acquireTokenSilent` method. When this method is called, the library first checks the cache in browser storage to see if a non-expired access token exists and returns it. If no access token is found for the given parameters, it will throw an `InteractionRequiredAuthError`, which should be handled with an interactive token request method (`acquireTokenPopup` or `acquireTokenRedirect`). If an access token is found but it's expired, it attempts to use its refresh token to get a fresh access token. If the refresh token's 24-hour lifetime has also expired, MSAL.js will open a hidden iframe to silently request a new authorization code by leveraging the existing active session with Azure AD (if any), which will then be exchanged for a fresh set of tokens (access _and_ refresh tokens). For more information about single sign-on (SSO) session and token lifetime values in Azure AD, see [Token lifetimes](active-directory-configurable-token-lifetimes.md). For more information on MSAL.js cache lookup policy, see: [Acquiring an Access Token](https://github.com/AzureAD/microsoft-authentication-library-for-js/blob/dev/lib/msal-browser/docs/acquire-token.md#acquiring-an-access-token).

+ Title: Microsoft identity platform scopes and permissions

+description: Learn about openID connect scopes and permissions in the Microsoft identity platform endpoint.

+# Scopes and permissions in the Microsoft identity platform

+The Microsoft identity platform implements the [OAuth 2.0](active-directory-v2-protocols.md) authorization protocol. OAuth 2.0 is a method through which a third-party app can access web-hosted resources on behalf of a user. Any web-hosted resource that integrates with the Microsoft identity platform has a resource identifier, or *application ID URI*.

+In this article, you'll learn about scopes and permissions in the identity platform.

+The following list shows are some examples of Microsoft web-hosted resources:

+- Microsoft Graph: `https://graph.microsoft.com`

+- Microsoft 365 Mail API: `https://outlook.office.com`

+- Azure Key Vault: `https://vault.azure.net`

+The same is true for any third-party resources that have integrated with the Microsoft identity platform. Any of these resources also can define a set of permissions that can be used to divide the functionality of that resource into smaller chunks. As an example, [Microsoft Graph](https://graph.microsoft.com) has defined permissions to do the following tasks, among others:

+- Read a user's calendar

+- Write to a user's calendar

+- Send mail as a user

+Because of these types of permission definitions, the resource has fine-grained control over its data and how API functionality is exposed. A third-party app can request these permissions from users and administrators, who must approve the request before the app can access data or act on a user's behalf.

+When a resource's functionality is chunked into small permission sets, third-party apps can be built to request only the permissions that they need to perform their function. Users and administrators can know what data the app can access. And they can be more confident that the app isn't behaving with malicious intent. Developers should always abide by the principle of least privilege, asking for only the permissions they need for their applications to function.

+In OAuth 2.0, these types of permission sets are called *scopes*. They're also often referred to as *permissions*. In the Microsoft identity platform, a permission is represented as a string value. An app requests the permissions it needs by specifying the permission in the `scope` query parameter. Identity platform supports several well-defined [OpenID Connect scopes](#openid-connect-scopes) and resource-based permissions (each permission is indicated by appending the permission value to the resource's identifier or application ID URI). For example, the permission string `https://graph.microsoft.com/Calendars.Read` is used to request permission to read users calendars in Microsoft Graph.

+In requests to the authorization server, for the Microsoft Identity platform, if the resource identifier is omitted in the scope parameter, the resource is assumed to be Microsoft Graph. For example, `scope=User.Read` is equivalent to `https://graph.microsoft.com/User.Read`.

+## Admin-restricted permissions

+Permissions in the Microsoft identity platform can be set to admin restricted. For example, many higher-privilege Microsoft Graph permissions require admin approval. If your app requires admin-restricted permissions, an organization's administrator must consent to those scopes on behalf of the organization's users. The following section gives examples of these kinds of permissions:

+- Read all user's full profiles by using `User.Read.All`

+- Write data to an organization's directory by using `Directory.ReadWrite.All`

+- Read all groups in an organization's directory by using `Groups.Read.All`

+> [!NOTE]

+>In requests to the authorization, token or consent endpoints for the Microsoft Identity platform, if the resource identifier is omitted in the scope parameter, the resource is assumed to be Microsoft Graph. For example, `scope=User.Read` is equivalent to `https://graph.microsoft.com/User.Read`.

+Although a consumer user might grant an application access to this kind of data, organizational users can't grant access to the same set of sensitive company data. If your application requests access to one of these permissions from an organizational user, the user receives an error message that says they're not authorized to consent to your app's permissions.

+If the application requests application permissions and an administrator grants these permissions this grant isn't done on behalf of any specific user. Instead, the client application is granted permissions *directly*. These types of permissions should only be used by daemon services and other non-interactive applications that run in the background. For more information on the direct access scenario, see [Access scenarios in the Microsoft identity platform](permissions-consent-overview.md).

+For a step by step guide on how to expose scopes in a web API, see [Configure an application to expose a web API](quickstart-configure-app-expose-web-apis.md).

+## OpenID Connect scopes

+The Microsoft identity platform implementation of OpenID Connect has a few well-defined scopes that are also hosted on Microsoft Graph: `openid`, `email`, `profile`, and `offline_access`. The `address` and `phone` OpenID Connect scopes aren't supported.

+If you request the OpenID Connect scopes and a token, you'll get a token to call the [UserInfo endpoint](userinfo.md).

+### openid

+If an app signs in by using [OpenID Connect](active-directory-v2-protocols.md), it must request the `openid` scope. The `openid` scope appears on the work account consent page as the **Sign you in** permission.

+By using this permission, an app can receive a unique identifier for the user in the form of the `sub` claim. The permission also gives the app access to the UserInfo endpoint. The `openid` scope can be used at the Microsoft identity platform token endpoint to acquire ID tokens. The app can use these tokens for authentication.

+### email

+The `email` scope can be used with the `openid` scope and any other scopes. It gives the app access to the user's primary email address in the form of the `email` claim.

+The `email` claim is included in a token only if an email address is associated with the user account, which isn't always the case. If your app uses the `email` scope, the app needs to be able to handle a case in which no `email` claim exists in the token.

+### profile

+The `profile` scope can be used with the `openid` scope and any other scope. It gives the app access to a large amount of information about the user. The information it can access includes, but isn't limited to, the user's given name, surname, preferred username, and object ID.

+For a complete list of the `profile` claims available in the `id_tokens` parameter for a specific user, see the [`id_tokens` reference](id-tokens.md).

+### offline_access

+The [`offline_access` scope](https://openid.net/specs/openid-connect-core-1_0.html#OfflineAccess) gives your app access to resources on behalf of the user for an extended time. On the consent page, this scope appears as the **Maintain access to data you have given it access to** permission.

+When a user approves the `offline_access` scope, your app can receive refresh tokens from the Microsoft identity platform token endpoint. Refresh tokens are long-lived. Your app can get new access tokens as older ones expire.

+> [!NOTE]

+> This permission currently appears on all consent pages, even for flows that don't provide a refresh token (such as the [implicit flow](v2-oauth2-implicit-grant-flow.md)). This setup addresses scenarios where a client can begin within the implicit flow and then move to the code flow where a refresh token is expected.

+On the Microsoft identity platform (requests made to the v2.0 endpoint), your app must explicitly request the `offline_access` scope, to receive refresh tokens. So when you redeem an authorization code in the [OAuth 2.0 authorization code flow](active-directory-v2-protocols.md), you'll receive only an access token from the `/token` endpoint.

+The access token is valid for a short time. It usually expires in one hour. At that point, your app needs to redirect the user back to the `/authorize` endpoint to get a new authorization code. During this redirect, depending on the type of app, the user might need to enter their credentials again or consent again to permissions.

+For more information about how to get and use refresh tokens, see the [Microsoft identity platform protocol reference](active-directory-v2-protocols.md).

+## The .default scope

+The `.default` scope is used to refer generically to a resource service (API) in a request, without identifying specific permissions. If consent is necessary, using `.default` signals that consent should be prompted for all required permissions listed in the application registration (for all APIs in the list).

+The scope parameter value is constructed by using the identifier URI for the resource and `.default`, separated by a forward slash (`/`). For example, if the resource's identifier URI is `https://contoso.com`, the scope to request is `https://contoso.com/.default`. For cases where you must include a second slash to correctly request the token, see the [section about trailing slashes](#trailing-slash-and-default).

+Using `scope={resource-identifier}/.default` is functionally the same as `resource={resource-identifier}` on the v1.0 endpoint (where `{resource-identifier}` is the identifier URI for the API, for example `https://graph.microsoft.com` for Microsoft Graph).

+The `.default` scope can be used in any OAuth 2.0 flow and to initiate [admin consent](v2-admin-consent.md). Its use is required in the [On-Behalf-Of flow](v2-oauth2-on-behalf-of-flow.md) and [client credentials flow](v2-oauth2-client-creds-grant-flow.md).

+Clients can't combine static (`.default`) consent and dynamic consent in a single request. So `scope=https://graph.microsoft.com/.default Mail.Read` results in an error because it combines scope types.

+### .default when the user has already given consent

+The `.default` scope parameter only triggers a consent prompt if consent hasn't been granted for any delegated permission between the client and the resource, on behalf of the signed-in user.

+If consent exists, the returned token contains all scopes granted for that resource for the signed-in user. However, if no permission has been granted for the requested resource (or if the `prompt=consent` parameter has been provided), a consent prompt is shown for all required permissions configured on the client application registration, for all APIs in the list.

+For example, if the scope `https://graph.microsoft.com/.default` is requested, your application is requesting an access token for the Microsoft Graph API. If at least one delegated permission has been granted for Microsoft Graph on behalf of the signed-in user, the sign-in will continue and all Microsoft Graph delegated permissions that have been granted for that user will be included in the access token. If no permissions have been granted for the requested resource (Microsoft Graph, in this example), then a consent prompt will be presented for all required permissions configured on the application, for all APIs in the list.

+#### Example 1: The user, or tenant admin, has granted permissions

+In this example, the user or a tenant administrator has granted the `Mail.Read` and `User.Read` Microsoft Graph permissions to the client.

+If the client requests `scope=https://graph.microsoft.com/.default`, no consent prompt is shown, regardless of the contents of the client application's registered permissions for Microsoft Graph. The returned token contains the scopes `Mail.Read` and `User.Read`.

+#### Example 2: The user hasn't granted permissions between the client and the resource

+In this example, the user hasn't granted consent between the client and Microsoft Graph, nor has an administrator. The client has registered for the permissions `User.Read` and `Contacts.Read`. It has also registered for the Azure Key Vault scope `https://vault.azure.net/user_impersonation`.

+When the client requests a token for `scope=https://graph.microsoft.com/.default`, the user sees a consent page for the Microsoft Graph `User.Read` and `Contacts.Read` scopes, and for the Azure Key Vault `user_impersonation` scope. The returned token contains only the `User.Read` and `Contacts.Read` scopes, and it can be used only against Microsoft Graph.

+#### Example 3: The user has consented, and the client requests more scopes

+In this example, the user has already consented to `Mail.Read` for the client. The client has registered for the `Contacts.Read` scope.

+The client first performs a sign-in with `scope=https://graph.microsoft.com/.default`. Based on the `scopes` parameter of the response, the application's code detects that only `Mail.Read` has been granted. The client then initiates a second sign-in using `scope=https://graph.microsoft.com/.default`, and this time forces consent using `prompt=consent`. If the user is allowed to consent for all the permissions that the application registered, they'll be shown the consent prompt. (If not, they'll be shown an error message or the [admin consent request](../manage-apps/configure-admin-consent-workflow.md) form.) Both `Contacts.Read` and `Mail.Read` will be in the consent prompt. If consent is granted and the sign-in continues, the token returned is for Microsoft Graph, and contains `Mail.Read` and `Contacts.Read`.

+### Using the .default scope with the client

+In some cases, a client can request its own `.default` scope. The following example demonstrates this scenario.

+The scenario accommodates some legacy clients that are moving from Azure AD Authentication Library (ADAL) to the Microsoft Authentication Library (MSAL). This setup *shouldn't* be used by new clients that target the Microsoft identity platform.

+```http

+// Line breaks are for legibility only.

+GET https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize

+ ?response_type=token //Code or a hybrid flow is also possible here

+ &client_id=9ada6f8a-6d83-41bc-b169-a306c21527a5

+ &scope=9ada6f8a-6d83-41bc-b169-a306c21527a5/.default

+ &redirect_uri=https%3A%2F%2Flocalhost

+ &state=1234

+```

+This code example produces a consent page for all registered permissions if the preceding descriptions of consent and `.default` apply to the scenario. Then the code returns an `id_token`, rather than an access token.

+### Client credentials grant flow and .default

+Another use of `.default` is to request app roles (also known as application permissions) in a non-interactive application like a daemon app that uses the [client credentials](v2-oauth2-client-creds-grant-flow.md) grant flow to call a web API.

+To define app roles (application permissions) for a web API, see [Add app roles in your application](howto-add-app-roles-in-azure-ad-apps.md).

+Client credentials requests in your client service *must* include `scope={resource}/.default`. Here, `{resource}` is the web API that your app intends to call, and wishes to obtain an access token for. Issuing a client credentials request by using individual application permissions (roles) is *not* supported. All the app roles (application permissions) that have been granted for that web API are included in the returned access token.

+To grant access to the app roles you define, including granting admin consent for the application, see [Configure a client application to access a web API](quickstart-configure-app-access-web-apis.md).

+### Trailing slash and .default

+Some resource URIs have a trailing forward slash, for example, `https://contoso.com/` as opposed to `https://contoso.com`. The trailing slash can cause problems with token validation. Problems occur primarily when a token is requested for Azure Resource Manager (`https://management.azure.com/`). In this case, a trailing slash on the resource URI means the slash must be present when the token is requested. So when you request a token for `https://management.azure.com/` and use `.default`, you must request `https://management.azure.com//.default` (notice the double slash!). In general, if you verify that the token is being issued, and if the token is being rejected by the API that should accept it, consider adding a second forward slash and trying again.

+## Next steps

+- [Requesting permissions through consent in the identity platform](consent-types-developer.md)

+- [ID tokens in the Microsoft identity platform](id-tokens.md)

+- [Access tokens in the Microsoft identity platform](access-tokens.md)

+Most applications require access to protected data, and the owner of that data needs to [consent](consent-types-developer.md) to that access. Consent can be granted in several ways, including by a tenant administrator who can consent for *all* users in an Azure AD tenant, or by the application users themselves who can grant access.

+The information in an ID token is a superset of the information available on UserInfo endpoint. Because you can get an ID token at the same time you get a token to call the UserInfo endpoint, we suggest getting the user's information from the token instead of calling the UserInfo endpoint. Using the ID token instead of calling the UserInfo endpoint eliminates up to two network requests, reducing latency in your application.

+Static permissions also enable administrators to [consent on behalf of all users](#requesting-consent-for-an-entire-tenant) in the organization.

+The `.default` scope can be used in any OAuth 2.0 flow and to initiate [admin consent](v2-admin-consent.md). Its use is required in the [On-Behalf-Of flow](v2-oauth2-on-behalf-of-flow.md) and [client credentials flow](v2-oauth2-client-creds-grant-flow.md).

+If consent does exist, the returned token contains all scopes granted for that resource for the signed-in user. However, if no permission has been granted for the requested resource (or if the `prompt=consent` parameter has been provided), a consent prompt is shown for all required permissions configured on the client application registration, for all APIs in the list.

+## October 2022

+### Updated articles

+- [Access Azure AD protected resources from an app in Google Cloud](workload-identity-federation-create-trust-gcp.md)

+- [Configure an app to trust an external identity provider](workload-identity-federation-create-trust.md)

+- [Configure a user-assigned managed identity to trust an external identity provider (preview)](workload-identity-federation-create-trust-user-assigned-managed-identity.md)

+- [Configuration requirements and troubleshooting tips for Xamarin Android with MSAL.NET](msal-net-xamarin-android-considerations.md)

+- [Customize claims emitted in tokens for a specific app in a tenant](active-directory-claims-mapping.md)

+- [Desktop app that calls web APIs: Acquire a token using Device Code flow](scenario-desktop-acquire-token-device-code-flow.md)

+- [Desktop app that calls web APIs: Acquire a token using integrated Windows authentication](scenario-desktop-acquire-token-integrated-windows-authentication.md)

+- [Desktop app that calls web APIs: Acquire a token using Username and Password](scenario-desktop-acquire-token-username-password.md)

+- [Making your application multi-tenant](howto-convert-app-to-be-multi-tenant.md)

+- [Microsoft identity platform and OAuth 2.0 On-Behalf-Of flow](v2-oauth2-on-behalf-of-flow.md)

+- [Prompt behavior with MSAL.js](msal-js-prompt-behavior.md)

+- [Quickstart: Register an application with the Microsoft identity platform](quickstart-register-app.md)

+- [Tutorial: Sign in users and call the Microsoft Graph API from a JavaScript single-page application](tutorial-v2-javascript-spa.md)

+- [Tutorial: Sign in users and call the Microsoft Graph API from a React single-page app (SPA) using auth code flow](tutorial-v2-react.md)

+> If your organization uses proxy servers that intercept SSL traffic for scenarios like data loss prevention or Azure AD tenant restrictions, ensure that traffic to `https://device.login.microsoftonline.com` is excluded from TLS break-and-inspect. Failure to exclude this URL may cause interference with client certificate authentication, cause issues with device registration, and device-based Conditional Access.

+## October 2022

+### Updated articles

+- [Tutorial: Bulk invite Azure AD B2B collaboration users](tutorial-bulk-invite.md)

+- [Quickstart: Add a guest user and send an invitation](b2b-quickstart-add-guest-users-portal.md)

+- [Define custom attributes for user flows](user-flow-add-custom-attributes.md)

+- [Create dynamic groups in Azure Active Directory B2B collaboration](use-dynamic-groups.md)

+- [Properties of an Azure Active Directory B2B collaboration user](user-properties.md)

+- [Authentication and Conditional Access for External Identities](authentication-conditional-access.md)

+- [Leave an organization as an external user](leave-the-organization.md)

+- [Azure Active Directory External Identities: What's new](whats-new-docs.md)

+- [Federation with SAML/WS-Fed identity providers for guest users](direct-federation.md)

+- [Example: Configure SAML/WS-Fed based identity provider federation with AD FS](direct-federation-adfs.md)

+- [The elements of the B2B collaboration invitation email - Azure Active Directory](invitation-email-elements.md)

+- [Configure Microsoft cloud settings for B2B collaboration (Preview)](cross-cloud-settings.md)

+- [Add Microsoft account (MSA) as an identity provider for External Identities](microsoft-account.md)

+- [How users in your organization can invite guest users to an app](add-users-information-worker.md)

+- Disable Soft Matching on your tenant. Soft Matching is a great feature to help transfering source of autority for existing cloud managed objects to Azure AD Connect, but it comes with certain security risks. If you do not require it, you should [disable Soft Matching](how-to-connect-syncservice-features.md#blocksoftmatch).

+- Disable Hard Match Takeover. Hard match takeover allows Azure AD Connect to take control of a cloud managed object and changing the source of authority for the object to Active Directory. Once the source of authority of an object is taken over by Azure AD Connect, changes made to the Active Directory object that is linked to the Azure AD object will overwrite the original Azure AD data - including the password hash, if Password Hash Sync is enabled. An attacker could use this capability to take over control of cloud managed objects. To mitigate this risk, [disable hard match takeover](https://learn.microsoft.com/powershell/module/msonline/set-msoldirsyncfeature?view=azureadps-1.0#example-3-block-cloud-object-takeover-through-hard-matching-for-the-tenant).

+ Title: Overview of user and admin consent

+description: Learn about the fundamental concepts of user and admin consent in Azure AD

+# User and admin consent in Azure Active Directory

+In this article, youΓÇÖll learn the foundational concepts and scenarios around user and admin consent in Azure Active Directory (Azure AD).

+Consent is a process where users can grant permission for an application to access a protected resource. To indicate the level of access required, an application requests the API permissions it requires. For example, an application can request the permission to see a signed-in user's profile and read the contents of the user's mailbox.

+Consent can be initiated in various ways. For example, users can be prompted for consent when they attempt to sign in to an application for the first time. Depending on the permissions they require, some applications might require an administrator to be the one who grants consent.

+## User consent

+A user can authorize an application to access some data at the protected resource, while acting as that user. The permissions that allow this type of access are called "delegated permissions."

+User consent is usually initiated when a user signs in to an application. After the user has provided sign-in credentials, they're checked to determine whether consent has already been granted. If no previous record of user or admin consent for the required permissions exists, the user is directed to the consent prompt window to grant the application the requested permissions.

+User consent by non-administrators is possible only in organizations where user consent is allowed for the application and for the set of permissions the application requires. If user consent is disabled, or if users aren't allowed to consent for the requested permissions, they won't be prompted for consent. If users are allowed to consent and they accept the requested permissions, the consent is recorded and they usually don't have to consent again on future sign-ins to the same application.

+### User consent settings

+Users are in control of their data. A Privileged Administrator can configure whether non-administrator users are allowed to grant user consent to an application. This setting can take into account aspects of the application and the application's publisher, and the permissions being requested.

+As an administrator, you can choose whether user consent is allowed. If you choose to allow user consent, you can also choose what conditions must be met before an application can be consented to by a user.

+By choosing which application consent policies apply for all users, you can set limits on when users are allowed to grant consent to applications and on when theyΓÇÖll be required to request administrator review and approval. The Azure portal provides the following built-in options:

+- *You can disable user consent*. Users can't grant permissions to applications. Users continue to sign in to applications they've previously consented to or to applications that administrators have granted consent to on their behalf, but they won't be allowed to consent to new permissions to applications on their own. Only users who have been granted a directory role that includes the permission to grant consent can consent to new applications.

+- *Users can consent to applications from verified publishers or your organization, but only for permissions you select*. All users can consent only to applications that were published by a [verified publisher](../develop/publisher-verification-overview.md) and applications that are registered in your tenant. Users can consent only to the permissions that you've classified as *low impact*. You must [classify permissions](configure-permission-classifications.md) to select which permissions users are allowed to consent to.

+- *Users can consent to all applications*. This option allows all users to consent to any permissions that don't require admin consent, for any application.

+For most organizations, one of the built-in options will be appropriate. Some advanced customers might want more control over the conditions that govern when users are allowed to consent. These customers can [create custom app consent policy](manage-app-consent-policies.md#create-a-custom-app-consent-policy) and configure those policies to apply to user consent.

+## Admin consent

+During admin consent, a Privileged Administrator may grant an application access on behalf of other users (usually, on behalf of the entire organization). Also during admin consent, applications or services provide direct access to an API, which can be used by the application if there's no signed-in user.

+When your organization purchases a license or subscription for a new application, you might proactively want to set up the application so that all users in the organization can use it. To avoid the need for user consent, an administrator can grant consent for the application on behalf of all users in the organization.

+After an administrator grants admin consent on behalf of the organization, users aren't usually prompted for consent for that application. In certain cases, a user might be prompted for consent even after consent was granted by an administrator. An example might be if an application requests another permission that the administrator hasn't already granted.

+Granting admin consent on behalf of an organization is a sensitive operation, potentially allowing the application's publisher access to significant portions of the organization's data, or the permission to do highly privileged operations. Examples of such operations might be role management, full access to all mailboxes or all sites, and full user impersonation.

+Before you grant tenant-wide admin consent, ensure that you trust the application and the application publisher, for the level of access you're granting. If you aren't confident that you understand who controls the application and why the application is requesting the permissions, do *not* grant consent.

+For step-by-step guidance on whether to grant an application admin consent, see [Evaluating a request for tenant-wide admin consent](manage-consent-requests.md#evaluate-a-request-for-tenant-wide-admin-consent).

+For step-by-step instructions for granting tenant-wide admin consent from the Azure portal, see [Grant tenant-wide admin consent to an application](grant-admin-consent.md).

+### Grant consent on behalf of a specific user

+Instead of granting consent for an entire organization, an admin can also use the [Microsoft Graph API](/graph/use-the-api) to grant consent to delegated permissions on behalf of a single user. For a detailed example that uses Microsoft Graph PowerShell, see [Grant consent on behalf of a single user by using PowerShell](manage-consent-requests.md).

+### Limit user access to an application

+User access to applications can still be limited, even when tenant-wide admin consent has been granted. Configure the applicationΓÇÖs properties to require user assignment to limit user access to the application. For more information, see [Methods for assigning users and groups](assign-user-or-group-access-portal.md).

+For a broader overview, including how to handle other complex scenarios, see [Use Azure AD for application access management](what-is-access-management.md).

+## Admin consent workflow

+The admin consent workflow gives users a way to request admin consent for applications when they aren't allowed to consent themselves. When the admin consent workflow is enabled, users are presented with an "Approval required" window for requesting admin approval for access to the application.

+After users submit the admin consent request, the admins who have been designated as reviewers receive a notification. The users are notified after a reviewer has acted on their request. For step-by-step instructions for configuring the admin consent workflow by using the Azure portal, see [configure the admin consent workflow](configure-admin-consent-workflow.md).

+### How users request admin consent

+After the admin consent workflow is enabled, users can request admin approval for an application that they're unauthorized to consent to. Here are the steps in the process:

+1. A user attempts to sign in to the application.

+1. An **Approval required** message appears. The user types a justification for needing access to the application and then selects "Request approval."

+1. A **Request sent** message confirms that the request was submitted to the admin. If the user sends several requests, only the first request is submitted to the admin.

+1. The user receives an email notification when the request is approved, denied, or blocked.

+## Next steps

+- [Configure user consent settings](configure-user-consent.md)

+- [Configure the admin consent workflow](configure-admin-consent-workflow.md)

+## October 2022

+### Updated articles

+- [Configure how users consent to applications](configure-user-consent.md)

+- [Tutorial: Configure F5 BIG-IP Access Policy Manager for Kerberos authentication](f5-big-ip-kerberos-advanced.md)

+- [Tutorial: Configure F5 BIG-IP Easy Button for Kerberos single sign-on](f5-big-ip-kerberos-easy-button.md)

+- [Tutorial: Configure F5 BIG-IP Easy Button for header-based and LDAP single sign-on](f5-big-ip-ldap-header-easybutton.md)

+- [Tutorial: Migrate your applications from Okta to Azure Active Directory](migrate-applications-from-okta-to-azure-active-directory.md)

+- [Tutorial: Configure Secure Hybrid Access with Azure Active Directory and Silverfort](silverfort-azure-ad-integration.md)

+description: Learn how to view Conditional Access policies in Azure AD sign-in logs so that you can assess the effect of those policies.

+With Conditional Access policies, you can control how your users get access to the resources of your Azure tenant. As a tenant admin, you need to be able to determine what effect your Conditional Access policies have on sign-ins to your tenant, so that you can take action if necessary.

+The sign-in logs in Azure Active Directory (Azure AD) give you the information that you need to assess the effect of your policies. This article explains how to view applied Conditional Access policies in those logs.

+- *Tenant administrators* who need to verify that Conditional Access policies have the intended effect on the users of a tenant.

+For more information about this cmdlet, see [Get-MgAuditLogSignIn](/powershell/module/microsoft.graph.reports/get-mgauditlogsignin).

+* The following roles in Azure Active Directory (if you're accessing Log Analytics through Azure Active Directory portal)

+5. Once you've configured the alert, select **Create alert** to enable it.

+* **Provisioning analysis**: This [workbook](../app-provisioning/application-provisioning-log-analytics.md) shows reports related to auditing provisioning activity. Activities can include the number of new users provisioned, provisioning failures, number of users updated, update failures, the number of users de-provisioned, and corresponding failures.

+* **Sign-ins Events**: This workbook shows the most relevant reports related to monitoring sign-in activity, such as sign-ins by application, user, device, and a summary view tracking the number of sign-ins over time.

+* **Conditional access insights**: The Conditional Access insights and reporting [workbook](../conditional-access/howto-conditional-access-insights-reporting.md) enables you to understand the effect of Conditional Access policies in your organization over time.

+The [Azure Active Directory (Azure AD) reporting APIs](./concept-reporting-api.md) provide you with programmatic access to the data through a set of REST-based APIs. You can call these APIs from many programming languages and tools.

+You need these values when configuring calls to the reporting API. We recommend using a certificate because it's more secure.

+1. In the [Azure portal](https://portal.azure.com), on the left navigation pane, select **Azure Active Directory**.

+1. In the [Azure portal](https://portal.azure.com), on the left navigation pane, select **Azure Active Directory**.

+3. Select **Certificates and Secrets** on the **API Application** page, in the **Client Secrets** section, select **+ New Client Secret**.

+ c. Select **Save**.

+### Error: Tenant isn't B2C or tenant doesn't have premium license

+### Error: The allowed roles doesn't include User.

+### Error: Application missing Azure AD 'Read directory data' permission

+1. Select **Audit logs** from the **Activity** section of Azure Active Directory.

+2. Select your directory from the top-right corner, then select **Azure Active Directory** from the left navigation pane.

+The Azure Active Directory (Azure AD) log analytics views helps you analyze and search the Azure AD activity logs in your Azure AD tenant. Azure AD activity logs include:

+1. Navigate to the [Azure portal](https://portal.azure.com) and select **All services**.

+1. Type **Log Analytics** in the text box, and select **Log Analytics workspaces**. Select the workspace you routed the activity logs to, as part of the prerequisites.

+1. Select **View Designer** > **Import** > **Choose File** to import the views from your local computer.

+1. Select the views you downloaded from the prerequisites and select **Save** to save the import. Complete this step for the **Azure AD Account Provisioning Events** view and the **Sign-ins Events** view.

+1. Navigate to the [Azure portal](https://portal.azure.com) and select **All services**.

+1. Type **Log Analytics** in the text box, and select **Log Analytics workspaces**. Select the workspace you routed the activity logs to, as part of the prerequisites.

+1. Once you're in the workspace, select **Workspace Summary**. You should see the following three views:

+ * **Azure AD Account Provisioning Events**: This view shows reports related to the auditing provisioning activity. Activities can include the number of new users provisioned, provisioning failures, number of users updated, update failures, the number of users de-provisioned and their corresponding failures.

+ * **Sign-ins Events**: This view shows the most relevant reports related to monitoring sign-in activity, such as sign-ins by application, user, device, and a summary view tracking the number of sign-ins over time.

+1. Select either of these views to jump in to the individual reports. You can also set alerts on any of the report parameters. For example, let's set an alert for every time there's a sign-in error.

+1. Select the **Sign-ins Events** > **Sign-in errors over time** > **Analytics** to open the details page, with the actual query behind the report.

+1. Select **Set Alert**, and then select **Whenever the Custom log search is &lt;logic undefined&gt;** under the **Alert criteria** section. Since we want to alert whenever there's a sign-in error, set the **Threshold** of the default alert logic to **1** and then select **Done**.

+1. Enter a name and description for the alert and set the severity to **Warning**.

+1. Select the action group to alert, such as a team you want to notify via email or text message. Learn how to [create and manage action groups in the Azure portal](../../azure-monitor/alerts/action-groups.md).

+1. Select **Create alert rule** to create the alert. Now you'll be alerted every time there's a sign-in error.

+* A configured instance of ArcSight Syslog NG Daemon SmartConnector (SmartConnector) or ArcSight Load Balancer. If the events are sent to ArcSight Load Balancer, they're sent to the SmartConnector by the Load Balancer.

+Download and open the [configuration guide for ArcSight SmartConnector for Azure Monitor Event Hubs](https://community.microfocus.com/t5/ArcSight-Connectors/SmartConnector-for-Microsoft-Azure-Monitor-Event-Hub/ta-p/1671292). This guide contains the steps you need to install and configure the ArcSight SmartConnector for Azure Monitor.

+3. Use the steps in the **Verifying the Deployment in Azure** to make sure the connector is set up and functions correctly. Verify the following prerequisites:

+4. Finally, complete the post-deployment steps in the **Post-Deployment Configurations** of the configuration guide. This section explains how to perform another configuration if you are on an App Service Plan to prevent the function apps from going idle after a timeout period, configure streaming of resource logs from the event hub, and update the SysLog NG Daemon SmartConnector keystore certificate to associate it with the newly created storage account.

+5. The configuration guide also explains how to customize the connector properties in Azure, and how to upgrade and uninstall the connector. There's also a section on performance improvements, including upgrading to an [Azure Consumption plan](https://azure.microsoft.com/pricing/details/functions) and configuring an ArcSight Load Balancer if the event load is greater than what a single Syslog NG Daemon SmartConnector can handle.

+[Configuration guide for ArcSight SmartConnector for Azure Monitor Event Hubs](https://community.microfocus.com/t5/ArcSight-Connectors/SmartConnector-for-Microsoft-Azure-Monitor-Event-Hub/ta-p/1671292)

+4. Identify the failed sign-in you want to investigate. Select it to open up the other details window with more information about the failed sign-in. Note down the **Sign-in error code** and **Failure reason**.

+## What is Azure Monitor workbooks for Azure AD reports?

+For one-off investigations with a limited scope, the Azure portal is often the easiest way to find the data you need. However, there are also business problems requiring a more complex analysis of the data in your activity logs. One common example for a scenario that requires a trend analysis is related to blocking legacy authentication in your Azure AD tenant.

+How can you determine whether it's safe to block legacy authentication in an environment? Answering this question requires an analysis of the sign-ins in your environment for a certain timeframe. Workbooks provide a flexible canvas for data analysis and the creation of rich visual reports within the Azure portal. They allow you to tap into multiple data sources from across Azure, and combine them into unified interactive experiences.

+- **Public templates** published to a [gallery](../../azure-monitor/visualize/workbooks-overview.md#the-gallery) that serve as a good starting point when you're just getting started with workbooks.

+- Following roles in Azure Active Directory (if you're accessing Log Analytics through Azure Active Directory portal)

+ - [Risky sign-ins](../identity-protection/overview-identity-protection.md) - A risky sign-in is an indicator for a sign-in attempt that might have been performed by someone who isn't the legitimate owner of a user account.

+This article lists the audit activities that can be logged in your audit logs.

+|Access Reviews|Review RBAC Role membership|

+|Authentication|Create a custom domain in the tenant|

+|Authorization|User Authorization: API is disabled for tenant feature set|

+|Authorization|Disable Desktop SSO|

+|Authorization|Disable Desktop SSO for a specific domain|

+|Authorization|Enable Desktop SSO|

+|Directory Management|Enable Desktop SSO for a specific domain|

+As an identity admin, you may need to track the Azure Active Directory (Azure AD) service-level agreement (SLA) performance to make sure Azure AD can support your vital apps. This article shows how the Azure AD service has performed according to the [SLA for Azure Active Directory (Azure AD)](https://azure.microsoft.com/support/legal/sla/active-directory/v1_1/).

+- **User authentication:** Users are able to sign in to the Azure AD service.

+- **App access:** Azure AD successfully emits the authentication and authorization tokens required for users to sign in to applications connected to the service.

+The SLA attainment is truncated at three places after the decimal. Numbers are not rounded up, so actual SLA attainment is higher than indicated.

+description: Describe the Azure AD sign-in log schema for use in Azure Monitor

+This article describes the Azure Active Directory (Azure AD) sign-in log schema in Azure Monitor. Information related to sign-ins is provided under the *Properties* attribute of the `records` object.

+| authProcessingDetails | Azure AD app authentication library | Contains Family, Library, and Platform information in format: "Family: Microsoft Authentication Library: ADAL.JS 1.0.0 Platform: JS" |

+| riskLevelAggregated | riskLevel | Aggregated risk level. The possible values are: `none`, `low`, `medium`, `high`, `hidden`, and `unknownFutureValue`. The value `hidden` means the user or sign-in wasn't enabled for Azure AD Identity Protection. **Note:** Details for this property are only available for Azure AD Premium P2 customers. All other customers will be returned `hidden`. |

+| riskLevelDuringSignIn | riskLevel | Risk level during sign-in. The possible values are: `none`, `low`, `medium`, `high`, `hidden`, and `unknownFutureValue`. The value `hidden` means the user or sign-in wasn't enabled for Azure AD Identity Protection. **Note:** Details for this property are only available for Azure AD Premium P2 customers. All other customers will be returned `hidden`. |

+Azure AD logs all sign-ins into an Azure tenant for compliance. As an IT administrator, you need to know what the values in the sign-in logs mean, so that you can interpret the log values correctly. [Learn how to access, view, and analyze Azure AD sign-in logs](concept-sign-ins.md)

+### Tenant

+### Sign-in

+The sign-in identifier is a string the user provides to Azure AD to identify itself when attempting to sign-in. It's usually a user principal name (UPN), but can be another identifier such as a phone number.

+### Authentication requirement

+This attribute shows the highest level of authentication needed through all the sign-in steps for the sign-in to succeed. Graph API supports `$filter` (`eq` and `startsWith` operators only).

+### Sign-in event types

+### User type

+### Cross-tenant access type

+### Conditional access evaluation

+* [Learn about exporting Azure AD sign-in logs](concept-activity-logs-azure-monitor.md)

+* [Explore the sign-in diagnostic in Azure AD](overview-sign-in-diagnostics.md)

+To install the public preview release, use the following:

+For more information on how to connect to Azure AD using PowerShell, see the article [Azure AD PowerShell for Graph](/powershell/azure/active-directory/install-adv2).

+In this article, you learn about the data retention policies for the different activity reports in Azure Active Directory (Azure AD).

+| Azure AD Free| The first time you open [Azure Active Directory](https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/Overview) or use the [reporting APIs](./overview-reports.md) |

+**No**, you can't. Azure stores up to seven days of activity data for a free version. When you switch from a free to a premium version, you can only see up to 7 days of data.

+- [Risky sign-ins](../identity-protection/overview-identity-protection.md) - A risky sign-in is an indicator for a sign-in attempt that might have been performed by someone who isn't the legitimate owner of a user account.

+## I have a lot of changes to my users and I'm not sure what the cause of it is.

+I check the Azure AD audit logs, and see multiple user updates occurring in my Azure AD tenant. These **Update User** events don't display **Actor** information, which causes uncertainty as to what/who triggered the mass changes to users.

+ A common reason behind mass object changes is a non-synchronous backend operation called **ProxyCalc**. **ProxyCalc** is the logic that determines the appropriate **UserPrincipalName** and **Proxy Addresses** that are updated in Azure AD users, groups, or contacts. The design behind **ProxyCalc** is to ensure that all **UserPrincipalName** and **Proxy Addresses** are consistent in Azure AD at any time. **ProxyCalc** must be triggered by an explicit change like a verified domain change and doesn't perpetually run in the background as a task.

+#### Notes:

+ProxyCalc doesn't cause changes to certain objects that:

+- don't have an active Exchange license

+- aren't considered a shared resource. Shared Resource is when **CloudMSExchRecipientDisplayType** contains one of the following values: **MailboxUser (shared)**, **PublicFolder**, **ConferenceRoomMailbox**, **EquipmentMailbox**, **ArbitrationMailbox**, **RoomList**, **TeamMailboxUser**, **Group mailbox**, **Scheduling mailbox**, **ACLableMailboxUser**, **ACLableTeamMailboxUser**

+Additionally, in most cases, there are no changes to users as their **UserPrincipalName** and **Proxy Addresses** are consistent, so we're working to display in Audit Logs only those updates that caused an actual change to the object. This action will prevent noise in the audit logs and help admins correlate the remaining user changes to verified domain change event as explained above.

+### Error: Application missing Azure AD 'Read directory data' permission

+I performed some actions in the Azure portal and expected to see the audit logs for those actions in the `Activity logs > Audit Logs`, but I canΓÇÖt find them.

+Wait for 15 minutes to two hours and see if the actions appear in the log. If you donΓÇÖt see the logs even after two hours, [file a support request,](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest) and we'll look into it.

+I recently signed into the Azure portal and expected to see the sign-in logs for those actions in the `Activity logs > Sign-ins`, but I canΓÇÖt find them.

+Wait for 15 minutes to two hours and see if the actions appear in the log. If you donΓÇÖt see the logs even after two hours, [file a support request,](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest) and we'll look into it.

+| Sign-in Activity | Not available. You can access your own sign-ins for 7 days from the individual user profile | 30 days | 30 days |

+You can use [Azure AD Reporting APIs](concept-reporting-api.md) to fetch up to a million records at any given point.

+Overprompting users can affect your user's productivity and often leads users getting phished for MFA. To be clear, MFA is essential! We are not talking about if you should require MFA but how frequently you should prompt your users.

+The prompts by application list view shows additional information such as timestamps, and request IDs that help with investigations.

+If data isn't showing up or seems to be showing up incorrectly, confirm that you have set the **Log Analytics Workspace** and **Subscriptions** on the proper resources.

+- To understand more about the different policies that affect MFA prompts, see [Optimize reauthentication prompts and understand session lifetime for Azure AD Multi-Factor Authentication](../authentication/concepts-azure-multi-factor-authentication-prompts-session-lifetime.md).

+- To learn more about the different vulnerabilities of different MFA methods, see [All your creds belong to us!](https://aka.ms/allyourcreds).

+- Number of sign-ins by applications that aren't impacted by conditional access policies

+- Number of sign-ins by location that weren't affected by conditional access policies

+Have you ever wondered how you can determine whether it's safe to turn off legacy authentication in your tenant? The sign-ins using legacy authentication workbook helps you to answer this question.

+Single-factor authentication (for example, username and password) doesnΓÇÖt provide the required level of protection for todayΓÇÖs computing environments. Passwords are bad as they're easy to guess and humans are bad at choosing good passwords.

+- Doesn't support multi-factor authentication (MFA) or other strong authentication methods.

+- Some clients can use both legacy authentication or modern authentication depending on client configuration. If you see ΓÇ£modern mobile/desktop clientΓÇ¥ or ΓÇ£browserΓÇ¥ for a client in the Azure AD logs, it's using modern authentication. If it has a specific client or protocol name, such as ΓÇ£Exchange ActiveSyncΓÇ¥, it's using legacy authentication to connect to Azure AD. The client types in conditional access, and the Azure AD reporting page in the Azure portal demarcate modern authentication clients and legacy authentication clients for you, and only legacy authentication is captured in this workbook.

+As an IT administrator, you need to be able to identify compromises in your environment to ensure that you can keep it in a healthy state.

+If your organization is new to Azure monitor workbooks, you need to integrate your Azure AD sign-in and audit logs with Azure Monitor before accessing the workbook. This integration allows you to store, and query, and visualize your logs using workbooks for up to two years. Only sign-in and audit events created after Azure Monitor integration will be stored, so the workbook won't contain insights prior to that date. Learn more about the prerequisites to Azure Monitor workbooks for Azure Active Directory. If you've previously integrated your Azure AD sign-in and audit logs with Azure Monitor, you can use the workbook to assess past information.

+- **Modified application and service principal credentials/authentication methods** - This report flags actors who have recently changed many service principal credentials, and how many of each type of service principal credentials have been changed.

+- Top actors and the number of credentials modifications they performed

+In cases where the attacker can't find a service principal or an application with a high privilege set of permissions through which to gain access, they'll often attempt to add the permissions to another service principal or app.

+- Add another SAML IDP that is controlled by the attacker as a trusted authentication source.

+- **Modified application and service principal credentials** to look out for credentials being added to service principals that aren't frequently used in your organization. Use the filters present in this section to further investigate any of the suspicious actors or service principals that were modified.

+Deployments are typically created and managed with `kubectl create` or `kubectl apply`. Create a deployment by defining a manifest file in the YAML format.

+A breakdown of the deployment specifications in the YAML manifest file is as follows:

+| Specification | Description |

+| -- | - |

+| `.apiVersion` | Specifies the API group and API resource you want to use when creating the resource. |

+| `.kind` | Specifies the type of resource you want to create. |

+| `.metadata.name` | Specifies the image to run. This file will run the *nginx* image from Docker Hub. |

+| `.spec.replicas` | Specifies how many pods to create. This file will create three deplicated pods. |

+| `.spec.selector` | Specifies which pods will be affected by this deployment. |

+| `.spec.selector.matchLabels` | Contains a map of *{key, value}* pairs that allows the deployment to find and manage the created pods. |

+| `.spec.selector.matchLabels.app` | Has to match `.spec.template.metadata.labels`. |

+| `.spec.template.labels` | Specifies the *{key, value}* pairs attached to the object. |

+| `.spec.template.app` | Has to match `.spec.selector.matchLabels`. |

+| `.spec.spec.containers` | Specifies the list of containers belonging to the pod. |

+| `.spec.spec.containers.name` | Specifies the name of the container specified as a DNS label. |

+| `.spec.spec.containers.image` | Specifies the container image name. |

+| `.spec.spec.containers.ports` | Specifies the list of ports to expose from the container. |

+| `.spec.spec.containers.ports.containerPort` | Specifies the number of port to expose on the pod's IP address. |

+| `.spec.spec.resources` | Specifies the compute resources required by the container. |

+| `.spec.spec.resources.requests` | Specifies the minimum amount of compute resources required. |

+| `.spec.spec.resources.requests.cpu` | Specifies the minimum amount of CPU required. |

+| `.spec.spec.resources.requests.memory` | Specifies the minimum amount of memory required. |

+| `.spec.spec.resources.limits` | Specifies the maximum amount of compute resources allowed. This limit is enforced by the kubelet. |

+| `.spec.spec.resources.limits.cpu` | Specifies the maximum amount of CPU allowed. This limit is enforced by the kubelet. |

+| `.spec.spec.resources.limits.memory` | Specifies the maximum amount of memory allowed. This limit is enforced by the kubelet. |

+ Title: Deploy an Azure container offer from Azure Marketplace

+description: Learn how to deploy Azure container offers from Azure Marketplace on an Azure Kubernetes Service (AKS) cluster.

+# Deploy a container offer from Azure Marketplace (preview)

+[Azure Marketplace][azure-marketplace] is an online store that contains thousands of IT software applications and services built by industry-leading technology companies. In Azure Marketplace, you can find, try, buy, and deploy the software and services that you need to build new solutions and manage your cloud infrastructure. The catalog includes solutions for different industries and technical areas, free trials, and consulting services from Microsoft partners.

+Included among these solutions are Kubernetes application-based container offers. These offers contain applications that are meant to run on Kubernetes clusters such as Azure Kubernetes Service (AKS). In this article, you'll learn how to:

+- Browse offers in Azure Marketplace.

+- Purchase an application.

+- Deploy the application on your AKS cluster.

+- Monitor usage and billing information.

+> This feature is currently supported only in the following regions:

+> - East US

+Before you deploy a container offer, you must register the `Microsoft.ContainerService` and `Microsoft.KubernetesConfiguration` providers on your subscription by using the `az provider register` command:

+## Select and deploy a Kubernetes offer

+1. In the [Azure portal](https://ms.portal.azure.com/), search for **Marketplace** on the top search bar. In the results, under **Services**, select **Marketplace**.

+1. You can search for an offer or publisher directly by name, or you can browse all offers. To find Kubernetes application offers, use the **Product Type** filter for **Azure Containers**.

+ :::image type="content" source="./media/deploy-marketplace/browse-marketplace-inline.png" alt-text="Screenshot of Azure Marketplace offers in the Azure portal, with the filter for product type set to Azure containers." lightbox="./media/deploy-marketplace/browse-marketplace-full.png":::

+ > [!IMPORTANT]

+ > The **Azure Containers** category includes both Kubernetes applications and standalone container images. This walkthrough is specific to Kubernetes applications. If you find that the steps to deploy an offer differ in some way, you're most likely trying to deploy a container image-based offer instead of a Kubernetes application-based offer.

+ >

+ > To ensure that you're searching for Kubernetes applications, include the term **KubernetesApps** in your search.

+1. After you decide on an application, select the offer.

+1. On the **Plans + Pricing** tab, select an option. Ensure that the terms are acceptable, and then select **Create**.

+ :::image type="content" source="./media/deploy-marketplace/plans-pricing-inline.png" alt-text="Screenshot of the offer purchasing page in the Azure portal, including plan and pricing information." lightbox="./media/deploy-marketplace/plans-pricing-full.png":::

+1. Follow each page in the wizard, all the way through **Review + Create**. Fill in information for your resource group, your cluster, and any configuration options that the application requires. You can decide to deploy on a new AKS cluster or use an existing cluster.

+ :::image type="content" source="./media/deploy-marketplace/purchase-experience-inline.png" alt-text="Screenshot of the Azure portal wizard for deploying a new offer, with the selector for creating a new cluster or using an existing cluster." lightbox="./media/deploy-marketplace/purchase-experience-full.png":::

+ When the application is deployed, the portal shows **Your deployment is complete**, along with details of the deployment.

+ :::image type="content" source="./media/deploy-marketplace/deployment-inline.png" alt-text="Screenshot of the Azure portal that shows a successful resource deployment to the cluster." lightbox="./media/deploy-marketplace/deployment-full.png":::

+1. Verify the deployment by using the following command to list the extensions that are running on your cluster:

+ ```azurecli-interactive

+ az k8s-extension list --cluster-name <clusterName> --resource-group <resourceGroupName> --cluster-type managedClusters

+ ```

+## Manage the offer lifecycle

+For lifecycle management, an Azure Kubernetes offer is represented as a cluster extension for AKS. For more information, seeΓÇ»[Cluster extensions for AKS][cluster-extensions].

+Purchasing an offer from Azure Marketplace creates a new instance of the extension on your AKS cluster. You can view the extension instance from the cluster by using the following command:

+## Monitor billing and usage information

+To monitor billing and usage information for the offer that you deployed:

+1. In the Azure portal, go to the page for your cluster's resource group.

+1. Select **Cost Management** > **Cost analysis**. Under **Product**, you can see a cost breakdown for the plan that you selected.

+ :::image type="content" source="./media/deploy-marketplace/billing-inline.png" alt-text="Screenshot of the Azure portal page for a resource group, with billing information broken down by offer plan." lightbox="./media/deploy-marketplace/billing-full.png":::

+## Remove an offer

+You can delete a purchased plan for an Azure container offer by deleting the extension instance on the cluster. For example:

+```azurecli-interactive

+az k8s-extension delete --name <extension-name> --cluster-name <clusterName> --resource-group <resourceGroupName> --cluster-type managedClusters

+```

+## Next steps

+[billing]: ../cost-management-billing/costs/quick-acm-cost-analysis.md

+ For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+ For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+ For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+ For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+ For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+For a breakdown of YAML manifest files, see [Deployments and YAML manifests](../concepts-clusters-workloads.md#deployments-and-yaml-manifests).

+3. In scenarios where the VNet containing your cluster has custom DNS settings (4), cluster deployment fails unless the private DNS zone is linked to the VNet that contains the custom DNS resolvers (5). This link can be created manually after the private zone is created during cluster provisioning or via automation upon detection of creation of the zone using event-based deployment mechanisms (for example, Azure Event Grid and Azure Functions). To avoid cluster failure during initial deployment, the cluster can be deployed with the private DNS zone resource ID. This only works with resource type Microsoft.ContainerService/managedCluster and API version 2022-07-01. Using an older version with an ARM template or Bicep resource definition is not supported.

+[use-custom-domains]: coredns-custom.md#use-custom-domains

+* **Administrators** - Azure subscription administrators are members of this group. Administrators manage API Management service instances, creating the APIs, operations, and products that are used by developers. You can't add users to this group.

+ > [!NOTE]

+ > You can change the administrator [email settings](api-management-howto-configure-notifications.md#configure-email-settings) that are used in notifications sent to developers from your API Management instance.

+* Learn how to manage the administrator [email settings](api-management-howto-configure-notifications.md#configure-email-settings) that are used in notifications to developers from your API Management instance.

+You can also disable certificate chain validation by using the [Backend](/rest/api/apimanagement/current-ga/backend) REST API.

+ 1. On the **Managed identities** page of your API Management instance, enable a system-assigned or user-assigned [managed identity](api-management-howto-use-managed-service-identity.md). Note the principal ID on that page.

+ 1. Give the list and get secrets permissions to this principal ID on the Azure Key Vault containing the certificate.

+Configure a CNAME record that points from your custom domain name (for example, `api.contoso.com`) to your API Management service hostname (for example, `<apim-service-name>.azure-api.net`). A CNAME record is more stable than an A-record in case the IP address changes. For more information, see [IP addresses of Azure API Management](api-management-howto-ip-addresses.md#changes-to-the-ip-addresses) and the [API Management FAQ](./api-management-faq.yml#how-can-i-secure-the-connection-between-the-api-management-gateway-and-my-backend-services-).

+* The [Azure APIOps Toolkit][5] provides a workflow built on top of a [git][21] source code control system (such as [GitHub][22] or [Azure Repos][23]). It uses an _extractor_ similar to the DevOps Resource Toolkit to produce an API definition that is then applied to a target API Management service. APIOps supports REST and GraphQL APIs at this time.

+[28]: /azure/architecture/example-scenario/devops/automated-api-deployments-apiops

+ Title: Configure virtual network integration with application and configuration routing.

+description: This how-to article walks you through configuring routing on a regional virtual network integration.

+Through application routing or configuration routing options, you can configure what traffic will be sent through the virtual network integration. See the [overview section](./overview-vnet-integration.md#routes) for more details.

+Your app is already integrated using the regional virtual network integration feature.

+## Configure application routing

+Application routing defines what traffic is routed from your app and into the virtual network. We recommend that you use the **Route All** site setting to enable routing of all traffic. Using the configuration setting allows you to audit the behavior with [a built-in policy](https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F33228571-70a4-4fa1-8ca1-26d0aba8d6ef). The existing `WEBSITE_VNET_ROUTE_ALL` app setting can still be used, and you can enable all traffic routing with either setting.

+### Configure in the Azure portal

+### Configure with the Azure CLI

+You can also configure **Route All** by using the Azure CLI.

+```azurecli-interactive

+az resource update --resource-group <group-name> --name <app-name> --resource-type "Microsoft.Web/sites" --properties.vnetRouteAllEnabled [true|false]

+```

+## Configure configuration routing

+When you're using virtual network integration, you can configure how parts of the configuration traffic are managed. By default, configuration traffic will go directly over the public route, but for the mentioned individual components, you can actively configure it to be routed through the virtual network integration.

+### Container image pull

+Routing container image pull over virtual network integration can be configured using the Azure CLI.

+az resource update --resource-group <group-name> --name <app-name> --resource-type "Microsoft.Web/sites" --properties.vnetImagePullEnabled [true|false]

+We recommend that you use the site property to enable routing image pull traffic through the virtual network integration. Using the configuration setting allows you to audit the behavior with Azure Policy. The existing `WEBSITE_PULL_IMAGE_OVER_VNET` app setting with the value `true` can still be used, and you can enable routing through the virtual network with either setting.

+### Content storage

+Routing content storage over virtual network integration can be configured using the Azure CLI. In addition to enabling the feature, you must also ensure that any firewall or Network Security Group configured on traffic from the subnet allow traffic to port 443 and 445.

+```azurecli-interactive

+az resource update --resource-group <group-name> --name <app-name> --resource-type "Microsoft.Web/sites" --properties.vnetContentStorageEnabled [true|false]

+We recommend that you use the site property to enable content storage traffic through the virtual network integration. Using the configuration setting allows you to audit the behavior with Azure Policy. The existing `WEBSITE_CONTENTOVERVNET` app setting with the value `1` can still be used, and you can enable routing through the virtual network with either setting.

+In addition to enabling access, you need to ensure that you have [configured DNS if you are using ILB App Service Environment](./networking.md#dns-configuration-for-ftp-access) and that the [necessary ports](./networking.md#ports-and-network-restrictions) are unblocked.

+If the certificate used for the custom domain suffix contains a Subject Alternate Name (SAN) entry for **.scm.CUSTOM-DOMAIN*, the scm site will then also be reachable from *APP-NAME.scm.CUSTOM-DOMAIN*. You can only access scm over custom domain using basic authentication. Single sign-on is only possible with the default root domain.

+Your certificate must be a wildcard certificate for the selected custom domain name. For example, *internal-contoso.com* would need a certificate covering **.internal-contoso.com*. If the certificate used custom domain suffix contains a Subject Alternate Name (SAN) entry for scm, for example **.scm.internal-contoso.com*, the scm site will also available using the custom domain suffix.

+1. Optionally create a zone for scm sub-domain with a * A record that points to the inbound IP address used by your App Service Environment

+1. Optionally create an A record in that zone that points *.scm to the inbound IP address used by your App Service Environment.

+> [!NOTE]

+> For FTP access, even if you want to disallow standard FTP on port 21, you still need to allow traffic from the LoadBalancer to the App Service Environment subnet range, as this is used for internal health ping traffic for the ftp service specifically.

+* **X-Azure-FDID**. [Custom header](../frontdoor/front-door-http-headers-protocol.md#from-the-front-door-to-the-backend) for identifying the reverse proxy instance. Azure Front Door will send a guid identifying the instance, but it can also be used by third party proxies to identify the specific instance. Accepts any string up to 64 characters in length.

+* **X-FD-HealthProbe**. [Custom header](../frontdoor/front-door-http-headers-protocol.md#from-the-front-door-to-the-backend) for identifying the health probe of the reverse proxy. Azure Front Door will send "1" to uniquely identify a health probe request. The header can also be used by third party proxies to identify health probes. Accepts any string up to 64 characters in length.

+The feature supports two virtual interface per worker. Two virtual interfaces per worker means two regional virtual network integrations per App Service plan. The apps in the same App Service plan can only use one of the virtual network integrations to a specific subnet. If you need an app to connect to additional virtual networks or additional subnets in the same virtual network, you need to create another App Service plan. The virtual interfaces used isn't a resource that customers have direct access to.

+Learn [how to configure application routing](./configure-vnet-integration-routing.md#configure-application-routing).

+To route content storage traffic through the virtual network integration, you must ensure that the routing setting is configured. Learn [how to configure content storage routing](./configure-vnet-integration-routing.md#content-storage).

+In addition to configuring the routing, you must also ensure that any firewall or Network Security Group configured on traffic from the subnet allow traffic to port 443 and 445.

+When using custom containers, you can pull the container over the virtual network integration. To route the container pull traffic through the virtual network integration, you must ensure that the routing setting is configured. Learn [how to configure image pull routing](./configure-vnet-integration-routing.md#container-image-pull).

+* You can have two regional virtual network integration per App Service plan. Multiple apps in the same App Service plan can use the same virtual network integration.

+Upon deletion of an App Service app or App Service Environment (ASE), the corresponding DNS is reserved. During the reservation period, reuse of the DNS is forbidden except for subscriptions belonging to tenant of the subscription originally owning the DNS.

+After the reservation expires, the DNS is free to be claimed by any subscription. By Name Reservation Service, the customer is afforded some time to either clean-up any associations/pointers to said DNS or reclaim the DNS in Azure. The DNS name being reserved for web apps can be derived by appending 'azurewebsites.net' and for ASE can be derived by appending 'appserviceenvironment.net'. Name Reservation Service is enabled by default on Azure App Service and doesn't require more configuration.

+ Title: Azure TPM VBS attestation usage

+description: Learn about how to apply TPM and VBS attestation

+# Using TPM/VBS attestation

+Attestation can be integrated into various applications and services, catering to different use cases. Azure Attestation service, which acts the remote attestation service can be used for desired purposes by updating the attestation policy. The policy engine works as processor, which takes the incoming payload as evidence and performs the validations as authored in the policy. This architecture simplifies the workflow and enables the service owner to purpose build solutions for the varied platforms and use cases.The workflow remains the same as described in [Azure attestation workflow](workflow.md). The attestation policy needs to be crafted as per the validations required.

+Attesting a platform has its own challenges with its varied components of boot and setup, one needs to rely on a hardware root-of-trust anchor which can be used to verify the first steps of the boot and extend that trust upwards into every layer on your system. A hardware TPM provides such an anchor for a remote attestation solution. Azure Attestation provides a highly scalable measured boot and runtime integrity measurement attestation solution with a revocation framework to give you full control over platform attestation.

+## Attestation steps

+Attestation Setup has two setups. One pertaining to the service setup and one pertaining to the client setup.

+Detailed information about the workflow is described in [Azure attestation workflow](workflow.md).

+### Service endpoint setup:

+This is the first step for any attestation to be performed. Setting up an endpoint, this can be performed either via code or using the Azure portal.

+Here's how you can set up an attestation endpoint using Portal

+1 Prerequisite: Access to the Microsoft Azure Active Directory(Azure AD) tenant and subscription under which you want to create the attestation endpoint.

+Learn more about setting up an [Azure AD tenant](../active-directory/develop/quickstart-create-new-tenant.md).

+2 Create an endpoint under the desired resource group, with the desired name.

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE5azcU]

+3 Add Attestation Contributor Role to the Identity who will be responsible to update the attestation policy.

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE5aoRj]

+4 Configure the endpoint with the required policy.

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE5aoRk]

+Sample policies can be found in the [policy section](tpm-attestation-sample-policies.md).

+> [!NOTE]

+> TPM endpoints are designed to be provisioned without a default attestation policy.

+### Client setup:

+A client to communicate with the attestation service endpoint needs to ensure it's following the protocol as described in the [protocol documentation](virtualization-based-security-protocol.md). Use the [Attestation Client NuGet](https://www.nuget.org/packages/Microsoft.Attestation.Client) to ease the integration.

+

+1 Prerequisite: An Azure AD identity is needed to access the TPM endpoint.

+Learn more [Azure AD identity tokens](../active-directory/develop/v2-overview.md).

+2 Add Attestation Reader Role to the identity that will be need for authentication against the endpoint. Azure i

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE5aoRi]

+## Execute the attestation workflow:

+Using the [Client](https://github.com/microsoft/Attestation-Client-Samples) to trigger an attestation flow. A successful attestation will result in an attestation report (encoded JWT token). Parsing the JWT token, the contents of the report can be easily validated against expected outcome.

+> [!VIDEO https://www.microsoft.com/en-us/videoplayer/embed/RE5azcT]

+Here's a sample of the contents of the attestation report.

+Using the Open ID [metadata endpoint](/rest/api/attestation/metadata-configuration/get?tabs=HTTP) contains properties, which describe the attestation service.The signing keys describe the keys, which will be used to sign tokens generated by the attestation service. All tokens emitted by the attestation service will be signed by one of the certificates listed in the attestation signing keys.

+## Next steps

+- [Set up Azure Attestation using PowerShell](quickstart-powershell.md)

+- [Attest an SGX enclave using code samples](/samples/browse/?expanded=azure&terms=attestation)

+- [Learn more about policy](policy-reference.md)

+ Title: Examples of an Azure SGX Attestation policy

+| Secure boot |Device boots using only software that's trusted by the OEM. | ` c:[type == "events", issuer=="AttestationService"] => add(type = "efiConfigVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG' && ProcessedData.VariableGuid == '8BE4DF61-93CA-11D2-AA0D-00E098032B8C']"));`<br>` => issue(type = "secureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] \| length(@) == `1` && @[0].ProcessedData.VariableData == 'AQ'")));`<br>` ![type=="secureBootEnabled", issuer=="AttestationPolicy"] => issue(type="secureBootEnabled", value=false);` |

+| Code integrity |Code integrity is a feature that validates the integrity of a driver or system file each time it's loaded into memory.| `// Retrieve bool properties `<br>` c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="codeIntegrityEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_CODEINTEGRITY")));`<br>`c:[type=="codeIntegrityEnabledSet", issuer=="AttestationPolicy"] => issue(type="codeIntegrityEnabled", value=ContainsOnlyValue(c.value, true));`<br>`![type=="codeIntegrityEnabled", issuer=="AttestationPolicy"] => issue(type="codeIntegrityEnabled", value=false);` |

+|BitLocker [Boot state] |Used for encryption of device drives.| `// Bitlocker Boot Status, The first non zero measurement or zero.`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => issue(type="bitlockerStatus", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_BITLOCKER_UNLOCK \| @[? Value != `0`].Value \| @[0]")));`<br>`![type=="bitlockerStatus"] => issue(type="bitlockerStatus", value=0);` |

+| Early Launch Antimalware (ELAM) | ELAM protects against loading unsigned or malicious drivers during boot. | `// Elam Driver (windows defender) Loaded.`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => issue(type="elamDriverLoaded", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_LOADEDMODULE_AGGREGATION[] \| [? EVENT_IMAGEVALIDATED == `true` && (equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\wdboot.sys') \|\| equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\wd\\wdboot.sys'))] \| @ != `null`")));`<br>`![type=="elamDriverLoaded", issuer=="AttestationPolicy"] => issue(type="elamDriverLoaded", value=false);` |

+| Boot debugging |Allows the user to connect to a boot debugger. Can be used to bypass secure boot and other boot protections. | `// Boot debugging`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="bootDebuggingEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_BOOTDEBUGGING")));`<br>`c:[type=="bootDebuggingEnabledSet", issuer=="AttestationPolicy"] => issue(type="bootDebuggingDisabled", value=ContainsOnlyValue(c.value, false));`<br>`![type=="bootDebuggingDisabled", issuer=="AttestationPolicy"] => issue(type="bootDebuggingDisabled", value=false);` |

+| Kernel debugging | Allows the user to connect a kernel debugger. Grants access to all system resources (less virtualization-based security [VBS] protected resources). | `// Kernel Debugging`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="osKernelDebuggingEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_OSKERNELDEBUG")));`<br>`c:[type=="osKernelDebuggingEnabledSet", issuer=="AttestationPolicy"] => issue(type="osKernelDebuggingDisabled", value=ContainsOnlyValue(c.value, false));`<br>`![type=="osKernelDebuggingDisabled", issuer=="AttestationPolicy"] => issue(type="osKernelDebuggingDisabled", value=false);` |

+|Data Execution Prevention (DEP) policy | DEP policy is a set of hardware and software technologies that perform extra checks on memory to help prevent malicious code from running on a system. | `// DEP Policy`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => issue(type="depPolicy", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_DATAEXECUTIONPREVENTION.Value \| @[-1]")));`<br>`![type=="depPolicy"] => issue(type="depPolicy", value=0);` |

+| Test and flight signing | Enables the user to run test-signed code. | `// Test Signing `<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>` c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="testSigningEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_TESTSIGNING")));`<br>` c:[type=="testSigningEnabledSet", issuer=="AttestationPolicy"] => issue(type="testSigningDisabled", value=ContainsOnlyValue(c.value, false));`<br>` ![type=="testSigningDisabled", issuer=="AttestationPolicy"] => issue(type="testSigningDisabled", value=false);`<br>`//Flight Signingc:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="flightSigningEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "\[\*\].EVENT_FLIGHTSIGNING")));`<br>`c:[type=="flightSigningEnabledSet", issuer=="AttestationPolicy"] => issue(type="flightSigningNotEnabled", value=ContainsOnlyValue(c.value, false));`<br>`![type=="flightSigningNotEnabled", issuer=="AttestationPolicy"] => issue(type="flightSigningNotEnabled", value=false);` |

+| Virtual Secure Mode/VBS | VBS uses the Windows hypervisor to create this virtual secure mode that's used to protect vital system and operating system resources and credentials. | `// VSM enabled `<br>` c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="vsmEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "\[\*\].EVENT_VBS_VSM_REQUIRED")));`<br>`c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="vsmEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_VBS_MANDATORY_ENFORCEMENT")));`<br>`c:[type=="vsmEnabledSet", issuer=="AttestationPolicy"] => issue(type="vsmEnabled", value=ContainsOnlyValue(c.value, true));`<br>`![type=="vsmEnabled", issuer=="AttestationPolicy"] => issue(type="vsmEnabled", value=false);`<br>`c:[type=="vsmEnabled", issuer=="AttestationPolicy"] => issue(type="vbsEnabled", value=c.value);` |

+| Hypervisor-protected code integrity (HVCI) | HVCI is a feature that validates the integrity of a system file each time it's loaded into memory.| `// HVCI`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="hvciEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_VBS_HVCI_POLICY \| @[?String == 'HypervisorEnforcedCodeIntegrityEnable'].Value")));`<br>`c:[type=="hvciEnabledSet", issuer=="AttestationPolicy"] => issue(type="hvciEnabled", value=ContainsOnlyValue(c.value, 1));`<br>`![type=="hvciEnabled", issuer=="AttestationPolicy"] => issue(type="hvciEnabled", value=false);` |

+| Input-output memory management unit (IOMMU) | IOMMU translates virtual to physical memory addresses for Direct memory access-capable device peripherals. IOMMU protects sensitive memory regions. | `// IOMMU`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="iommuEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_VBS_IOMMU_REQUIRED")));`<br>`c:[type=="iommuEnabledSet", issuer=="AttestationPolicy"] => issue(type="iommuEnabled", value=ContainsOnlyValue(c.value, true));`<br>`![type=="iommuEnabled", issuer=="AttestationPolicy"] => issue(type="iommuEnabled", value=false);` |

+| PCR value evaluation | PCRs contain measurements of components that are made during the boot. These measurements can be used to verify the components against golden or known measurements. | `//PCRS are only read-only and thus cannot be used with issue operation, but they can be used to validate expected/golden measurements.`<br>`c:[type == "pcrs", issuer=="AttestationService"] && c1:[type=="pcrMatchesExpectedValue", value==JsonToClaimValue(JmesPath(c.value, "PCRs[? Index == `0`].Digests.SHA1 \| @[0] == `\"KCk6Ow\"`"))] => issue(claim = c1);` |

+| Boot Manager version | The security version number of the Boot Manager that was loaded during initial boot on the attested device. | `// Find the first EVENT_APPLICATION_SVN. That value is the Boot Manager SVN`<br>`// Find the first EV_SEPARATOR in PCR 12, 13, Or 14`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="evSeparatorSeq", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_SEPARATOR' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `14`)] \| @[0].EventSeq"));`<br>`c:[type=="evSeparatorSeq", value != "null", issuer=="AttestationPolicy"] => add(type="beforeEvSepClause", value=AppendString(AppendString("Events[? EventSeq < `", c.value), "`"));`<br>`[type=="evSeparatorSeq", value=="null", issuer=="AttestationPolicy"] => add(type="beforeEvSepClause", value="Events[? `true` ");`<br>`// Find the first EVENT_APPLICATION_SVN. That value is the Boot Manager SVNc:[type=="beforeEvSepClause", issuer=="AttestationPolicy"] => add(type="bootMgrSvnSeqQuery", value=AppendString(c.value, " && EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `12` && ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_APPLICATION_SVN] \| @[0].EventSeq"));`<br>`c1:[type=="bootMgrSvnSeqQuery", issuer=="AttestationPolicy"] && c2:[type=="events", issuer=="AttestationService"] => add(type="bootMgrSvnSeq", value=JmesPath(c2.value, c1.value));`<br>`c:[type=="bootMgrSvnSeq", value!="null", issuer=="AttestationPolicy"] => add(type="bootMgrSvnQuery", value=AppendString(AppendString("Events[? EventSeq == `", c.value), "`].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_APPLICATION_SVN \| @[0]"));`<br>`c1:[type=="bootMgrSvnQuery", issuer=="AttestationPolicy"] && c2:[type=="events", issuer=="AttestationService"] => issue(type="bootMgrSvn", value=JsonToClaimValue(JmesPath(c2.value, c1.value)));` |

+| Safe mode | Safe mode is a troubleshooting option for Windows that starts your computer in a limited state. Only the basic files and drivers necessary to run Windows are started. | `// Safe mode`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="safeModeEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_SAFEMODE")));`<br>`c:[type=="safeModeEnabledSet", issuer=="AttestationPolicy"] => issue(type="notSafeMode", value=ContainsOnlyValue(c.value, false));`<br>`![type=="notSafeMode", issuer=="AttestationPolicy"] => issue(type="notSafeMode", value=true);` |

+| WinPE boot | Windows pre-installation Environment (Windows PE) is a minimal operating system with limited services that's used to prepare a computer for Windows installation, to copy disk images from a network file server, and to initiate Windows setup. | `// Win PE`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `19` \|\| PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));`<br>`c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="winPEEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "\[\*\].EVENT_WINPE")));`<br>`c:[type=="winPEEnabledSet", issuer=="AttestationPolicy"] => issue(type="notWinPE", value=ContainsOnlyValue(c.value, false));`<br>`![type=="notWinPE", issuer=="AttestationPolicy"] => issue(type="notWinPE", value=true);` |

+| Code integrity (CI) policy | Hash of CI policy that's controlling the security of the boot environment. | `// CI Policy`<br>`c :[type=="events", issuer=="AttestationService"] => issue(type="codeIntegrityPolicy", value=JsonToClaimValue(JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `13`].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_SI_POLICY[].RawData")));`|

+| Secure Boot Configuration Policy Hash (SBCPHash) | SBCPHash is the fingerprint of the Custom SBCP that was loaded during boot in Windows devices, except PCs. | `// Secure Boot Custom Policy`<br>`c:[type=="events", issuer=="AttestationService"] => issue(type="secureBootCustomPolicy", value=JsonToClaimValue(JmesPath(c.value, "Events[? EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG' && PcrIndex == `7` && ProcessedData.UnicodeName == 'CurrentPolicy' && ProcessedData.VariableGuid == '77FA9ABD-0359-4D32-BD60-28F4E78F784B'].ProcessedData.VariableData \| @[0]")));` |

+| System Guard (DRTM Validation and SMM Levels) | Ensure System Guard has been validated during boot and corresponding System Management Mode Level | ` // Extract the DRTM state auth event. `<br>`// The rule attempts to find the valid DRTM state auth event by applying following conditions:`<br>`// 1. There is only one DRTM state auth event in the events log`<br>`// 2. The EVENT_DRTM_STATE_AUTH.SignatureValid field in the DRTM state auth event is set to true`<br><br>` c:[type=="events", issuer=="AttestationService"] => add(type="validDrtmStateAuthEvent", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `20` && ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid != `null`] \| length(@) == `1` && @[0] \| @.{EventSeq:EventSeq, SignatureValid:ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid}"));`<br><br>` // Check if Signature is valid in extracted state auth events`<br>`c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=JsonToClaimValue(JmesPath(c.value, "SignatureValid")));`<br>`![type=="drtmMleValid", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=false);`<br><br>`// Get the sequence number of the DRTM state auth event.`<br>`// The sequence number is used to ensure that the SMM event appears before the last DRTM state auth event.`<br>`[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] && c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => add(type="validDrtmStateAuthEventSeq", value=JmesPath(c.value, "EventSeq"));`<br><br>` // Create query for SMM event`<br>`// The query is constructed to find the SMM level from the SMM level event that appears exactly once before the valid DRTM state auth event in the event log`<br>`[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] && c:[type=="validDrtmStateAuthEventSeq", issuer=="AttestationPolicy"] => add(type="smmQuery", value=AppendString(AppendString("Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `20` && EventSeq <`", c.value), "`].ProcessedData.EVENT_DRTM_SMM \| length(@) == `1` && @[0] \| @.Value"));`<br><br>`// Extract SMM value`<br>`[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] &&`<br>` c1:[type=="smmQuery", issuer=="AttestationPolicy"] &&`<br>` c2:[type=="events", issuer=="AttestationService"] => issue(type="smmLevel", value=JsonToClaimValue(JmesPath(c2.value, c1.value)));`<br>` ` |

+| Boot application subversion | The version of the Boot Manager that's running on the device. | `// Find the first EV_SEPARATOR in PCR 12, 13, Or 14, the ordering of the events is critical to ensure correctness.`<br>`c:[type=="events", issuer=="AttestationService"] => add(type="evSeparatorSeq", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_SEPARATOR' && (PcrIndex == `12` \|\| PcrIndex == `13` \|\| PcrIndex == `14`)] \| @[0].EventSeq"));`<br>`c:[type=="evSeparatorSeq", value != "null", issuer=="AttestationPolicy"] => add(type="beforeEvSepClause", value=AppendString(AppendString("Events[? EventSeq < `", c.value), "`"));`<br>`[type=="evSeparatorSeq", value=="null", issuer=="AttestationPolicy"] => add(type="beforeEvSepClause", value="Events[? `true` ");`<br>` // No restriction of EV_SEPARATOR in case it is not present// Find the first EVENT_TRANSFER_CONTROL with value 1 or 2 in PCR 12 that is before the EV_SEPARATORc1:[type=="beforeEvSepClause", issuer=="AttestationPolicy"] && c2:[type=="bootMgrSvnSeq", value != "null", issuer=="AttestationPolicy"] => add(type="beforeEvSepAfterBootMgrSvnClause", value=AppendString(AppendString(AppendString(c1.value, "&& EventSeq >= `"), c2.value), "`"));`<br>`c:[type=="beforeEvSepAfterBootMgrSvnClause", issuer=="AttestationPolicy"] => add(type="tranferControlQuery", value=AppendString(c.value, " && EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `12`&& (ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_TRANSFER_CONTROL.Value == `1` \|\| ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_TRANSFER_CONTROL.Value == `2`)] \| @[0].EventSeq"));`<br>`c1:[type=="tranferControlQuery", issuer=="AttestationPolicy"] && c2:[type=="events", issuer=="AttestationService"] => add(type="tranferControlSeq", value=JmesPath(c2.value, c1.value));`<br>`// Find the first non-null EVENT_MODULE_SVN in PCR 13 after the transfer control.c:[type=="tranferControlSeq", value!="null", issuer=="AttestationPolicy"] => add(type="afterTransferCtrlClause", value=AppendString(AppendString(" && EventSeq > `", c.value), "`"));`<br>`c1:[type=="beforeEvSepClause", issuer=="AttestationPolicy"] && c2:[type=="afterTransferCtrlClause", issuer=="AttestationPolicy"] => add(type="moduleQuery", value=AppendString(AppendString(c1.value, c2.value), " && EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `13` && ((ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_LOADEDMODULE_AGGREGATION[].EVENT_MODULE_SVN \| @[0]) \|\| (ProcessedData.EVENT_LOADEDMODULE_AGGREGATION[].EVENT_MODULE_SVN \| @[0]))].EventSeq \| @[0]"));`<br>`c1:[type=="moduleQuery", issuer=="AttestationPolicy"] && c2:[type=="events", issuer=="AttestationService"] => add(type="moduleSeq", value=JmesPath(c2.value, c1.value));`<br>`// Find the first EVENT_APPLICATION_SVN after EV_EVENT_TAG in PCR 12. That value is Boot App SVNc:[type=="moduleSeq", value!="null", issuer=="AttestationPolicy"] => add(type="applicationSvnAfterModuleClause", value=AppendString(AppendString(" && EventSeq > `", c.value), "`"));`<br>`c1:[type=="beforeEvSepClause", issuer=="AttestationPolicy"] && c2:[type=="applicationSvnAfterModuleClause", issuer=="AttestationPolicy"] => add(type="bootAppSvnQuery", value=AppendString(AppendString(c1.value, c2.value), " && EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `12`].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_APPLICATION_SVN \| @[0]"));`<br>`c1:[type=="bootAppSvnQuery", issuer=="AttestationPolicy"] && c2:[type=="events", issuer=="AttestationService"] => issue(type="bootAppSvn", value=JsonToClaimValue(JmesPath(c2.value, c1.value)));` |

+| Boot revision list | Boot revision list used to direct the device to an enterprise honeypot to further monitor the device's activities. | `// Boot Rev List Info `<br>`c:[type=="events", issuer=="AttestationService"] => issue(type="bootRevListInfo", value=JsonToClaimValue(JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `13`].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_BOOT_REVOCATION_LIST.RawData \| @[0]")));` |

+[type=="aikValidated", value==true] &&

+};

+authorizationrules {

+c:[type=="efiConfigVariables", issuer=="AttestationPolicy"]=> add(type = "secureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] | length(@) == `1` && @[0].ProcessedData.VariableData == 'AQ'")));

+c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` || PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY"));

+c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` || PcrIndex == `13` || PcrIndex == `19` || PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));

+c:[type=="boolProperties", issuer=="AttestationPolicy"] => issue(type="MaliciousDriverLoaded", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_LOADEDMODULE_AGGREGATION[] | [? EVENT_IMAGEVALIDATED == true && (equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\malicious.sys') || equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\wd\\malicious.sys'))] | @ != null ")));

+## Extending the protection from malicious boot attacks via Integrity Measurement Architecture(IMA) on Linux

+Linux systems follow a similar boot process to Windows, and with TPM attestation the protection profile can be extended to beyond boot into the kernel as well using Integrity Measurement Architecture(IMA). IMA subsystem was designed to detect if files have been accidentally or maliciously altered, both remotely and locally, it maintains a runtime measurement list and, if anchored in a hardware Trusted Platform Module(TPM), an aggregate integrity value over this list provides the benefit of resiliency from software attacks. Recent enhancements in the IMA subsystem also allow for non file based attributes to be measured and attested remotely. Azure attestation supports non file based measurements to be attested remotely to provide a holistic view of system integrity.

+Enabling IMA with the following ima-policy will enable measurement of non file attributes while still enabling local file integrity attestation.

+Using the following Attestation policy, you can now validate the secureboot, kernel signature, kernel version, kernel cmdline passed in by grub and other key security attributes supported by IMA.

+```

+version = 1.2;

+configurationrules

+{

+};

+authorizationrules

+{

+ [type == "aikValidated", value==true]

+ => permit();

+};

+issuancerules {

+ // Retrieve all EFI Boot variables with event = 'EV_EFI_VARIABLE_BOOT'

+ c:[type == "events", issuer=="AttestationService"] => add(type ="efiBootVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_BOOT']"));

+ // Retrieve all EFI Driver Config variables with event = 'EV_EFI_VARIABLE_DRIVER_CONFIG'

+ c:[type == "events", issuer=="AttestationService"] => add(type ="efiConfigVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG']"));

+ // Grab all IMA events

+ c:[type=="events", issuer=="AttestationService"] => add(type="imaMeasurementEvents", value=JmesPath(c.value, "Events[?EventTypeString == 'IMA_MEASUREMENT_EVENT']"));

+ // Look for "Boot Order" from EFI Boot Data

+ c:[type == "efiBootVariables", issuer=="AttestationPolicy"] => add(type = "bootOrderFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'BootOrder'] | length(@) == `1` && @[0].PcrIndex == `1` && @[0].ProcessedData.VariableData"));

+ c:[type=="bootOrderFound", issuer=="AttestationPolicy"] => issue(type="bootOrder", value=JsonToClaimValue(c.value));

+ ![type=="bootOrderFound", issuer=="AttestationPolicy"] => issue(type="bootOrder", value=0);

+ // Look for "Secure Boot" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => issue(type = "secureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData == 'AQ'")));

+ ![type=="secureBootEnabled", issuer=="AttestationPolicy"] => issue(type="secureBootEnabled", value=false);

+ // Look for "Platform Key" from EFI Boot Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "platformKeyFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'PK'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="platformKeyFound", issuer=="AttestationPolicy"] => issue(type="platformKey", value=JsonToClaimValue(c.value));

+ ![type=="platformKeyFound", issuer=="AttestationPolicy"] => issue(type="platformKey", value=0);

+

+ // Look for "Key Exchange Key" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "keyExchangeKeyFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'KEK'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="keyExchangeKeyFound", issuer=="AttestationPolicy"] => issue(type="keyExchangeKey", value=JsonToClaimValue(c.value));

+ ![type=="keyExchangeKeyFound", issuer=="AttestationPolicy"] => issue(type="keyExchangeKey", value=0);

+ // Look for "Key Database" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "keyDatabaseFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'db'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="keyDatabaseFound", issuer=="AttestationPolicy"] => issue(type="keyDatabase", value=JsonToClaimValue(c.value));

+ ![type=="keyDatabaseFound", issuer=="AttestationPolicy"] => issue(type="keyDatabase", value=0);

+ // Look for "Forbidden Signatures" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "forbiddenSignaturesFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'dbx'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="forbiddenSignaturesFound", issuer=="AttestationPolicy"] => issue(type="forbiddenSignatures", value=JsonToClaimValue(c.value));

+ ![type=="forbiddenSignaturesFound", issuer=="AttestationPolicy"] => issue(type="forbiddenSignatures", value=0);

+ // Look for "Kernel Version" in IMA Measurement events

+ c:[type=="imaMeasurementEvents", issuer=="AttestationPolicy"] => add(type="kernelVersionsFound", value=JmesPath(c.value, "[].ProcessedData.KernelVersion"));

+ c:[type=="kernelVersionsFound", issuer=="AttestationPolicy"] => issue(type="kernelVersions", value=JsonToClaimValue(c.value));

+ ![type=="kernelVersionsFound", issuer=="AttestationPolicy"] => issue(type="kernelVersions", value=0);

+ // Look for "Built-In Trusted Keys" in IMA Measurement events

+ c:[type=="imaMeasurementEvents", issuer=="AttestationPolicy"] => add(type="builtintrustedkeysFound", value=JmesPath(c.value, "[? ProcessedData.Keyring == '.builtin_trusted_keys'].ProcessedData.CertificateSubject"));

+ c:[type=="builtintrustedkeysFound", issuer=="AttestationPolicy"] => issue(type="builtintrustedkeys", value=JsonToClaimValue(c.value));

+ ![type=="builtintrustedkeysFound", issuer=="AttestationPolicy"] => issue(type="builtintrustedkeys", value=0);

+};

+```

+Note: Support for non-file based measurements are only available from linux kernel version: 5.15

+## TPM Key attestation support

+Numerous applications rely on foundational credential management of keys and certs for protections against credential theft, and one of main ways of ensuring the credential security is the reliance of key storage providers that provide additional security from malware and attacks. Windows implements various cryptographic providers that can be either software or hardware based.

+The two most important ones are:

+* Microsoft Software Key Storage Provider: Standard provider, which stores keys software based and supports CNG (Crypto-Next Generation)

+* Microsoft Platform Crypto Provider: Hardware based which stores keys on a TPM (trusted platform module) and supports CNG as well

+Whenever a Storage provider is used, itΓÇÖs usually to create a pub/priv key pair that is chained to a root of trust. At creation more properties can also be used to enable certain aspects of the key storage, exportability, etc. Key attestation in this context, is the technical ability to prove to a replying party that a private key was generated inside, and is managed inside, and in a not exportable form. Such attestation clubbed with other information can help protect from credential theft and replay type of attack.

+TPMs also provide the capability ability to attest that keys are resident in a TPM, enabling higher security assurance, backed up by non-exportability, anti-hammering, and isolation of keys. A common use case is for applications that issue digital signature certificate for subscriber keys, verifying that the subscribers private signature key is generated and managed in an approved TPM.

+One can easily attest to the fact the keys are resident in a valid TPM with appropriate Nonexportability flags using a policy as below.

+```

+version=1.2;

+authorizationrules

+{

+ => permit();

+};

+issuancerules

+{

+ // Key Attest Policy

+ // -- Validating key types

+ c:[type=="x-ms-tpm-request-key", issuer=="AttestationService"] => add(type="requestKeyType", value=JsonToClaimValue(JmesPath(c.value, "jwk.kty")));

+ c:[type=="x-ms-tpm-other-keys", issuer=="AttestationService"] => add(type="otherKeysTypes", value=JsonToClaimValue(JmesPath(c.value, "[*].jwk.kty")));

+ c:[type=="requestKeyType", issuer=="AttestationPolicy", value=="RSA"] => issue(type="requestKeyType", value="RSA");

+ c:[type=="otherKeysTypes", issuer=="AttestationPolicy", value=="RSA"] => issue(type="otherKeysTypes", value="RSA");

+ // -- Validating tpm_quote attributes

+ c:[type=="x-ms-tpm-request-key", issuer=="AttestationService"] => add(type="requestKeyQuote", value=JmesPath(c.value, "info.tpm_quote"));

+ c:[type=="requestKeyQuote", issuer=="AttestationPolicy"] => add(type="requestKeyQuoteHashAlg", value=JsonToClaimValue(JmesPath(c.value, "hash_alg")));

+ c:[type=="requestKeyQuoteHashAlg", issuer=="AttestationPolicy", value=="sha-256"] => issue(type="requestKeyQuoteHashAlg", value="sha-256");

+ // -- Validating tpm_certify attributes

+ c:[type=="x-ms-tpm-request-key", issuer=="AttestationService"] => add(type="requestKeyCertify", value=JmesPath(c.value, "info.tpm_certify"));

+ c:[type=="requestKeyCertify", issuer=="AttestationPolicy"] => add(type="requestKeyCertifyNameAlg", value=JsonToClaimValue(JmesPath(c.value, "name_alg")));

+ c:[type=="requestKeyCertifyNameAlg", issuer=="AttestationPolicy", value==11] => issue(type="requestKeyCertifyNameAlg", value=11);

+ c:[type=="requestKeyCertify", issuer=="AttestationPolicy"] => add(type="requestKeyCertifyObjAttr", value=JsonToClaimValue(JmesPath(c.value, "obj_attr")));

+ c:[type=="requestKeyCertifyObjAttr", issuer=="AttestationPolicy", value==50] => issue(type="requestKeyCertifyObjAttr", value=50);

+ c:[type=="requestKeyCertify", issuer=="AttestationPolicy"] => add(type="requestKeyCertifyAuthPolicy", value=JsonToClaimValue(JmesPath(c.value, "auth_policy")));

+ c:[type=="requestKeyCertifyAuthPolicy", issuer=="AttestationPolicy", value=="AQIDBA"] => issue(type="requestKeyCertifyAuthPolicy", value="AQIDBA");

+ c:[type=="x-ms-tpm-other-keys", issuer=="AttestationService"] => add(type="otherKeysCertify", value=JmesPath(c.value, "[*].info.tpm_certify"));

+ c:[type=="otherKeysCertify", issuer=="AttestationPolicy"] => add(type="otherKeysCertifyNameAlgs", value=JsonToClaimValue(JmesPath(c.value, "[*].name_alg")));

+ c:[type=="otherKeysCertifyNameAlgs", issuer=="AttestationPolicy", value==11] => issue(type="otherKeysCertifyNameAlgs", value=11);

+ c:[type=="otherKeysCertify", issuer=="AttestationPolicy"] => add(type="otherKeysCertifyObjAttr", value=JsonToClaimValue(JmesPath(c.value, "[*].obj_attr")));

+ c:[type=="otherKeysCertifyObjAttr", issuer=="AttestationPolicy", value==50] => issue(type="otherKeysCertifyObjAttr", value=50);

+ c:[type=="otherKeysCertify", issuer=="AttestationPolicy"] => add(type="otherKeysCertifyAuthPolicy", value=JsonToClaimValue(JmesPath(c.value, "[*].auth_policy")));

+ c:[type=="otherKeysCertifyAuthPolicy", issuer=="AttestationPolicy", value=="AQIDBA"] => issue(type="otherKeysCertifyAuthPolicy", value="AQIDBA");

+};

+```

+- [Try out TPM attestation](azure-tpm-vbs-attestation-usage.md)

+ Title: Examples of an Azure TPM Attestation policy

+description: Examples of Azure Attestation policy for TPM endpoint.

+# Examples of an attestation policy for TPM endpoint

+Attestation policy is used to process the attestation evidence and determine whether Azure Attestation will issue an attestation token. Attestation token generation can be controlled with custom policies. Below are some examples of an attestation policy.

+## Sample policy for TPM using Policy version 1.0

+```

+version=1.0;

+authorizationrules {

+ => permit();

+};

+issuancerules

+{

+[type=="aikValidated", value==true]&&

+[type=="secureBootEnabled", value==true] &&

+[type=="bootDebuggingDisabled", value==true] &&

+[type=="vbsEnabled", value==true] &&

+[type=="notWinPE", value==true] &&

+[type=="notSafeMode", value==true] => issue(type="PlatformAttested", value=true);

+};

+```

+A simple TPM attestation policy that can be used to verify minimal aspects of the boot.

+## Sample policy for TPM using Policy version 1.2

+The policy uses the TPM version to restrict attestation calls. The issuancerules looks at various properties measured during boot.

+```

+version=1.2;

+configurationrules{

+};

+authorizationrules {

+ => permit();

+};

+issuancerules{

+c:[type == "aikValidated", issuer=="AttestationService"] =>issue(type="aikValidated", value=c.value);

+// SecureBoot enabled

+c:[type == "events", issuer=="AttestationService"] => add(type = "efiConfigVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG' && ProcessedData.VariableGuid == '8BE4DF61-93CA-11D2-AA0D-00E098032B8C']"));

+c:[type == "efiConfigVariables", issuer=="AttestationPolicy"]=> issue(type = "SecureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] | length(@) == `1` && @[0].ProcessedData.VariableData == 'AQ'")));

+![type=="SecureBootEnabled", issuer=="AttestationPolicy"] => issue(type="SecureBootEnabled", value=false);

+// Retrieve bool properties Code integrity

+c:[type=="events", issuer=="AttestationService"] => add(type="boolProperties", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` || PcrIndex == `13` || PcrIndex == `19` || PcrIndex == `20`)].ProcessedData.EVENT_TRUSTBOUNDARY"));

+c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="codeIntegrityEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_CODEINTEGRITY")));

+c:[type=="codeIntegrityEnabledSet", issuer=="AttestationPolicy"] => issue(type="CodeIntegrityEnabled", value=ContainsOnlyValue(c.value, true));

+![type=="CodeIntegrityEnabled", issuer=="AttestationPolicy"] => issue(type="CodeIntegrityEnabled", value=false);

+// Bitlocker Boot Status, The first non zero measurement or zero.

+c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` || PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY"));

+c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="BitlockerStatus", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_BITLOCKER_UNLOCK | @[? Value != `0`].Value | @[0]")));

+[type=="BitlockerStatus", issuer=="AttestationPolicy"] => issue(type="BitlockerStatus", value=true);

+![type=="BitlockerStatus", issuer=="AttestationPolicy"] => issue(type="BitlockerStatus", value=false);

+// Elam Driver (windows defender) Loaded

+c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="elamDriverLoaded", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_LOADEDMODULE_AGGREGATION[] | [? EVENT_IMAGEVALIDATED == `true` && (equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\wdboot.sys') || equals_ignore_case(EVENT_FILEPATH, '\\windows\\system32\\drivers\\wd\\wdboot.sys'))] | @ != `null`")));

+[type=="elamDriverLoaded", issuer=="AttestationPolicy"] => issue(type="ELAMDriverLoaded", value=true);

+![type=="elamDriverLoaded", issuer=="AttestationPolicy"] => issue(type="ELAMDriverLoaded", value=false);

+};

+```

+### Attestation policy to authorize only those TPMs that match known PCR hashes.

+```

+version=1.2;

+authorizationrules

+{

+ c:[type == "pcrs", issuer=="AttestationService"] => add(type="pcr0Validated", value=JsonToClaimValue(JmesPath(c.value, "PCRs[? Index == `0`].Digests.SHA256 | @[0] =='4c833b1c361fceffd8dc0f81eec76081b71e1a0eb2193caed0b6e1c7735ec33e' ")));

+ c:[type == "pcrs", issuer=="AttestationService"] => add(type="pcr1Validated", value=JsonToClaimValue(JmesPath(c.value, "PCRs[? Index == `1`].Digests.SHA256 | @[0] =='8c25e3be6ad6f5bd33c9ae40d5d5461e88c1a7366df0c9ee5c7e5ff40d3d1d0e' ")));

+ c:[type == "pcrs", issuer=="AttestationService"] => add(type="pcr2Validated", value=JsonToClaimValue(JmesPath(c.value, "PCRs[? Index == `2`].Digests.SHA256 | @[0] =='3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969' ")));

+ c:[type == "pcrs", issuer=="AttestationService"] => add(type="pcr3Validated", value=JsonToClaimValue(JmesPath(c.value, "PCRs[? Index == `3`].Digests.SHA256 | @[0] =='3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969' ")));

+

+ [type=="pcr0Validated", value==true] &&

+ [type=="pcr1Validated", value==true] &&

+ [type=="pcr2Validated", value==true] &&

+ [type=="pcr3Validated", value==true] => permit();

+};

+issuancerules

+{

+};

+```

+### Attestation policy to validate System Guard is enabled as expected and has been validated for its state.

+```

+version = 1.2;

+authorizationrules

+{

+ => permit();

+};

+issuancerules

+{

+ // Extract the DRTM state auth event

+ // The rule attempts to find the valid DRTM state auth event by applying following conditions:

+ // 1. There is only one DRTM state auth event in the events log

+ // 2. The EVENT_DRTM_STATE_AUTH.SignatureValid field in the DRTM state auth event is set to true

+ c:[type=="events", issuer=="AttestationService"] => add(type="validDrtmStateAuthEvent", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `20` && ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid != `null`] | length(@) == `1` && @[0] | @.{EventSeq:EventSeq, SignatureValid:ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid}"));

+ // Check if Signature is valid in extracted state auth events

+ c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=JsonToClaimValue(JmesPath(c.value, "SignatureValid")));

+ ![type=="drtmMleValid", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=false);

+ // Get the sequence number of the DRTM state auth event.

+ // The sequence number is used to ensure that the SMM event appears before the last DRTM state auth event.

+ [type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] &&

+ c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => add(type="validDrtmStateAuthEventSeq", value=JmesPath(c.value, "EventSeq"));

+ // Create query for SMM event

+ // The query is constructed to find the SMM level from the SMM level event that appears exactly once before the valid DRTM state auth event in the event log

+ [type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] &&

+ c:[type=="validDrtmStateAuthEventSeq", issuer=="AttestationPolicy"] => add(type="smmQuery", value=AppendString(AppendString("Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == `20` && EventSeq < `", c.value), "`].ProcessedData.EVENT_DRTM_SMM | length(@) == `1` && @[0] | @.Value"));

+ // Extract SMM value

+ [type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] &&

+ c1:[type=="smmQuery", issuer=="AttestationPolicy"] &&

+ c2:[type=="events", issuer=="AttestationService"] => issue(type="smmLevel", value=JsonToClaimValue(JmesPath(c2.value, c1.value)));

+};

+```

+### Attestation policy to validate VBS enclave for its validity and identity.

+```

+version=1.2;

+authorizationrules {

+ [type=="vsmReportPresent", value==true] &&

+ [type=="enclaveAuthorId", value=="AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"] &&

+ [type=="enclaveImageId", value=="AQEAAAAAAAAAAAAAAAAAAA"] &&

+ [type=="enclaveSvn", value>=0] => permit();

+};

+issuancerules

+{

+};

+```

+### Attestation policy to issue all incoming claims produced by the service.

+```

+version = 1.2;

+configurationrules

+{

+};

+authorizationrules

+{

+ => permit();

+};

+issuancerules

+{

+ c:[type=="bootEvents", issuer=="AttestationService"] => issue(type="outputBootEvents", value=c.value);

+ c:[type=="events", issuer=="AttestationService"] => issue(type="outputEvents", value=c.value);

+};

+```

+### Attestation policy to produce some critical security evaluation claims for Windows.

+```

+version=1.2;

+authorizationrules {

+ => permit();

+};

+issuancerules{

+// SecureBoot enabled

+c:[type == "events", issuer=="AttestationService"] => add(type = "efiConfigVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG' && ProcessedData.VariableGuid == '8BE4DF61-93CA-11D2-AA0D-00E098032B8C']")); c:[type == "efiConfigVariables", issuer=="AttestationPolicy"]=> issue(type = "secureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] | length(@) == `1` && @[0].ProcessedData.VariableData == 'AQ'"))); ![type=="secureBootEnabled", issuer=="AttestationPolicy"] => issue(type="secureBootEnabled", value=false);

+// Boot debugging

+c:[type=="boolProperties", issuer=="AttestationPolicy"] => add(type="bootDebuggingEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_BOOTDEBUGGING"))); c:[type=="bootDebuggingEnabledSet", issuer=="AttestationPolicy"] => issue(type="bootDebuggingDisabled", value=ContainsOnlyValue(c.value, false)); ![type=="bootDebuggingDisabled", issuer=="AttestationPolicy"] => issue(type="bootDebuggingDisabled", value=false);

+// Virtualization Based Security enabled

+c:[type=="events", issuer=="AttestationService"] => add(type="srtmDrtmEventPcr", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == `12` || PcrIndex == `19`)].ProcessedData.EVENT_TRUSTBOUNDARY")); c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="vbsEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_VBS_VSM_REQUIRED"))); c:[type=="srtmDrtmEventPcr", issuer=="AttestationPolicy"] => add(type="vbsEnabledSet", value=JsonToClaimValue(JmesPath(c.value, "[*].EVENT_VBS_MANDATORY_ENFORCEMENT"))); c:[type=="vbsEnabledSet", issuer=="AttestationPolicy"] => issue(type="vbsEnabled", value=ContainsOnlyValue(c.value, true)); ![type=="vbsEnabled", issuer=="AttestationPolicy"] => issue(type="vbsEnabled", value=false); c:[type=="vbsEnabled", issuer=="AttestationPolicy"] => issue(type="vbsEnabled", value=c.value);

+// System Guard and SMM value

+c:[type=="events", issuer=="AttestationService"] => add(type="validDrtmStateAuthEvent", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == '20' && ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid !=null] | length(@) == '1' && @[0] | @.{EventSeq:EventSeq, SignatureValid:ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRTM_STATE_AUTH.SignatureValid}"));

+// Check if Signature is valid in extracted state auth events

+c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=JsonToClaimValue(JmesPath(c.value, "SignatureValid")));

+![type=="drtmMleValid", issuer=="AttestationPolicy"] => issue(type="drtmMleValid", value=false);

+// Get the sequence number of the DRTM state auth event.

+// The sequence number is used to ensure that the SMM event appears before the last DRTM state auth event.

+[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] && c:[type=="validDrtmStateAuthEvent", issuer=="AttestationPolicy"] => add(type="validDrtmStateAuthEventSeq", value=JmesPath(c.value, "EventSeq"));

+// Create query for SMM event

+// The query is constructed to find the SMM level from the SMM level event that appears exactly once before the valid DRTM state auth event in the event log

+[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] && c:[type=="validDrtmStateAuthEventSeq", issuer=="AttestationPolicy"] => add(type="smmQuery", value=AppendString(AppendString("Events[? EventTypeString == 'EV_EVENT_TAG' && PcrIndex == 20 && EventSeq <", c.value), "].ProcessedData.EVENT_DRTM_SMM | length(@) == 1 && @[0] | @.Value"));

+// Extract SMM value

+[type=="drtmMleValid", value==true, issuer=="AttestationPolicy"] &&

+c1:[type=="smmQuery", issuer=="AttestationPolicy"] &&

+c2:[type=="events", issuer=="AttestationService"] => issue(type="smmLevel", value=JsonToClaimValue(JmesPath(c2.value, c1.value)));

+};

+```

+### Attestation policy to validate boot related firmware and early boot driver signers on linux

+```

+version = 1.2;

+configurationrules

+{

+};

+authorizationrules

+{

+ [type == "aikValidated", value==true]

+ => permit();

+};

+issuancerules {

+ // Retrieve all EFI Boot variables with event = 'EV_EFI_VARIABLE_BOOT'

+ c:[type == "events", issuer=="AttestationService"] => add(type ="efiBootVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_BOOT']"));

+ // Retrieve all EFI Driver Config variables with event = 'EV_EFI_VARIABLE_DRIVER_CONFIG'

+ c:[type == "events", issuer=="AttestationService"] => add(type ="efiConfigVariables", value = JmesPath(c.value, "Events[?EventTypeString == 'EV_EFI_VARIABLE_DRIVER_CONFIG']"));

+ // Grab all IMA events

+ c:[type=="events", issuer=="AttestationService"] => add(type="imaMeasurementEvents", value=JmesPath(c.value, "Events[?EventTypeString == 'IMA_MEASUREMENT_EVENT']"));

+ // Look for "Boot Order" from EFI Boot Data

+ c:[type == "efiBootVariables", issuer=="AttestationPolicy"] => add(type = "bootOrderFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'BootOrder'] | length(@) == `1` && @[0].PcrIndex == `1` && @[0].ProcessedData.VariableData"));

+ c:[type=="bootOrderFound", issuer=="AttestationPolicy"] => issue(type="bootOrder", value=JsonToClaimValue(c.value));

+ ![type=="bootOrderFound", issuer=="AttestationPolicy"] => issue(type="bootOrder", value=0);

+ // Look for "Secure Boot" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => issue(type = "secureBootEnabled", value = JsonToClaimValue(JmesPath(c.value, "[?ProcessedData.UnicodeName == 'SecureBoot'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData == 'AQ'")));

+ ![type=="secureBootEnabled", issuer=="AttestationPolicy"] => issue(type="secureBootEnabled", value=false);

+ // Look for "Platform Key" from EFI Boot Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "platformKeyFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'PK'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="platformKeyFound", issuer=="AttestationPolicy"] => issue(type="platformKey", value=JsonToClaimValue(c.value));

+ ![type=="platformKeyFound", issuer=="AttestationPolicy"] => issue(type="platformKey", value=0);

+

+ // Look for "Key Exchange Key" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "keyExchangeKeyFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'KEK'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="keyExchangeKeyFound", issuer=="AttestationPolicy"] => issue(type="keyExchangeKey", value=JsonToClaimValue(c.value));

+ ![type=="keyExchangeKeyFound", issuer=="AttestationPolicy"] => issue(type="keyExchangeKey", value=0);

+ // Look for "Key Database" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "keyDatabaseFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'db'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="keyDatabaseFound", issuer=="AttestationPolicy"] => issue(type="keyDatabase", value=JsonToClaimValue(c.value));

+ ![type=="keyDatabaseFound", issuer=="AttestationPolicy"] => issue(type="keyDatabase", value=0);

+ // Look for "Forbidden Signatures" from EFI Driver Configuration Data

+ c:[type == "efiConfigVariables", issuer=="AttestationPolicy"] => add(type = "forbiddenSignaturesFound", value = JmesPath(c.value, "[?ProcessedData.UnicodeName == 'dbx'] | length(@) == `1` && @[0].PcrIndex == `7` && @[0].ProcessedData.VariableData"));

+ c:[type=="forbiddenSignaturesFound", issuer=="AttestationPolicy"] => issue(type="forbiddenSignatures", value=JsonToClaimValue(c.value));

+ ![type=="forbiddenSignaturesFound", issuer=="AttestationPolicy"] => issue(type="forbiddenSignatures", value=0);

+ // Look for "Kernel Version" in IMA Measurement events

+ c:[type=="imaMeasurementEvents", issuer=="AttestationPolicy"] => add(type="kernelVersionsFound", value=JmesPath(c.value, "[].ProcessedData.KernelVersion"));

+ c:[type=="kernelVersionsFound", issuer=="AttestationPolicy"] => issue(type="kernelVersions", value=JsonToClaimValue(c.value));

+ ![type=="kernelVersionsFound", issuer=="AttestationPolicy"] => issue(type="kernelVersions", value=0);

+ // Look for "Built-In Trusted Keys" in IMA Measurement events

+ c:[type=="imaMeasurementEvents", issuer=="AttestationPolicy"] => add(type="builtintrustedkeysFound", value=JmesPath(c.value, "[? ProcessedData.Keyring == '.builtin_trusted_keys'].ProcessedData.CertificateSubject"));

+ c:[type=="builtintrustedkeysFound", issuer=="AttestationPolicy"] => issue(type="builtintrustedkeys", value=JsonToClaimValue(c.value));

+ ![type=="builtintrustedkeysFound", issuer=="AttestationPolicy"] => issue(type="builtintrustedkeys", value=0);

+};

+```

+### Attestation policy to issue the list of drivers loaded during boot.

+```

+version = 1.2;

+configurationrules

+{

+};

+authorizationrules

+{

+ => permit();

+};

+issuancerules {

+c:[type=="events", issuer=="AttestationService"] => issue(type="alldriverloads", value=JmesPath(c.value, "Events[? EventTypeString == 'EV_EVENT_TAG' ].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_LOADEDMODULE_AGGREGATION[].EVENT_FILEPATH"));

+c:[type=="events", issuer=="AttestationService"] => issue(type="DriverLoadPolicy", value=JmesPath(c.value, "events[? EventTypeString == 'EV_EVENT_TAG' && (PcrIndex == '13')].ProcessedData.EVENT_TRUSTBOUNDARY.EVENT_DRIVER_LOAD_POLICY.String"));

+};

+```

+### Attestation policy for Key attestation, validating keys and properties of the key.

+```

+version=1.2;

+authorizationrules

+{

+ // Key Attest Policy

+ // -- Validating key types

+ c:[type=="x-ms-tpm-request-key", issuer=="AttestationService"] => add(type="requestKeyType", value=JsonToClaimValue(JmesPath(c.value, "jwk.kty")));

+ c:[type=="requestKeyType", issuer=="AttestationPolicy", value=="RSA"] => issue(type="requestKeyType", value="RSA");

+ // -- Validating tpm_certify attributes

+ c:[type=="x-ms-tpm-request-key", issuer=="AttestationService"] => add(type="requestKeyCertify", value=JmesPath(c.value, "info.tpm_certify"));

+ c:[type=="requestKeyCertify", issuer=="AttestationPolicy"] => add(type="requestKeyCertifyObjAttr", value=JsonToClaimValue(JmesPath(c.value, "obj_attr")));

+ c:[type=="requestKeyCertifyObjAttr", issuer=="AttestationPolicy", value==50] => issue(type="requestKeyCertifyObjAttrVerified", value=true);

+

+ c:[type=="requestKeyCertifyObjAttrVerified", issuer=="AttestationPolicy" , value==true] => permit();

+};

+issuancerules

+{

+

+};

+```

+ Title: Virtualization-based security (VBS) protocol for Azure Attestation

+The protocol has 2 messages exchanges:

+* Init Message

+* Request Message

+Message to establish context for the request message.

+## Request Message

+Payload containing the data that is to be attested by the attestation service.

+Note: Support for IMA measurement logs and Keys has been added to Request message, and can be found in the Request Message V2 section

+## Request message v1

+##### JWS protected header

+##### JWS payload

+## Request message v2

+```

+{

+ "request": "<JWS>"

+}

+```

+**request** (JWS): Request consists of a JSON Web Signature (JWS) structure. The JWS Protected Header and JWS Payload are shown below. As in any JWS structure, the final value consists of:

+BASE64URL(UTF8(JWS Protected Header)) || '.' ||

+BASE64URL(JWS Payload) || '.' ||

+BASE64URL(JWS Signature)

+##### JWS protected header

+```

+{

+ "alg": "PS256",

+ "typ": "attReqV2"

+ // no "kid" parameter as the key specified by request_key MUST sign this JWS to prove possession.

+}

+```

+#### JWS payload

+JWS payload can be of type basic or vsm. Basic is used when attestation evidence does not include VSM data.

+Basic example:

+```

+{

+ "att_type": "basic",

+ "att_data": {

+ "rp_id": "<URL>",

+ "rp_data": "<BASE64URL(RPCUSTOMDATA)>",

+ "challenge": "<BASE64URL(CHALLENGE)>",

+ "tpm_att_data": {

+ "current_attestation": {

+ "logs": [

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG1)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG2)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG3)>"

+ }

+ ],

+ "aik_cert": "<BASE64URL(AIKCERTIFICATE)>",

+ // aik_pub is represented as a JSON Web Key (JWK) object (RFC 7517).

+ "aik_pub": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "pcrs": [

+ {

+ "algorithm": 4, // TPM_ALG_SHA1

+ "values": [

+ {

+ "index": 0,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 5,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ },

+ {

+ "algorithm": 11, // TPM_ALG_SHA256

+ "values": [

+ {

+ "index": 2,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 1,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ }

+ ],

+ "quote": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ },

+ "boot_attestation": {

+ "logs": [

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(BOOT_LOG1)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(BOOT_LOG2)>"

+ }

+ ],

+ "aik_cert": "<BASE64URL(AIKCERTIFICATE)>",

+ // aik_pub is represented as a JSON Web Key (JWK) object (RFC 7517).

+ "aik_pub": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "pcrs": [

+ {

+ "algorithm": 4, // TPM_ALG_SHA1

+ "values": [

+ {

+ "index": 0,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 5,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ },

+ {

+ "algorithm": 11, // TPM_ALG_SHA256

+ "values": [

+ {

+ "index": 2,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 1,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ }

+ ],

+ "quote": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ }

+ },

+ "request_key": {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_quote": {

+ "hash_alg": "sha-256"

+ }

+ }

+ },

+ "other_keys": [

+ {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_certify": {

+ "public": "<BASE64URL(TPMT_PUBLIC)>",

+ "certification": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ }

+ }

+ },

+ {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ }

+ }

+ ],

+ "custom_claims": [

+ {

+ "name": "<name>",

+ "value": "<value>",

+ "value_type": "<value_type>"

+ },

+ {

+ "name": "<name>",

+ "value": "<value>",

+ "value_type": "<value_type>"

+ }

+ ],

+ "service_context": "<BASE64URL(SERVICECONTEXT)>"

+ }

+}

+```

+TPM + VBS enclave example:

+```

+{

+ "att_type": "vbs",

+ "att_data": {

+ "report_signed": {

+ "rp_id": "<URL>",

+ "rp_data": "<BASE64URL(RPCUSTOMDATA)>",

+ "challenge": "<BASE64URL(CHALLENGE)>",

+ "tpm_att_data": {

+ "current_attestation": {

+ "logs": [

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG1)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG2)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(CURRENT_LOG3)>"

+ }

+ ],

+ "aik_cert": "<BASE64URL(AIKCERTIFICATE)>",

+ // aik_pub is represented as a JSON Web Key (JWK) object (RFC 7517).

+ "aik_pub": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "pcrs": [

+ {

+ "algorithm": 4, // TPM_ALG_SHA1

+ "values": [

+ {

+ "index": 0,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 5,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ },

+ {

+ "algorithm": 11, // TPM_ALG_SHA256

+ "values": [

+ {

+ "index": 2,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 1,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ }

+ ],

+ "quote": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ },

+ "boot_attestation": {

+ "logs": [

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(BOOT_LOG1)>"

+ },

+ {

+ "type": "TCG",

+ "log": "<BASE64URL(BOOT_LOG2)>"

+ }

+ ],

+ "aik_cert": "<BASE64URL(AIKCERTIFICATE)>",

+ // aik_pub is represented as a JSON Web Key (JWK) object (RFC 7517).

+ "aik_pub": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "pcrs": [

+ {

+ "algorithm": 4, // TPM_ALG_SHA1

+ "values": [

+ {

+ "index": 0,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 5,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ },

+ {

+ "algorithm": 11, // TPM_ALG_SHA256

+ "values": [

+ {

+ "index": 2,

+ "digest": "<BASE64URL(DIGEST)>"

+ },

+ {

+ "index": 1,

+ "digest": "<BASE64URL(DIGEST)>"

+ }

+ ]

+ }

+ ],

+ "quote": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ }

+ },

+ "request_key": {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_quote": {

+ "hash_alg": "sha-256"

+ }

+ }

+ },

+ "other_keys": [

+ {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_certify": {

+ "public": "<BASE64URL(TPMT_PUBLIC)>",

+ "certification": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ }

+ }

+ },

+ {

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ }

+ }

+ ],

+ "custom_claims": [

+ {

+ "name": "<name>",

+ "value": "<value>",

+ "value_type": "<value_type>"

+ },

+ {

+ "name": "<name>",

+ "value": "<value>",

+ "value_type": "<value_type>"

+ }

+ ],

+ "service_context": "<BASE64URL(SERVICECONTEXT)>"

+ },

+ "vsm_report": {

+ "enclave": {

+ "report": "<BASE64URL(REPORT)>"

+ }

+ }

+ }

+}

+```

+**rp_id** (StringOrURI): Relying party identifier. Used by the service in the computation of the machine ID claim.

+**rp_data** (BASE64URL(OCTETS)): Opaque data passed by the relying party. This is normally used by the relying party as a nonce to guarantee freshness of the report.

+**challenge** (BASE64URL(OCTETS)): Random value issued by the service.

+- ***current_attestation*** (Object): Contains logs and TPM quote for the current state of the system (either boot or resume). The nonce received from the service must be passed to the TPM2_Quote command in the 'qualifyingData' parameter.

+- ***boot_attestation*** (Object): This is optional and contains logs and the TPM quote saved before the system hibernated and resumed. boot_attestation info must be associated with the same cold boot cycle (i.e. the system was only hibernated and resumed between them).

+- ***logs*** (Array(Object)): Array of logs. Each element of the array contains a log and the array must be in the order used for measurements.

+- ***aik_cert*** (BASE64URL(OCTETS)): The X.509 certificate representing the AIK.

+- ***aik_pub*** (JWK): The public part of the AIK represented as a JSON Web Key (JWK) object (RFC 7517).

+- ***pcrs*** (Array(Object)): Contains the set quoted. Each element of the array represents a PCR bank and the array must be in the order used to create the quote. A PCR bank is defined by its algorithm and its values (only the values quoted should be in the list).

+**vsm_report** (VSM Report Object): The VSM attestation report. See the VSM REPORT OBJECT section.

+**request_key** (Key object): Key used to sign the request. If a TPM is present (request contains TPM quote), request_key must either be bound to the TPM via quote or be resident in the TPM (see KEY OBJECT).

+**other_keys** (Array(Key object)): Array of keys to be sent to the service. Maximum of 2 keys.

+**custom_claims** (Array(Object)): Array of custom enclave claims sent to the service that can be evaluated by the policy.

+- ***name*** (String): Name of the claim. This name will be appended to a url determined by the Attestation Service (to avoid conflicts) and the concatenated string becomes the type of the claim that can be used in the policy.

+- ***value*** (String): Value of the claim.

+- ***value_type*** (String): Data type of the claimΓÇÖs value.

+**service_context** (BASE64URL(OCTETS)): Opaque, encrypted context created by the service which includes, among others, the challenge and an expiration time for that challenge.

+## Key object

+**jwk** (Object): The public part of the key represented as a JSON Web Key (JWK) object (RFC 7517).

+**info** (Object): Extra information about the key.

+No extra information:(Info object can be empty or missing from request)

+ΓÇó Key bound to the TPM via quote:

+- ***tpm_quote*** (Object): Data for the TPM quote binding method.

+- ***hash_alg*** (String): The algorithm used to create the hash passed to the TPM2_Quote command in the 'qualifyingData' parameter. The hash is computed by HASH[UTF8(jwk) || 0x00 || <OCTETS(service challenge)>]. Note: UTF8(jwk) must be the exact string that will be sent on the wire as the service will compute the hash using the exact string received in the request without modifications.

+>> Note: This binding method cannot be used for keys in the other_keys array.

+ΓÇó Key certified to be resident in the TPM:

+- ***tpm_certify*** (Object): Data for the TPM certification binding method.

+ΓÇ£publicΓÇ¥ (BASE64URL(OCTETS)): TPMT_PUBLIC structure representing the public area of the key in the TPM.

+- ***certification*** (BASE64URL(OCTETS)): TPMS_ATTEST returned by the TPM2_Certify command. The challenge received from the service must be passed to the TPM2_Certify command in the 'qualifyingData' parameter. The AIK provided in the request must be used to certify the key.

+- ***signature*** (BASE64URL(OCTETS)): TPMT_SIGNATURE returned by the TPM2_Certify command. The challenge received from the service must be passed to the TPM2_Certify command in the 'qualifyingData' parameter. The AIK provided in the request must be used to certify the key.

+>> Note: When this binding method is used for the request_key, the 'qualifyingData' parameter value passed to the TPM2_Quote command is simply the challenge received from the service.

+Examples:

+Key not bound to the TPM:

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ }

+}

+```

+Key bound to the TPM via quote (either resident in a VBS enclave or not):

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_quote":

+ "hash_alg": "sha-256"

+ }

+ }

+}

+```

+Key certified to be resident in the TPM:

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_certify": {

+ "public": "<BASE64URL(TPMT_PUBLIC)>",

+ "certification": "<BASE64URL(TPMS_ATTEST)>",

+ "signature": "<BASE64URL(TPMT_SIGNATURE)>"

+ }

+ }

+}

+```

+## Policy key object

+The policy key object is the version of the key object used as input claims in the policy. It is processed by the service in order to make it more readable and easier to evaluate by policy rules.

+ΓÇó Key not bound to the TPM:

+Same as the respective key object.

+Example:

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ }

+}

+```

+ΓÇó Key bound to the TPM via quote (either resident in a VBS enclave or not):

+Same as the respective key object.

+Example:

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_quote":

+ "hash_alg": "sha-256"

+ }

+ }

+}

+```

+ΓÇó Key certified to be resident in the TPM:

+***jwk*** (Object): Same as the respective key object.

+***info.tpm_certify*** (Object):

+- ***name_alg*** (Integer): UINT16 value representing a hash algorithm defined by the TPM_ALG_ID constants.

+- ***obj_attr*** (Integer): UINT32 value representing the attributes of the key object defined by TPMA_OBJECT

+- ***auth_policy*** (BASE64URL(OCTETS)): Optional policy for using this key object.

+Example:

+```

+{

+ "jwk": {

+ "kty": "RSA",

+ "n": "<Base64urlUInt(MODULUS)>",

+ "e": "<Base64urlUInt(EXPONENT)>"

+ },

+ "info": {

+ "tpm_certify": {

+ "name_alg": 11, // 0xB (TPM_ALG_SHA256)

+ "obj_attr": 50, // 0x32 (fixedTPM | fixedParent | sensitiveDataOrigin)

+ "auth_policy": "<BASE64URL(AUTH_POLICY)>"

+ }

+ }

+}

+```

+## VBS report object

+### Enclave attestation:

+***enclave*** (Object): Data for VSM enclave attestation.

+- ***report*** (BASE64URL(OCTETS)): The VSM enclave attestation report as returned by function EnclaveGetAttestationReport. The EnclaveData parameter must be the SHA-512 hash of the value of report_signed (including the opening and closing braces). The hash function input is UTF8(report_signed).

+Examples:

+Enclave attestation:

+```

+{

+ "enclave": {

+ "report": "<BASE64URL(REPORT)>"

+ }

+}

+```

+## Report message

+Direction

+Attestation Service -> Client

+Payload

+```

+{

+ "report": "<JWT>"

+}

+```

+***report*** (JWT): The attestation report in JSON Web Token (JWT) format (RFC 7519).

+Automation accounts currently support the following regions:

+### Public preview of PowerShell 7.2 and Python 3.10

+Azure Automation now supports runbooks in latest Runtime versions - PowerShell 7.2 and Python 3.10 in public preview. This enables creation and execution of runbooks for orchestration of management tasks. These new runtimes are currently supported only for Cloud jobs in five regions - West Central US, East US, South Africa North, North Europe, Australia, and Southeast. [Learn more](automation-runbook-types.md).

+Build your own disaster recovery strategy to handle a region-wide or zone-wide failure [Learn more](https://learn.microsoft.com/azure/automation/automation-disaster-recovery).

+az customlocation delete -n <customLocationName> -g <resourceGroupName>

+Required parameters:

+| Parameter name | Description |

+|-||

+| `--name, --n` | Name of the custom location |

+| `--resource-group, --g` | Resource group of the custom location |

+ - Ability to log in as a local user or an [Azure user (Linux only)](../../active-directory/devices/howto-vm-sign-in-azure-ad-linux.md)

+### Install local command line tool

+This functionality is currently packaged in an Azure CLI extension and an Azure PowerShell module.

+#### [Install Azure CLI extension](#tab/azure-cli)

+```az extension add --name ssh```

+> The Azure CLI extension version must be greater than 1.1.0.

+#### [Install Azure PowerShell module](#tab/azure-powershell)

+```Install-Module -Name AzPreview -Scope CurrentUser -Repository PSGallery -Force```

+### Enable functionality on your Arc-enabled server

+In order to use the SSH connect feature, you must enable connections on the hybrid agent.

+> [!NOTE]

+> The following actions must be completed in an elevated terminal session.

+View your current incoming connections:

+```azcmagent config list```

+If you have existing ports, you'll need to include them in the following command.

+To add access to SSH connections, run the following:

+```azcmagent config set incomingconnections.ports 22<,other open ports,...>```

+If you're using a non-default port for your SSH connection, replace port 22 with your desired port in the previous command.

+> The following steps will not need to be run for most users.

+### Register the HybridConnectivity resource provider

+> [!NOTE]

+> This is a one-time operation that needs to be performed on each subscription.

+Check if the HybridConnectivity resource provider (RP) has been registered:

+```az provider show -n Microsoft.HybridConnectivity```

+If the RP hasn't been registered, run the following:

+```az provider register -n Microsoft.HybridConnectivity```

+This operation can take 2-5 minutes to complete. Before moving on, check that the RP has been registered.

+To view examples, view the Az CLI documentation page for [az ssh](/cli/azure/ssh) or the Azure PowerShell documentation page for [Az.Ssh](/powershell/module/az.ssh).

+## WEBSITE\_SLOT\_NAME

+Read-only. Name of the current deployment slot. The name of the production slot is `Production`.

+|Key|Sample value|

+|||

+|WEBSITE_SLOT_NAME|`Production`|

+ Title: How to create a dataset using a GeoJson package

+description: Learn how to create a dataset using a GeoJson package embedding the module's JavaScript libraries.

+# Create a dataset using a GeoJson package (Preview)

+Azure Maps Creator enables users to import their indoor map data in GeoJSON format with [Facility Ontology 2.0][Facility Ontology], which can then be used to create a [dataset][dataset-concept].

+> [!NOTE]

+> This article explains how to create a dataset from a GeoJSON package. For information on additional steps required to complete an indoor map, see [Next steps](#next-steps).

+## Prerequisites

+- Basic understanding of [Creator for indoor maps](creator-indoor-maps.md).

+- Basic understanding of [Facility Ontology 2.0][Facility Ontology].

+- [Azure Maps account][Azure Maps account].

+- [Azure Maps Creator resource][Creator resource].

+- [Subscription key][Subscription key].

+- Zip package containing all required GeoJSON files. If you don't have GeoJSON

+ files, you can download the [Contoso building sample][Contoso building sample].

+>[!IMPORTANT]

+>

+> - This article uses the `us.atlas.microsoft.com` geographical URL. If your Creator service wasn't created in the United States, you must use a different geographical URL. For more information, see [Access to Creator Services](how-to-manage-creator.md#access-to-creator-services).

+> - In the URL examples in this article you will need to replace `{Azure-Maps-Primary-Subscription-key}` with your primary subscription key.

+## Create dataset using the GeoJSON package

+For more information on the GeoJSON package, see the [Geojson zip package requirements](#geojson-zip-package-requirements) section.

+### Upload the GeoJSON package

+Use the [Data Upload API](/rest/api/maps/data-v2/upload) to upload the Drawing package to Azure Maps Creator account.

+The Data Upload API is a long running transaction that implements the pattern defined in [Creator Long-Running Operation API V2](creator-long-running-operation-v2.md).

+To upload the GeoJSON package:

+1. Execute the following HTTP POST request that uses the [Data Upload API](/rest/api/maps/data-v2/upload):

+ ```http

+ https://us.atlas.microsoft.com/mapData?api-version=2.0&dataFormat=zip&subscription-key={Your-Azure-Maps-Primary-Subscription-key}

+ ```

+ 1. Set `Content-Type` in the **Header** to `application/zip`.

+1. Copy the value of the `Operation-Location` key in the response header. The `Operation-Location` key is also known as the `status URL` and is required to check the status of the upload, which is explained in the next section.

+### Check the GeoJSON package upload status

+To check the status of the GeoJSON package and retrieve its unique identifier (`udid`):

+1. Execute the following HTTP GET request that uses the status URL you copied as the last step in the previous section of this article. The request should look like the following URL:

+```http

+https://us.atlas.microsoft.com/mapData/operations/{operationId}?api-version=2.0&subscription-key={Your-Azure-Maps-Primary-Subscription-key}

+```

+1. Copy the value of the `Resource-Location` key in the response header, which is the `resource location URL`. The `resource location URL` contains the unique identifier (`udid`) of the GeoJSON package resource.

+### Create a dataset

+<!--

+A dataset is a collection of map features, such as buildings, levels, and rooms. To create a dataset from your GeoJSON, use the new [Dataset Create API][Dataset Create 2022-09-01-preview]. The Dataset Create API takes the `udid` you got in the previous section and returns the `datasetId` of the new dataset.

+-->

+A dataset is a collection of map features, such as buildings, levels, and rooms. To create a dataset from your GeoJSON, use the new create dataset API. The create dataset API takes the `udid` you got in the previous section and returns the `datasetId` of the new dataset.

+> [!IMPORTANT]

+> This is different from the [previous version][Dataset Create] in that it doesn't require a `conversionId` from a converted Drawing package.

+To create a dataset:

+1. Enter the following URL to the dataset service. The request should look like the following URL (replace {udid} with the `udid` obtained in [Check the GeoJSON package upload status](#check-the-geojson-package-upload-status) section):

+<!--1. Enter the following URL to the [Dataset service][Dataset Create 2022-09-01-preview]. The request should look like the following URL (replace {udid} with the `udid` obtained in [Check the GeoJSON package upload status](#check-the-geojson-package-upload-status) section):-->

+ ```http

+ https://us.atlas.microsoft.com/datasets?api-version=2022-09-01-preview&udid={udid}&subscription-key={Your-Azure-Maps-Primary-Subscription-key}

+ ```

+1. Copy the value of the `Operation-Location` key in the response header. The `Operation-Location` key is also known as the `status URL` and is required to check the status of the dataset creation process and to get the `datasetId`, which is required to create a tileset.

+### Check the dataset creation status

+To check the status of the dataset creation process and retrieve the `datasetId`:

+1. Enter the status URL you copied in [Create a dataset](#create-a-dataset). The request should look like the following URL:

+ ```http

+ https://us.atlas.microsoft.com/datasets/operations/{operationId}?api-version=2022-09-01-preview&subscription-key={Your-Azure-Maps-Primary-Subscription-key}

+ ```

+1. In the Header of the HTTP response, copy the value of the unique identifier contained in the `Resource-Location` key.

+ > `https://us.atlas.microsoft.com/datasets/**c9c15957-646c-13f2-611a-1ea7adc75174**?api-version=2022-09-01-preview`

+See [Next steps](#next-steps) for links to articles to help you complete your indoor map.

+## Add data to an existing dataset

+<!--

+Data can be added to an existing dataset by providing the `datasetId` parameter to the [dataset create API][Dataset Create 2022-09-01-preview] along with the unique identifier of the data you wish to add. The unique identifier can be either a `udid` or `conversionId`. This creates a new dataset consisting of the data (facilities) from both the existing dataset and the new data being imported. Once the new dataset has been created successfully, the old dataset can be deleted.

+-->

+Data can be added to an existing dataset by providing the `datasetId` parameter to the create dataset API along with the unique identifier of the data you wish to add. The unique identifier can be either a `udid` or `conversionId`. This creates a new dataset consisting of the data (facilities) from both the existing dataset and the new data being imported. Once the new dataset has been created successfully, the old dataset can be deleted.

+One thing to consider when adding to an existing dataset is how the feature IDs are created. If a dataset is created from a converted drawing package, the feature IDs are generated automatically. When a dataset is created from a GeoJSON package, feature IDs must be provided in the GeoJSON file. When appending to an existing dataset, the original dataset drives the way feature IDs are created. If the original dataset was created using a `udid`, it uses the IDs from the GeoJSON, and will continue to do so with all GeoJSON packages appended to that dataset in the future. If the dataset was created using a `conversionId`, IDs will be internally generated, and will continue to be internally generated with all GeoJSON packages appended to that dataset in the future.

+### Add to dataset created from a GeoJSON source

+If your original dataset was created from a GoeJSON source and you wish to add another facility created from a drawing package, you can append it to your existing dataset by referencing its `conversionId`, as demonstrated by this HTTP POST request:

+```http

+https://us.atlas.microsoft.com/datasets?api-version=2022-09-01-preview&conversionId={conversionId}&outputOntology=facility-2.0&datasetId={datasetId}

+```

+| Identifier | Description |

+|--|-|

+| conversionId | The ID returned when converting your drawing package. For more information, see [Convert a Drawing package][conversion]. |

+| datasetId | The dataset ID returned when creating the original dataset from a GeoJSON package). |

+<!--For more information, see [][].-->

+## Geojson zip package requirements

+The GeoJSON zip package consists of one or more [RFC 7946][RFC 7946] compliant GeoJSON files, one for each feature class, all in the root directory (subdirectories aren't supported), compressed with standard Zip compression and named using the `.ZIP` extension.

+Each feature class file must match its definition in the [Facility ontology 2.0][Facility ontology] and each feature must have a globally unique identifier.

+Feature IDs can only contain alpha-numeric (a-z, A-Z, 0-9), hyphen (-), dot (.) and underscore (_) characters.

+> [!TIP]

+> If you want to be certain you have a globally unique identifier (GUID), consider creating it by running a GUID generating tool such as the Guidgen.exe command line program (Available with [Visual Studio][Visual Studio]). Guidgen.exe never produces the same number twice, no matter how many times it is run or how many different machines it runs on.

+### Facility ontology 2.0 validations in the Dataset

+[Facility ontology][Facility ontology] defines how Azure Maps Creator internally stores facility data, divided into feature classes, in a Creator dataset. When importing a GeoJSON package, anytime a feature is added or modified, a series of validations run. This includes referential integrity checks as well as geometry and attribute validations. These validations are described in more detail below.

+- The maximum number of features that can be imported into a dataset at a time is 150,000.

+- The facility area can be between 4 and 4,000 Sq Km.

+- The top level element is [facility][facility], which defines each building in the file *facility.geojson*.

+- Each facility has one or more levels, which are defined in the file *levels.goejson*.

+ - Each level must be inside the facility.

+- Each [level][level] contain [units][unit], [structures][structure], [verticalPenetrations][verticalPenetration] and [openings][opening]. All of the items defined in the level must be fully contained within the Level geometry.

+ - `unit` can consist of an array of items such as hallways, offices and courtyards, which are defined by [area][areaElement], [line][lineElement] or [point][pointElement] elements. Units are defined in the file *unit.goejson*.

+ - All `unit` elements must be fully contained within their level and intersect with their children.

+ - `structure` defines physical, non-overlapping areas that can't be navigated through, such as a wall. Structures are defined in the file *structure.goejson*.

+ - `verticalPenetration` represents a method of navigating vertically between levels, such as stairs and elevators and are defined in the file *verticalPenetration.geojson*.

+ - verticalPenetrations can't intersect with other verticalPenetrations on the same level.

+ - `openings` define traversable boundaries between two units, or a `unit` and `verticalPenetration` and are defined in the file *opening.geojson*.

+ - Openings can't intersect with other openings on the same level.

+ - Every `opening` must be associated with at least one `verticalPenetration` or `unit`.

+## Next steps

+> [!div class="nextstepaction"]

+> [Create a tileset](tutorial-creator-indoor-maps.md#create-a-tileset)

+> [!div class="nextstepaction"]

+> [Query datasets with WFS API](tutorial-creator-wfs.md)

+> [!div class="nextstepaction"]

+> [Create a feature stateset](tutorial-creator-feature-stateset.md)

+[Contoso building sample]: https://github.com/Azure-Samples/am-creator-indoor-data-examples

+[unit]: creator-facility-ontology.md?pivots=facility-ontology-v2#unit

+[structure]: creator-facility-ontology.md?pivots=facility-ontology-v2#structure

+[level]: creator-facility-ontology.md?pivots=facility-ontology-v2#level

+[facility]: creator-facility-ontology.md?pivots=facility-ontology-v2#facility

+[verticalPenetration]: creator-facility-ontology.md?pivots=facility-ontology-v2#verticalpenetration

+[opening]: creator-facility-ontology.md?pivots=facility-ontology-v2#opening

+[areaElement]: creator-facility-ontology.md?pivots=facility-ontology-v2#areaelement

+[lineElement]: creator-facility-ontology.md?pivots=facility-ontology-v2#lineelement

+[pointElement]: creator-facility-ontology.md?pivots=facility-ontology-v2#pointelement

+[conversion]: tutorial-creator-indoor-maps.md#convert-a-drawing-package

+[Azure Maps account]: quick-demo-map-app.md#create-an-azure-maps-account

+[Creator resource]: how-to-manage-creator.md

+[Subscription key]: quick-demo-map-app.md#get-the-primary-key-for-your-account

+[Facility Ontology]: creator-facility-ontology.md?pivots=facility-ontology-v2

+[RFC 7946]: https://www.rfc-editor.org/rfc/rfc7946.html

+[dataset-concept]: creator-indoor-maps.md#datasets

+<!--[Dataset Create 2022-09-01-preview]: /rest/api/maps/v20220901preview/dataset/create-->

+[Visual Studio]: https://visualstudio.microsoft.com/downloads/

+> [!TIP]

+> You can also create a dataset from a GeoJSON package. For more information, see [Create a dataset using a GeoJson package (Preview)](how-to-dataset-geojson.md).

+- Azure Diagnostics Extension can be used only with Azure virtual machines. The Log Analytics agent can be used with virtual machines in Azure, other clouds, and on-premises.

+- Azure Diagnostics extension sends data to Azure Storage, [Azure Monitor Metrics](../essentials/data-platform-metrics.md) (Windows only) and Azure Event Hubs. The Log Analytics agent collects data to [Azure Monitor Logs](../logs/data-platform-logs.md).

+- The Log Analytics agent is required for retired [solutions](../insights/solutions.md), [VM insights](../vm/vminsights-overview.md), and other services such as [Microsoft Defender for Cloud](../../security-center/index.yml).

+* Use different [solutions](../insights/solutions.md) to monitor a particular service or application.

+For apps written by using [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) or [WorkerService](worker-service.md#add-telemetry-processors), adding a new telemetry processor is done by using the `AddApplicationInsightsTelemetryProcessor` extension method on `IServiceCollection`, as shown. This method is called in the `ConfigureServices` method of your `Startup.cs` class.

+For apps written using [ASP.NET Core](asp-net-core.md#adding-telemetryinitializers) or [WorkerService](worker-service.md#add-telemetry-initializers), adding a new telemetry initializer is done by adding it to the Dependency Injection container, as shown. Accomplish this step in the `Startup.ConfigureServices` method.

+ Title: Monitor Azure App Service performance in .NET Core | Microsoft Docs

+description: Application performance monitoring for Azure App Service using ASP.NET Core. Chart load and response time, dependency information, and set alerts on performance.

+# Application monitoring for Azure App Service and ASP.NET Core

+Enabling monitoring on your ASP.NET Core-based web applications running on [Azure App Service](../../app-service/index.yml) is now easier than ever. Previously, you needed to manually instrument your app. Now, the latest extension/agent is built into the App Service image by default. This article walks you through enabling Azure Monitor Application Insights monitoring. It also provides preliminary guidance for automating the process for large-scale deployments.

+> Only .NET Core [Long Term Support](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) is supported for auto-instrumentation on Windows.

+[Trim self-contained deployments](/dotnet/core/deploying/trimming/trim-self-contained) is *not supported*. Use [manual instrumentation](./asp-net-core.md) via code instead.

+> [!NOTE]

+> Auto-instrumentation used to be known as "codeless attach" before October 2021.

+See the following [Enable monitoring](#enable-monitoring) section to begin setting up Application Insights with your App Service resource.

+[Trim self-contained deployments](/dotnet/core/deploying/trimming/trim-self-contained) is *not supported*. Use [manual instrumentation](./asp-net-core.md) via code instead.

+> [!NOTE]

+> Linux auto-instrumentation App Service portal enablement is in public preview. These preview versions are provided without a service level agreement. Certain features might not be supported or might have constrained capabilities.

+See the following [Enable monitoring](#enable-monitoring) section to begin setting up Application Insights with your App Service resource.

+### Enable monitoring

+1. Select **Application Insights** in the left pane for your app service. Then select **Enable**.

+ :::image type="content"source="./media/azure-web-apps/enable.png" alt-text=" Screenshot that shows the Application Insights tab with Enable selected.":::

+1. Create a new resource or select an existing Application Insights resource for this application.

+ > When you select **OK** to create a new resource, you're prompted to **Apply monitoring settings**. Selecting **Continue** links your new Application Insights resource to your app service. Your app service then restarts.

+ :::image type="content"source="./media/azure-web-apps/change-resource.png" alt-text="Screenshot that shows the Change your resource dropdown.":::

+1. After you specify which resource to use, you can choose how you want Application Insights to collect data per platform for your application. ASP.NET Core collection options are **Recommended** or **Disabled**.

+ :::image type="content"source="./media/azure-web-apps-net-core/instrument-net-core.png" alt-text=" Screenshot that shows instrumenting your application section.":::

+Client-side monitoring is enabled by default for ASP.NET Core apps with **Recommended** collection, regardless of whether the app setting `APPINSIGHTS_JAVASCRIPT_ENABLED` is present.

+If you want to disable client-side monitoring:

+1. Select **Settings** > **Configuration**.

+1. Under **Application settings**, create a **New application setting** with the following information:

+ - **Name**: `APPINSIGHTS_JAVASCRIPT_ENABLED`

+ - **Value**: `false`

+1. **Save** the settings. Restart your app.

+To enable telemetry collection with Application Insights, only the application settings must be set.

+|XDT_MicrosoftApplicationInsights_PreemptSdk | For ASP.NET Core apps only. Enables Interop (interoperation) with the Application Insights SDK. Loads the extension side by side with the SDK and uses it to send telemetry. (Disables the Application Insights SDK.) |`1`|

+## Upgrade monitoring extension/agent - .NET

+To upgrade the monitoring extension/agent, follow the steps in the next sections.

+Starting with version 2.8.9, the preinstalled site extension is used. If you're using an earlier version, you can update via one of two ways:

+* [Upgrade by enabling via the portal](#enable-auto-instrumentation-monitoring): Even if you have the Application Insights extension for App Service installed, the UI shows only the **Enable** button. Behind the scenes, the old private site extension will be removed.

+ 1. Set the application settings to enable the preinstalled site extension `ApplicationInsightsAgent`. For more information, see [Enable through PowerShell](#enable-through-powershell).

+ 1. Manually remove the private site extension named **Application Insights extension for Azure App Service**.

+If the upgrade is done from a version prior to 2.5.1, check that the `ApplicationInsights` DLLs are removed from the application bin folder. For more information, see [Troubleshooting steps](#troubleshooting).

+> When you create a web app with the `ASP.NET Core` runtimes in App Service, it deploys a single static HTML page as a starter website. We *do not* recommend that you troubleshoot an issue with the default template. Deploy an application before you troubleshoot an issue.

+What follows is our step-by-step troubleshooting guide for extension/agent-based monitoring for ASP.NET Core-based applications running on App Service.

+1. Check that the `ApplicationInsightsAgent_EXTENSION_VERSION` app setting is set to a value of `~2`.

+1. Browse to `https://yoursitename.scm.azurewebsites.net/ApplicationInsights`.

+ :::image type="content"source="./media/azure-web-apps/app-insights-sdk-status.png" alt-text="Screenshot that shows the link above the results page."border ="false":::

+ - Confirm that **Application Insights Extension Status** is `Pre-Installed Site Extension, version 2.8.x.xxxx, is running.`

+ If it isn't running, follow the instructions in the section [Enable Application Insights monitoring](#enable-auto-instrumentation-monitoring).

+ - Confirm that the status source exists and looks like `Status source D:\home\LogFiles\ApplicationInsights\status\status_RD0003FF0317B6_4248_1.json`.

+ If a similar value isn't present, it means the application isn't currently running or isn't supported. To ensure that the application is running, try manually visiting the application URL/application endpoints, which will allow the runtime information to become available.

+ - Confirm that **IKeyExists** is `True`. If it's `False`, add `APPINSIGHTS_INSTRUMENTATIONKEY` and `APPLICATIONINSIGHTS_CONNECTION_STRING` with your ikey GUID to your application settings.

+ - If your application refers to any Application Insights packages, enabling the App Service integration might not take effect and the data might not appear in Application Insights. An example would be if you've previously instrumented, or attempted to instrument, your app with the [ASP.NET Core SDK](./asp-net-core.md). To fix the issue, in the portal, turn on **Interop with Application Insights SDK**. You'll start seeing the data in Application Insights.

+

+ > This functionality is in preview.

+ :::image type="content"source="./media/azure-web-apps-net-core/interop.png" alt-text=" Screenshot that shows the interop setting enabled.":::

+ The data will now be sent by using a codeless approach, even if the Application Insights SDK was originally used or attempted to be used.

+ > If the application used the Application Insights SDK to send any telemetry, the telemetry will be disabled. In other words, custom telemetry (for example, any `Track*()` methods) and custom settings (such as sampling) will be disabled.

+1. Check that the `ApplicationInsightsAgent_EXTENSION_VERSION` app setting is set to a value of `~2`.

+1. Browse to `https://your site name.scm.azurewebsites.net/ApplicationInsights`.

+ * The status source exists and looks like `Status source /var/log/applicationinsights/status_abcde1234567_89_0.json`.

+ * The value `Auto-Instrumentation enabled successfully` is displayed. If a similar value isn't present, it means the application isn't running or isn't supported. To ensure that the application is running, try manually visiting the application URL/application endpoints, which will allow the runtime information to become available.

+ * **IKeyExists** is `True`. If it's `False`, add `APPINSIGHTS_INSTRUMENTATIONKEY` and `APPLICATIONINSIGHTS_CONNECTION_STRING` with your ikey GUID to your application settings.

+ :::image type="content" source="media/azure-web-apps-net-core/auto-instrumentation-status.png" alt-text="Screenshot that shows the auto-instrumentation status webpage." lightbox="media/azure-web-apps-net-core/auto-instrumentation-status.png":::

+When you create a web app with the ASP.NET Core runtimes in App Service, it deploys a single static HTML page as a starter website. The static webpage also loads an ASP.NET-managed web part in IIS. This behavior allows for testing codeless server-side monitoring but doesn't support automatic client-side monitoring.

+If you want to test out codeless server and client-side monitoring for ASP.NET Core in an App Service web app, we recommend that you follow the official guides for [creating an ASP.NET Core web app](../../app-service/quickstart-dotnetcore.md). Then use the instructions in the current article to enable monitoring.

+PHP and WordPress sites aren't supported. There's currently no officially supported SDK/agent for server-side monitoring of these workloads. To manually instrument client-side transactions on a PHP or WordPress site by adding the client-side JavaScript to your webpages, use the [JavaScript SDK](./javascript.md).

+The following table provides an explanation of what these values mean, their underlying causes, and recommended fixes.

+|Problem value |Explanation |Fix |

+| `AppAlreadyInstrumented:true` | This value indicates that the extension detected that some aspect of the SDK is already present in the application and will back off. It can be because of a reference to `Microsoft.ApplicationInsights.AspNetCore` or `Microsoft.ApplicationInsights`. | Remove the references. Some of these references are added by default from certain Visual Studio templates. Older versions of Visual Studio reference `Microsoft.ApplicationInsights`. |

+|`AppAlreadyInstrumented:true` | This value can also be caused by the presence of `Microsoft.ApplicationsInsights` DLL in the app folder from a previous deployment. | Clean the app folder to ensure that these DLLs are removed. Check both your local app's bin directory and the *wwwroot* directory on the App Service. (To check the wwwroot directory of your App Service web app, select **Advanced Tools (Kudu**) > **Debug console** > **CMD** > **home\site\wwwroot**). |

+|`IKeyExists:false`|This value indicates that the instrumentation key isn't present in the app setting `APPINSIGHTS_INSTRUMENTATIONKEY`. Possible causes include accidentally removing the values or forgetting to set the values in automation script. | Make sure the setting is present in the App Service application settings. |

+For the latest updates and bug fixes, see the [Release notes](web-app-extension-release-notes.md).

+* [Run the Profiler on your live app](./profiler.md).

+* Use [Application Insights for JavaScript apps and webpages](javascript.md) to get client telemetry from the browsers that visit a webpage.

+ Title: Application Insights telemetry data model | Microsoft Docs

+description: This article presents an overview of the Application Insights telemetry data model.

+[Application Insights](./app-insights-overview.md) sends telemetry from your web application to the Azure portal so that you can analyze the performance and usage of your application. The telemetry model is standardized, so it's possible to create platform and language-independent monitoring.

+Data collected by Application Insights models this typical application execution pattern.

+

+The following types of telemetry are used to monitor the execution of your app. Three types are automatically collected by the Application Insights SDK from the web application framework:

+* [Request](data-model-request-telemetry.md): Generated to log a request received by your app. For example, the Application Insights web SDK automatically generates a Request telemetry item for each HTTP request that your web app receives.

+ An *operation* is made up of the threads of execution that process a request. You can also [write code](./api-custom-events-metrics.md#trackrequest) to monitor other types of operation, such as a "wake up" in a web job or function that periodically processes data. Each operation has an ID. The ID can be used to [group](./correlation.md) all telemetry generated while your app is processing the request. Each operation either succeeds or fails and has a duration of time.

+* [Exception](data-model-exception-telemetry.md): Typically represents an exception that causes an operation to fail.

+* [Dependency](data-model-dependency-telemetry.md): Represents a call from your app to an external service or storage, such as a REST API or SQL. In ASP.NET, dependency calls to SQL are defined by `System.Data`. Calls to HTTP endpoints are defined by `System.Net`.

+Application Insights provides three data types for custom telemetry:

+* [Trace](data-model-trace-telemetry.md): Used either directly or through an adapter to implement diagnostics logging by using an instrumentation framework that's familiar to you, such as `Log4Net` or `System.Diagnostics`.

+* [Event](data-model-event-telemetry.md): Typically used to capture user interaction with your service to analyze usage patterns.

+* [Metric](data-model-metric-telemetry.md): Used to report periodic scalar measurements.

+Every telemetry item can define the [context information](data-model-context.md) like application version or user session ID. Context is a set of strongly typed fields that unblocks certain scenarios. When application version is properly initialized, Application Insights can detect new patterns in application behavior correlated with redeployment.

+You can use session ID to calculate an outage or an issue impact on users. Calculating the distinct count of session ID values for a specific failed dependency, error trace, or critical exception gives you a good understanding of an impact.

+The Application Insights telemetry model defines a way to [correlate](./correlation.md) telemetry to the operation of which it's a part. For example, a request can make a SQL Database call and record diagnostics information. You can set the correlation context for those telemetry items that tie it back to the request telemetry.

+The Application Insights data model is a basic yet powerful way to model your application telemetry. We strive to keep the model simple and slim to support essential scenarios and allow the schema to be extended for advanced use.

+To report data model or schema problems and suggestions, use our [GitHub repository](https://github.com/microsoft/ApplicationInsights-dotnet/issues/new/choose).

+- [Write custom telemetry](./api-custom-events-metrics.md).

+- Use [sampling](./sampling.md) to minimize the amount of telemetry based on data model.

+The following example shows how to add/remove counters. This customization would be done in the `ConfigureServices` method of your application after Application Insights telemetry collection is enabled using either `AddApplicationInsightsTelemetry()` or `AddApplicationInsightsWorkerService()`. Following is an example code from an ASP.NET Core application. For other type of applications, refer to [this](worker-service.md#configure-or-remove-default-telemetry-modules) document.

+description: Export diagnostic and usage data to storage in Azure and download it from there.

+Do you want to keep your telemetry for longer than the standard retention period? Or do you want to process it in some specialized way? Continuous export is ideal for this purpose. The events you see in the Application Insights portal can be exported to storage in Azure in JSON format. From there, you can download your data and write whatever code you need to process it.

+> * When you [migrate to a workspace-based Application Insights resource](convert-classic-resource.md), you must use [diagnostic settings](#diagnostic-settings-based-export) for exporting telemetry. All [workspace-based Application Insights resources](./create-workspace-resource.md) must use [diagnostic settings](./create-workspace-resource.md#export-telemetry).

+> * Diagnostic settings export might increase costs. For more information, see [Diagnostic settings-based export](export-telemetry.md#diagnostic-settings-based-export).

+* The **Export** button at the top of a metrics or search tab lets you transfer tables and charts to an Excel spreadsheet.

+* [Log Analytics](../logs/log-query-overview.md) provides a powerful query language for telemetry. It can also export results.

+* If you're looking to [explore your data in Power BI](../logs/log-powerbi.md), you can do that without using continuous export if you've [migrated to a workspace-based resource](convert-classic-resource.md).

+* The [Data Access REST API](https://dev.applicationinsights.io/) lets you access your telemetry programmatically.

+* You can also access setup for [continuous export via PowerShell](/powershell/module/az.applicationinsights/new-azapplicationinsightscontinuousexport).

+After continuous export copies your data to storage, where it can stay as long as you like, it's still available in Application Insights for the usual [retention period](./data-retention-privacy.md).

+## Supported regions

+Continuous export is supported in the following regions:

+> Continuous export will continue to work for applications in East US and West Europe if the export was configured before February 23, 2021. New continuous export rules can't be configured on any application in East US or West Europe, no matter when the application was created.

+## Continuous export advanced storage configuration

+Continuous export *doesn't support* the following Azure Storage features or configurations:

+* Use of [Azure Virtual Network/Azure Storage firewalls](../../storage/common/storage-network-security.md) with Azure Blob Storage.

+## <a name="setup"></a> Create a continuous export

+> An application can't export more than 3 TB of data per day. If more than 3 TB per day is exported, the export will be disabled. To export without a limit, use [diagnostic settings-based export](#diagnostic-settings-based-export).

+1. In the Application Insights resource for your app under **Configure** on the left, open **Continuous export** and select **Add**.

+1. Choose the telemetry data types you want to export.

+1. Create or select an [Azure Storage account](../../storage/common/storage-introduction.md) where you want to store the data. For more information on storage pricing options, see the [Pricing page](https://azure.microsoft.com/pricing/details/storage/).

+ Select **Add** > **Export destination** > **Storage account**. Then either create a new store or choose an existing store.

+ > By default, the storage location will be set to the same geographical region as your Application Insights resource. If you store in a different region, you might incur transfer charges.

+1. Create or select a container in the storage.

+> After you've created your export, newly ingested data will begin to flow to Azure Blob Storage. Continuous export only transmits new telemetry that's created or ingested after continuous export was enabled. Any data that existed prior to enabling continuous export won't be exported. There's no supported way to retroactively export previously created data by using continuous export.

+After the first export is finished, you'll find the following structure in your Blob Storage container. (This structure varies depending on the data you're collecting.)

+| [Event](export-data-model.md#events) | Custom events generated by [TrackEvent()](./api-custom-events-metrics.md#trackevent).

+### Edit continuous export

+Select **Continuous export** and select the storage account to edit.

+### Stop continuous export

+To stop the export, select **Disable**. When you select **Enable** again, the export restarts with new data. You won't get the data that arrived in the portal while export was disabled.

+To add or change exports, you need Owner, Contributor, or Application Insights Contributor access rights. [Learn about roles][roles].

+> If your application sends a lot of data, the sampling feature might operate and send only a fraction of the generated telemetry. [Learn more about sampling.](./sampling.md)

+You can inspect the storage directly in the portal. Select **Home** on the leftmost menu. At the top where it says **Azure services**, select **Storage accounts**. Select the storage account name, and on the **Overview** page select **Services** > **Blobs**. Finally, select the container name.

+To inspect Azure Storage in Visual Studio, select **View** > **Cloud Explorer**. If you don't have that menu command, you need to install the Azure SDK. Open the **New Project** dialog, expand **Visual C#/Cloud**, and select **Get Microsoft Azure SDK for .NET**.

+When you open your blob store, you'll see a container with a set of blob files. You'll see the URI of each file derived from your Application Insights resource name, its instrumentation key, and telemetry type, date, and time. The resource name is all lowercase, and the instrumentation key omits dashes.

+

+The date and time are UTC and are when the telemetry was deposited in the store, not the time it was generated. For this reason, if you write code to download the data, it can move linearly through the data.

+Where:

+* `blobCreationTimeUtc` is the time when the blob was created in the internal staging storage.

+* `blobDeliveryTimeUtc` is the time when the blob is copied to the export destination storage.

+The data is formatted so that:

+* Each blob is a text file that contains multiple `\n`-separated rows. It contains the telemetry processed over a time period of roughly half a minute.

+* Each row represents a telemetry data point, such as a request or page view.

+* Each row is an unformatted JSON document. If you want to view the rows, open the blob in Visual Studio and select **Edit** > **Advanced** > **Format File**.

+ 

+For a detailed data model reference for the property types and values, see [Application Insights export data model](export-data-model.md).

+## Process the data

+On a small scale, you can write some code to pull apart your data and read it into a spreadsheet. For example:

+For a larger code sample, see [Using a worker role][exportasa].

+You're responsible for managing your storage capacity and deleting old data, if necessary.

+## Regenerate your storage key

+Select the **Continuous Export** tab and edit your export. Edit the**Export Destination** value, but leave the same storage selected. Select **OK** to confirm.

+Continuous export will restart.

+For export samples, see:

+On larger scales, consider [HDInsight](https://azure.microsoft.com/services/hdinsight/) Hadoop clusters in the cloud. HDInsight provides various technologies for managing and analyzing big data. You can use it to process data that's been exported from Application Insights.

+This section provides answers to common questions.

+### Can I get a one-time download of a chart?

+You can do that. At the top of the tab, select **Export Data**.

+### I set up an export, but why is there no data in my store?

+Did Application Insights receive any telemetry from your app since you set up the export? You'll only receive new data.

+### I tried to set up an export, but why was I denied access?

+If the account is owned by your organization, you have to be a member of the Owners or Contributors groups.

+### Can I export straight to my own on-premises store?

+No. Our export engine currently only works with Azure Storage at this time.

+### Is there any limit to the amount of data you put in my store?

+No. We'll keep pushing data in until you delete the export. We'll stop if we hit the outer limits for Blob Storage, but that limit is huge. It's up to you to control how much storage you use.

+### How many blobs should I see in the storage?

+ * For every data type you selected to export, a new blob is created every minute, if data is available.

+ * For applications with high traffic, extra partition units are allocated. In this case, each unit creates a blob every minute.

+### I regenerated the key to my storage, or changed the name of the container, but why doesn't the export work?

+Edit the export and select the **Export destination** tab. Leave the same storage selected as before, and select **OK** to confirm. Export will restart. If the change was within the past few days, you won't lose data.

+### Can I pause the export?

+Yes. Select **Disable**.

+* [Export to SQL by using Stream Analytics][exportasa]

+* [Detailed data model reference for property types and values](export-data-model.md)

+## Diagnostic settings-based export

+Diagnostic settings export is preferred because it provides extra features:

+ > * Azure Storage accounts with virtual networks, firewalls, and private links.

+ > * Export to Azure Event Hubs.

+ > Extra costs might be incurred because of an increase in calls to the destination, such as a storage account.

+1. [Migrate application to workspace based](convert-classic-resource.md).

+1. [Enable diagnostic settings export](create-workspace-resource.md#export-telemetry). Select **Diagnostic settings** > **Add diagnostic setting** from within your Application Insights resource.

+> If you want to store diagnostic logs in a Log Analytics workspace, there are two points to consider to avoid seeing duplicate data in Application Insights:

+>

+> * The Application Insights user can't have access to both workspaces. Set the Log Analytics [access control mode](../logs/log-analytics-workspace-overview.md#permissions) to **Requires workspace permissions**. Through [Azure role-based access control](./resources-roles-access-control.md), ensure the user only has access to the Log Analytics workspace the Application Insights resource is based on.

+>

+> These steps are necessary because Application Insights accesses telemetry across Application Insight resources, including Log Analytics workspaces, to provide complete end-to-end transaction operations and accurate application maps. Because diagnostic logs use the same table names, duplicate telemetry can be displayed if the user has access to multiple resources that contain the same data.

+[roles]: ./resources-roles-access-control.md

+ Title: Application Insights IP address collection | Microsoft Docs

+By default, IP addresses are temporarily collected but not stored in Application Insights. This process follows some basic steps.

+When telemetry is sent to Azure, Application Insights uses the IP address to do a geolocation lookup. Application Insights uses the results of this lookup to populate the fields `client_City`, `client_StateOrProvince`, and `client_CountryOrRegion`. The address is then discarded, and `0.0.0.0` is written to the `client_IP` field.

+To remove geolocation data, see the following articles:

+* **Browser telemetry**: Application Insights collects the sender's IP address. The ingestion endpoint calculates the IP address.

+* **Server telemetry**: The Application Insights telemetry module temporarily collects the client IP address. The IP address isn't collected locally when the `X-Forwarded-For` header is set. When the incoming IP address list has more than one item, the last IP address is used to populate geolocation fields.

+This behavior is by design to help avoid unnecessary collection of personal data and IP address location information. Whenever possible, we recommend avoiding the collection of personal data.

+> Although the default is to not collect IP addresses, you can override this behavior. We recommend verifying that the collection doesn't break any compliance requirements or local regulations.

+> To learn more about handling personal data in Application Insights, see [Guidance for personal data](../logs/personal-data-mgmt.md).

+When IP addresses aren't collected, city and other geolocation attributes populated by our pipeline by using the IP address also aren't collected. You can mask IP collection at the source. There are two ways to do it. You can:

+* Remove the client IP initializer. For more information, see [Configuration with Applications Insights Configuration](configuration-with-applicationinsights-config.md).

+* Provide your own custom initializer. For more information, see an [API filtering example](api-filtering-sampling.md).

+To enable IP collection and storage, the `DisableIpMasking` property of the Application Insights component must be set to `true`. You can set this property through Azure Resource Manager templates (ARM templates) or by calling the REST API.

+### ARM template

+If you need to modify the behavior for only a single Application Insights resource, use the Azure portal.

+1. Go to your Application Insights resource, and then select **Automation** > **Export template**.

+1. Select **Deploy**.

+ 

+1. Select **Edit template**.

+ 

+ > If you experience the error shown in the preceding screenshot, you can resolve it. It states: "The resource group is in a location that is not supported by one or more resources in the template. Please choose a different resource group." Temporarily select a different resource group from the dropdown list and then re-select your original resource group.

+1. In the JSON template, locate `properties` inside `resources`. Add a comma to the last JSON field, and then add the following new line: `"DisableIpMasking": true`. Then select **Save**.

+1. Select **Review + create** > **Create**.

+1. After the deployment is complete, new telemetry data will be recorded.

+ A list of properties is returned as a result. One of the properties should read `DisableIpMasking: true`. If you run the PowerShell commands before you deploy the new property with Azure Resource Manager, the property won't exist.

+The following [REST API](/rest/api/azure/) payload makes the same modifications:

+If you need a more flexible alternative than `DisableIpMasking`, you can use a [telemetry initializer](./api-filtering-sampling.md#addmodify-properties-itelemetryinitializer) to copy all or part of the IP address to a custom field.

+Unlike the server-side SDKs, the client-side JavaScript SDK doesn't calculate an IP address. By default, IP address calculation for client-side telemetry occurs at the ingestion endpoint in Azure.

+If you want to calculate the IP address directly on the client side, you need to add your own custom logic and use the result to set the `ai.location.ip` tag. When `ai.location.ip` is set, the ingestion endpoint doesn't perform IP address calculation, and the provided IP address is used for the geolocation lookup. In this scenario, the IP address is still zeroed out by default.

+To keep the entire IP address calculated from your custom logic, you could use a telemetry initializer that would copy the IP address data that you provided in `ai.location.ip` to a separate custom field. But again, unlike the server-side SDKs, the client-side SDK won't calculate the address for you if it can't rely on third-party libraries or your own custom logic.

+If client-side data traverses a proxy before forwarding to the ingestion endpoint, IP address calculation might show the IP address of the proxy and not the client.

+Newly collected IP addresses will appear in the `customDimensions_client-ip` column. The default `client-ip` column will still have all four octets zeroed out.

+* Learn more about how [IP address collection](https://apmtips.com/posts/2016-07-05-client-ip-address/) works in Application Insights. This article is an older external blog post written by one of our engineers. It predates the current default behavior where the IP address is recorded as `0.0.0.0`. The article goes into greater depth on the mechanics of the built-in telemetry initializer.

+More information on configuring WorkerService applications can be found in our guidance on [configuring telemetry modules in WorkerServices](./worker-service.md#configure-or-remove-default-telemetry-modules).

+ASP.NET Core applications may be configured in code or through the `appsettings.json` file. For more information, see [Configuration in ASP.NET Core](https://learn.microsoft.com/aspnet/core/fundamentals/configuration).

+[Application Insights SDK for Worker Service](https://www.nuget.org/packages/Microsoft.ApplicationInsights.WorkerService) is a new SDK, which is best suited for non-HTTP workloads like messaging, background tasks, and console applications. These types of applications don't have the notion of an incoming HTTP request like a traditional ASP.NET/ASP.NET Core web application. For this reason, using Application Insights packages for [ASP.NET](asp-net.md) or [ASP.NET Core](asp-net-core.md) applications isn't supported.

+The new SDK doesn't do any telemetry collection by itself. Instead, it brings in other well-known Application Insights auto collectors like [DependencyCollector](https://www.nuget.org/packages/Microsoft.ApplicationInsights.DependencyCollector/), [PerfCounterCollector](https://www.nuget.org/packages/Microsoft.ApplicationInsights.PerfCounterCollector/), and [ApplicationInsightsLoggingProvider](https://www.nuget.org/packages/Microsoft.Extensions.Logging.ApplicationInsights). This SDK exposes extension methods on `IServiceCollection` to enable and configure telemetry collection.

+The [Application Insights SDK for Worker Service](https://www.nuget.org/packages/Microsoft.ApplicationInsights.WorkerService) is best suited for non-HTTP applications no matter where or how they run. If your application is running and has network connectivity to Azure, telemetry can be collected. Application Insights monitoring is supported everywhere .NET Core is supported. This package can be used in the newly introduced [.NET Core Worker Service](https://devblogs.microsoft.com/aspnet/dotnet-core-workers-in-azure-container-instances), [background tasks in ASP.NET Core](/aspnet/core/fundamentals/host/hosted-services), and console apps like .NET Core and .NET Framework.

+You must have a valid Application Insights connection string. This string is required to send any telemetry to Application Insights. If you need to create a new Application Insights resource to get a connection string, see [Create an Application Insights resource](./create-new-resource.md).

+## Use Application Insights SDK for Worker Service

+ The following snippet shows the changes that must be added to your project's `.csproj` file:

+

+ ```xml

+ <ItemGroup>

+ <PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.13.1" />

+ </ItemGroup>

+ ```

+1. Configure the connection string in the `APPLICATIONINSIGHTS_CONNECTION_STRING` environment variable or in configuration (`appsettings.json`).

+ :::image type="content" source="media/migrate-from-instrumentation-keys-to-connection-strings/migrate-from-instrumentation-keys-to-connection-strings.png" alt-text="Screenshot displaying Application Insights overview and connection string." lightbox="media/migrate-from-instrumentation-keys-to-connection-strings/migrate-from-instrumentation-keys-to-connection-strings.png":::

+1. Retrieve an `ILogger` instance or `TelemetryClient` instance from the Dependency Injection (DI) container by calling `serviceProvider.GetRequiredService<TelemetryClient>();` or by using Constructor Injection. This step will trigger setting up of `TelemetryConfiguration` and auto-collection modules.

+## .NET Core LTS Worker Service application

+The full example is shared at the [NuGet website](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/WorkerService).

+1. Download and install .NET Core [Long Term Support (LTS)](https://dotnet.microsoft.com/platform/support/policy/dotnet-core).

+1. Create a new Worker Service project either by using a Visual Studio new project template or the command line `dotnet new worker`.

+1. Install the [Microsoft.ApplicationInsights.WorkerService](https://www.nuget.org/packages/Microsoft.ApplicationInsights.WorkerService) package to the application.

+1. Add `services.AddApplicationInsightsTelemetryWorkerService();` to the `CreateHostBuilder()` method in your `Program.cs` class, as in this example:

+ ```csharp

+ public static IHostBuilder CreateHostBuilder(string[] args) =>

+ Host.CreateDefaultBuilder(args)

+ .ConfigureServices((hostContext, services) =>

+ {

+ services.AddHostedService<Worker>();

+ services.AddApplicationInsightsTelemetryWorkerService();

+ });

+ ```

+1. Modify your `Worker.cs` as per the following example:

+ ```csharp

+ using Microsoft.ApplicationInsights;

+ using Microsoft.ApplicationInsights.DataContracts;

+

+ public class Worker : BackgroundService

+ private readonly ILogger<Worker> _logger;

+ private TelemetryClient _telemetryClient;

+ private static HttpClient _httpClient = new HttpClient();

+

+ public Worker(ILogger<Worker> logger, TelemetryClient tc)

+ _logger = logger;

+ _telemetryClient = tc;

+ }

+

+ protected override async Task ExecuteAsync(CancellationToken stoppingToken)

+ {

+ while (!stoppingToken.IsCancellationRequested)

+ _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);

+

+ using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))

+ {

+ _logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");

+ _logger.LogInformation("Calling bing.com");

+ var res = await _httpClient.GetAsync("https://bing.com");

+ _logger.LogInformation("Calling bing completed with status:" + res.StatusCode);

+ _telemetryClient.TrackEvent("Bing call event completed");

+ }

+

+ await Task.Delay(1000, stoppingToken);

+ ```

+1. Set up the connection string.

+ :::image type="content" source="media/migrate-from-instrumentation-keys-to-connection-strings/migrate-from-instrumentation-keys-to-connection-strings.png" alt-text="Screenshot that shows Application Insights overview and connection string." lightbox="media/migrate-from-instrumentation-keys-to-connection-strings/migrate-from-instrumentation-keys-to-connection-strings.png":::

+ > [!NOTE]

+ > We recommend that you specify the connection string in configuration. The following code sample shows how to specify a connection string in `appsettings.json`. Make sure `appsettings.json` is copied to the application root folder during publishing.

+ ```json

+ "ApplicationInsights":

+ {

+ "ConnectionString" : "InstrumentationKey=00000000-0000-0000-0000-000000000000;"

+ },

+ "Logging":

+ "LogLevel":

+ {

+ "Default": "Warning"

+ }

+ ```

+Typically, `APPLICATIONINSIGHTS_CONNECTION_STRING` specifies the connection string for applications deployed to web apps as web jobs.

+[This document](/aspnet/core/fundamentals/host/hosted-services?tabs=visual-studio) describes how to create background tasks in an ASP.NET Core application.

+The full example is shared at this [GitHub page](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/BackgroundTasksWithHostedService).

+1. Add `services.AddApplicationInsightsTelemetryWorkerService();` to the `ConfigureServices()` method, as in this example:

+ ```csharp

+ public static async Task Main(string[] args)

+ var host = new HostBuilder()

+ .ConfigureAppConfiguration((hostContext, config) =>

+ {

+ config.AddJsonFile("appsettings.json", optional: true);

+ })

+ .ConfigureServices((hostContext, services) =>

+ {

+ services.AddLogging();

+ services.AddHostedService<TimedHostedService>();

+

+ // connection string is read automatically from appsettings.json

+ services.AddApplicationInsightsTelemetryWorkerService();

+ })

+ .UseConsoleLifetime()

+ .Build();

+

+ using (host)

+ {

+ // Start the host

+ await host.StartAsync();

+

+ // Wait for the host to shutdown

+ await host.WaitForShutdownAsync();

+ }

+ ```

+ The following code is for `TimedHostedService`, where the background task logic resides:

+ ```csharp

+ using Microsoft.ApplicationInsights;

+ using Microsoft.ApplicationInsights.DataContracts;

+

+ public class TimedHostedService : IHostedService, IDisposable

+ private readonly ILogger _logger;

+ private Timer _timer;

+ private TelemetryClient _telemetryClient;

+ private static HttpClient httpClient = new HttpClient();

+

+ public TimedHostedService(ILogger<TimedHostedService> logger, TelemetryClient tc)

+ {

+ _logger = logger;

+ this._telemetryClient = tc;

+ }

+

+ public Task StartAsync(CancellationToken cancellationToken)

+ {

+ _logger.LogInformation("Timed Background Service is starting.");

+

+ _timer = new Timer(DoWork, null, TimeSpan.Zero,

+ TimeSpan.FromSeconds(1));

+

+ return Task.CompletedTask;

+ }

+

+ private void DoWork(object state)

+ _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);

+

+ using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))

+ {

+ _logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");

+ _logger.LogInformation("Calling bing.com");

+ var res = httpClient.GetAsync("https://bing.com").GetAwaiter().GetResult();

+ _logger.LogInformation("Calling bing completed with status:" + res.StatusCode);

+ _telemetryClient.TrackEvent("Bing call event completed");

+ }

+ ```

+1. Set up the connection string.

+ Use the same `appsettings.json` from the preceding .NET Core [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) Worker Service example.

+## .NET Core/.NET Framework console application

+As mentioned in the beginning of this article, the new package can be used to enable Application Insights telemetry from even a regular console application. This package targets [`NetStandard2.0`](/dotnet/standard/net-standard), so it can be used for console apps in .NET Core [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) or higher, and .NET Framework [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) or higher.

+The full example is shared at this [GitHub page](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/ConsoleApp).

+1. Modify *Program.cs* as shown in the following example:

+ ```csharp

+ using Microsoft.ApplicationInsights;

+ using Microsoft.ApplicationInsights.DataContracts;

+ using Microsoft.Extensions.DependencyInjection;

+ using Microsoft.Extensions.Logging;

+ using System;

+ using System.Net.Http;

+ using System.Threading.Tasks;

+

+ namespace WorkerSDKOnConsole

+ class Program

+ static async Task Main(string[] args)

+ // Create the DI container.

+ IServiceCollection services = new ServiceCollection();

+

+ // Being a regular console app, there is no appsettings.json or configuration providers enabled by default.

+ // Hence instrumentation key/ connection string and any changes to default logging level must be specified here.

+ services.AddLogging(loggingBuilder => loggingBuilder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>("Category", LogLevel.Information));

+ services.AddApplicationInsightsTelemetryWorkerService("instrumentation key here");

+

+ // To pass a connection string

+ // - aiserviceoptions must be created

+ // - set connectionstring on it

+ // - pass it to AddApplicationInsightsTelemetryWorkerService()

+

+ // Build ServiceProvider.

+ IServiceProvider serviceProvider = services.BuildServiceProvider();

+

+ // Obtain logger instance from DI.

+ ILogger<Program> logger = serviceProvider.GetRequiredService<ILogger<Program>>();

+

+ // Obtain TelemetryClient instance from DI, for additional manual tracking or to flush.

+ var telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();

+

+ var httpClient = new HttpClient();

+

+ while (true) // This app runs indefinitely. Replace with actual application termination logic.

+ logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);

+

+ // Replace with a name which makes sense for this operation.

+ using (telemetryClient.StartOperation<RequestTelemetry>("operation"))

+ {

+ logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is captured by Application Insights");

+ logger.LogInformation("Calling bing.com");

+ var res = await httpClient.GetAsync("https://bing.com");

+ logger.LogInformation("Calling bing completed with status:" + res.StatusCode);

+ telemetryClient.TrackEvent("Bing call event completed");

+ }

+

+ await Task.Delay(1000);

+

+ // Explicitly call Flush() followed by sleep is required in console apps.

+ // This is to ensure that even if application terminates, telemetry is sent to the back-end.

+ telemetryClient.Flush();

+ Task.Delay(5000).Wait();

+ ```

+This console application also uses the same default `TelemetryConfiguration`. It can be customized in the same way as the examples in earlier sections.

+Run your application. The workers from all the preceding examples make an HTTP call every second to bing.com and also emit few logs by using `ILogger`. These lines are wrapped inside the `StartOperation` call of `TelemetryClient`, which is used to create an operation. In this example, `RequestTelemetry` is named "operation."

+Application Insights collects these ILogger logs, with a severity of Warning or above by default, and dependencies. They're correlated to `RequestTelemetry` with a parent-child relationship. Correlation also works across process/network boundaries. For example, if the call was made to another monitored component, it's correlated to this parent as well.

+This custom operation of `RequestTelemetry` can be thought of as the equivalent of an incoming web request in a typical web application. It isn't necessary to use an operation, but it fits best with the [Application Insights correlation data model](./correlation.md). `RequestTelemetry` acts as the parent operation and every telemetry generated inside the worker iteration is treated as logically belonging to the same operation.

+This approach also ensures all the telemetry generated, both automatic and manual, will have the same `operation_id`. Because sampling is based on `operation_id`, the sampling algorithm either keeps or drops all the telemetry from a single iteration.

+The following sections list the full telemetry automatically collected by Application Insights.

+[Live Metrics](./live-stream.md) can be used to quickly verify if Application Insights monitoring is configured correctly. Although it might take a few minutes before telemetry appears in the portal and analytics, Live Metrics shows CPU usage of the running process in near real time. It can also show other telemetry like Requests, Dependencies, and Traces.

+Logs emitted via `ILogger` with the severity Warning or greater are automatically captured. Follow [ILogger docs](ilogger.md#logging-level) to customize which log levels are captured by Application Insights.

+Dependency collection is enabled by default. The article [Dependency tracking in Application Insights](asp-net-dependencies.md#automatically-tracked-dependencies) explains the dependencies that are automatically collected and also contains steps to do manual tracking.

+`EventCounterCollectionModule` is enabled by default, and it will collect a default set of counters from .NET Core [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) apps. The [EventCounter](eventcounters.md) tutorial lists the default set of counters collected. It also has instructions on how to customize the list.

+### Manually track other telemetry

+Although the SDK automatically collects telemetry as explained, in most cases, you'll need to send other telemetry to Application Insights. The recommended way to track other telemetry is by obtaining an instance of `TelemetryClient` from Dependency Injection and then calling one of the supported `TrackXXX()` [API](api-custom-events-metrics.md) methods on it. Another typical use case is [custom tracking of operations](custom-operations-tracking.md). This approach is demonstrated in the preceding worker examples.

+The default `TelemetryConfiguration` used by the Worker Service SDK is similar to the automatic configuration used in an ASP.NET or ASP.NET Core application, minus the telemetry initializers used to enrich telemetry from `HttpContext`.

+You can customize the Application Insights SDK for Worker Service to change the default configuration. Users of the Application Insights ASP.NET Core SDK might be familiar with changing configuration by using ASP.NET Core built-in [dependency injection](/aspnet/core/fundamentals/dependency-injection). The Worker Service SDK is also based on similar principles. Make almost all configuration changes in the `ConfigureServices()` section by calling appropriate methods on `IServiceCollection`, as detailed in the next section.

+> When you use this SDK, changing configuration by modifying `TelemetryConfiguration.Active` isn't supported and changes won't be reflected.

+### Use ApplicationInsightsServiceOptions

+The following table lists commonly used settings in `ApplicationInsightsServiceOptions`.

+|EnableQuickPulseMetricStream | Enable/Disable the Live Metrics feature. | True

+|EnableAdaptiveSampling | Enable/Disable Adaptive Sampling. | True

+|EnableHeartbeat | Enable/Disable the Heartbeats feature, which periodically (15-min default) sends a custom metric named "HeartBeatState" with information about the runtime like .NET version and Azure environment, if applicable. | True

+|AddAutoCollectedMetricExtractor | Enable/Disable the AutoCollectedMetrics extractor, which is a telemetry processor that sends pre-aggregated metrics about Requests/Dependencies before sampling takes place. | True

+|EnableDiagnosticsTelemetryModule | Enable/Disable `DiagnosticsTelemetryModule`. Disabling this setting will cause the following settings to be ignored: `EnableHeartbeat`, `EnableAzureInstanceMetadataTelemetryModule`, and `EnableAppServicesHeartbeatTelemetryModule`. | True

+For the most up-to-date list, see the [configurable settings in `ApplicationInsightsServiceOptions`](https://github.com/microsoft/ApplicationInsights-dotnet/blob/develop/NETCORE/src/Shared/Extensions/ApplicationInsightsServiceOptions.cs).

+The Application Insights SDK for Worker Service supports both fixed-rate and adaptive sampling. Adaptive sampling is enabled by default. Sampling can be disabled by using the `EnableAdaptiveSampling` option in [ApplicationInsightsServiceOptions](#use-applicationinsightsserviceoptions).

+To configure other sampling settings, you can use the following example:

+ // The following adds adaptive sampling with 15 items per sec.

+For more information, see the [Sampling](#sampling) document.

+### Add telemetry initializers

+Add any new telemetry initializer to the `DependencyInjection` container and the SDK automatically adds them to `TelemetryConfiguration`.

+### Remove telemetry initializers

+ // Remove a specific built-in telemetry initializer.

+ // Remove all initializers.

+### Add telemetry processors

+You can add custom telemetry processors to `TelemetryConfiguration` by using the extension method `AddApplicationInsightsTelemetryProcessor` on `IServiceCollection`. You use telemetry processors in [advanced filtering scenarios](./api-filtering-sampling.md#itelemetryprocessor-and-itelemetryinitializer) to allow for more direct control over what's included or excluded from the telemetry you send to Application Insights. Use the following example:

+### Configure or remove default telemetry modules

+The following auto-collection modules are enabled by default. These modules are responsible for automatically collecting telemetry. You can disable or configure them to alter their default behavior.

+* `AppServicesHeartbeatTelemetryModule` (There's currently an issue involving this telemetry module. For a temporary workaround, see [GitHub Issue 1689](https://github.com/microsoft/ApplicationInsights-dotnet/issues/1689

+To configure any default telemetry module, use the extension method `ConfigureTelemetryModule<T>` on `IServiceCollection`, as shown in the following example:

+### Configure the telemetry channel

+The default channel is `ServerTelemetryChannel`. You can override it as the following example shows:

+If you want to disable telemetry conditionally and dynamically, you can resolve the `TelemetryConfiguration` instance with an ASP.NET Core dependency injection container anywhere in your code and set the `DisableTelemetry` flag on it.

+This section provides answers to common questions.

+| .NET Core app scenario | Package |

+Yes. The configuration will be shared with the rest of the web application.

+Get an instance of `TelemetryClient` by using constructor injection and call the required `TrackXXX()` method on it. We don't recommend creating new `TelemetryClient` instances. A singleton instance of `TelemetryClient` is already registered in the `DependencyInjection` container, which shares `TelemetryConfiguration` with the rest of the telemetry. Creating a new `TelemetryClient` instance is recommended only if it needs a configuration that's separate from the rest of the telemetry.

+Visual Studio IDE onboarding is currently supported only for ASP.NET/ASP.NET Core applications. This document will be updated when Visual Studio ships support for onboarding Worker Service applications.

+No. [Azure Monitor Application Insights Agent](./status-monitor-v2-overview.md) currently supports .NET [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) only.

+ ```csharp

+ using Microsoft.ApplicationInsights.Channel;

+ using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;

+ public void ConfigureServices(IServiceCollection services)

+ {

+ // The following will configure the channel to use the given folder to temporarily

+ // store telemetry items during network or Application Insights server issues.

+ // User should ensure that the given folder already exists

+ // and that the application has read/write permissions.

+ services.AddSingleton(typeof(ITelemetryChannel),

+ new ServerTelemetryChannel () {StorageFolder = "/tmp/myfolder"});

+ services.AddApplicationInsightsTelemetryWorkerService();

+ }

+ ```

+[.NET Core console application](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/ConsoleApp):

+Use this sample if you're using a console application written in either .NET Core (2.0 or higher) or .NET Framework (4.7.2 or higher).

+[ASP.NET Core background tasks with HostedServices](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/BackgroundTasksWithHostedService):

+Use this sample if you're in ASP.NET Core and creating background tasks in accordance with [official guidance](/aspnet/core/fundamentals/host/hosted-services).

+[.NET Core Worker Service](https://github.com/microsoft/ApplicationInsights-dotnet/tree/develop/examples/WorkerService):

+Use this sample if you have a .NET Core [LTS](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) Worker Service application in accordance with [official guidance](/aspnet/core/fundamentals/host/hosted-services?tabs=visual-studio#worker-service-template).

+[Read and contribute to the code](https://github.com/microsoft/ApplicationInsights-dotnet).

+For the latest updates and bug fixes, [see the Release Notes](./release-notes.md).

+* [Enrich or filter auto-collected telemetry](./api-filtering-sampling.md).

+Your cluster must be configured to send metrics to [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md). For more information, see [Collect Prometheus metrics with Container insights](container-insights-prometheus-metrics-addon.md).

+If you're going to configure the cluster to [collect Prometheus metrics](container-insights-prometheus.md) with [Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md), you must have an Azure Monitor workspace where Prometheus metrics are stored. You can let the onboarding experience create an Azure Monitor workspace in the default resource group of the AKS cluster subscription or use an existing Azure Monitor workspace.

+This article describes how to send Prometheus metrics to a Log Analytics workspace with the Container insights monitoring addon. You can also send metrics to Azure Monitor managed service for Prometheus with the metrics addon which that supports standard Prometheus features such as PromQL and Prometheus alert rules. See [Collect Prometheus metrics with Container insights](container-insights-prometheus.md).

+Container insights can also scrape Prometheus metrics from your cluster and send the data to either Azure Monitor Logs or to Azure Monitor managed service for Prometheus. This requires exposing the Prometheus metrics endpoint through your exporters or pods and then configuring one of the addons for the Azure Monitor agent used by Container insights as shown the following diagram.

+[Azure Monitor managed service for Prometheus](../essentials/prometheus-metrics-overview.md) is a fully managed Prometheus-compatible service that supports industry standard features such as PromQL, Grafana dashboards, and Prometheus alerts. This requires configuring the *metrics addon* for the Azure Monitor agent, which sends data to Prometheus.

+> [!TIP]

+> You don't need to enable Container insights to configure your AKS cluster to send data to managed Prometheus. See [Collect Prometheus metrics from AKS cluster (preview)](../essentials/prometheus-metrics-enable.md) for details on how to configure your cluster without enabling Container insights.

+Use the following procedure to add Promtheus collection to your cluster that's already using Container insights.

+1. Open the **Kubernetes services** menu in the Azure portal and select your AKS cluster.

+2. Click **Insights**.

+3. Click **Monitor settings**.

+ :::image type="content" source="media/container-insights-prometheus-metrics-addon/aks-cluster-monitor-settings.png" lightbox="media/container-insights-prometheus-metrics-addon/aks-cluster-monitor-settings.png" alt-text="Screenshot of button for monitor settings for an AKS cluster.":::

+4. Click the checkbox for **Enable Prometheus metrics** and select your Azure Monitor workspace.

+5. To send the collected metrics to Grafana, select a Grafana workspace. See [Create an Azure Managed Grafana instance](../../managed-grafan) for details on creating a Grafana workspace.

+ :::image type="content" source="media/container-insights-prometheus-metrics-addon/aks-cluster-monitor-settings-details.png" lightbox="media/container-insights-prometheus-metrics-addon/aks-cluster-monitor-settings-details.png" alt-text="Screenshot of monitor settings for an AKS cluster.":::

+6. Click **Configure** to complete the configuration.

+See [Collect Prometheus metrics from AKS cluster (preview)](../essentials/prometheus-metrics-enable.md) for details on [verifying your deployment](../essentials/prometheus-metrics-enable.md#verify-deployment) and [limitations](../essentials/prometheus-metrics-enable.md#limitations)

+## Send metrics to Azure Monitor Logs

+You may want to collect additional data in addition to the predefined set of data collected by Container insights. This data isn't used by Container insights views but is available for log queries and alerts like the other data it collects. This requires configuring the *monitoring addon* for the Azure Monitor agent, which is the one currently used by Container insights to send data to a Log Analytics workspace.

+### Prometheus scraping settings

+Active scraping of metrics from Prometheus is performed from one of two perspectives:

+- **Cluster-wide**: Defined in the ConfigMap section *[Prometheus data_collection_settings.cluster]*.

+- **Node-wide**: Defined in the ConfigMap section *[Prometheus_data_collection_settings.node]*.

+| Endpoint | Scope | Example |

+|-|-||

+| Pod annotation | Cluster-wide | `prometheus.io/scrape: "true"` <br>`prometheus.io/path: "/mymetrics"` <br>`prometheus.io/port: "8000"` <br>`prometheus.io/scheme: "http"` |

+| Kubernetes service | Cluster-wide | `http://my-service-dns.my-namespace:9100/metrics` <br>`https://metrics-server.kube-system.svc.cluster.local/metrics`ΓÇï |

+| URL/endpoint | Per-node and/or cluster-wide | `http://myurl:9101/metrics` |

+When a URL is specified, Container insights only scrapes the endpoint. When Kubernetes service is specified, the service name is resolved with the cluster DNS server to get the IP address. Then the resolved service is scraped.

+|Scope | Key | Data type | Value | Description |

+||--|--|-|-|

+| Cluster-wide | | | | Specify any one of the following three methods to scrape endpoints for metrics. |

+| | `urls` | String | Comma-separated array | HTTP endpoint (either IP address or valid URL path specified). For example: `urls=[$NODE_IP/metrics]`. ($NODE_IP is a specific Container insights parameter and can be used instead of a node IP address. Must be all uppercase.) |

+| | `kubernetes_services` | String | Comma-separated array | An array of Kubernetes services to scrape metrics from kube-state-metrics. Fully qualified domain names must be used here. For example,`kubernetes_services = ["https://metrics-server.kube-system.svc.cluster.local/metrics",http://my-service-dns.my-namespace.svc.cluster.local:9100/metrics]`|

+| | `monitor_kubernetes_pods` | Boolean | true or false | When set to `true` in the cluster-wide settings, the Container insights agent will scrape Kubernetes pods across the entire cluster for the following Prometheus annotations:<br> `prometheus.io/scrape:`<br> `prometheus.io/scheme:`<br> `prometheus.io/path:`<br> `prometheus.io/port:` |

+| | `prometheus.io/scrape` | Boolean | true or false | Enables scraping of the pod, and `monitor_kubernetes_pods` must be set to `true`. |

+| | `prometheus.io/scheme` | String | http or https | Defaults to scraping over HTTP. If necessary, set to `https`. |

+| | `prometheus.io/path` | String | Comma-separated array | The HTTP resource path from which to fetch metrics. If the metrics path isn't `/metrics`, define it with this annotation. |

+| | `prometheus.io/port` | String | 9102 | Specify a port to scrape from. If the port isn't set, it will default to 9102. |

+| | `monitor_kubernetes_pods_namespaces` | String | Comma-separated array | An allowlist of namespaces to scrape metrics from Kubernetes pods.<br> For example, `monitor_kubernetes_pods_namespaces = ["default1", "default2", "default3"]` |

+| Node-wide | `urls` | String | Comma-separated array | HTTP endpoint (either IP address or valid URL path specified). For example: `urls=[$NODE_IP/metrics]`. ($NODE_IP is a specific Container insights parameter and can be used instead of a node IP address. Must be all uppercase.) |

+| Node-wide or cluster-wide | `interval` | String | 60s | The collection interval default is one minute (60 seconds). You can modify the collection for either the *[prometheus_data_collection_settings.node]* and/or *[prometheus_data_collection_settings.cluster]* to time units such as s, m, and h. |

+| Node-wide or cluster-wide | `fieldpass`<br> `fielddrop`| String | Comma-separated array | You can specify certain metrics to be collected or not from the endpoint by setting the allow (`fieldpass`) and disallow (`fielddrop`) listing. You must set the allowlist first. |

+### Configure ConfigMaps

+Perform the following steps to configure your ConfigMap configuration file for your cluster. ConfigMaps is a global list and there can be only one ConfigMap applied to the agent. You can't have another ConfigMaps overruling the collections.

+1. [Download](https://aka.ms/container-azm-ms-agentconfig) the template ConfigMap YAML file and save it as c*ontainer-azm-ms-agentconfig.yaml*. If you've already deployed a ConfigMap to your cluster and you want to update it with a newer configuration, you can edit the ConfigMap file you've previously used.

+1. Edit the ConfigMap YAML file with your customizations to scrape Prometheus metrics.

+ #### [Cluster-wide](#tab/cluster-wide)

+ To collect Kubernetes services cluster-wide, configure the ConfigMap file by using the following example:

+ ```

+ prometheus-data-collection-settings: |- ΓÇï

+ # Custom Prometheus metrics data collection settings

+ [prometheus_data_collection_settings.cluster] ΓÇï

+ interval = "1m" ## Valid time units are s, m, h.

+ fieldpass = ["metric_to_pass1", "metric_to_pass12"] ## specify metrics to pass through ΓÇï

+ fielddrop = ["metric_to_drop"] ## specify metrics to drop from collecting

+ kubernetes_services = ["http://my-service-dns.my-namespace:9102/metrics"]

+ ```

+ #### [Specific URL](#tab/url)

+ To configure scraping of Prometheus metrics from a specific URL across the cluster, configure the ConfigMap file by using the following example:

+ ```

+ prometheus-data-collection-settings: |- ΓÇï

+ # Custom Prometheus metrics data collection settings

+ [prometheus_data_collection_settings.cluster] ΓÇï

+ interval = "1m" ## Valid time units are s, m, h.

+ fieldpass = ["metric_to_pass1", "metric_to_pass12"] ## specify metrics to pass through ΓÇï

+ fielddrop = ["metric_to_drop"] ## specify metrics to drop from collecting

+ urls = ["http://myurl:9101/metrics"] ## An array of urls to scrape metrics from

+ ```

+ #### [DaemonSet](#tab/deamonset)

+ To configure scraping of Prometheus metrics from an agent's DaemonSet for every individual node in the cluster, configure the following example in the ConfigMap:

+ ```

+ prometheus-data-collection-settings: |- ΓÇï

+ # Custom Prometheus metrics data collection settings ΓÇï

+ [prometheus_data_collection_settings.node] ΓÇï

+ interval = "1m" ## Valid time units are s, m, h.

+ urls = ["http://$NODE_IP:9103/metrics"] ΓÇï

+ fieldpass = ["metric_to_pass1", "metric_to_pass2"] ΓÇï

+ fielddrop = ["metric_to_drop"] ΓÇï

+ ```

+ `$NODE_IP` is a specific Container insights parameter and can be used instead of a node IP address. It must be all uppercase.

+ #### [Pod annotation](#tab/pod)

+ To configure scraping of Prometheus metrics by specifying a pod annotation:

+ 1. In the ConfigMap, specify the following configuration:

+ ```

+ prometheus-data-collection-settings: |- ΓÇï

+ # Custom Prometheus metrics data collection settings

+ [prometheus_data_collection_settings.cluster] ΓÇï

+ interval = "1m" ## Valid time units are s, m, h

+ monitor_kubernetes_pods = true

+ ```

+ 1. Specify the following configuration for pod annotations:

+ ```

+ - prometheus.io/scrape:"true" #Enable scraping for this pod ΓÇï

+ - prometheus.io/scheme:"http" #If the metrics endpoint is secured then you will need to set this to `https`, if not default ΓÇÿhttpΓÇÖΓÇï

+ - prometheus.io/path:"/mymetrics" #If the metrics path is not /metrics, define it with this annotation. ΓÇï

+ - prometheus.io/port:"8000" #If port is not 9102 use this annotationΓÇï

+ ```

+

+ If you want to restrict monitoring to specific namespaces for pods that have annotations, for example, only include pods dedicated for production workloads, set the `monitor_kubernetes_pod` to `true` in ConfigMap. Then add the namespace filter `monitor_kubernetes_pods_namespaces` to specify the namespaces to scrape from. An example is `monitor_kubernetes_pods_namespaces = ["default1", "default2", "default3"]`.

+2. Run the following kubectl command: `kubectl apply -f <configmap_yaml_file.yaml>`.

+

+ Example: `kubectl apply -f container-azm-ms-agentconfig.yaml`.

+The configuration change can take a few minutes to finish before taking effect. You must restart all Azure Monitor Agent pods manually. When the restarts are finished, a message appears that's similar to the following and includes the result `configmap "container-azm-ms-agentconfig" created`.

+### Verify configuration

+To verify the configuration was successfully applied to a cluster, use the following command to review the logs from an agent pod: `kubectl logs ama-logs-fdf58 -n=kube-system`.

+If there are configuration errors from the Azure Monitor Agent pods, the output will show errors similar to the following example:

+```

+***************Start Config Processing********************

+config::unsupported/missing config schema version - 'v21' , using defaults

+```

+Errors related to applying configuration changes are also available for review. The following options are available to perform additional troubleshooting of configuration changes and scraping of Prometheus metrics:

+- From an agent pod logs using the same `kubectl logs` command.

+- From Live Data (preview). Live Data (preview) logs show errors similar to the following example:

+ ```

+ 2019-07-08T18:55:00Z E! [inputs.prometheus]: Error in plugin: error making HTTP request to http://invalidurl:1010/metrics: Get http://invalidurl:1010/metrics: dial tcp: lookup invalidurl on 10.0.0.10:53: no such host

+ ```

+- From the **KubeMonAgentEvents** table in your Log Analytics workspace. Data is sent every hour with *Warning* severity for scrape errors and *Error* severity for configuration errors. If there are no errors, the entry in the table will have data with severity *Info*, which reports no errors. The **Tags** property contains more information about the pod and container ID on which the error occurred and also the first occurrence, last occurrence, and count in the last hour.

+- For Azure Red Hat OpenShift v3.x and v4.x, check the Azure Monitor Agent logs by searching the **ContainerLog** table to verify if log collection of openshift-azure-logging is enabled.

+Errors prevent Azure Monitor Agent from parsing the file, causing it to restart and use the default configuration. After you correct the errors in ConfigMap on clusters other than Azure Red Hat OpenShift v3.x, save the YAML file and apply the updated ConfigMaps by running the command `kubectl apply -f <configmap_yaml_file.yaml`.

+For Azure Red Hat OpenShift v3.x, edit and save the updated ConfigMaps by running the command `oc edit configmaps container-azm-ms-agentconfig -n openshift-azure-logging`.

+### Query Prometheus metrics data

+To view Prometheus metrics scraped by Azure Monitor and any configuration/scraping errors reported by the agent, review [Query Prometheus metrics data](container-insights-log-query.md#prometheus-metrics).

+### View Prometheus metrics in Grafana

+Container insights supports viewing metrics stored in your Log Analytics workspace in Grafana dashboards. We've provided a template that you can download from Grafana's [dashboard repository](https://grafana.com/grafana/dashboards?dataSource=grafana-azure-monitor-datasource&category=docker). Use the template to get started and reference it to help you learn how to query other data from your monitored clusters to visualize in custom Grafana dashboards.

+- [See the default configuration for Prometheus metrics](../essentials/prometheus-metrics-scrape-default.md).

+- [Customize Prometheus metric scraping for the cluster](../essentials/prometheus-metrics-scrape-configuration.md).

+Azure Monitor is based on a [common monitoring data platform](data-platform.md) that includes

+- [Metrics](essentials/data-platform-metrics.md)

+- [Logs](logs/data-platform-logs.md)

+- Traces

+- Changes. This platform allows data from multiple resources to be analyzed together using a common set of tools in Azure Monitor. Monitoring data may also be sent to other locations to support certain scenarios, and some resources may write to other locations before they can be collected into Logs or Metrics.

+> Access to data in the Log Analytics Workspaces is governed as outline [here](logs/manage-access.md).

+| Activity log | The Activity log is collected into its own data store that you can view from the Azure Monitor menu or use to create Activity log alerts. |[Query the Activity log with the Azure portal](essentials/activity-log.md#view-the-activity-log) |

+Enabling the Azure diagnostics extension for Azure Virtual machines allows you to collect logs and metrics from the guest operating system of Azure compute resources including Azure Cloud Service (classic) Web and Worker Roles, Virtual Machines, Virtual Machine Scale Sets, and Service Fabric.

+Detailed application monitoring in Azure Monitor is done with [Application Insights](/azure/application-insights/), which collects data from applications running on various platforms. The application can be running in Azure, another cloud, or on-premises.

+Other services in Azure write data to the Azure Monitor data platform. This allows you to analyze data collected by these services with data collected by Azure Monitor and apply the same analysis and visualization tools.

+| [Microsoft Defender for Cloud](../security-center/index.yml) | Azure Monitor Logs | Microsoft Defender for Cloud stores the security data it collects in a Log Analytics workspace, which allows it to be analyzed with other log data collected by Azure Monitor. | [Data collection in Microsoft Defender for Cloud](../security-center/security-center-enable-data-collection.md) |

+| [Microsoft Sentinel](../sentinel/index.yml) | Azure Monitor Logs | Microsoft Sentinel stores the data it collects from different data sources in a Log Analytics workspace, which allows it to be analyzed with other log data collected by Azure Monitor. | [Connect data sources](../sentinel/quickstart-onboard.md) |

+# Monitor Azure resources with Azure Monitor

+In this article, you learn about:

+> This article describes Azure Monitor concepts and walks you through different menu items. To jump right into using Azure Monitor features, start with [Analyze metrics for an Azure resource](../essentials/tutorial-metrics.md).

+To learn how to use Metrics Explorer, see [Analyze metrics for an Azure resource](../essentials/tutorial-metrics.md).

+To learn how to create alert rules and view alerts, see [Create a metric alert for an Azure resource](../alerts/tutorial-metric-alert.md) or [Create a log query alert for an Azure resource](../alerts/tutorial-log-alert.md).

+To learn how to use Metrics Explorer, see [Analyze metrics for an Azure resource](../essentials/tutorial-metrics.md).

+To learn how to create a diagnostic setting, see [Collect and analyze resource logs from an Azure resource](../essentials/tutorial-resource-logs.md).

+For a list of insights that are available and links to their documentation, see [Insights](../insights/insights-overview.md) and [core solutions](../insights/solutions.md).

+- [Collect Prometheus metrics for your AKS cluster](../essentials/prometheus-metrics-enable.md).

+- [Customize scraping of Prometheus metrics](prometheus-metrics-scrape-configuration.md).

+ Title: Enable Azure Monitor managed service for Prometheus (preview)

+description: Enable Azure Monitor managed service for Prometheus (preview) and configure data collection from your Azure Kubernetes Service (AKS) cluster.

+# Collect Prometheus metrics from AKS cluster (preview)

+This article describes how to configure your Azure Kubernetes Service (AKS) cluster to send data to Azure Monitor managed service for Prometheus. When you configure your AKS cluster to send data to Azure Monitor managed service for Prometheus, a containerized version of the [Azure Monitor agent](../agents/agents-overview.md) is installed with a metrics extension. You just need to specify the Azure Monitor workspace that the data should be sent to.

+> [!NOTE]

+> The process described here doesn't enable [Container insights](../containers/container-insights-overview.md) on the cluster even though the Azure Monitor agent installed in this process is the same one used by Container insights. See [Enable Container insights](../containers/container-insights-onboard.md) for different methods to enable Container insights on your cluster. See [Collect Prometheus metrics with Container insights](../containers/container-insights-prometheus.md) for details on adding Prometheus collection to a cluster that already has Container insights enabled.

+## Prerequisites

+- You must either have an [Azure Monitor workspace](azure-monitor-workspace-overview.md) or [create a new one](azure-monitor-workspace-overview.md#create-an-azure-monitor-workspace).

+- The cluster must use [managed identity authentication](../containers/container-insights-enable-aks.md#migrate-to-managed-identity-authentication).

+- The following resource providers must be registered in the subscription of the AKS cluster and the Azure Monitor Workspace.

+ - Microsoft.ContainerService

+ - Microsoft.Insights

+ - Microsoft.AlertsManagement

+## Enable Prometheus metric collection

+Use any of the following methods to install the Azure Monitor agent on your AKS cluster and send Prometheus metrics to your Azure Monitor workspace.

+### [Azure portal](#tab/azure-portal)

+1. Open the **Azure Monitor workspaces** menu in the Azure portal and select your cluster.

+2. Select **Managed Prometheus** to display a list of AKS clusters.

+3. Click **Configure** next to the cluster you want to enable.

+ :::image type="content" source="media/prometheus-metrics-enable/azure-monitor-workspace-configure-prometheus.png" lightbox="media/prometheus-metrics-enable/azure-monitor-workspace-configure-prometheus.png" alt-text="Screenshot of Azure Monitor workspace with Prometheus configuration.":::

+### [CLI](#tab/cli)

+#### Prerequisites

+- Register the `AKS-PrometheusAddonPreview` feature flag in the Azure Kubernetes clusters subscription with the following command in Azure CLI: `az feature register --namespace Microsoft.ContainerService --name AKS-PrometheusAddonPreview`.

+- The aks-preview extension needs to be installed using the command `az extension add --name aks-preview`. For more information on how to install a CLI extension, see [Use and manage extensions with the Azure CLI](/azure/azure-cli-extensions-overview).

+- Azure CLI version 2.41.0 or higher is required for this feature.

+#### Install metrics addon

+Use `az aks update` with the `-enable-azuremonitormetrics` option to install the metrics addon. Following are multiple options depending on the Azure Monitor workspace and Grafana workspace you want to use.

+**Create a new default Azure Monitor workspace.**<br>

+If no Azure Monitor Workspace is specified, then a default Azure Monitor Workspace will be created in the `DefaultRG-<cluster_region>` following the format `DefaultAzureMonitorWorkspace-<mapped_region>`.

+This Azure Monitor Workspace will be in the region specific in [Region mappings](#region-mappings).

+```azurecli

+az aks update --enable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group>

+```

+**Use an existing Azure Monitor workspace.**<br>

+If the Azure Monitor workspace is linked to one or more Grafana workspaces, then the data will be available in Grafana.

+```azurecli

+az aks update --enable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group> --azure-monitor-workspace-resource-id <workspace-name-resource-id>

+```

+**Use an existing Azure Monitor workspace and link with an existing Grafana workspace.**<br>

+This creates a link between the Azure Monitor workspace and the Grafana workspace.

+```azurecli

+az aks update --enable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group> --azure-monitor-workspace-resource-id <azure-monitor-workspace-name-resource-id> --grafana-resource-id <grafana-workspace-name-resource-id>

+```

+The output for each command will look similar to the following:

+```json

+"azureMonitorProfile": {

+ "metrics": {

+ "enabled": true,

+ "kubeStateMetrics": {

+ "metrican'tationsAllowList": "",

+ "metricLabelsAllowlist": ""

+ }

+ }

+}

+```

+#### Optional parameters

+Following are optional parameters that you can use with the previous commands.

+- `--ksm-metric-annotations-allow-list` is a comma-separated list of Kubernetes annotations keys that will be used in the resource's labels metric. By default the metric contains only name and namespace labels. To include additional annotations provide a list of resource names in their plural form and Kubernetes annotation keys, you would like to allow for them. A single `*` can be provided per resource instead to allow any annotations, but that has severe performance implications.

+- `--ksm-metric-labels-allow-list` is a comma-separated list of additional Kubernetes label keys that will be used in the resource's labels metric. By default the metric contains only name and namespace labels. To include additional labels provide a list of resource names in their plural form and Kubernetes label keys you would like to allow for them. A single `*` can be provided per resource instead to allow any labels, but that has severe performance implications.

+**Use annotations and labels.**

+```azurecli

+az aks update --enable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group> --ksm-metric-labels-allow-list "namespaces=[k8s-label-1,k8s-label-n]" --ksm-metric-annotations-allow-list "pods=[k8s-annotation-1,k8s-annotation-n]"

+```

+The output will be similar to the following:

+```json

+ "azureMonitorProfile": {

+ "metrics": {

+ "enabled": true,

+ "kubeStateMetrics": {

+ "metrican'tationsAllowList": "pods=[k8s-annotation-1,k8s-annotation-n]",

+ "metricLabelsAllowlist": "namespaces=[k8s-label-1,k8s-label-n]"

+ }

+ }

+ }

+```

+## [Resource Manager](#tab/resource-manager)

+### Prerequisites

+- Register the `AKS-PrometheusAddonPreview` feature flag in the Azure Kubernetes clusters subscription with the following command in Azure CLI: `az feature register --namespace Microsoft.ContainerService --name AKS-PrometheusAddonPreview`.

+- The Azure Monitor workspace and Azure Managed Grafana workspace must already be created.

+- The template needs to be deployed in the same resource group as the Azure Managed Grafana workspace.

+### Retrieve required values for Grafana resource

+From the **Overview** page for the Azure Managed Grafana instance in the Azure portal, select **JSON view**.

+ Copy the value of the `principalId` field for the `SystemAssigned` identity.

+```json

+"identity": {

+ "principalId": "00000000-0000-0000-0000-000000000000",

+ "tenantId": "00000000-0000-0000-0000-000000000000",

+ "type": "SystemAssigned"

+ },

+```

+If you're using an existing Azure Managed Grafana instance that already has been linked to an Azure Monitor workspace then you need the list of Grafana integrations. Copy the value of the `azureMonitorWorkspaceIntegrations` field. If it doesn't exist, then the instance hasn't been linked with any Azure Monitor workspace.

+```json

+"properties": {

+ "grafanaIntegrations": {

+ "azureMonitorWorkspaceIntegrations": [

+ {

+ "azureMonitorWorkspaceResourceId": "full_resource_id_1"

+ },

+ {

+ "azureMonitorWorkspaceResourceId": "full_resource_id_2"

+ }

+ ]

+ }

+}

+```

+### Assign role to system identity

+The Azure Managed Grafana resource requires the `Monitoring Data Reader` role to read data from the Azure Monitor Workspace.

+1. From the **Access control (IAM)** page for the Azure Managed Grafana instance in the Azure portal, select **Add** and then **Add role assignment**.

+2. Select `Monitoring Data Reader`.

+3. Select **Managed identity** and then **Select members**.

+4. Select the **system-assigned managed identity** with the `principalId` from the Grafana resource.

+5. Click **Select** and then **Review+assign**.

+### Download and edit template and parameter file

+1. Download the template at [https://aka.ms/azureprometheus-enable-arm-template](https://aka.ms/azureprometheus-enable-arm-template) and save it as **existingClusterOnboarding.json**.

+2. Download the parameter file at [https://aka.ms/azureprometheus-enable-arm-template-parameterss](https://aka.ms/azureprometheus-enable-arm-template-parameters) and save it as **existingClusterParam.json**.

+3. Edit the values in the parameter file.

+ | Parameter | Value |

+ |:|:|

+ | `azureMonitorWorkspaceResourceId` | Resource ID for the Azure Monitor workspace. Retrieve from the **JSON view** on the **Overview** page for the Azure Monitor workspace. |

+ | `azureMonitorWorkspaceLocation` | Location of the Azure Monitor workspace. Retrieve from the **JSON view** on the **Overview** page for the Azure Monitor workspace. |

+ | `clusterResourceId` | Resource ID for the AKS cluster. Retrieve from the **JSON view** on the **Overview** page for the cluster. |

+ | `clusterLocation` | Location of the AKS cluster. Retrieve from the **JSON view** on the **Overview** page for the cluster. |

+ | `metricLabelsAllowlist` | Comma-separated list of Kubernetes labels keys that will be used in the resource's labels metric. |

+ | `metricAnnotationsAllowList` | Comma-separated list of additional Kubernetes label keys that will be used in the resource's labels metric. |

+ | `grafanaResourceId` | Resource ID for the managed Grafana instance. Retrieve from the **JSON view** on the **Overview** page for the Grafana instance. |

+ | `grafanaLocation` | Location for the managed Grafana instance. Retrieve from the **JSON view** on the **Overview** page for the Grafana instance. |

+ | `grafanaSku` | SKU for the managed Grafana instance. Retrieve from the **JSON view** on the **Overview** page for the Grafana instance. Use the **sku.name**. |

+4. Open the template file and update the `grafanaIntegrations` property at the end of the file with the values that you retrieved from the Grafana instance. This will be similar to the following:

+ ```json

+ {

+ "type": "Microsoft.Dashboard/grafana",

+ "apiVersion": "2022-08-01",

+ "name": "[split(parameters('grafanaResourceId'),'/')[8]]",

+ "sku": {

+ "name": "[parameters('grafanaSku')]"

+ },

+ "location": "[parameters('grafanaLocation')]",

+ "properties": {

+ "grafanaIntegrations": {

+ "azureMonitorWorkspaceIntegrations": [

+ {

+ "azureMonitorWorkspaceResourceId": "full_resource_id_1"

+ },

+ {

+ "azureMonitorWorkspaceResourceId": "full_resource_id_2"

+ },

+ {

+ "azureMonitorWorkspaceResourceId": "[parameters('azureMonitorWorkspaceResourceId')]"

+ }

+ ]

+ }

+ }

+ ````

+In this json, `full_resource_id_1` and `full_resource_id_2` were already in the Azure Managed Grafana resource JSON, and they're added here to the ARM template. If you have no existing Grafana integrations, then don't include these entries for `full_resource_id_1` and `full_resource_id_2`.

+The final `azureMonitorWorkspaceResourceId` entry is already in the template and is used to link to the Azure Monitor Workspace resource ID provided in the parameters file.

+### Deploy template

+Deploy the template with the parameter file using any valid method for deploying Resource Manager templates. See [Deploy the sample templates](../resource-manager-samples.md#deploy-the-sample-templates) for examples of different methods.

+## Verify Deployment

+Run the following command to which verify that the daemon set was deployed properly:

+```

+kubectl get ds ama-metrics-node --namespace=kube-system

+```

+The output should resemble the following:

+```

+User@aksuser:~$ kubectl get ds ama-metrics-node --namespace=kube-system

+NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE

+ama-metrics-node 1 1 1 1 1 <none> 10h

+```

+Run the following command to which verify that the replica set was deployed properly:

+```

+kubectl get rs --namespace=kube-system

+```

+The output should resemble the following:

+```

+User@aksuser:~$kubectl get rs --namespace=kube-system

+NAME DESIRED CURRENT READY AGE

+ama-metrics-5c974985b8 1 1 1 11h

+ama-metrics-ksm-5fcf8dffcd 1 1 1 11h

+```

+## Limitations

+- Ensure that you update the `kube-state metrics` Annotations and Labels list with proper formatting. There's a limitation in the Resource Manager template deployments that require exact values in the `kube-state` metrics pods. If the kuberenetes pod has any issues with malformed parameters and isn't running, then the feature won't work as expected.

+- A data collection rule and data collection endpoint is created with the name `MSPROM-\<cluster-name\>-\<cluster-region\>`. These names can't currently be modified.

+- You must get the existing Azure Monitor workspace integrations for a Grafana workspace and update the Resource Manager template with it, otherwise it will overwrite and remove the existing integrations from the grafana workspace.

+- CPU and Memory requests and limits can't be changed for Container insights metrics addon. If changed, they'll be reconciled and replaced by original values in a few seconds.

+- Metrics addon doesn't work on AKS clusters configured with HTTP proxy.

+## Uninstall metrics addon

+Currently, Azure CLI is the only option to remove the metrics addon and stop sending Prometheus metrics to Azure Monitor managed service for Prometheus.

+If you don't already have it, install the aks-preview extension with the following command.

+The `aks-preview` extension needs to be installed using the following command. For more information on how to install a CLI extension, see [Use and manage extensions with the Azure CLI](/cli/azure/azure-cli-extensions-overview).

+```azurecli

+az extension add --name aks-preview

+```

+Use the following command to remove the agent from the cluster nodes and delete the recording rules created for the data being collected from the cluster. This doesn't remove the DCE, DCR, or the data already collected and stored in your Azure Monitor workspace.

+```azurecli

+az aks update --disable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group>

+```

+## Region mappings

+When you allow a default Azure Monitor workspace to be created when you install the metrics addon, it's created in the region listed in the following table.

+| AKS Cluster region | Azure Monitor workspace region |

+|--||

+|australiacentral |eastus|

+|australiacentral2 |eastus|

+|australiaeast |eastus|

+|australiasoutheast |eastus|

+|brazilsouth |eastus|

+|canadacentral |eastus|

+|canadaeast |eastus|

+|centralus |centralus|

+|centralindia |centralindia|

+|eastasia |westeurope|

+|eastus |eastus|

+|eastus2 |eastus2|

+|francecentral |westeurope|

+|francesouth |westeurope|

+|japaneast |eastus|

+|japanwest |eastus|

+|koreacentral |westeurope|

+|koreasouth |westeurope|

+|northcentralus |eastus|

+|northeurope |westeurope|

+|southafricanorth |westeurope|

+|southafricawest |westeurope|

+|southcentralus |eastus|

+|southeastasia |westeurope|

+|southindia |centralindia|

+|uksouth |westeurope|

+|ukwest |westeurope|

+|westcentralus |eastus|

+|westeurope |westeurope|

+|westindia |centralindia|

+|westus |westus|

+|westus2 |westus2|

+|westus3 |westus|

+|norwayeast |westeurope|

+|norwaywest |westeurope|

+|switzerlandnorth |westeurope|

+|switzerlandwest |westeurope|

+|uaenorth |westeurope|

+|germanywestcentral |westeurope|

+|germanynorth |westeurope|

+|uaecentral |westeurope|

+|eastus2euap |eastus2euap|

+|centraluseuap |westeurope|

+|brazilsoutheast |eastus|

+|jioindiacentral |centralindia|

+|swedencentral |westeurope|

+|swedensouth |westeurope|

+|qatarcentral |westeurope|

+## Next steps

+- [See the default configuration for Prometheus metrics](prometheus-metrics-scrape-default.md).

+- [Customize Prometheus metric scraping for the cluster](prometheus-metrics-scrape-configuration.md).

+You can create multiple Data Collection Rules that point to the same Data Collection Endpoint for metrics to be sent to additional Azure Monitor Workspaces from the same Kubernetes cluster. Currently, this is only available through onboarding through Resource Manager templates. You can follow the [regular onboarding process](prometheus-metrics-enable.md) and then edit the same Resource Manager templates to add additional DCRs for your additional Azure Monitor Workspaces. You'll need to edit the template to add an additional parameters for every additional Azure Monitor workspace, add another DCR for every additional Azure Monitor workspace, and add an additional Azure Monitor workspace integration for Grafana.

+- [Collect Prometheus metrics from AKS cluster](prometheus-metrics-enable.md).

+- Azure Kubernetes service (AKS)

+- Any Kubernetes cluster running self-managed Prometheus using [remote-write](https://aka.ms/azureprometheus-promio-prw).

+## Enable

+The only requirement to enable Azure Monitor managed service for Prometheus is to create an [Azure Monitor workspace](azure-monitor-workspace-overview.md), which is where Prometheus metrics are stored. Once this workspace is created, you can onboard services that collect Prometheus metrics.

+- To collect Prometheus metrics from your AKS cluster without using Container insights, see [Collect Prometheus metrics from AKS cluster (preview)](prometheus-metrics-enable.md).

+- To add collection of Prometheus metrics to your cluster using Container insights, see [Collect Prometheus metrics with Container insights](../containers/container-insights-prometheus.md#send-data-to-azure-monitor-managed-service-for-prometheus).

+- To configure remote-write to collect data from your self-managed Prometheus server, see [Azure Monitor managed service for Prometheus remote write - managed identity (preview)](prometheus-remote-write-managed-identity.md).

+- [Enable Azure Monitor managed service for Prometheus](prometheus-metrics-enable.md).

+This article provides instructions on customizing metrics scraping for a Kubernetes cluster with the [metrics addon](prometheus-metrics-enable.md) in Azure Monitor.

+This article lists the default targets, dashboards, and recording rules when you [configure Prometheus metrics to be scraped from an AKS cluster](prometheus-metrics-enable.md) for any AKS cluster.

+ Title: Analyze metrics for an Azure resource

+# Analyze metrics for an Azure resource

+Following is a video that shows a more extensive scenario than the procedure outlined in this tutorial. If you are new to metrics, we suggest you read through this article first and then view the video to see more specifics.

+To complete the steps in this tutorial, you need the following:

+ Title: Collect resource logs from an Azure resource

+description: Learn how to configure diagnostic settings to send resource logs from an Azure resource io a Log Analytics workspace where they can be analyzed with a log query.

+# Collect and analyze resource logs from an Azure resource

+To complete the steps in this tutorial, you need the following:

+ Title: Azure Monitor Insights Overview

+description: Lists available Azure Monitor "Insights" and other Azure product integrations

+# Azure Monitor Insights overview

+Some services have a curated monitoring experience. That is, Microsoft provides customized functionality meant to act as a starting point for monitoring those services. These experiences are collectively known as *curated visualizations* with the larger more complex of them being called *Insights*.

+The experiences collect and analyze a subset of available telemetry for a given service(s). Depending on the service, the experiences might also provide out-of-the-box alerting. They present the telemetry in a visual layout. The visualizations vary in size and scale.

+Some visualizations are considered part of Azure Monitor and follow the support and service level agreements for Azure. They're supported in all Azure regions where Azure Monitor is available. Other curated visualizations provide less functionality, might not scale, and might have different agreements. Some might be based solely on Azure Monitor Workbooks, while others might have an extensive custom experience.

+## Insights and curated visualizations

+The following table lists the available curated visualizations and information about them. **Most** of the list below can be found in the [Insights hub in the Azure portal](https://ms.portal.azure.com/#view/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/~/more). The table uses the same grouping as portal.

+>[!NOTE]

+> Another type of older visualization called *monitoring solutions* is no longer in active development. The replacement technology is the Azure Monitor Insights, as mentioned here. We suggest you use the Insights and not deploy new instances of solutions. For more information on the solutions, see [Monitoring solutions in Azure Monitor](solutions.md).

+|Name with docs link| State | [Azure portal link](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/more)| Description |

+|:--|:--|:--|:--|

+|Compute||||

+ | [Azure VM Insights](/azure/azure-monitor/insights/vminsights-overview) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/virtualMachines) | Monitors your Azure VMs and Virtual Machine Scale Sets at scale. It analyzes the performance and health of your Windows and Linux VMs and monitors their processes and dependencies on other resources and external processes. |

+| [Azure Container Insights](/azure/azure-monitor/insights/container-insights-overview) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/containerInsights) | Monitors the performance of container workloads that are deployed to managed Kubernetes clusters hosted on Azure Kubernetes Service. It gives you performance visibility by collecting metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring from Kubernetes clusters, these metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux. |

+|Networking||||

+ | [Azure Network Insights](../../network-watcher/network-insights-overview.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/networkInsights) | Provides a comprehensive view of health and metrics for all your network resources. The advanced search capability helps you identify resource dependencies, enabling scenarios like identifying resources that are hosting your website, by searching for your website name. |

+|Storage||||

+ | [Azure Storage Insights](/azure/azure-monitor/insights/storage-insights-overview) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/storageInsights) | Provides comprehensive monitoring of your Azure Storage accounts by delivering a unified view of your Azure Storage services performance, capacity, and availability. |

+| [Azure Backup](../../backup/backup-azure-monitoring-use-azuremonitor.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_DataProtection/BackupCenterMenuBlade/backupReportsConfigure/menuId/backupReportsConfigure) | Provides built-in monitoring and alerting capabilities in a Recovery Services vault. |

+|Databases||||

+| [Azure Cosmos DB Insights](../../cosmos-db/cosmosdb-insights-overview.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/cosmosDBInsights) | Provides a view of the overall performance, failures, capacity, and operational health of all your Azure Cosmos DB resources in a unified interactive experience. |

+| [Azure Monitor for Azure Cache for Redis (preview)](../../azure-cache-for-redis/redis-cache-insights-overview.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/redisCacheInsights) | Provides a unified, interactive view of overall performance, failures, capacity, and operational health. |

+| [Azure Data Explorer Insights](/azure/data-explorer/data-explorer-insights) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/adxClusterInsights) | Azure Data Explorer Insights provides comprehensive monitoring of your clusters by delivering a unified view of your cluster performance, operations, usage, and failures. |

+ | [Azure Monitor Log Analytics Workspace](../logs/log-analytics-workspace-insights-overview.md) | Preview | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/lawsInsights) | Log Analytics Workspace Insights (preview) provides comprehensive monitoring of your workspaces through a unified view of your workspace usage, performance, health, agent, queries, and change log. This article will help you understand how to onboard and use Log Analytics Workspace Insights (preview). |

+|Security||||

+ | [Azure Key Vault Insights (preview)](../../key-vault/key-vault-insights-overview.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/keyvaultsInsights) | Provides comprehensive monitoring of your key vaults by delivering a unified view of your Key Vault requests, performance, failures, and latency. |

+|Monitor||||

+ | [Azure Monitor Application Insights](../app/app-insights-overview.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/applicationsInsights) | Extensible application performance management service that monitors the availability, performance, and usage of your web applications whether they're hosted in the cloud or on-premises. It uses the powerful data analysis platform in Azure Monitor to provide you with deep insights into your application's operations. It enables you to diagnose errors without waiting for a user to report them. Application Insights includes connection points to various development tools and integrates with Visual Studio to support your DevOps processes. |

+| [Azure activity Log Insights](../essentials/activity-log-insights.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_DataProtection/BackupCenterMenuBlade/backupReportsConfigure/menuId/backupReportsConfigure) | Provides built-in monitoring and alerting capabilities in a Recovery Services vault. |

+ | [Azure Monitor for Resource Groups](resource-group-insights.md) | GA | No | Triage and diagnose any problems your individual resources encounter, while offering context for the health and performance of the resource group as a whole. |

+|Integration||||

+ | [Azure Service Bus Insights](../../service-bus-messaging/service-bus-insights.md) | Preview | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/serviceBusInsights) | Azure Service Bus Insights provide a view of the overall performance, failures, capacity, and operational health of all your Service Bus resources in a unified interactive experience. |

+ [Azure IoT Edge](../../iot-edge/how-to-explore-curated-visualizations.md) | GA | No | Visualize and explore metrics collected from the IoT Edge device right in the Azure portal by using Azure Monitor Workbooks-based public templates. The curated workbooks use built-in metrics from the IoT Edge runtime. These views don't need any metrics instrumentation from the workload modules. |

+|Workloads||||

+| [Azure SQL Insights (preview)](/azure/azure-sql/database/sql-insights-overview) | Preview | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/sqlWorkloadInsights) | A comprehensive interface for monitoring any product in the Azure SQL family. SQL Insights uses dynamic management views to expose the data you need to monitor health, diagnose problems, and tune performance. Note: If you're just setting up SQL monitoring, use SQL Insights instead of the SQL Analytics solution. |

+| [Azure Monitor for SAP solutions](../../virtual-machines/workloads/sap/monitor-sap-on-azure.md) | Preview | No | An Azure-native monitoring product for anyone running their SAP landscapes on Azure. It works with both SAP on Azure Virtual Machines and SAP on Azure Large Instances. Collects telemetry data from Azure infrastructure and databases in one central location and visually correlates the data for faster troubleshooting. You can monitor different components of an SAP landscape, such as Azure virtual machines (VMs), high-availability clusters, SAP HANA database, and SAP NetWeaver, by adding the corresponding provider for that component. |

+|Other||||

+ | [Azure Virtual Desktop Insights](../../virtual-desktop/azure-monitor.md) | GA | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_WVD/WvdManagerMenuBlade/insights/menuId/insights) | Azure Virtual Desktop Insights is a dashboard built on Azure Monitor Workbooks that helps IT professionals understand their Azure Virtual Desktop environments. |

+ | [Azure Stack HCI Insights](/azure-stack/hci/manage/azure-stack-hci-insights) | Preview | [Yes](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/azureStackHCIInsights) | Based on Azure Monitor Workbooks. Provides health, performance, and usage insights about registered Azure Stack HCI version 21H2 clusters that are connected to Azure and enrolled in monitoring. It stores its data in a Log Analytics workspace, which allows it to deliver powerful aggregation and filtering and analyze data trends over time. |

+|Not in Azure portal Insight hub||||

+| [Azure Monitor Workbooks for Azure Active Directory](../../active-directory/reports-monitoring/howto-use-azure-monitor-workbooks.md) | General availability (GA) | [Yes](https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/Workbooks) | Azure Active Directory provides workbooks to understand the effect of your Conditional Access policies, troubleshoot sign-in failures, and identify legacy authentications. |

+| [Azure HDInsight](../../hdinsight/log-analytics-migration.md#insights) | Preview | No | An Azure Monitor workbook that collects important performance metrics from your HDInsight cluster and provides the visualizations and dashboards for most common scenarios. Gives a complete view of a single HDInsight cluster including resource utilization and application status.|

+|Analytics||||

+## Next steps

+- Reference some of the insights listed above to review their functionality

+- Understand [what Azure Monitor can monitor](../monitor-reference.md)

+> Many monitoring solutions are no longer in active development. We suggest you check each solution to see if it has a replacement. We suggest you not deploy new instances of solutions that have other options, even if those solutions are still available. Many have been replaced by a [newer curated visualization or insight](insights-overview.md).

+| **Get insights** | Logs support [insights](../insights/insights-overview.md) that provide a customized monitoring experience for particular applications and services. |

+Azure Monitor data is collected and stored based on resource provider namespaces. Each resource in Azure has a unique ID. The resource provider namespace is part of all unique IDs. For example, a key vault resource ID would be similar to `/subscriptions/d03b04c7-d1d4-eeee-aaaa-87b6fcb38b38/resourceGroups/KeyVaults/providers/Microsoft.KeyVault/vaults/mysafekeys ` . *Microsoft.KeyVault* is the resource provider namespace. *Microsoft.KeyVault/vaults/* is the resource provider.

+For a list of Azure resource provider namespaces, see [Resource providers for Azure services](/azure/azure-resource-manager/management/azure-services-resource-providers).

+For a list of resource providers that support Azure Monitor

+- **Metrics** - See [Supported metrics in Azure Monitor](essentials/metrics-supported.md).

+- **Metric alerts** - See [Supported resources for metric alerts in Azure Monitor](/alerts/alerts-metric-near-real-time.md).

+- **Prometheus metrics** - See [TBD](essentials/FILL ME IN.md).

+- **Resource logs** - See [Supported categories for Azure Monitor resource logs](/essentials/resource-logs-categories.md).

+- **Activity log** - All entries in the activity log are available for query, alerting and routing to Azure Monitor Logs store regardless of resource provider.

+## Services that require agents

+Azure Monitor can't see inside a service running its own application, operating system or container. That type of service requires one or more agents to be installed. The agent then runs as well to collect metrics, logs, traces and changes and forward them to Azure Monitor. The following services require agents for this reason.

+- [Azure Cloud Services](../cloud-services-extended-support/index.yml)

+- [Azure Virtual Machines](../virtual-machines/index.yml)

+- [Azure Virtual Machine Scale Sets](../virtual-machine-scale-sets/index.yml)

+- [Azure Service Fabric](../service-fabric/index.yml)

+In addition, applications also require either the Application Insights SDK or auto-instrumentation (via an agent) to collect information and write it to the Azure Monitor data platform.

+## Services with Insights

+Some services have curated monitoring experiences call "insights". Insights are meant to be a starting point for monitoring a service or set of services. Some insights may also automatically pull additional data that's not captured or stored in Azure Monitor. For more information on monitoring insights, see [Insights Overview](insights/insights-overview.md).

+The services and [older monitoring solutions](insights/solutions.md) in the following table store their data in Azure Monitor Logs so that it can be analyzed with other log data collected by Azure Monitor.

+| **The following solutions also integrate with parts of Azure Monitor. Note that solutions, which are based on Azure Monitor Logs and Log Analytics, are no longer under active development. Use [Insights](insights/insights-overview.md) instead.** | |

+- The stores for the **[data platform](data-platform.md)** are at the center of the diagram. Azure Monitor stores these fundamental types of data: metrics, logs, traces, and changes.

+- The **[sources of monitoring data](data-sources.md)** that populate these data stores are on the left.

+- The different functions that Azure Monitor performs with this collected data are on the right. This includes such actions as analysis, alerting.

+- At the bottom is a layer of integration pieces. These are actually integrated throughout other parts of the diagram, but that is too complex to show visually.

+Natively, Azure Monitor stores data as metrics, logs, or changes. Traces are stored in the Logs store. Each storage platform is optimized for particular monitoring scenarios, and each supports different features in Azure Monitor. It's important for you to understand the differences between features such as data analysis, visualizations, or alerting, so that you can implement your required scenario in the most efficient and cost effective manner.

+| Metrics | Metrics are numerical values that describe some aspect of a system at a particular point in time. Metrics are collected at regular intervals and are identified with a timestamp, a name, a value, and one or more defining labels. Metrics can be aggregated using various algorithms, compared to other metrics, and analyzed for trends over time.<br><br>Metrics in Azure Monitor are stored in a time-series database, which is optimized for analyzing time-stamped data. For more information, see [Azure Monitor Metrics](essentials/data-platform-metrics.md). |

+| Logs | [Logs](logs/data-platform-logs.md) are events that occurred within the system. They can contain different kinds of data and may be structured or free-form text with a timestamp. They may be created sporadically as events in the environment generate log entries, and a system under heavy load will typically generate more log volume.<br><br>Azure Monitor stores logs in the Azure Monitor Logs store. The store allows you to segregate logs into separate "Log Analytics workspaces". There you can analyze them using the Log Analytics tool. Log Analytics workspaces are based on [Azure Data Explorer](/azure/data-explorer/), which provides a powerful analysis engine and the [Kusto rich query language](/azure/kusto/query/). For more information, see [Azure Monitor Logs](logs/data-platform-logs.md). |

+## What data can Azure Monitor collect?

+- **Container** - Data about containers and applications running inside containers, such as Azure Kubernetes.

+- **Azure resource** - Data about the operation of an Azure resource. For a list of the resources that have metrics and/or logs, see [What can you monitor with Azure Monitor?](monitor-reference.md).

+For more information, see [List of insights and curated visualizations using Azure Monitor](insights/insights-overview.md). Some of the larger insights are described here.

+Multiple APIs are available to read and write metrics and logs to and from Azure Monitor in addition to accessing generated alerts. You can also configure and retrieve alerts. With APIs, you have unlimited possibilities to build custom solutions that integrate with Azure Monitor.

+* [Best practices](/azure/architecture/best-practices/monitoring) for monitoring cloud applications and services.

+[New pricing has been introduced for these services](https://azure.microsoft.com/blog/introducing-a-new-way-to-purchase-azure-monitoring-services/), and the OMS bundling is no longer available for new customers. None of the services that were part of OMS have changed, except for the consolidation into Azure Monitor described above. The OMS portal was retired and is no longer available.

+ Title: Alert when Azure virtual is down

+# Create alert when Azure virtual machine is unavailable

+In this article, you learn how to:

+To complete the steps in this article you need the following:

+ Title: Enable monitoring for Azure virtual machine

+# Enable monitoring for Azure virtual machine

+To monitor the health and performance of an Azure virtual machine, you need to install an agent to collect data from its guest operating system. VM insights monitors the guest operating system and workloads running on Azure virtual machines. When you enable monitoring for an Azure virtual machine, it installs the necessary agents and starts collecting performance, process, and dependency information from the guest operating system.

+> * Inspect graphs analyzing performance data collected from the virtual machine.

+To complete this tutorial, you need the following:

+Select **Insights** from your virtual machine's menu in the Azure portal. If VM insights hasn't been enabled, you should see a screen similar to the following allowing you to enable monitoring. Click **Enable**.

+ Title: Collect guest logs and metrics from Azure virtual machine

+# Collect guest logs and metrics from Azure virtual machine

+* [Protecting HANA databases configured with HSR on Azure NetApp Files with AzAcSnap](https://techcommunity.microsoft.com/t5/running-sap-applications-on-the/protecting-hana-databases-configured-with-hsr-on-azure-netapp/ba-p/3654620)

+ * Ensure that DNS servers have network connectivity to the Azure NetApp Files delegated subnet hosting the Azure NetApp Files volumes.

+> Any resource, including a VPN Gateway, that is associated with a public IP Standard SKU address can't be moved across subscriptions. For virtual machines, you can [disassociate the public IP address](../../../virtual-network/ip-services/remove-public-ip-address-vm.md) before moving across subscriptions.

+> | publicipaddresses | Yes | Yes - see [Networking move guidance](./move-limitations/networking-move-limitations.md) | Yes<br/><br/> Use [Azure Resource Mover](../../resource-mover/tutorial-move-region-virtual-machines.md) to move public IP address configurations (IP addresses are not retained). |

+> | virtualnetworkgateways | Yes | Yes - see [Networking move guidance](./move-limitations/networking-move-limitations.md) | No |

+ Title: Indexing configuration guide

+description: This article explains the configuration options of indexing process with Azure Video Indexer.

+# The indexing configuration guide

+It's important to understand the configuration options to index efficiently while ensuring you meet your indexing objectives. When indexing videos, users can use the default settings or adjust many of the settings. Azure Video Indexer allows you to choose between a range of language, indexing, custom models, and streaming settings that have implications on the insights generated, cost, and performance.

+This article explains each of the options and the impact of each option to enable informed decisions when indexing. The article discusses the [Azure Video Indexer website](https://www.videoindexer.ai/) experience but the same options apply when submitting jobs through the API (see the [API guide](video-indexer-use-apis.md)). When indexing large volumes, follow the [at-scale guide](considerations-when-use-at-scale.md).

+The initial upload screen presents options to define the video name, source language, and privacy settings.

+All the other setting options appear if you select Advanced options.

+## Default settings

+By default, Azure Video Indexer is configured to a **Video source language** of English, **Privacy** of private, **Standard** audio and video setting, and **Streaming quality** of single bitrate.

+> [!TIP]

+> This topic describes each indexing option in detail.

+Below are a few examples of when using the default setting might not be a good fit:

+- If you need insights observed people or matched person that is only available through Advanced Video.

+- If you're only using Azure Video Indexer for transcription and translation, indexing of both audio and video isnΓÇÖt required, **Basic** for audio should suffice.

+- If you're consuming Azure Video Indexer insights but have no need to generate a new media file, streaming isn't necessary and **No streaming** should be selected to avoid the encoding job and its associated cost.

+- If a video is primarily in a language that isn't English.

+### Video source language

+If you're aware of the language spoken in the video, select the language from the video source language list. If you're unsure of the language of the video, choose **Auto-detect single language**. When uploading and indexing your video, Azure Video Indexer will use language identification (LID) to detect the videos language and generate transcription and insights with the detected language.

+If the video may contain multiple languages and you aren't sure which ones, select **Auto-detect multi-language**. In this case, multi-language (MLID) detection will be applied when uploading and indexing your video.

+While auto-detect is a great option when the language in your videos varies, there are two points to consider when using LID or MLID:

+- LID/MLID don't support all the languages supported by Azure Video Indexer.

+- The transcription is of a higher quality when you pre-select the videoΓÇÖs appropriate language.

+Learn more about [language support and supported languages](language-support.md).

+### Privacy

+This option allows you to determine if the insights should only be accessible to users in your Azure Video Indexer account or to anyone with a link.

+### Indexing options

+When indexing a video with the default settings, beware each of the audio and video indexing options may be priced differently. See [Azure Video Indexer pricing](https://azure.microsoft.com/pricing/details/video-indexer/) for details.

+Below are the indexing type options with details of their insights provided. To modify the indexing type, select **Advanced settings**.

+|Audio only|Video only |Audio & Video |

+||||

+|Basic |||

+|Standard| Standard |Standard |

+|Advanced |Advanced|Advanced |

+## Advanced settings

+### Audio only

+- **Basic**: Indexes and extract insights by using audio only (ignoring video) and provides the following insights: transcription, translation, formatting of output captions and subtitles, named entities (brands, locations, people), and topics.

+- **Standard**: Indexes and extract insights by using audio only (ignoring video) and provides the following insights: transcription, translation, formatting of output captions and subtitles, emotions, keywords, named entities (brands, locations, people), sentiments, speakers, and topics.

+- **Advanced**: Indexes and extract insights by using audio only (ignoring video) and provides the following insights: transcription, translation, formatting of output captions and subtitles, audio effects (preview), emotions, keywords, named entities (brands, locations, people), sentiments, speakers, and articles.

+### Video only

+- **Standard**: Indexes and extract insights by using video only (ignoring audio) and provides the following insights: labels (OCR), named entities (OCR - brands, locations, people), OCR, people, scenes (keyframes and shots), and topics (OCR).

+- **Advanced**: Indexes and extract insights by using video only (ignoring audio) and provides the following insights: labels (OCR), matched person (preview), named entities (OCR - brands, locations, people), OCR, observed people (preview), people, scenes (keyframes and shots), and topics (OCR).

+### Audio and Video

+- **Standard**: Indexes and extract insights by using audio and video and provides the following insights: transcription, translation, formatting of output captions and subtitles, audio effects (preview), emotions, keywords, named entities (brands, locations, people), OCR, people, sentiments, speakers, and topics.

+- **Advanced**: Indexes and extract insights by using audio and video and provides the following insights: transcription, translation, formatting of output captions and subtitles, audio effects (preview), emotions, keywords, matched person (preview), named entities (brands, locations, people), OCR, observed people (preview), people, sentiments, speakers, and topics.

+### Streaming quality options

+When indexing a video, you can decide if encoding of the file should occur which will enable streaming. The sequence is as follows:

+Upload > Encode (optional) > Index & Analysis > Publish for streaming (optional)

+Encoding and streaming operations are performed by and billed by Azure Media Services. There are two additional operations associated with the creation of a streaming video:

+- The creation of a Streaming Endpoint.

+- Egress traffic ΓÇô the volume depends on the number of video playbacks, video playback length, and the video quality (bitrate).

+

+There are several aspects that influence the total costs of the encoding job. The first is if the encoding is with single or adaptive streaming. This will create either a single output or multiple encoding quality outputs. Each output is billed separately and depends on the source quality of the video you uploaded to Azure Video Indexer.

+For Media Services encoding pricing details, see [pricing](https://azure.microsoft.com/pricing/details/media-services/#pricing).

+When indexing a video, default streaming settings are applied. Below are the streaming type options that can be modified if you, select **Advanced** settings and go to **Streaming quality**.

+|Single bitrate|Adaptive bitrate| No streaming |

+||||

+- **Single bitrate**: With Single Bitrate, the standard Media Services encoder cost will apply for the output. If the video height is greater than or equal to 720p HD, Azure Video Indexer encodes it with a resolution of 1280 x 720. Otherwise, it's encoded as 640 x 468. The default setting is content-aware encoding.

+- **Adaptive bitrate**: With Adaptive Bitrate, if you upload a video in 720p HD single bitrate to Azure Video Indexer and select Adaptive Bitrate, the encoder will use the [AdaptiveStreaming](/rest/api/media/transforms/create-or-update?tabs=HTTP#encodernamedpreset) preset. An output of 720p HD (no output exceeding 720p HD is created) and several lower quality outputs are created (for playback on smaller screens/low bandwidth environments). Each output will use the Media Encoder Standard base price and apply a multiplier for each output. The multiplier is 2x for HD, 1x for non-HD, and 0.25 for audio and billing is per minute of the input video.

+ **Example**: If you index a video in the US East region that is 40 minutes in length and is 720p HP and have selected the streaming option of Adaptive Bitrate, 3 outputs will be created - 1 HD (multiplied by 2), 1 SD (multiplied by 1) and 1 audio track (multiplied by 0.25). This will total to (2+1+0.25) * 40 = 130 billable output minutes.

+ Output minutes (standard encoder): 130 x $0.015/minute = $1.95.

+- **No streaming**: Insights are generated but no streaming operation is performed and the video isn't available on the Azure Video Indexer website. When No streaming is selected, you aren't billed for encoding.

+### Customizing content models - People/Animated characters and Brand categories

+Azure Video Indexer allows you to customize some of its models to be adapted to your specific use case. These models include animated characters, brands, language, and person. If you have customized models, this section enables you to configure if one of the created models should be used for the indexing.

+## Next steps

+Learn more about [language support and supported languages](language-support.md).

+| **Language** | **Code** | **Transcription** | **LID** | **MLID** | **Translation** | **Customization** (language model) |

+|::|:--:|:--:|:--:|:--:|:-:|::|

+| Chinese (Simplified) | `zh-Hans` | Γ£ö | Γ£ö\*<br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| | Γ£ö | Γ£ö |

+| English United States | `en-US` | Γ£ö | Γ£ö\*<br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid) | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö | Γ£ö |

+| French | `fr-FR` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö | Γ£ö |

+| German | `de-DE` | Γ£ö | Γ£ö \* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö \* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö | Γ£ö |

+| Italian | `it-IT` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid) | Γ£ö | Γ£ö | Γ£ö |

+| Japanese | `ja-JP` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid) | Γ£ö | Γ£ö | Γ£ö |

+| Portuguese | `pt-BR` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid) | Γ£ö | Γ£ö | Γ£ö |

+| Russian | `ru-RU` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid) | Γ£ö | Γ£ö | Γ£ö |

+| Slovenian as default languages, w | `sl-SI` | | | | Γ£ö |

+| Spanish | `es-ES` | Γ£ö | Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö\* <br/>[Change default languages supported by LID and MLID](#change-default-languages-supported-by-lid-and-mlid)| Γ£ö | Γ£ö |

+### Change default languages supported by LID and MLID

+Languages marked with * (in the table above) are used as default when auto-detecting languages by LID or/and MLID. You can specify to use other supported languages (listed in the table above) as default languages, when [uploading a video](https://api-portal.videoindexer.ai/api-details#api=Operations&operation=Upload-Video) with an API and passing the `customLanguages` parameter. The `customLanguages` parameter allows up to 10 languages to be identified by LID or MLID.

+> To change the default languages that you want for LID or MLID to use when auto-detecting, call [upload a video](https://api-portal.videoindexer.ai/api-details#api=Operations&operation=Upload-Video) and set the `customLanguages` parameter.

+An annotation is any type of additional information that is added to an already existing text, be it a transcription of an audio file or an original text file.

+[JetStream DR](https://www.jetstreamsoft.com/product-portfolio/jetstream-dr/) is a cloud-native disaster recovery solution designed to minimize downtime of virtual machines (VMs) if there is a disaster. Instances of JetStream DR are deployed at both the protected and recovery sites.

+| **JetStream Management Server Virtual Appliance (MSA)** | MSA enables both Day 0 and Day 2 configuration, such as primary sites, protection domains, and recovering VMs. The MSA is deployed from an OVA on a vSphere node by the cloud admin. The MSA collects and maintains statistics relevant to VM protection and implements a vCenter plugin that allows you to manage JetStream DR natively with the vSphere Client. The MSA doesn't handle replication data of protected VMs. |

+| **JetStream DR Virtual Appliance (DRVA)** | Linux-based Virtual Machine appliance receives protected VMs replication data from the source ESXi host. It maintains the replication log and manages the transfer of the VMs and their data to the object store such as Azure Blob Storage. Depending on the number of protected VMs and the amount of VM data to replicate, the private cloud admin can create one or more DRVA instances. |

+| **JetStream ESXi host components (IO Filter packages)** | JetStream software installed on each ESXi host configured for JetStream DR. The host driver intercepts the vSphere VMs IO and sends the replication data to the DRVA. The IO filters also monitor relevant events, such as vMotion, Storage vMotion, snapshots, etc. |

+| **JetStream Protected Domain** | Logical group of VMs that will be protected together using the same policies and runbook. The data for all VMs in a protection domain is stored in the same Azure Blob container instance. A single DRVA instance handles replication to remote DR storage for all VMs in a Protected Domain. |

+| **Azure Blob Storage containers** | The protected VMs replicated data is stored in Azure Blobs. JetStream software creates one Azure Blob container instance for each JetStream Protected Domain. |

+ - Create Protected Domains (groups of related VMs) and assign DRVAs and the Azure Blob Storage/ANF.

+Disks with Write Accelerator enabled | Azure VM with WA disk backup is available in all Azure public regions starting from May 18, 2022. If WA disk backup is not required as part of VM backup, you can choose to remove with [**Selective disk** feature](selective-disk-backup-restore.md). <br><br>**Important** <br> Virtual machines with WA disks need internet connectivity for a successful backup (even though those disks are excluded from the backup).

+## Fetch mapping details

+To fetch the geo-code mapping list, run the following command:

+```azurecli-interactive

+ az cli list-locations

+```

+- **Pool allocation mode:** When creating a Batch account, you can choose between two pool allocation modes: **Batch service** or **user subscription**. For most cases, you should use the default Batch service mode, in which pools are allocated behind the scenes in Batch-managed subscriptions. In the alternative user subscription mode, Batch VMs and other resources are created directly in your subscription when a pool is created. User subscription accounts are primarily used to enable a small but important subset of scenarios. For more information, see [configuration for user subscription mode](batch-account-create-portal.md#additional-configuration-for-user-subscription-mode).

+- **'virtualMachineConfiguration' or 'cloudServiceConfiguration':** While you can currently create pools using either configuration, new pools should be configured using 'virtualMachineConfiguration' and not 'cloudServiceConfiguration'. All current and new Batch features will be supported by Virtual Machine Configuration pools. Cloud Services Configuration pools don't support all features and no new capabilities are planned. You won't be able to create new 'cloudServiceConfiguration' pools or add new nodes to existing pools [after February 29, 2024](https://azure.microsoft.com/updates/azure-batch-cloudserviceconfiguration-pools-will-be-retired-on-29-february-2024/). For more information, see [Migrate Batch pool configuration from Cloud Services to Virtual Machine](batch-pool-cloud-service-to-virtual-machine-configuration.md).

+- **Job and task run time considerations:** If you have jobs comprised primarily of short-running tasks, and the expected total task counts are small, so that the overall expected run time of the job isn't long, don't allocate a new pool for each job. The allocation time of the nodes will diminish the run time of the job.

+- **Multiple compute nodes:** Individual nodes aren't guaranteed to always be available. While uncommon, hardware failures, operating system updates, and a host of other issues can cause individual nodes to be offline. If your Batch workload requires deterministic, guaranteed progress, you should allocate pools with multiple nodes.

+- **Images with impending end-of-life (EOL) dates:** We strongly recommended avoiding images with impending Batch support end of life (EOL) dates. These dates can be discovered via the [`ListSupportedImages` API](/rest/api/batchservice/account/listsupportedimages), [PowerShell](/powershell/module/az.batch/get-azbatchsupportedimage), or [Azure CLI](/cli/azure/batch/pool/supported-images). It's your responsibility to periodically refresh your view of the EOL dates pertinent to your pools and migrate your workloads before the EOL date occurs. If you're using a custom image with a specified node agent, ensure that you follow Batch support end-of-life dates for the image for which your custom image is derived or aligned with.

+- **Unique resource names:** Batch resources (jobs, pools, etc.) often come and go over time. For example, you may create a pool on Monday, delete it on Tuesday, and then create another similar pool on Thursday. Each new resource you create should be given a unique name that you haven't used before. You can create uniqueness by using a GUID (either as the entire resource name, or as a part of it) or by embedding the date and time that the resource was created in the resource name. Batch supports [DisplayName](/dotnet/api/microsoft.azure.batch.jobspecification.displayname), which can give a resource a more readable name even if the actual resource ID is something that isn't human-friendly. Using unique names makes it easier for you to differentiate which particular resource did something in logs and metrics. It also removes ambiguity if you ever have to file a support case for a resource.

+- **Continuity during pool maintenance and failure:** It's best to have your jobs use pools dynamically. If your jobs use the same pool for everything, there's a chance that jobs won't run if something goes wrong with the pool. This principle is especially important for time-sensitive workloads. For example, select or create a pool dynamically when you schedule each job, or have a way to override the pool name so that you can bypass an unhealthy pool.

+- **Business continuity during pool maintenance and failure:** There are many reasons why a pool may not grow to the size you desire, such as internal errors or capacity constraints. Make sure you can retarget jobs at a different pool (possibly with a different VM size using [UpdateJob](/dotnet/api/microsoft.azure.batch.protocol.joboperationsextensions.update)) if necessary. Avoid relying on a static pool ID with the expectation that it will never be deleted and never change.

+For the purposes of isolation, if your scenario requires isolating jobs or tasks from each other, do so by having them in separate pools. A pool is the security isolation boundary in Batch, and by default, two pools aren't visible or able to communicate with each other. Avoid using separate Batch accounts as a means of security isolation unless the larger environment from which the Batch account operates in requires isolation.

+Batch node agents aren't automatically upgraded for pools that have non-zero compute nodes. To ensure your Batch pools receive the latest security fixes and updates to the Batch node agent, you need to either resize the pool to zero compute nodes or recreate the pool. It's recommended to monitor the [Batch Node Agent release notes](https://github.com/Azure/Batch/blob/master/changelogs/nodeagent/CHANGELOG.md) to understand changes to new Batch node agent versions. Checking regularly for updates when they were released enables you to plan upgrades to the latest agent version.

+Before you recreate or resize your pool, you should download any node agent logs for debugging purposes if you're experiencing issues with your Batch pool or compute nodes. This is further discussed in the [Nodes](#nodes) section.

+- **Pool efficiency and billing:** Batch itself incurs no extra charges. However, you do incur charges for Azure resources utilized, such as compute, storage, networking and any other resources that may be required for your Batch workload. You're billed for every compute node in the pool, regardless of the state it's in. For more information, see [Cost analysis and budgets for Azure Batch](budget.md).

+Pool allocation failures can happen at any point during first allocation or subsequent resizes. These failures can be due to temporary capacity exhaustion in a region or failures in other Azure services that Batch relies on. Your core quota isn't a guarantee but rather a limit.

+It's possible for Batch pools to experience downtime events in Azure. Understanding that problems can arise and you should develop your workflow to be resilient to re-executions. If nodes fail, Batch automatically attempts to recover these compute nodes on your behalf. This recovery may trigger rescheduling any running task on the node that is restored or on a different, available node. To learn more about interrupted tasks, see [Designing for retries](#design-for-retries-and-re-execution).

+When you create an Azure Batch pool using the Virtual Machine Configuration, you specify a VM image that provides the operating system for each compute node in the pool. You can create the pool with a supported Azure Marketplace image, or you can [create a custom image with an Azure Compute Gallery image](batch-sig-images.md). While you can also use a [managed image](batch-custom-images.md) to create a custom image pool, we recommend creating custom images using the Azure Compute Gallery whenever possible. Using the Azure Compute Gallery helps you provision pools faster, scale larger quantities of VMs, and improves reliability when provisioning VMs.

+Using a job to run a single task is inefficient. For example, it's more efficient to use a single job containing 1000 tasks rather than creating 100 jobs that contain 10 tasks each. If you used 1000 jobs, each with a single task that would be the least efficient, slowest, and most expensive approach to take.

+Avoid designing a Batch solution that requires thousands of simultaneously active jobs. There's no quota for tasks, so executing many tasks under as few jobs as possible efficiently uses your [job and job schedule quotas](batch-quota-limit.md#resource-quotas).

+A job doesn't automatically move to completed state unless explicitly terminated. This action can be automatically triggered through the [onAllTasksComplete](/dotnet/api/microsoft.azure.batch.common.onalltaskscomplete) property or [maxWallClockTime](/rest/api/batchservice/job/add#jobconstraints).

+There's a default [active job and job schedule quota](batch-quota-limit.md#resource-quotas). Jobs and job schedules in completed state don't count towards this quota.

+Compute nodes are by their nature ephemeral. Batch features such as [autopool](nodes-and-pools.md#autopools) and [autoscale](nodes-and-pools.md#automatic-scaling-policy) can make it easy for nodes to disappear. When nodes leave a pool (due to a resize or a pool delete), all the files on those nodes are also deleted. Because of this behavior, a task should move its output off of the node it's running on, and to a durable store before it completes. Similarly, if a task fails, it should move logs required to diagnose the failure to a durable store.

+Batch has integrated support Azure Storage to upload data via [OutputFiles](batch-task-output-files.md), and with various shared file systems, or you can perform the upload yourself in your tasks.

+Delete tasks when they're no longer needed, or set a [retentionTime](/dotnet/api/microsoft.azure.batch.taskconstraints.retentiontime) task constraint. If a `retentionTime` is set, Batch automatically cleans up the disk space used by the task when the `retentionTime` expires.

+Deleting tasks accomplishes two things:

+- Ensures that you don't have a build-up of tasks in the job. This action will help avoid difficulty in finding the task you're interested in as you'll have to filter through the Completed tasks.

+- Cleans up the corresponding task data on the node (provided `retentionTime` hasn't already been hit). This action helps ensure that your nodes don't fill up with task data and run out of disk space.

+Batch supports oversubscribing tasks on nodes (running more tasks than a node has cores). It's up to you to ensure that your tasks are right-sized for the nodes in your pool. For example, you may have a degraded experience if you attempt to schedule eight tasks that each consume 25% CPU usage onto one node (in a pool with `taskSlotsPerNode = 8`).

+Tasks should be designed to withstand failure and accommodate retry. This principle is especially important for long running tasks. Ensure that your tasks generate the same, single result even if they're run more than once. One way to achieve this outcome is to make your tasks "goal seeking." Another way is to make sure your tasks are idempotent (tasks will have the same outcome no matter how many times they're run).

+Tasks that only run for one to two seconds aren't ideal. Try to do a significant amount of work in an individual task (10 second minimum, going up to hours or days). If each task is executing for one minute (or more), then the scheduling overhead as a fraction of overall compute time is small.

+As with other tasks, the node [start task](jobs-and-tasks.md#start-task) should be idempotent, as it will be rerun every time the node boots. An idempotent task is simply one that produces a consistent result when run multiple times.

+Sometimes there's a need to run another agent alongside the Batch agent in the node. For example, you may want to gather data from the node and report it. We recommend that these agents be deployed as OS services, such as a Windows service or a Linux `systemd` service.

+These services must not take file locks on any files in Batch-managed directories on the node, because otherwise Batch will be unable to delete those directories due to the file locks. For example, if installing a Windows service in a start task, instead of launching the service directly from the start task working directory, copy the files elsewhere (or if the files exist just skip the copy). Then install the service from that location. When Batch reruns your start task, it will delete the start task working directory and create it again.

+### Temporary disks and `AZ_BATCH_NODE_ROOT_DIR`

+Batch relies on VM temporary disks, for Batch-compatible VM sizes, to store metadata related to task execution along with any artifacts of each task

+execution on this temporary disk. Examples of these temporary disk mount points or directories are: `/mnt/batch`, `/mnt/resource/batch`, and `D:\batch\tasks`.

+Replacing, remounting, junctioning, symlinking, or otherwise redirecting these mount points and directories or any of the parent directories

+isn't supported and can lead to instability. If you require more disk space, consider using a VM size or family that has temporary

+disk space that meets your requirements or [attaching data disks](/rest/api/batchservice/pool/add#datadisk). For more information, see the next

+section about attaching and preparing data disks for compute nodes.

+### Attaching and preparing data disks

+Each individual compute node will have the exact same data disk specification attached if specified as part of the Batch pool instance. Only

+new data disks may be attached to Batch pools. These data disks attached to compute nodes aren't automatically partitioned, formatted or

+mounted. It's your responsibility to perform these operations as part of your [start task](jobs-and-tasks.md#start-task). These start tasks

+must be crafted to be idempotent. A re-execution of the start task after the compute node has been provisioned is possible. If the start

+task isn't idempotent, potential data loss can occur on the data disks.

+#### Preparing data disks in Linux Batch pools

+Azure data disks in Linux are presented as block devices and assigned a typical `sd[X]` identifier. You shouldn't rely on static `sd[X]`

+assignments as these labels are dynamically assigned at boot time and aren't guaranteed to be consistent between the first and any subsequent

+boots. You should identify your attached disks through the mappings presented in `/dev/disk/azure/scsi1/`. For example, if you specified LUN 0

+for your data disk in the AddPool API, then this disk would manifest as `/dev/disk/azure/scsi1/lun0`. As an example, if you were to list this

+directory, you could potentially see:

+```

+user@host:~$ ls -l /dev/disk/azure/scsi1/

+total 0

+lrwxrwxrwx 1 root root 12 Oct 31 15:16 lun0 -> ../../../sdc

+```

+There's no need to translate the reference back to the `sd[X]` mapping in your preparation script, instead refer to the device directly.

+In this example, this device would be `/dev/disk/azure/scsi1/lun0`. You could provide this ID directly to `fdisk`, `mkfs`, and any other

+tooling required for your workflow.

+For more information about Azure data disks in Linux, see this [article](../virtual-machine-scale-sets/tutorial-use-disks-cli.md).

+#### Preparing data disks in Windows Batch pools

+Azure data disks attached to Batch Windows compute nodes are presented unpartitioned and unformatted. You'll need to enumerate disks

+with `RAW` partitions for actioning as part of your start task. This information can be retrieved using the `Get-Disk` PowerShell cmdlet.

+As an example, you could potentially see:

+```

+PS C:\Windows\system32> Get-Disk

+Number Friendly Name Serial Number HealthStatus OperationalStatus Total Size Partition

+ Style

+ - - -- - -

+0 Virtual HD Healthy Online 30 GB MBR

+1 Virtual HD Healthy Online 32 GB MBR

+2 Msft Virtu... Healthy Online 64 GB RAW

+```

+Where disk number 2 is the uninitialized data disk attached to this compute node. These disks can then be initialized, partitioned,

+and formatted as required for your workflow.

+For more information about Azure data disks in Linux, see this [article](../virtual-machine-scale-sets/tutorial-use-disks-powershell.md).

+When provisioning [Batch pools in a virtual network](batch-virtual-network.md), ensure that you're closely following the guidelines regarding the use of the `BatchNodeManagement.<region>` service tag, ports, protocols and direction of the rule. Use of the service tag is highly recommended; don't use underlying Batch service IP addresses as they can change over time. Using Batch service IP addresses directly can cause instability, interruptions, or outages for your Batch pools.

+For User Defined Routes (UDRs), it's recommended to use `BatchNodeManagement.<region>` [service tags](../virtual-network/virtual-networks-udr-overview.md#service-tags-for-user-defined-routes) instead of Batch service IP addresses as they can change over time.

+Ensure that your systems honor DNS Time-to-Live (TTL) for your Batch account service URL. Additionally, ensure that your Batch service clients and other connectivity mechanisms to the Batch service don't rely on IP addresses.

+If your requests receive 5xx level HTTP responses and there's a "Connection: close" header in the response, your Batch service client should observe the recommendation by closing the existing connection, re-resolving DNS for the Batch account service URL, and attempt following requests on a new connection.

+Typically, virtual machines in a Batch pool are accessed through public IP addresses that can change over the lifetime of the pool. This dynamic nature can make it difficult to interact with a database or other external service that limits access to certain IP addresses. To address this concern, you can create a pool using a set of static public IP addresses that you control. For more information, see [Create an Azure Batch pool with specified public IP addresses](create-pool-public-ip.md).

+You can't use the normal "ping"/ICMP protocol with cloud services, because the ICMP protocol isn't permitted through the Azure load balancer. For more information, see [Connectivity and networking for Azure Cloud Services](../cloud-services/cloud-services-connectivity-and-networking-faq.yml#can-i-ping-a-cloud-service-).

+Azure Batch creates and manages a set of users and groups on the VM, which shouldn't be altered:

+The automated cleanup for the working directory will be blocked if you run a service on Windows from the start task working directory, due to the folder still being in use. This action will lead to degraded performance. To fix this issue, change the directory for that service to a separate directory that isn't managed by Batch.

+For additional information on HTTP headers supported in Azure CDN, see [Front Door to backend](../frontdoor/front-door-http-headers-protocol.md#from-the-front-door-to-the-backend).

+ token="Bearer $(az account get-access-token --resource "https://management.azure.com/" | jq -r ".accessToken")"

+ curl -X DELETE https://management.azure.com/providers/Microsoft.Portal/usersettings/cloudconsole?api-version=2017-12-01-preview -H Authorization:"$token"

+ Title: Create an Anomaly Detector resource

+description: Create an Anomaly Detector resource

+# Create and Anomaly Detector resource

+Anomaly Detector service is a cloud-based Cognitive Service that uses machine-learning models to detect anomalies in your time series data. Here, you'll learn how to create an Anomaly Detector resource in the Azure portal.

+## Create an Anomaly Detector resource in Azure portal

+1. Create an Azure subscription if you don't have one - [Create one for free](https://azure.microsoft.com/free/cognitive-services)

+1. Once you have your Azure subscription, [create an Anomaly Detector resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesAnomalyDetector) in the Azure portal, and fill out the following fields:

+ - **Subscription**: Select your current subscription.

+ - **Resource group**: The [Azure resource group](/azure/cloud-adoption-framework/govern/resource-consistency/resource-access-management#what-is-an-azure-resource-group) that will contain your resource. You can create a new group or add it to a pre-existing group.

+ - **Region**: Select your local region, see supported [Regions](../regions.md).

+ - **Name**: Enter a name for your resource. We recommend using a descriptive name, for example *multivariate-msft-test*.

+ - **Pricing tier**: The cost of your resource depends on the pricing tier you choose and your usage. For more information, see [pricing details](https://azure.microsoft.com/pricing/details/cognitive-services/anomaly-detector/). You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

+> [!div class="mx-imgBorder"]

+> 

+1. Select **Identity** in the banner above and make sure you set the status as **On** which enables Anomaly Detector to visit your data in Azure in a secure way, then select **Review + create.**

+> [!div class="mx-imgBorder"]

+> 

+1. Wait a few seconds until validation passed, and select **Create** button from the bottom-left corner.

+1. After you select create, you'll be redirected to a new page that says Deployment in progress. After a few seconds, you'll see a message that says, Your deployment is complete, then select **Go to resource**.

+## Get Endpoint URL and keys

+In your resource, select **Keys and Endpoint** on the left navigation bar, copy the **key** (both key1 and key2 will work) and **endpoint** values from your Anomaly Detector resource.. You'll need the key and endpoint values to connect your application to the Anomaly Detector API.

+> [!div class="mx-imgBorder"]

+> 

+That's it! You could start preparing your data for further steps!

+## Next steps

+* [Join us to get more supports!](https://aka.ms/adadvisorsjoin)

+ Title: Prepare your data and upload to Storage Account

+description: Prepare your data and upload to Storage Account

+# Prepare your data and upload to Storage Account

+Multivariate Anomaly Detection requires training to process your data, and an Azure Storage Account to store your data for further training and inference steps.

+## Data preparation

+First you need to prepare your data for training and inference.

+### Input data schema

+Multivariate Anomaly Detection supports two types of data schemas: **OneTable** and **MultiTable**. You could use either of these schemas to prepare your data and upload to Storage Account for further training and inference.

+#### Schema 1: OneTable

+**OneTable** is one CSV file that contains all the variables that you want to train a Multivariate Anomaly Detection model and one `timestamp` column. Download [One Table sample data](https://mvaddataset.blob.core.windows.net/public-sample-data/sample_data_5_3000.csv)

+* The `timestamp` values should conform to *ISO 8601*; the values of other variables in other columns could be *integers* or *decimals* with any number of decimal places.

+* Variables for training and variables for inference should be consistent. For example, if you're using `series_1`, `series_2`, `series_3`, `series_4`, and `series_5` for training, you should provide exactly the same variables for inference.

+ ***Example：***

+

+#### Schema 2: MultiTable

+**MultiTable** is multiple CSV files in one file folder, and each CSV file contains only two columns of one variable, with the exact column names of: **timestamp** and **value**. Download [Multiple Tables sample data](https://mvaddataset.blob.core.windows.net/public-sample-data/sample_data_5_3000.zip) and unzip it.

+* The `timestamp` values should conform to *ISO 8601*; the `value` could be *integers* or *decimals* with any number of decimal places.

+* The name of the csv file will be used as the variable name and should be unique. For example, *temperature.csv* and *humidity.csv*.

+* Variables for training and variables for inference should be consistent. For example, if you're using `series_1`, `series_2`, `series_3`, `series_4`, and `series_5` for training, you should provide exactly the same variables for inference.

+ ***Example：***

+

+> [!div class="mx-imgBorder"]

+> 

+> [!NOTE]

+> If your timestamps have hours, minutes, and/or seconds, ensure that they're properly rounded up before calling the APIs.

+> For example, if your data frequency is supposed to be one data point every 30 seconds, but you're seeing timestamps like "12:00:01" and "12:00:28", it's a strong signal that you should pre-process the timestamps to new values like "12:00:00" and "12:00:30".

+> For details, please refer to the ["Timestamp round-up" section](../concepts/best-practices-multivariate.md#timestamp-round-up) in the best practices document.

+## Upload your data to Storage Account

+Once you prepare your data with either of the two schemas above, you could upload your CSV file (OneTable) or your data folder (MultiTable) to your Storage Account.

+1. [Create a Storage Account](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM), fill out the fields, which are similar to the steps when creating Anomaly Detector resource.

+ > [!div class="mx-imgBorder"]

+ > 

+2. Select **Container** to the left in your Storage Account resource and select **+Container** to create one that will store your data.

+3. Upload your data to the container.

+ **Upload *OneTable* data**

+ Go to the container that you created, and select **Upload**, then choose your prepared CSV file and upload.

+ Once your data is uploaded, select your CSV file and copy the **blob URL** through the small blue button. (Please paste the URL somewhere convenient for further steps.)

+ > [!div class="mx-imgBorder"]

+ > 

+ **Upload *MultiTable* data**

+ Go to the container that you created, and select **Upload**, then select **Advanced**, and initiate a folder name in **Upload to folder**, and select all the variables in separate CSV files and upload.

+ Once your data is uploaded, go into the folder, and select one CSV file in the folder, copy the **blob URL** and only keep the part before the name of this CSV file, so the final blob URL should ***link to the folder***. (Please paste the URL somewhere convenient for further steps.)

+ > [!div class="mx-imgBorder"]

+ > 

+4. Grant Anomaly Detector access to read the data in your Storage Account.

+ * In your container, select **Access Control(IAM)** to the left, select **+ Add** to **Add role assignment**. If you see the add role assignment is disabled, please contact your Storage Account owner to add Owner role to your Container.

+ > [!div class="mx-imgBorder"]

+ > 

+ * Search role of **Storage Blob Data Reader**, **click on it** and then select **Next**. Technically, the roles highlighted below and the *Owner* role all should work.

+ > [!div class="mx-imgBorder"]

+ > 

+ * Select assign access to **Managed identity**, and **Select Members**, then choose the anomaly detector resource that you created earlier, then select **Review + assign**.

+## Next steps

+* [Train a multivariate anomaly detection model](train-model.md)

+* [Best practices of multivariate anomaly detection](../concepts/best-practices-multivariate.md)

+ Title: Embedded Speech - Speech service

+description: Embedded Speech is designed for on-device scenarios where cloud connectivity is intermittent or unavailable.

+zone_pivot_groups: programming-languages-set-thirteen

+# Embedded Speech (preview)

+Embedded Speech is designed for on-device [speech-to-text](speech-to-text.md) and [text-to-speech](text-to-speech.md) scenarios where cloud connectivity is intermittent or unavailable. For example, you can use embedded speech in medical equipment, a voice enabled air conditioning unit, or a car that might travel out of range. You can also develop hybrid cloud and offline solutions. For scenarios where your devices must be in a secure environment like a bank or government entity, you should first consider [disconnected containers](/azure/cognitive-services/containers/disconnected-containers).

+> [!IMPORTANT]

+> Microsoft limits access to embedded speech. You can apply for access through the Azure Cognitive Services [embedded speech limited access review](https://aka.ms/csgate-embedded-speech). For more information, see [Limited access for embedded speech](/legal/cognitive-services/speech-service/embedded-speech/limited-access-embedded-speech?context=/azure/cognitive-services/speech-service/context/context).

+## Platform requirements

+Embedded speech is included with the Speech SDK (version 1.24.1 and higher) for C#, C++, and Java. Refer to the general [Speech SDK installation requirements](quickstarts/setup-platform.md) for programming language and target platform specific details.

+**Choose your target environment**

+# [Android](#tab/android)

+Requires Android 7.0 (API level 24) or higher on ARM64 (`arm64-v8a`) or ARM32 (`armeabi-v7a`) hardware.

+Embedded TTS with neural voices is only supported on ARM64.

+# [Linux](#tab/linux)

+Requires Linux on x64, ARM64, or ARM32 hardware with [supported Linux distributions](quickstarts/setup-platform.md?tabs=linux).

+Embedded speech isn't supported on RHEL/CentOS 7.

+Embedded TTS with neural voices isn't supported on ARM32.

+# [macOS](#tab/macos)

+Requires 10.14 or newer on x64 or ARM64 hardware.

+# [Windows](#tab/windows)

+Requires Windows 10 or newer on x64 or ARM64 hardware.

+The latest [Microsoft Visual C++ Redistributable for Visual Studio 2015-2022](/cpp/windows/latest-supported-vc-redist?view=msvc-170&preserve-view=true) must be installed regardless of the programming language used with the Speech SDK.

+The Speech SDK for Java doesn't support Windows on ARM64.

+## Limitations

+Embedded speech is only available with C#, C++, and Java SDKs. The other Speech SDKs, Speech CLI, and REST APIs don't support embedded speech.

+Embedded speech recognition only supports mono 16 bit, 16-kHz PCM-encoded WAV audio.

+Embedded neural voices only support 24-kHz sample rate.

+## Models and voices

+For embedded speech, you'll need to download the speech recognition models for [speech-to-text](speech-to-text.md) and voices for [text-to-speech](text-to-speech.md). Instructions will be provided upon successful completion of the [limited access review](https://aka.ms/csgate-embedded-speech) process.

+## Embedded speech configuration

+For cloud connected applications, as shown in most Speech SDK samples, you use the `SpeechConfig` object with a Speech resource key and region. For embedded speech, you don't use a Speech resource. Instead of a cloud resource, you use the [models and voices](#models-and-voices) that you downloaded to your local device.

+Use the `EmbeddedSpeechConfig` object to set the location of the models or voices. If your application is used for both speech-to-text and text-to-speech, you can use the same `EmbeddedSpeechConfig` object to set the location of the models and voices.

+```csharp

+// Provide the location of the models and voices.

+List<string> paths = new List<string>();

+paths.Add("C:\\dev\\embedded-speech\\stt-models");

+paths.Add("C:\\dev\\embedded-speech\\tts-voices");

+var embeddedSpeechConfig = EmbeddedSpeechConfig.FromPaths(paths.ToArray());

+// For speech-to-text

+embeddedSpeechConfig.SetSpeechRecognitionModel(

+ "Microsoft Speech Recognizer en-US FP Model V8",

+ Environment.GetEnvironmentVariable("MODEL_KEY"));

+// For text-to-speech

+embeddedSpeechConfig.SetSpeechSynthesisVoice(

+ "Microsoft Server Speech Text to Speech Voice (en-US, JennyNeural)",

+ Environment.GetEnvironmentVariable("VOICE_KEY"));

+embeddedSpeechConfig.SetSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat.Riff24Khz16BitMonoPcm);

+```

+You can find ready to use embedded speech samples at [GitHub](https://aka.ms/csspeech/samples).

+- [C# (.NET 6.0)](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/csharp/dotnetcore/embedded-speech)

+- [C# for Unity](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/csharp/unity/embedded-speech)

+> [!TIP]

+> The `GetEnvironmentVariable` function is defined in the [speech-to-text quickstart](get-started-speech-to-text.md) and [text-to-speech quickstart](get-started-text-to-speech.md).

+```cpp

+// Provide the location of the models and voices.

+vector<string> paths;

+paths.push_back("C:\\dev\\embedded-speech\\stt-models");

+paths.push_back("C:\\dev\\embedded-speech\\tts-voices");

+auto embeddedSpeechConfig = EmbeddedSpeechConfig::FromPaths(paths);

+// For speech-to-text

+embeddedSpeechConfig->SetSpeechRecognitionModel((

+ "Microsoft Speech Recognizer en-US FP Model V8",

+ GetEnvironmentVariable("MODEL_KEY"));

+// For text-to-speech

+embeddedSpeechConfig->SetSpeechSynthesisVoice(

+ "Microsoft Server Speech Text to Speech Voice (en-US, JennyNeural)",

+ GetEnvironmentVariable("VOICE_KEY"));

+embeddedSpeechConfig->SetSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat::Riff24Khz16BitMonoPcm);

+```

+You can find ready to use embedded speech samples at [GitHub](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/cpp/embedded-speech)

+```java

+// Provide the location of the models and voices.

+List<String> paths = new ArrayList<>();

+paths.add("C:\\dev\\embedded-speech\\stt-models");

+paths.add("C:\\dev\\embedded-speech\\tts-voices");

+var embeddedSpeechConfig = EmbeddedSpeechConfig.fromPaths(paths);

+// For speech-to-text

+embeddedSpeechConfig.setSpeechRecognitionModel(

+ "Microsoft Speech Recognizer en-US FP Model V8",

+ System.getenv("MODEL_KEY"));

+// For text-to-speech

+embeddedSpeechConfig.setSpeechSynthesisVoice(

+ "Microsoft Server Speech Text to Speech Voice (en-US, JennyNeural)",

+ System.getenv("VOICE_KEY"));

+embeddedSpeechConfig.setSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat.Riff24Khz16BitMonoPcm);

+```

+You can find ready to use embedded speech samples at [GitHub](https://aka.ms/csspeech/samples).

+- [Java (JRE)](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/java/jre/embedded-speech)

+- [Java for Android](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/java/android/embedded-speech)

+## Hybrid speech

+Hybrid speech with the `HybridSpeechConfig` object uses the cloud speech service by default and embedded speech as a fallback in case cloud connectivity is limited or slow.

+With hybrid speech configuration for [speech-to-text](speech-to-text.md) (recognition models), embedded speech is used when connection to the cloud service fails after repeated attempts. Recognition may continue using the cloud service again if the connection is later resumed.

+With hybrid speech configuration for [text-to-speech](text-to-speech.md) (voices), embedded and cloud synthesis are run in parallel and the result is selected based on which one gives a faster response. The best result is evaluated on each synthesis request.

+## Cloud speech

+For cloud speech, you use the `SpeechConfig` object, as shown in the [speech-to-text quickstart](get-started-speech-to-text.md) and [text-to-speech quickstart](get-started-text-to-speech.md). To run the quickstarts for embedded speech, you can replace `SpeechConfig` with `EmbeddedSpeechConfig` or `HybridSpeechConfig`. Most of the other speech recognition and synthesis code are the same, whether using cloud, embedded, or hybrid configuration.

+## Next steps

+- [Quickstart: Recognize and convert speech to text](get-started-speech-to-text.md)

+- [Quickstart: Convert text to speech](get-started-text-to-speech.md)

+description: Custom Translator offers similar capabilities to what Microsoft Translator Hub does for Statistical Machine Translation (SMT), but exclusively for Neural Machine Translation (NMT) systems.

+Translation systems built with [Custom Translator](https://portal.customtranslator.azure.ai) are available through the same cloud-based, secure, high performance, highly scalable Microsoft Translator [Text API V3](../reference/v3-0-translate.md?tabs=curl) that powers billions of translations every day.

+* [**Quickstarts**](./v2.0/quickstart.md) are getting-started instructions to guide you through making requests to the service.

+* [**How-to guides**](./v2.0/how-to/create-manage-workspace.md) contain instructions for using the feature in more specific or customized ways.

+|[Build systems that knows your business terminology](./v2.0/beginners-guide.md) | Customize and build translation systems using parallel documents that understand the terminologies used in your own business and industry. |

+|[Use a dictionary to build your models](./v2.0/how-to/train-custom-model.md#when-to-select-dictionary-only-training) | If you don't have training data set, you can train a model with only dictionary data. |

+|[Collaborate with others](./v2.0/how-to/create-manage-workspace.md#manage-workspace-settings) | Collaborate with your team by sharing your work with different people. |

+|[Access your custom translation model](./v2.0/how-to/translate-with-custom-model.md) | Your custom translation model can be accessed anytime by your existing applications/ programs via Microsoft Translator Text API V3. |

+The secure [Custom Translator](https://portal.customtranslator.azure.ai) portal enables users to upload training data, train systems, test systems, and deploy them to a production environment through an intuitive user interface. The system will then be available for use at scale within a few hours (actual time depends on training data size).

+[Custom Translator](https://portal.customtranslator.azure.ai) can also be programmatically accessed through a [dedicated API](https://custom-api.cognitive.microsofttranslator.com/swagger/). The API allows users to manage creating or updating training through their own app or webservice.

+Custom systems can be seamlessly accessed and integrated into any product or business workflow and on any device via the Microsoft Translator Text REST API.

+* Read about [pricing details](https://azure.microsoft.com/pricing/details/cognitive-services/translator-text-api/).

+* With [Quickstart](./v2.0/quickstart.md) learn to build a translation model in Custom Translator.

+ Title: Custom Translator for beginners

+description: A user guide for understanding the end-to-end customized machine translation process.

+# Custom Translator for beginners

+ [Custom Translator](../overview.md) enables you to a build translation system that reflects your business, industry, and domain-specific terminology and style. Training and deploying a custom system is easy and doesn't require any programming skills. The customized translation system seamlessly integrates into your existing applications, workflows, and websites and is available on Azure through the same cloud-based [Microsoft Text Translator API](../../reference/v3-0-translate.md?tabs=curl) service that powers billions of translations every day.

+The platform enables users to build and publish custom translation systems to and from English. The Custom Translator supports more than three dozen languages that map directly to the languages available for NMT. For a complete list, *see* [Translator language support](../../language-support.md).

+## Is a custom translation model the right choice for me?

+A well-trained custom translation model provides more accurate domain-specific translations because it relies on previously translated in-domain documents to learn preferred translations. Translator uses these terms and phrases in context to produce fluent translations in the target language while respecting context-dependent grammar.

+Training a full custom translation model requires a substantial amount of data. If you don't have at least 10,000 sentences of previously trained documents, you won't be able to train a full-language translation model. However, you can either train a dictionary-only model or use the high-quality, out-of-the-box translations available with the Text Translator API.

+## What does training a custom translation model involve?

+Building a custom translation model requires:

+* Understanding your use-case.

+* Obtaining in-domain translated data (preferably human translated).

+* The ability to assess translation quality or target language translations.

+## How do I evaluate my use-case?

+Having clarity on your use-case and what success looks like is the first step towards sourcing proficient training data. Here are a few considerations:

+* What is your desired outcome and how will you measure it?

+* What is your business domain?

+* Do you have in-domain sentences of similar terminology and style?

+* Does your use-case involve multiple domains? If yes, should you build one translation system or multiple systems?

+* Do you have requirements impacting regional data residency at-rest and in-transit?

+* Are the target users in one or multiple regions?

+## How should I source my data?

+Finding in-domain quality data is often a challenging task that varies based on user classification. Here are some questions you can ask yourself as you evaluate what data may be available to you:

+* Enterprises often have a wealth of translation data that has accumulated over many years of using human translation. Does your company have previous translation data available that you can use?

+* Do you have a vast amount of monolingual data? Monolingual data is data in only one language. If so, can you get translations for this data?

+* Can you crawl online portals to collect source sentences and synthesize target sentences?

+## What should I use for training material?

+| Source | What it does | Rules to follow |

+||||

+| Bilingual training documents | Teaches the system your terminology and style. | **Be liberal**. Any in-domain human translation is better than machine translation. Add and remove documents as you go and try to improve the [BLEU score](../what-is-bleu-score.md?WT.mc_id=aiml-43548-heboelma). |

+| Tuning documents | Trains the Neural Machine Translation parameters. | **Be strict**. Compose them to be optimally representative of what you are going to translation in the future. |

+| Test documents | Calculate the [BLEU score](../what-is-bleu-score.md?WT.mc_id=aiml-43548-heboelma).| **Be strict**. Compose test documents to be optimally representative of what you plan to translate in the future. |

+| Phrase dictionary | Forces the given translation 100% of the time. | **Be restrictive**. A phrase dictionary is case-sensitive and any word or phrase listed is translated in the way you specify. In many cases, it's better to not use a phrase dictionary and let the system learn. |

+| Sentence dictionary | Forces the given translation 100% of the time. | **Be strict**. A sentence dictionary is case-insensitive and good for common in domain short sentences. For a sentence dictionary match to occur, the entire submitted sentence must match the source dictionary entry. If only a portion of the sentence matches, the entry won't match. |

+## What is a BLEU score?

+BLEU (Bilingual Evaluation Understudy) is an algorithm for evaluating the precision or accuracy of text that has been machine translated from one language to another. Custom Translator uses the BLEU metric as one way of conveying translation accuracy.

+A BLEU score is a number between zero and 100. A score of zero indicates a low quality translation where nothing in the translation matched the reference. A score of 100 indicates a perfect translation that is identical to the reference. It's not necessary to attain a score of 100 - a BLEU score between 40 and 60 indicates a high-quality translation.

+[Read more](../what-is-bleu-score.md?WT.mc_id=aiml-43548-heboelma)

+## What happens if I don't submit tuning or testing data?

+Tuning and test sentences are optimally representative of what you plan to translate in the future. If you don't submit any tuning or testing data, Custom Translator will automatically exclude sentences from your training documents to use as tuning and test data.

+| System-generated | Manual-selection |

+|||

+| Convenient. | Enables fine-tuning for your future needs.|

+| Good, if you know that your training data is representative of what you are planning to translate. | Provides more freedom to compose your training data.|

+| Easy to redo when you grow or shrink the domain. | Allows for more data and better domain coverage.|

+|Changes each training run.| Remains static over repeated training runs|

+## How is training material processed by Custom Translator?

+To prepare for training, documents undergo a series of processing and filtering steps. These steps are explained below. Knowledge of the filtering process may help with understanding the sentence count displayed as well as the steps you can take to prepare training documents for training with Custom Translator.

+* ### Sentence alignment

+ If your document isn't in XLIFF, XLSX, TMX, or ALIGN format, Custom Translator aligns the sentences of your source and target documents to each other, sentence-by-sentence. Translator doesn't perform document alignmentΓÇöit follows your naming convention for the documents to find a matching document in the other language. Within the source text, Custom Translator tries to find the corresponding sentence in the target language. It uses document markup like embedded HTML tags to help with the alignment.

+ If you see a large discrepancy between the number of sentences in the source and target documents, your source document may not be parallel, or couldn't be aligned. The document pairs with a large difference (>10%) of sentences on each side warrant a second look to make sure they're indeed parallel.

+* ### Extracting tuning and testing data

+ Tuning and testing data is optional. If you don't provide it, the system will remove an appropriate percentage from your training documents to use for tuning and testing. The removal happens dynamically as part of the training process. Since this step occurs as part of training, your uploaded documents aren't affected. You can see the final used sentence counts for each category of dataΓÇötraining, tuning, testing, and dictionaryΓÇöon the Model details page after training has succeeded.

+* ### Length filter

+ * Removes sentences with only one word on either side.

+ * Removes sentences with more than 100 words on either side. Chinese, Japanese, Korean are exempt.

+ * Removes sentences with fewer than three characters. Chinese, Japanese, Korean are exempt.

+ * Removes sentences with more than 2000 characters for Chinese, Japanese, Korean.

+ * Removes sentences with less than 1% alphanumeric characters.

+ * Removes dictionary entries containing more than 50 words.

+* ### White space

+ * Replaces any sequence of white-space characters including tabs and CR/LF sequences with a single space character.

+ * Removes leading or trailing space in the sentence.

+* ### Sentence end punctuation

+ * Replaces multiple sentence-end punctuation characters with a single instance. Japanese character normalization.

+ * Converts full width letters and digits to half-width characters.

+* ### Unescaped XML tags

+ Transforms unescaped tags into escaped tags:

+ | Tag | Becomes |

+ |||

+ | \&lt; | \&amp;lt; |

+ | \&gt; | \&amp;gt; |

+ | \&amp; | \&amp;amp; |

+* ### Invalid characters

+ Custom Translator removes sentences that contain Unicode character U+FFFD. The character U+FFFD indicates a failed encoding conversion.

+## What steps should I take before uploading data?

+* Remove sentences with invalid encoding.

+* Remove Unicode control characters.

+* If feasible, align sentences (source-to-target).

+* Remove source and target sentences that don't match the source and target languages.

+* When source and target sentences have mixed languages, ensure that untranslated words are intentional, for example, names of organizations and products.

+* Correct grammatical and typographical errors to prevent teaching these errors to your model.

+* Though our training process handles source and target lines containing multiple sentences, it's better to have one source sentence mapped to one target sentence.

+## How do I evaluate the results?

+After your model is successfully trained, you can view the model's BLEU score and baseline model BLEU score on the model details page. We use the same set of test data to generate both the model's BLEU score and the baseline BLEU score. This data will help you make an informed decision regarding which model would be better for your use-case.

+## Next steps

+> [!div class="nextstepaction"]

+> [Try our Quickstart](quickstart.md)

+ Title: Create and manage a project

+description: How to create and manage a project in the Azure Cognitive Services Custom Translator.

+# Create and manage a project

+A project contains translation models for one language pair. Each project includes all documents that were uploaded into that workspace with the correct language pair.

+Creating a project is the first step in building and publishing a model.

+## Create a project

+1. After you sign in, your default workspace is loaded. To create a project in different workspace, select **My workspaces**, then select a workspace name.

+1. Select **Create project**.

+1. Enter the following details about your project in the creation dialog:

+ - **Project name (required):** Give your project a unique, meaningful name. It's not necessary to mention the languages within the title.

+ - **Language pair (required):** Select the source and target languages from the dropdown list

+ - **Domain (required):** Select the domain from the dropdown list that's most appropriate for your project. The domain describes the terminology and style of the documents you intend to translate.

+ >[!Note]

+ >Select **Show advanced options** to add project label, project description, and domain description

+ - **Project label:** The project label distinguishes between projects with the same language pair and domain. As a best practice, here are a few tips:

+ - Use a label *only* if you're planning to build multiple projects for the same language pair and same domain and want to access these projects with a different Domain ID.

+ - Don't use a label if you're building systems for one domain only.

+ - A project label isn't required and not helpful to distinguish between language pairs.

+ - You can use the same label for multiple projects.

+ - **Project description:** A short summary about the project. This description has no influence over the behavior of the Custom Translator or your resulting custom system, but can help you differentiate between different projects.

+ - **Domain description:** Use this field to better describe the particular field or industry in which you're working. or example, if your category is medicine, you might add details about your subfield, such as surgery or pediatrics. The description has no influence over the behavior of the Custom Translator or your resulting custom system.

+1. Select **Create project**.

+ :::image type="content" source="../media/how-to/create-project-dialog.png" alt-text="Screenshot illustrating the create project fields.":::

+## Edit a project

+To modify the project name, project description, or domain description:

+1. Select the workspace name.

+1. Select the project name, for example, *English-to-German*.

+1. The **Edit and Delete** buttons should now be visible.

+ :::image type="content" source="../media/how-to/edit-project-dialog-1.png" alt-text="Screenshot illustrating the edit project fields":::

+1. Select **Edit** and fill in or modify existing text.

+ :::image type="content" source="../media/how-to/edit-project-dialog-2.png" alt-text="Screenshot illustrating detailed edit project fields.":::

+1. Select **Edit project** to save.

+## Delete a project

+1. Follow the [**Edit a project**](#edit-a-project) steps 1-3 above.

+1. Select **Delete** and read the delete message before you select **Delete project** to confirm.

+ :::image type="content" source="../media/how-to/delete-project-1.png" alt-text="Screenshot illustrating delete project fields.":::

+ >[!Note]

+ >If your project has a published model or a model that is currently in training, you will only be able to delete your project once your model is no longer published or training.

+ >

+ > :::image type="content" source="../media/how-to/delete-project-2.png" alt-text="Screenshot illustrating the unable to delete message.":::

+## Next steps

+- Learn [how to manage project documents](create-manage-training-documents.md).

+- Learn [how to train a model](train-custom-model.md).

+- Learn [how to test and evaluate model quality](view-model-test-translation.md).

+- Learn [how to publish model](publish-model.md).

+- Learn [how to translate with custom models](translate-with-custom-model.md).

+ Title: Build and upload training documents

+description: How to build and upload parallel documents (two documents where one is the origin and the other is the translation) using Custom Translator.

+# Build and manage training documents

+[Custom Translator](../../overview.md) enables you to build translation models that reflect your business, industry, and domain-specific terminology and style. Training and deploying a custom model is easy and doesn't require any programming skills. Custom Translator allows you to upload parallel files, translation memory files, or zip files.

+[Parallel documents](../../what-are-parallel-documents.md) are pairs of documents where one (target) is a translation of the other (source). One document in the pair contains sentences in the source language and the other document contains those sentences translated into the target language.

+Before uploading your documents, review the [document formats and naming convention guidance](../../document-formats-naming-convention.md) to make sure your file format is supported by Custom Translator.

+## How to create document sets

+Finding in-domain quality data is often a challenging task that varies based on user classification. Here are some questions you can ask yourself as you evaluate what data may be available to you:

+- Enterprises often have a wealth of translation data that has accumulated over many years of using human translation. Does your company have previous translation data available that you can use?

+- Do you have a vast amount of monolingual data? Monolingual data is data in only one language. If so, can you get translations for this data?

+- Can you crawl online portals to collect source sentences and synthesize target sentences?

+### Training material for each document types

+| Source | What it does | Rules to follow |

+||||

+| Bilingual training documents | Teaches the system your terminology and style. | **Be liberal**. Any in-domain human translation is better than machine translation. Add and remove documents as you go and try to improve the [BLEU score](../../what-is-bleu-score.md?WT.mc_id=aiml-43548-heboelma). |

+| Tuning documents | Trains the Neural Machine Translation parameters. | **Be strict**. Compose them to be optimally representative of what you are going to translation in the future. |

+| Test documents | Calculate the [BLEU score](../beginners-guide.md#what-is-a-bleu-score).| **Be strict**. Compose test documents to be optimally representative of what you plan to translate in the future. |

+| Phrase dictionary | Forces the given translation 100% of the time. | **Be restrictive**. A phrase dictionary is case-sensitive and any word or phrase listed is translated in the way you specify. In many cases, it's better to not use a phrase dictionary and let the system learn. |

+| Sentence dictionary | Forces the given translation 100% of the time. | **Be strict**. A sentence dictionary is case-insensitive and good for common in domain short sentences. For a sentence dictionary match to occur, the entire submitted sentence must match the source dictionary entry. If only a portion of the sentence matches, the entry won't match. |

+## How to upload documents

+Document types are associated with the language pair selected when you create a project.

+1. Sign-in to [Custom Translator](https://portal.customtranslator.azure.ai) portal. Your default workspace is loaded and a list of previously created projects are displayed.

+1. Select the desired project **Name**. By default, the **Manage documents** blade is selected and a list of previously uploaded documents is displayed.

+1. Select **Add document set** and choose the document type:

+ - Training set

+ - Testing set

+ - Tuning set

+ - Dictionary set:

+ - Phrase Dictionary

+ - Sentence Dictionary

+1. Select **Next**.

+ :::image type="content" source="../media/how-to/upload-1.png" alt-text="Screenshot illustrating the document upload link.":::

+ >[!Note]

+ >Choosing **Dictionary set** launches **Choose type of dictionary** dialog.

+ >Choose one and select **Next**

+1. Select your documents format from the radio buttons.

+ :::image type="content" source="../media/how-to/upload-2.png" alt-text="Screenshot illustrating the upload document page.":::

+ - For **Parallel documents**, fill in the `Document set name` and select **Browse files** to select source and target documents.

+ - For **Translation memory (TM)** file or **Upload multiple sets with ZIP**, select **Browse files** to select the file

+1. Select **Upload**.

+At this point, Custom Translator is processing your documents and attempting to extract sentences as indicated in the upload notification. Once done processing, you'll see the upload successful notification.

+ :::image type="content" source="../media/quickstart/document-upload-notification.png" alt-text="Screenshot illustrating the upload document processing dialog window.":::

+## Next steps

+- Learn [how to train a model](train-custom-model.md).

+- Learn [how to test and evaluate model quality](view-model-test-translation.md).

+- Learn [how to publish model](publish-model.md).

+- Learn [how to translate with custom models](translate-with-custom-model.md).

+ Title: Create and manage a workspace

+description: How to create and manage workspaces

+# Create and manage a workspace

+ Workspaces are places to manage your documents, projects, and models. When you create a workspace, you can choose to use the workspace independently, or share it with teammates to divide up the work.

+## Create workspace

+1. After you sign in to Custom Translator, you'll be asked for permission to read your profile from the Microsoft identity platform to request your user access token and refresh token. Both tokens are needed for authentication and to ensure that you aren't signed out during your live session or while training your models. </br>Select **Yes**.

+ :::image type="content" source="../media/quickstart/first-time-user.png" alt-text="Screenshot illustrating first-time sign-in.":::

+1. Select **My workspaces**

+1. Select **Create a new workspace**

+1. Type a **Workspace name** and select **Next**

+1. Select "Global" for **Select resource region** from the dropdown list.

+1. Copy/paste your Translator Services key.

+1. Select **Next**.

+1. Select **Done**

+ > [!NOTE]

+ > Region must match the region that was selected during the resource creation. You can use **KEY 1** or **KEY 2**.

+ > [!NOTE]

+ > All uploaded customer content, custom model binaries, custom model configurations, and training logs are kept encrypted-at-rest in the selected region.

+ :::image type="content" source="../media/quickstart/resource-key.png" alt-text="Screenshot illustrating the resource key.":::

+ :::image type="content" source="../media/quickstart/create-workspace-1.png" alt-text="Screenshot illustrating workspace creation.":::

+## Manage workspace settings

+Select a workspace and navigate to **Workspace settings**. You can manage the following workspace settings:

+* Change the resource key for global regions. If you're using a regional specific resource, you can't change your resource key.

+* Change the workspace name.

+* [Share the workspace with others](#share-workspace-for-collaboration).

+* Delete the workspace.

+### Share workspace for collaboration

+The person who created the workspace is the owner. Within **Workspace settings**, an owner can designate three different roles for a collaborative workspace:

+* **Owner**. An owner has full permissions within the workspace.

+* **Editor**. An editor can add documents, train models, and delete documents and projects. They can't modify who the workspace is shared with, delete the workspace, or change the workspace name.

+* **Reader**. A reader can view (and download if available) all information in the workspace.

+> [!NOTE]

+> The Custom Translator workspace sharing policy has changed. For additional security measures, you can share a workspace only with people who have recently signed in to the Custom Translator portal.

+1. Select **Share**.

+1. Complete the **email address** field for collaborators.

+1. Select **role** from the dropdown list.

+1. Select **Share**.

+### Remove somebody from a workspace

+1. Select **Share**.

+2. Select the **X** icon next to the **Role** and email address that you want to remove.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn how to manage projects](create-manage-project.md)

+ Title: Publish a custom model

+description: This article explains how to publish a custom model using the Azure Cognitive Services Custom Translator.

+# Publish a custom model

+Publishing your model makes it available for use with the Translator API. A project might have one or many successfully trained models. You can only publish one model per project; however, you can publish a model to one or multiple regions depending on your needs. For more information, see [Translator pricing](https://azure.microsoft.com/pricing/details/cognitive-services/translator/#pricing).

+## Publish your trained model

+You can publish one model per project to one or multiple regions.

+1. Select the **Publish model** blade.

+1. Select *en-de with sample data* and select **Publish**.

+1. Check the desired region(s).

+1. Select **Publish**. The status should transition from _Deploying_ to _Deployed_.

+ :::image type="content" source="../media/quickstart/publish-model.png" alt-text="Screenshot illustrating the publish model blade.":::

+## Replace a published model

+To replace a published model, you can exchange the published model with a different model in the same region(s):

+1. Select the replacement model.

+1. Select **Publish**.

+1. Select **publish** once more in the **Publish model** dialog window.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn how to translate documents with custom models](translate-with-custom-model.md)

+ Title: Train model

+description: How to train a custom model

+# Train a custom model

+A model provides translations for a specific language pair. The outcome of a successful training is a model. To train a custom model, three mutually exclusive document types are required: training, tuning, and testing. If only training data is provided when queuing a training, Custom Translator will automatically assemble tuning and testing data. It will use a random subset of sentences from your training documents, and exclude these sentences from the training data itself. A minimum of 10,000 parallel training sentences are required to train a full model.

+## Create model

+1. Select the **Train model** blade.

+1. Type the **Model name**.

+1. Keep the default **Full training** selected or select **Dictionary-only training**.

+ >[!Note]

+ >Full training displays all uploaded document types. Dictionary-only displays dictionary documents only.

+1. Under **Select documents**, select the documents you want to use to train the model, for example, `sample-English-German` and review the training cost associated with the selected number of sentences.

+1. Select **Train now**.

+1. Select **Train** to confirm.

+ >[!Note]

+ >**Notifications** displays model training in progress, e.g., **Submitting data** state. Training model takes few hours, subject to the number of selected sentences.

+ :::image type="content" source="../media/quickstart/train-model.png" alt-text="Screenshot illustrating the train model blade.":::

+## When to select dictionary-only training

+For better results, we recommended letting the system learn from your training data. However, when you don't have enough parallel sentences to meet the 10,000 minimum requirements, or sentences and compound nouns must be rendered as-is, use dictionary-only training. Your model will typically complete training much faster than with full training. The resulting models will use the baseline models for translation along with the dictionaries you've added. You won't see BLEU scores or get a test report.

+> [!Note]

+>Custom Translator doesn't sentence-align dictionary files. Therefore, it is important that there are an equal number of source and target phrases/sentences in your dictionary documents and that they are precisely aligned. If not, the document upload will fail.

+## Model details

+1. After successful model training, select the **Model details** blade.

+1. Select the **Model Name** to review training date/time, total training time, number of sentences used for training, tuning, testing, dictionary, and whether the system generated the test and tuning sets. You'll use `Category ID` to make translation requests.

+1. Evaluate the model [BLEU score](../beginners-guide.md#what-is-a-bleu-score). Review the test set: the **BLEU score** is the custom model score and the **Baseline BLEU** is the pre-trained baseline model used for customization. A higher **BLEU score** means higher translation quality using the custom model.

+ :::image type="content" source="../media/quickstart/model-details.png" alt-text="Screenshot illustrating model details fields.":::

+## Duplicate model

+1. Select the **Model details** blade.

+1. Hover over the model name and check the selection button.

+1. Select **Duplicate**.

+1. Fill in **New model name**.

+1. Keep **Train immediately** checked if no further data will be selected or uploaded, otherwise, check **Save as draft**

+1. Select **Save**

+ > [!Note]

+ >

+ > If you save the model as `Draft`, **Model details** is updated with the model name in `Draft` status.

+ >

+ > To add more documents, select on the model name and follow `Create model` section above.

+ :::image type="content" source="../media/how-to/duplicate-model.png" alt-text="Screenshot illustrating the duplicate model blade.":::

+## Next steps

+- Learn [how to test and evaluate model quality](view-model-test-translation.md).

+- Learn [how to publish model](publish-model.md).

+- Learn [how to translate with custom models](translate-with-custom-model.md).

+ Title: Translate text with a custom model

+description: How to make translation requests using custom models published with the Azure Cognitive Services Custom Translator.

+# Translate text with a custom model

+After you publish your custom model, you can access it with the Translator API by using the `Category ID` parameter.

+## How to translate

+1. Use the `Category ID` when making a custom translation request via Microsoft Translator [Text API V3](../../../reference/v3-0-translate.md?tabs=curl). The `Category ID` is created by concatenating the WorkspaceID, project label, and category code. Use the `CategoryID` with the Text Translator API to get custom translations.

+ ```http

+ https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&to=de&category=a2eb72f9-43a8-46bd-82fa-4693c8b64c3c-TECH

+ ```

+ More information about the Translator Text API can be found on the [Translator API Reference](../../../reference/v3-0-translate.md) page.

+1. You may also want to download and install our free [DocumentTranslator app for Windows](https://github.com/MicrosoftTranslator/DocumentTranslation/releases).

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn more about building and publishing custom models](../beginners-guide.md)

+ Title: View custom model details and test translation

+description: How to test custom model BLEU score and model translation

+# View custom model details and test translation

+Once your model has successfully trained, you can use translations to evaluate the quality of your model. In order to make an informed decision about whether to use our standard model or your custom model, you should evaluate the delta between your custom model [**BLEU score**](#bleu-score) and our standard model **Baseline BLEU**. If your models have been trained on a narrow domain, and your training data is consistent with the test data, you can expect a high BLEU score.

+## BLEU score

+BLEU (Bilingual Evaluation Understudy) is an algorithm for evaluating the precision or accuracy of text that has been machine translated from one language to another. Custom Translator uses the BLEU metric as one way of conveying translation accuracy.

+A BLEU score is a number between zero and 100. A score of zero indicates a low-quality translation where nothing in the translation matched the reference. A score of 100 indicates a perfect translation that is identical to the reference. It's not necessary to attain a score of 100ΓÇöa BLEU score between 40 and 60 indicates a high-quality translation.

+[Read more](../../what-is-bleu-score.md?WT.mc_id=aiml-43548-heboelma)

+## Model details

+1. Select the **Model details** blade.

+1. Select the model name. Review the training date/time, total training time, number of sentences used for training, tuning, testing, and dictionary. Check whether the system generated the test and tuning sets. You'll use the `Category ID` to make translation requests.

+1. Evaluate the model [BLEU](../beginners-guide.md#what-is-a-bleu-score) score. Review the test set: the **BLEU score** is the custom model score and the **Baseline BLEU** is the pre-trained baseline model used for customization. A higher **BLEU score** means there's high translation quality using the custom model.

+ :::image type="content" source="../media/quickstart/model-details.png" alt-text="Screenshot illustrating the model detail.":::

+## Test quality of your model's translation

+1. Select **Test model** blade.

+1. Select model **Name**.

+1. Human evaluate translation from your **Custom model** and the **Baseline model** (our pre-trained baseline used for customization) against **Reference** (target translation from the test set).

+1. If you're satisfied with the training results, place a deployment request for the trained model.

+## Next steps

+- Learn [how to publish/deploy a custom model](publish-model.md).

+- Learn [how to translate documents with a custom model](translate-with-custom-model.md).

+ Title: "Quickstart: Build, deploy, and use a custom model - Custom Translator"

+description: A step-by-step guide to building a translation system using the Custom Translator portal v2.

+# Quickstart: Build, publish, and translate with custom models

+Translator is a cloud-based neural machine translation service that is part of the Azure Cognitive Services family of REST API that can be used with any operating system. Translator powers many Microsoft products and services used by thousands of businesses worldwide to perform language translation and other language-related operations. In this quickstart, you'll learn to build custom solutions for your applications across all [supported languages](../../language-support.md).

+## Prerequisites

+ To use the [Custom Translator](https://portal.customtranslator.azure.ai/) portal, you'll need the resources:

+* A [Microsoft account](https://signup.live.com).

+* Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/)

+* Once you have an Azure subscription, [create a Translator resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesTextTranslation) in the Azure portal to get your key and endpoint. After it deploys, select **Go to resource**.

+ * You'll need the key and endpoint from the resource to connect your application to the Translator service. You'll paste your key and endpoint into the code below later in the quickstart. You can find these values on the Azure portal **Keys and Endpoint** page:

+ :::image type="content" source="../../media/keys-and-endpoint-portal.png" alt-text="Screenshot: Azure portal keys and endpoint page.":::

+ See [how to create a Translator resource](../../how-to-create-translator-resource.md).

+Once you have the above prerequisites, sign in to the [Custom Translator](https://portal.customtranslator.azure.ai/) portal to create workspaces, build projects, upload files, train models, and publish your custom solution.

+You can read an overview of translation and custom translation, learn some tips, and watch a getting started video in the [Azure AI technical blog](https://techcommunity.microsoft.com/t5/azure-ai/customize-a-translation-to-make-sense-in-a-specific-context/ba-p/2811956).

+>[!Note]

+>Custom Translator does not support creating workspace for a Translator Text API resource created inside an [Enabled VNet](../../../../api-management/api-management-using-with-vnet.md?tabs=stv2).

+## Process summary

+1. [**Create a workspace**](#create-a-workspace). A workspace is a work area for composing and building your custom translation system. A workspace can contain multiple projects, models, and documents. All the work you do in Custom Translator is done inside a specific workspace.

+1. [**Create a project**](#create-a-project). A project is a wrapper for models, documents, and tests. Each project includes all documents that are uploaded into that workspace with the correct language pair. For example, if you have both an English-to-Spanish project and a Spanish-to-English project, the same documents will be included in both projects.

+1. [**Upload parallel documents**](#upload-documents). Parallel documents are pairs of documents where one (target) is the translation of the other (source). One document in the pair contains sentences in the source language and the other document contains sentences translated into the target language. It doesn't matter which language is marked as "source" and which language is marked as "target"ΓÇöa parallel document can be used to train a translation system in either direction.

+1. [**Train your model**](#train-your-model). A model is the system that provides translation for a specific language pair. The outcome of a successful training is a model. When you train a model, three mutually exclusive document types are required: training, tuning, and testing. If only training data is provided when queuing a training, Custom Translator will automatically assemble tuning and testing data. It will use a random subset of sentences from your training documents, and exclude these sentences from the training data itself. A 10,000 parallel sentence is the minimum requirement to train a model.

+1. [**Test (human evaluate) your model**](#test-your-model). The testing set is used to compute the [BLEU](beginners-guide.md#what-is-a-bleu-score) score. This score indicates the quality of your translation system.

+1. [**Publish (deploy) your trained model**](#publish-your-model). Your custom model is made available for runtime translation requests.

+1. [**Translate text**](#translate-text). Use the cloud-based, secure, high performance, highly scalable Microsoft Translator [Text API V3](../../reference/v3-0-translate.md?tabs=curl) to make translation requests.

+## Create a workspace

+1. After your sign-in to Custom Translator, you'll be asked for permission to read your profile from the Microsoft identity platform to request your user access token and refresh token. Both tokens are needed for authentication and to ensure that you aren't signed out during your live session or while training your models. </br>Select **Yes**.

+ :::image type="content" source="media/quickstart/first-time-user.png" alt-text="Screenshot illustrating how to create a workspace.":::

+1. Select **My workspaces**

+1. Select **Create a new workspace**

+1. Type _Contoso MT models_ for **Workspace name** and select **Next**

+1. Select "Global" for **Select resource region** from the dropdown list.

+1. Copy/paste your Translator Services key.

+1. Select **Next**.

+1. Select **Done**

+ >[!Note]

+ > Region must match the region that was selected during the resource creation. You can use **KEY 1** or **KEY 2.**

+ :::image type="content" source="media/quickstart/resource-key.png" alt-text="Screenshot illustrating the resource key.":::

+ :::image type="content" source="media/quickstart/create-workspace-1.png" alt-text="Screenshot illustrating workspace creation.":::

+## Create a project

+Once the workspace is created successfully, you'll be taken to the **Projects** page.

+You'll create English-to-German project to train a custom model with only a [training](../training-and-model.md#training-document-type-for-custom-translator) document type.

+1. Select **Create project**.

+1. Type *English-to-German* for **Project name**.

+1. Select *English (en)* as **Source language** from the dropdown list.

+1. Select *German (de)* as **Target language** from the dropdown list.

+1. Select *General* for **Domain** from the dropdown list.

+1. Select **Create project**

+ :::image type="content" source="media/quickstart/create-project.png" alt-text="Screenshot illustrating how to create a project.":::

+## Upload documents

+In order to create a custom model, you need to upload all or a combination of [training](../training-and-model.md#training-document-type-for-custom-translator), [tuning](../training-and-model.md#tuning-document-type-for-custom-translator), [testing](../training-and-model.md#testing-dataset-for-custom-translator), and [dictionary](../what-is-dictionary.md) document types.

+In this quickstart, you'll upload [training](../training-and-model.md#training-document-type-for-custom-translator) documents for customization.

+>[!Note]

+> You can use our sample training, phrase and sentence dictionaries dataset, [Customer sample English-to-German datasets](https://github.com/MicrosoftTranslator/CustomTranslatorSampleDatasets), for this quickstart. However, for production, it's better to upload your own training dataset.

+1. Select *English-to-German* project name.

+1. Select **Manage documents** from the left navigation menu.

+1. Select **Add document set**.

+1. Check the **Training set** box and select **Next**.

+1. Keep **Parallel documents** checked and type *sample-English-German*.

+1. Under the **Source (English - EN) file**, select **Browse files** and select *sample-English-German-Training-en.txt*.

+1. Under **Target (German - EN) file**, select **Browse files** and select *sample-English-German-Training-de.txt*.

+1. Select **Upload**

+ >[!Note]

+ >You can upload the sample phrase and sentence dictionaries dataset. This step is left for you to complete.

+ :::image type="content" source="media/quickstart/upload-model.png" alt-text="Screenshot illustrating how to upload documents.":::

+## Train your model

+Now you're ready to train your English-to-German model.

+1. Select **Train model** from the left navigation menu.

+1. Type *en-de with sample data* for **Model name**.

+1. Keep **Full training** checked.

+1. Under **Select documents**, check *sample-English-German* and review the training cost associated with the selected number of sentences.

+1. Select **Train now**.

+1. Select **Train** to confirm.

+ >[!Note]

+ >**Notifications** displays model training in progress, e.g., **Submitting data** state. Training model takes few hours, subject to the number of selected sentences.

+ :::image type="content" source="media/quickstart/train-model.png" alt-text="Screenshot illustrating how to create a model.":::

+1. After successful model training, select **Model details** from the left navigation menu.

+1. Select the model name *en-de with sample data*. Review training date/time, total training time, number of sentences used for training, tuning, testing, and dictionary. Check whether the system generated the test and tuning sets. You'll use the `Category ID` to make translation requests.

+1. Evaluate the model [BLEU](beginners-guide.md#what-is-a-bleu-score) score. The test set **BLEU score** is the custom model score and **Baseline BLEU** is the pre-trained baseline model used for customization. A higher **BLEU score** means higher translation quality using the custom model.

+ >[!Note]

+ >If you train with our shared customer sample datasets, BLEU score will be different than the image.

+ :::image type="content" source="media/quickstart/model-details.png" alt-text="Screenshot illustrating model details.":::

+## Test your model

+Once your training has completed successfully, inspect the test set translated sentences.

+1. Select **Test model** from the left navigation menu.

+2. Select "en-de with sample data"

+3. Human evaluate translation from **New model** (custom model), and **Baseline model** (our pre-trained baseline used for customization) against **Reference** (target translation from the test set)

+## Publish your model

+Publishing your model makes it available for use with the Translator API. A project might have one or many successfully trained models. You can only publish one model per project; however, you can publish a model to one or multiple regions depending on your needs. For more information, see [Translator pricing](https://azure.microsoft.com/pricing/details/cognitive-services/translator/#pricing).

+1. Select **Publish model** from the left navigation menu.

+1. Select *en-de with sample data* and select **Publish**.

+1. Check the desired region(s).

+1. Select **Publish**. The status should transition from _Deploying_ to _Deployed_.

+ :::image type="content" source="media/quickstart/publish-model.png" alt-text="Screenshot illustrating how to deploy a trained model.":::

+## Translate text

+1. Developers should use the `Category ID` when making translation requests with Microsoft Translator [Text API V3](../../reference/v3-0-translate.md?tabs=curl). More information about the Translator Text API can be found on the [API Reference](../../reference/v3-0-reference.md) webpage.

+1. Business users may want to download and install our free [DocumentTranslator app for Windows](https://github.com/MicrosoftTranslator/DocumentTranslator/releases/tag/V2.9.4).

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn how to manage workspaces](how-to/create-manage-workspace.md)

+* To learn more, see our Custom Translator [documentation](custom-translator/overview.md) and try our [quickstart](custom-translator/v2.0/quickstart.md) for step-by-step instructions.

+|[Anomaly Detector](../anomaly-detector/index.yml "Anomaly Detector") | The Anomaly Detector service allows you to monitor and detect abnormalities in your time series data.|

+- [Embedded Speech](/legal/cognitive-services/speech-service/embedded-speech/limited-access-embedded-speech?context=/azure/cognitive-services/speech-service/context/context): All features

+- [Embedded Speech](https://aka.ms/csgate-embedded-speech): All features

+- [Embedded Speech](https://aka.ms/csgate-embedded-speech): All features

+* There is a new endpoint URL and request format for making REST API calls to prebuilt Language service features. See the following quickstart guides and reference documentation for information on structuring your API calls. All text analytics 3.2-preview.2 API users can begin migrating their workloads to this new endpoint.

+* [Anomaly Detector](./anomaly-detector/index.yml)

+Rooms can also be accessed using the [Azure Communication Services UI Library](https://azure.github.io/communication-ui-library/?path=/docs/rooms--page). The UI Library enables developers to add a call client that is Rooms enabled into their application with only a couple lines of code.

+- Review the [Network requirements for media and signaling](../voice-video-calling/network-requirements.md).

+ Title: Media streaming overview

+description: Conceptual information about using Media Streaming APIs with Call Automation.

+# Media streaming overview - audio subscription

+> [!IMPORTANT]

+> Functionality described on this document is currently in private preview. Private preview includes access to SDKs and documentation for testing purposes that are not yet available publicly.

+> Apply to become an early adopter by filling out the form for [preview access to Azure Communication Services](https://aka.ms/ACS-EarlyAdopter).

+Azure Communication Services provides developers with Media Streaming capabilities to get real-time access to media streams to capture, analyze and process audio content during active calls. In today's world consumption of live audio and video is prevalent, this content could be in the forms of online meetings, online conferences, online schooling, customer support, etc. This consumption has only been exacerbated by the recent events of Covid-19, with many of the worlds work force working remotely from home. With media streaming access, developers can now build server applications to capture and analyze audio streams for each of the participants on the call in real-time. Developers can also combine media streaming with other call automation actions or use their own AI models to analyze audio streams for use cases such as NLP for conversation analysis or provide real-time insights and suggestions to their agents while they are in an active interaction with their end users.

+This private preview supports the ability for developers to get access to real-time audio streams over a websocket to analyze each participants audio in mixed and unmixed formats

+## Common use cases

+Audio streams can be used in many ways, below are some examples of how developers may wish to use the audio streams in their applications.

+### Real-time call assistance

+**Improved AI powered suggestions** - Use real-time audio streams of active interactions between agents and customers to gauge the intent of the call and how your agents can provide a better experience to their customer through active suggestions using your own AI model to analyze the call.

+### Authentication

+**Biometric authentication** ΓÇô Use the audio streams to carry out authentication using caller biometrics such as voice recognition.

+### Interpretations

+**Real-time translation** ΓÇô Use audio streams to send to human or AI translators who can consume this audio content and provide translations.

+## Sample architecture for subscribing to audio streams from an ongoing call

+[](./media/media-streaming-flow.png#lightbox)

+## Supported formats

+### Mixed format

+Contains mixed audio of all participants on the call.

+

+### Unmixed

+Contains audio per participant per channel, with support for up to four channels for four dominant speakers. You will also get a participantRawID that you can use to determine the speaker.

+## Additional information

+The table below describes information that will help developers convert the media packets into audible content that can be used by their applications.

+- Framerate: 50 frames per second

+- Packet stream rate: 20 ms rate

+- Data packet: 64 Kbytes

+- Audio metric: 16-bit PCM mono at 16000 hz

+- Public string data is a base64 string that should be converted into a byte array to create raw PCM file. You can then use the following configuration in Audacity to run the file.

+## Next Steps

+Check out the [Media Streaming quickstart](../../quickstarts/voice-video-calling/media-streaming.md) to learn more.

+If you run into the issue "The app is trying to access a service '1fd5118e-2576-4263-8130-9503064c837a'(Azure Communication Services) that your organization '{GUID}' lacks a service principal for. Contact your IT Admin to review the configuration of your service subscriptions or consent to the application to create the required service principal." your Azure Active Directory tenant lacks a service principal for the Azure Communication Services application. To fix this issue, use PowerShell as an Azure AD administrator to connect to your tenant. Replace `Tenant_ID` with an ID of your AAD tenancy.

+```script

+Connect-AzureAD -TenantId "Tenant_ID"

+```

+If the command is not found, start PowerShell as an administrator and install the Azure AD package.

+```script

+Install-Module AzureAD

+```

+Then execute the following command to add a service principal to your tenant. Do not modify the GUID of the App ID.

+```script

+New-AzureADServicePrincipal -AppId "1fd5118e-2576-4263-8130-9503064c837a"

+```

+ Title: Media streaming quickstart

+description: Provides a quick start for developers to get audio streams through media streaming APIs from ACS calls.

+zone_pivot_groups: acs-csharp-java

+# Quickstart: Subscribing to audio streams from an ongoing call

+> [!IMPORTANT]

+> Functionality described on this document is currently in private preview. Private preview includes access to SDKs and documentation for testing purposes that are not yet available publicly.

+> Apply to become an early adopter by filling out the form for [preview access to Azure Communication Services](https://aka.ms/ACS-EarlyAdopter).

+Get started with using audio streams through Azure Communication Services Media Streaming API. This quickstart assumes you're already familiar with Call Automation APIs to build an automated call routing solution.

+## Message schema

+When ACS has received the URL for your WebSocket server, it will create a connection to it. Once ACS has successfully connected to your WebSocket server, it will send through the first data packet which contains metadata regarding the incoming media packets.

+``` code

+/**

+ * The first message upon WebSocket connection will be the metadata packet

+ * which contains the subscriptionId and audio format

+ */

+public class AudioMetadataSample {

+ public string kind; // What kind of data this is, e.g. AudioMetadata, AudioData.

+ public AudioMetadata audioMetadata;

+}

+public class AudioMetadata {

+ public string subscriptionId // unique identifier for a subscription request

+ public string encoding; // PCM only supported

+ public int sampleRate; // 16000 default

+ public int channels; // 1 default

+ public int length; // 640 default

+}

+```

+## Audio streaming schema

+After sending through the metadata packet, ACS will start streaming audio media to your WebSocket server. Below is an example of what the media object your server will receive looks like.

+``` code

+/**

+ * The audio buffer object which is then serialized to JSON format

+ */

+public class AudioDataSample {

+ public string kind; // What kind of data this is, e.g. AudioMetadata, AudioData.

+ public AudioData audioData;

+}

+public class AudioData {

+ public string data; // Base64 Encoded audio buffer data

+ public string timestamp; // In ISO 8601 format (yyyy-mm-ddThh:mm:ssZ)

+ public string participantRawID;

+ public boolean silent; // Indicates if the received audio buffer contains only silence.

+}

+```

+Example of audio data being streamed

+``` json

+{

+ "kind": "AudioData",

+ "audioData": {

+ "timestamp": "2022-10-03T19:16:12.925Z",

+ "participantRawID": "8:acs:3d20e1de-0f28-41c5-84a0-4960fde5f411_0000000b-faeb-c708-99bf-a43a0d0036b0",

+ "data": "5ADwAOMA6AD0AOIA4ADkAN8AzwDUANEAywC+ALQArgC0AKYAnACJAIoAlACWAJ8ApwCiAKkAqgCqALUA0wDWANAA3QDVAN0A8wDzAPAA7wDkANkA1QDPAPIA6QDmAOcA0wDYAPMA8QD8AP0AAwH+AAAB/QAAAREBEQEDAQoB9wD3APsA7gDxAPMA7wDpAN0A6gD5APsAAgEHAQ4BEAETARsBMAFHAUABPgE2AS8BKAErATEBLwE7ASYBGQEAAQcBBQH5AAIBBwEMAQ4BAAH+APYA6gDzAPgA7gDkAOUA3wDcANQA2gDWAN8A3wDcAMcAxwDIAMsA1wDfAO4A3wDUANQA3wDvAOUA4QDpAOAA4ADhAOYA5wDkAOUA1gDxAOcA4wDpAOEA4gD0APoA7wD9APkA6ADwAPIA7ADrAPEA6ADfANQAzQDLANIAzwDaANcA3QDZAOQA4wDXANwA1ADbAOsA7ADyAPkA7wDiAOIA6gDtAOsA7gDeAOIA4ADeANUA6gD1APAA8ADgAOQA5wDgAPgA8ADnAN8A5gDgAOoA6wDcAOgA2gDZANUAyQDPANwA3gDgAO4A8QDyAAQBEwEDAewA+gDpAN4A6wDeAO8A8QDwAO8ABAEKAQUB/gD5AAMBAwEIARoBFAEeARkBDgH8AP0A+gD8APcA+gDrAO0A5wDcANEA0QDHAM4A0wDUAM4A0wDZANQAxgDSAM4A1ADVAOMA4QDhANUA2gDjAOYA5wDrANQA5wDrAMsAxQDWANsA5wDpAOEA4QDFAMoA0QDKAMgAwgDNAMsAwgCwAKkAtwCrAKoAsACgAJ4AlQCeAKAAoQCmAKwApwCsAK0AnQCVAA==",

+ "silent": false

+ }

+}

+```

+## Stop audio streaming

+Audio streaming will automatically stop when the call ends or is canceled.

+## Clean up resources

+If you want to clean up and remove a Communication Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it. Learn more about [cleaning up resources](../create-communication-resource.md#clean-up-resources).

+## Next steps

+- Learn more about [Media Streaming](../../concepts/voice-video-calling/media-streaming.md).

+- Learn more about [Call Automation](../../concepts/voice-video-calling/call-automation.md) and its features.

+- Learn more about [Play action](../../concepts/voice-video-calling/play-action.md).

+- Learn more about [Recognize action](../../concepts/voice-video-calling/recognize-action.md).

+> Starting October 2021, new container registries allow a maximum of 200 private endpoints. Registries created earlier allow a maximum of 10 private endpoints. Use the az acr show-usage command to see the limit for your registry. Please open a support ticket if this limit needs to be increased to 200 private endpoints.

+* Learn more about [Microsoft Defender for container registries](https://learn.microsoft.com/azure/defender-for-cloud/defender-for-containers-va-acr)

+* Learn more about [container security in Microsoft Defender for Cloud](https://learn.microsoft.com/azure/defender-for-cloud/defender-for-containers-introduction)

+Microsoft Defender for Cloud scans images that are pushed to a registry, imported into a registry, or any images pulled within the last 30 days. If vulnerabilities are detected, [recommended remediations](https://learn.microsoft.com/azure/defender-for-cloud/defender-for-containers-va-acr#view-and-remediate-findings) appear in Microsoft Defender for Cloud.

+For details, see [Use Microsoft Defender for container registries](https://learn.microsoft.com/azure/defender-for-cloud/defender-for-containers-va-acr).

+| ``url`` | ``COSMOS_CONNECTION_STRING`` environment variable | API for MongoDB connection string to use for all requests |

+ Title: Choose shard count - Azure Cosmos DB for PostgreSQL

+description: Pick the right shard count for distributed tables

+# Choose shard count

+Choosing the shard count for each distributed table is a balance between the

+flexibility of having more shards, and the overhead for query planning and

+execution across them. If you decide to change the shard count of a table after

+distributing, you can use the

+[alter_distributed_table](reference-functions.md#alter_distributed_table)

+function.

+## Multi-tenant SaaS use case

+The optimal choice varies depending on your access patterns for the data. For

+instance, in the Multi-Tenant SaaS Database use-case we recommend choosing

+between **32 - 128** shards. For smaller workloads say <100 GB, you could start with

+32 shards and for larger workloads you could choose 64 or 128. This choice gives you

+the leeway to scale from 32 to 128 worker machines.

+## Real-time analytics use case

+In the Real-Time Analytics use-case, shard count should be related to the total

+number of cores on the workers. To ensure maximum parallelism, you should create

+enough shards on each node such that there is at least one shard per CPU core.

+We typically recommend creating a high number of initial shards, for example,

+**2x or 4x the number of current CPU cores**. Having more shards allows for

+future scaling if you add more workers and CPU cores.

+Keep in mind that, for each query, Azure Cosmos DB for PostgreSQL opens one

+database connection per shard, and that these connections are limited. Be

+careful to keep the shard count small enough that distributed queries wonΓÇÖt

+often have to wait for a connection. Put another way, the connections needed,

+`(max concurrent queries * shard count)`, shouldn't exceed the total

+connections possible in the system, `(number of workers * max_connections per

+worker)`.

+## Next steps

+- Learn more about cluster [performance options](resources-compute.md).

+- [Scale a cluster](howto-scale-grow.md) up or out

+- [Rebalance shards](howto-scale-rebalance.md)

+> [!NOTE]

+>

+> In real applications, when your workload fits in 64 vCores, 256GB RAM and 2TB

+> storage, you can use a single-node cluster. In this case, distributing tables

+> is optional. Later, you can distribute tables as needed using

+> [create_distributed_table_concurrently](reference-functions.md#create_distributed_table_concurrently).

+**shard\_count:** (Optional) the number of shards to create for the new

+distributed table. When specifying `shard_count` you canΓÇÖt specify a value of

+`colocate_with` other than none. To change the shard count of an existing table

+or colocation group, use the [alter_distributed_table](#alter_distributed_table

+function.

+Possible values for `shard_count` are between 1 and 64000. For guidance on

+choosing the optimal value, see [Shard Count](howto-shard-count.md).

+### create\_distributed\_table\_concurrently

+This function has the same interface and purpose as

+[create_distributed_function](#create_distributed_table), but doesn't block

+writes during table distribution.

+However, `create_distributed_table_concurrently` has a few limitations:

+* You can't use the function in a transaction block, which means you can only

+ distribute one table at a time. (You *can* use the function on

+ time-partitioned tables, though.)

+* You can't use `create_distributed_table_concurrently` when the table is

+ referenced by a foreign key, or references another local table. However,

+ foreign keys to reference tables work, and you can create foreign keys to other

+ distributed tables after table distribution completes.

+* If you don't have a primary key or replica identity on your table, then

+ update and delete commands will fail during the table distribution due to

+ limitations on logical replication.

+> - **You must have owner or Reservation administrator access on the Reservation Order to exchange or refund an existing reservation**. You can [Add or change users who can manage a reservation](./manage-reserved-vm-instance.md#who-can-manage-a-reservation-by-default).

+- Enterprise Agreement and Microsoft Customer Agreement billing contributors can manage all reservations from Cost Management + Billing > Reservation Transactions > select the blue banner.

+- A Reservation administrator for reservations in their Azure Active Directory (Azure AD) tenant (directory).

+- A Reservation reader has read-only access to reservations in their Azure Active Directory tenant (directory).

+ Title: Monitor Azure Data Factory and Azure Synapse Analytics pipelines with annotations and user properties

+description: Advanced monitoring with annotations and user properties

+# Monitor Azure Data Factory and Azure Synapse Analytics pipelines with annotations and user properties

+When monitoring your data pipelines, you may want to be able to filter and monitor a certain group of activities, such as those of a project or specific department's pipelines. You may also need to further monitor activities based on dynamic properties. You can achieve these things by leveraging annotations and user properties.

+## Annotations

+Azure Data Factory annotations are tags that you can add to your Azure Data Factory or Azure Synapse Analytics entities to easily identify them. An annotation allows you to classify or group different entities in order to easily monitor or filter them after an execution. Annotations only allow you to define static values and can be added to pipelines, datasets, linked services and triggers.

+## User properties

+User properties are key-value pairs defined at the activity level. By adding user properties, you can view additional information about activities under activity runs window that may help you to monitor your activity executions.

+User properties allow you to define dynamic values and can be added to any activity, up to 5 per activity, under User Properties tab.

+## Create and use annotations and user properties

+As we discussed, annotations are static values that you can assign to pipelines, datasets, linked services, and triggers. Let's assume you want to filter for pipelines that belong to the same business unit or project name. We will first create the annotation. Click on the Properties icon, + New button and name your annotation appropriately. We advise being consistent with your naming.

+

+When you go to the Monitor tab, you can filter under Pipeline runs for this Annotation:

+

+If you want to monitor for dynamic values at the activity level, you can do so by leveraging the User properties. You can add these under any activity by clicking on the Activity box, User properties tab and the + New button:

+

+For Copy Activity specifically, you can auto-generate these:

+

+To monitor User properties, go to the Activity runs monitoring view. Here you will see all the properties you added.

+

+You can remove some from the view if you click on the Bookmark sign:

+

+## Next steps

+To learn more about monitoring see [Visually monitor Azure Data Factory.](./monitor-visually.md)

+>This connector doesn't support retrieving views, custom objects or custom data extensions.

+In this scenario, you want to copy data from AWS S3 to Azure Blob storage and transform the data with Azure Databricks on an hourly schedule for 8 hours per day for 30 days.

+| Run Pipeline | 3 Activity runs **per execution** (1 for trigger run, 2 for activity runs) = 720 activity runs, rounded up since the calculator only allows increments of 1000. |

+| Copy Data Assumption: DIU hours **per execution** = 10 min | 10 min \ 60 min \* 4 Azure Integration Runtime (default DIU setting = 4) For more information on data integration units and optimizing copy performance, see [this article](copy-activity-performance.md) |

+| Execute Databricks activity Assumption: external execution hours **per execution** = 10 min | 10 min \ 60 min External Pipeline Activity Execution |

+**Total scenario pricing for 30 days: $41.01**

+In this scenario, you want to copy data from AWS S3 to Azure Blob storage and transform with Azure Databricks (with dynamic parameters in the script) on an hourly schedule for 8 hours each day over 30 days.

+- One schedule trigger to execute the pipeline every hour for 8 hours per day. When you want to run a pipeline, you can either [trigger it immediately or schedule it](concepts-pipeline-execution-triggers.md). In addition to the pipeline itself, each trigger instance counts as a single Activity run.

+| Run Pipeline | 4 Activity runs **per execution** (1 for trigger run, 3 for activity runs) = 960 activity runs, rounded up since the calculator only allows increments of 1000. |

+| Copy Data Assumption: DIU hours **per execution** = 10 min | 10 min \ 60 min \* 4 Azure Integration Runtime (default DIU setting = 4) For more information on data integration units and optimizing copy performance, see [this article](copy-activity-performance.md) |

+| Execute Lookup activity Assumption: pipeline activity hours **per execution** = 1 min | 1 min / 60 min Pipeline Activity execution |

+| Execute Databricks activity Assumption: external execution hours **per execution** = 10 min | 10 min / 60 min External Pipeline Activity execution |

+**Total scenario pricing for 30 days: $41.03**

+In this scenario, you want to delete original files on Azure Blob Storage and copy data from Azure SQL Database to Azure Blob Storage on an hourly schedule for 8 hours per day. We'll calculate the price for 30 days. You'll do this execution twice on different pipelines for each run. The execution time of these two pipelines is overlapping.

+| Run Pipeline | 6 Activity runs **per execution** (2 for trigger runs, 4 for activity runs) = 1440, rounded up since the calculator only allows increments of 1000.|

+| Execute Delete Activity: pipeline execution time **per execution** = 7 min. If the Delete Activity execution in the first pipeline is from 10:00 AM UTC to 10:05 AM UTC and the Delete Activity execution in second pipeline is from 10:02 AM UTC to 10:07 AM UTC. | Total 7 min / 60 min \* 240 montly executions = 28 pipeline activity execution hours in Managed VNET. Pipeline activity supports up to 50 concurrent executions in Managed VNET. There's a 60 minutes Time To Live (TTL) for pipeline activity. |

+| Copy Data Assumption: DIU execution time **per execution** = 10 min if the Copy execution in first pipeline is from 10:06 AM UTC to 10:15 AM UTC and the Copy Activity execution in second pipeline is from 10:08 AM UTC to 10:17 AM UTC. | 10 min \ 60 min * 4 Azure Integration Runtime (default DIU setting = 4) For more information on data integration units and optimizing copy performance, see [this article](copy-activity-performance.md) |

+**Total scenario pricing for 30 days: $42.14**

+In this scenario, you want to get delta changes from one table in SAP ECC via SAP CDC connector, do a few necessary transforms in flight, and then write data to Azure Data Lake Gen2 storage in ADF mapping dataflow daily. We will calculate prices for execution on a schedule once per hour for 8 hours over 30 days.

+| Run Pipeline | 2 Activity runs **per execution** (1 for trigger run, 1 for activity run) = 480, rounded up since the calculator only allows increments of 1000. |

+| Data Flow: execution hours of general compute with 8 cores **per execution** = 15 mins | 15 min / 60 min |

+| Self-Hosted Integration Runtime: data movement execution hours60 **per execution** = 15 mins | 15 min / 60 min |

+**Total scenario pricing for 30 days: $138.66**

+| Run Pipeline | 2 Activity runs **per execution** (1 for the trigger to run, 1 for activity to run) = 480 Activity runs, rounded up since the calculator only allows increments of 1000. |

+| Copy Data Assumption: DIU hours **per execution** | 0.5 hours \* 4 Azure Integration Runtime (default DIU setting = 4) For more information on data integration units and optimizing copy performance, see [this article](copy-activity-performance.md) |

+| Total execution hours: 8 executions per day for 30 days | 240 executions * 2 DIU/run = 480 DIUs |

+In this scenario, you want to transform data in Blob Store visually in ADF mapping data flows on an hourly schedule for 8 hours per day over 30 days.

+| Run Pipeline | 2 Activity runs **per execution** (1 for trigger run, 1 for activity runs) = 480 activity runs, rounded up since the calculator only allows increments of 1000. |

+| Data Flow Assumptions: General purpose 16 vCore hours **per execution** = 10 min + 10 min TTL | 20 min \ 60 min |

+**Total scenario pricing for 30 days: $350.76**

+| Block Blob storage Premium | |

+1. Create an environment by using a *catalog-item* ('infra-as-code' template defined in the [manifest.yaml](configure-catalog-item.md#add-a-new-catalog-item) file) from the list of available catalog items.

+ If the specific *catalog-item* requires any parameters use `--parameters` and provide the parameters as a json-string or json-file, for example:

+ --catalog-item-name <catalog-item-name> --catalog-name <catalog-name>

+ |**Folder path**|Provide the repo relative path in which the [catalog item](concept-environments-key-concepts.md#catalog-items) exist.|

+1. Select the specific repo, and then select **Sync**.

+

+ :::image type="content" source="media/configure-catalog-item/sync-catalog-items.png" alt-text="Screenshot showing how to sync the catalog." :::

+ Title: How to manage a dev box project

+description: This article describes how to create, and delete Microsoft Dev Box Preview dev box projects.

+<!-- Intent: As a dev infrastructure manager, I want to be able to manage dev box projects so that I can provide appropriate dev boxes to my users. -->

+# Manage a dev box project

+A project is the point of access to Microsoft Dev Box Preview for the development team members. A project contains dev box pools, which specify the dev box definitions and network connections used when dev boxes are created. Dev managers can configure the project with dev box pools that specify dev box definitions appropriate for their team's workloads. Dev box users create dev boxes from the dev box pools they have access to through their project memberships.

+Each project is associated with a single dev center. When you associate a project with a dev center, all the settings at the dev center level will be applied to the project automatically.

+## Project admins

+Microsoft Dev Box makes it possible for you to delegate administration of projects to a member of the project team. Project administrators can assist with the day-to-day management of projects for their team, like creating and managing dev box pools. To provide users permissions to manage projects, add them to the DevCenter Project Admin role. The tasks in this quickstart can be performed by project admins.

+To learn how to add a user to the Project Admin role, see [Provide access to a dev box project](#provide-access-to-a-dev-box-project).

+## Permissions

+To manage a dev box project, you need the following permissions:

+|Action|Permission required|

+|--|--|

+|Create or delete dev box project|Owner, Contributor, or Write permissions on the dev center in which you want to create the project. |

+|Update a dev box project|Owner, Contributor, or Write permissions on the project.|

+|Create, delete, and update dev box pools in the project|Owner, Contributor, or DevCenter Project Admin.|

+|Manage a dev box within the project|Owner, Contributor, or DevCenter Project Admin.|

+|Add a dev box user to the project|Owner permissions on the project.|

+## Create a dev box project

+The following steps show you how to create and configure a project in dev box.

+1. In the [Azure portal](https://portal.azure.com), in the search box, type *Projects* and then select **Projects** from the list.

+1. On the Projects page, select **+Create**.

+

+1. On the **Create a project** page, on the **Basics** tab, enter the following values:

+ |Name|Value|

+ |-|-|

+ |**Subscription**|Select the subscription in which you want to create the project.|

+ |**Resource group**|Select an existing resource group or select **Create new**, and enter a name for the resource group.|

+ |**Dev center**|Select the dev center to which you want to associate this project. All the dev center level settings will be applied to the project.|

+ |**Name**|Enter a name for your project. |

+ |**Description**|Enter a brief description of the project. |

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/dev-box-project-create.png" alt-text="Screenshot of the Create a dev box project basics tab.":::

+1. [Optional] On the **Tags** tab, enter a name and value pair that you want to assign.

+1. Select **Review + Create**.

+1. On the **Review** tab, select **Create**.

+1. Confirm that the project is created successfully by checking the notifications. Select **Go to resource**.

+1. Verify that you see the **Project** page.

+## Delete a dev box project

+You can delete a dev box project when you're no longer using it. Deleting a project is permanent and cannot be undone. You cannot delete a project that has dev box pools associated with it.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. In the search box, type *Projects* and then select **Projects** from the list.

+1. Open the project you want to delete.

+

+1. Select the dev box project you want to delete and then select **Delete**.

+

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/delete-project.png" alt-text="Screenshot of the list of existing dev box pools, with the one to be deleted selected.":::

+1. In the confirmation message, select **Confirm**.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/confirm-delete-project.png" alt-text="Screenshot of the Delete dev box pool confirmation message.":::

+## Provide access to a dev box project

+Before users can create dev boxes based on the dev box pools in a project, you must provide access for them through a role assignment. The Dev Box User role enables dev box users to create, manage and delete their own dev boxes. You must have sufficient permissions to a project before you can add users to it.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+

+1. In the search box, type *Projects* and then select **Projects** from the list.

+1. Select the project you want to provide your team members access to.

+

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/projects-grid.png" alt-text="Screenshot of the list of existing projects.":::

+1. Select **Access Control (IAM)** from the left menu.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/access-control-tab.png" alt-text="Screenshot showing the Project Access control page with the Access Control link highlighted.":::

+1. Select **Add** > **Add role assignment**.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/add-role-assignment.png" alt-text="Screenshot showing the Add menu with Add role assignment highlighted.":::

+1. On the Add role assignment page, search for *devcenter dev box user*, select the **DevCenter Dev Box User** built-in role, and then select **Next**.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/dev-box-user-role.png" alt-text="Screenshot showing the Add role assignment search box highlighted.":::

+1. On the Members page, select **+ Select Members**.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/dev-box-user-select-members.png" alt-text="Screenshot showing the Members tab with Select members highlighted.":::

+1. On the **Select members** pane, select the Active Directory Users or Groups you want to add, and then select **Select**.

+ :::image type="content" source="./media/how-to-manage-dev-box-projects/select-members-search.png" alt-text="Screenshot showing the Select members pane with a user account highlighted.":::

+1. On the Add role assignment page, select **Review + assign**.

+The user will now be able to view the project and all the pools within it. They can create dev boxes from any of the pools and manage those dev boxes from the [developer portal](https://aka.ms/devbox-portal).

+## Next steps

+- [Manage dev box pools](./how-to-manage-dev-box-pools.md)

+- [Create dev box definitions](./quickstart-configure-dev-box-service.md#create-a-dev-box-definition)

+- [Configure an Azure Compute Gallery](./how-to-configure-azure-compute-gallery.md)

+`MATCH` supports any query that finds a path between twins within a range of hops, based on certain relationship conditions.

+It contains these placeholders:

+* `twin_or_twin_collection` (x2): The `MATCH` clause requires one operand to represent a single twin. The other operand can represent another single twin, or a collection of twins.

+* `relationship_condition`: In this space, define a condition that describes the relationship between the twins or twin collections. The condition can [specify relationship direction](#specify-relationship-direction), [specify relationship name](#specify-relationship-name), [specify number of hops](#specify-number-of-hops), [specify relationship properties](#assign-query-variable-to-relationship-and-specify-relationship-properties), or [any combination of these options](#combine-match-operations).

+* `twin_ID`: Here, specify a `$dtId` within one of the twin collections so that one of the operands represents a single twin.

+You can leave one of the twin collections blank in order to allow any twin to work in that spot.

+This section shows the syntax for different directions of relationships. The placeholder values that should be replaced with your values are `source_twin_or_twin_collection` and `target_twin_or_twin_collection`.

+For a single name, use the following syntax. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1`, `relationship_name`, and `twin_or_twin_collection_2`.

+For multiple possible names use the following syntax. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1`, `relationship_name_option_1`, `relationship_name_option_2`, `twin_or_twin_collection_2`, and the note to continue the pattern as needed for the number of relationship names you want to enter.

+To specify an exact number of hops, use the following syntax. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1`, `number_of_hops`, and `twin_or_twin_collection_2`.

+To specify a range of hops, use the following syntax. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1`, `starting_limit`, `ending_limit` and `twin_or_twin_collection_2`. The starting limit isn't included in the range, while the ending limit is included.

+To assign a query variable to the relationship, put the variable name in the square brackets (`[]`). The placeholder values shown below that should be replaced with your values are `twin_or_twin_collection_1`, `relationship_variable`, and `twin_or_twin_collection_2`.

+To specify relationship direction, relationship name, and number of hops within a single query, use the following syntax within the relationship condition. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1` and `twin_or_twin_collection_2`, `optional_left_angle_bracket` and `optional_right_angle_bracket`, `relationship_name(s)`, and `number_of_hops`.

+To specify relationship direction, relationship name, and a query variable for the relationship within a single query, use the following syntax within the relationship condition. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1` and `twin_or_twin_collection_2`, `optional_left_angle_bracket` and `optional_right_angle_bracket`, `relationship_variable`, and `relationship_name(s)`.

+You can chain multiple relationship conditions together, like this. The placeholder values that should be replaced with your values are `twin_or_twin_collection_1`, all instances of `relationship_condition`, and `twin_or_twin_collection_2`.

+- The following IP address space is reserved and can't be used for the DNS resolver service: 10.0.1.0 - 10.0.16.255.

+ - Do not use these class C networks or subnets within these networks for DNS resolver subnets: 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24, 10.0.4.0/24, 10.0.5.0/24, 10.0.6.0/24, 10.0.7.0/24, 10.0.8.0/24, 10.0.9.0/24, 10.0.10.0/24, 10.0.11.0/24, 10.0.12.0/24, 10.0.13.0/24, 10.0.14.0/24, 10.0.15.0/24, 10.0.16.0/24.

+- Rulesets can have up to 25 rules.

+ExpressRoute allows you to create a connection between your on-premises network and the Microsoft cloud in four different ways, [CloudExchange Co-location](#CloudExchange), [Point-to-point Ethernet Connection](#Ethernet), [Any-to-any (IPVPN) Connection](#IPVPN), and [ExpressRoute Direct](#Direct). Connectivity providers may offer more than one connectivity models. You can work with your connectivity provider to pick the model that works best for you.

+If you're co-located in a facility with a cloud exchange, you can request for virtual cross-connections to the Microsoft cloud through the co-location providerΓÇÖs Ethernet exchange. Co-location providers can offer either Layer 2 cross-connections, or managed Layer 3 cross-connections between your infrastructure in the co-location facility and the Microsoft cloud.

+You can connect your on-premises datacenters or offices to the Microsoft cloud through point-to-point Ethernet links. Point-to-point Ethernet providers can offer Layer 2 connections, or managed Layer 3 connections between your site and the Microsoft cloud.

+You can integrate your WAN with the Microsoft cloud. IPVPN providers (typically MPLS VPN) offer any-to-any connectivity between your branch offices and datacenters. The Microsoft cloud can be interconnected to your WAN to make it appear like any other branch office. WAN providers typically offer managed Layer 3 connectivity. ExpressRoute capabilities and features are all identical across all of the above connectivity models.

+You can connect directly into the Microsoft global network at a peering location strategically distributed across the world. [ExpressRoute Direct](expressroute-erdirect-about.md) provides dual 100-Gbps or 10-Gbps connectivity that supports Active/Active connectivity at scale.

+ExpressRoute Direct gives you the ability to connect directly into the Microsoft global network at peering locations strategically distributed around the world. ExpressRoute Direct provides dual 100-Gbps or 10-Gbps connectivity, that supports Active/Active connectivity at scale. You can work with any service provider to set up ExpressRoute Direct.

+Key features that ExpressRoute Direct provides include, but not limited to:

+* Large data ingestion into services like Azure Storage and Azure Cosmos DB.

+* Physical isolation for industries that regulates and require dedicated or isolated connectivity such as banks, government, and retail companies.

+* Granular control of circuit distribution based on business unit.

+Before you can set up ExpressRoute Direct, you must first enroll your subscription. Run the following commands using Azure PowerShell:

+1. Register your subscription to **AllowExpressRoutePorts** using the following command:

+When you start using ExpressRoute Direct and notice that there aren't any available ports for your chosen peering location, submit a support request to request for more inventory.

+| ExpressRoute using a service provider | ExpressRoute Direct |

+| Uses a service provider to enable fast onboarding and connectivity into existing infrastructure | Requires 100-Gbps or 10-Gbps infrastructure and full management of all layers |

+| Integrates with hundreds of providers including Ethernet and MPLS | Direct and Dedicated capacity for regulated industries and large data ingestion |

+| Circuits SKUs ranging from 50 Mbps to 10 Gbps | Customer may select a combination of the following circuit SKUs on 100-Gbps ExpressRoute Direct: <ul><li>5 Gbps</li><li>10 Gbps</li><li>40 Gbps</li><li>100 Gbps</li></ul> Customer may select a combination of the following circuit SKUs on 10-Gbps ExpressRoute Direct:<ul><li>1 Gbps</li><li>2 Gbps</li><li>5 Gbps</li><li>10 Gbps</li></ul>

+| Optimized for a single tenant | Optimized for single tenant with multiple business units and multiple work environments

+Azure ExpressRoute allows you to extend your on-premises network into the Microsoft cloud over a private connection made possible through a connectivity provider. With ExpressRoute, you can establish connections to Microsoft cloud services, such as Microsoft Azure, and Microsoft 365.

+Each peering location has access to the Microsoft global network and can access any region in a geopolitical zone by default. You can access any global regions when you set up a premium circuit.

+The functionality in most scenarios is equivalent to circuits that use an ExpressRoute service provider to operate. To support further granularity and new capabilities offered using ExpressRoute Direct, there are certain key capabilities that exist only with ExpressRoute Direct circuits.

+ExpressRoute Direct supports large data ingestion scenarios into services such as Azure storage. ExpressRoute circuits with 100-Gbps ExpressRoute Direct also support **40 Gbps** and **100 Gbps** circuit bandwidth. The physical port pairs are **100 Gbps or 10 Gbps** only and can have multiple virtual circuits.

+### Circuit sizes

+| 100-Gbps ExpressRoute Direct | 10-Gbps ExpressRoute Direct |

+| Subscribed Bandwidth: 200 Gbps | Subscribed Bandwidth: 20 Gbps |

+> [!NOTE]

+> You can provision logical ExpressRoute circuits on top of your selected ExpressRoute Direct resource of 10-Gbps or 100-Gbps up to the subscribed Bandwidth of 20Gbps or 200Gbps. For example,you can provision two 10 Gbps ExpressRoute circuits within a single 10 Gbps ExpressRoute Direct resource (port pair). Configuring circuits that over-subscribe the ExpressRoute Direct resource is only avialable with Azure PowerShell and Azure CLI.

+* **QinQ VLAN Tagging** allows for isolated routing domains on a per ExpressRoute circuit basis. Azure dynamically gives an S-Tag at circuit creation and can't be changed. Each peering on the circuit (Private and Microsoft) will use a unique C-Tag as the VLAN. The C-Tag isn't required to be unique across circuits on the ExpressRoute Direct ports.

+ExpressRoute Direct provides the same enterprise-grade SLA with Active/Active redundant connections into the Microsoft Global Network. ExpressRoute infrastructure is redundant and connectivity into the Microsoft Global Network is redundant and diverse and scales correctly with customer requirements. For more information, see [ExpressRoute SLA](https://azure.microsoft.com/support/legal/sla/expressroute/v1_3/).

+1. Enter a unique name for your IP Group, and then select a region.

+ ResourceGroupName = 'Test-FW-RG'

+ Location = 'East US'

+ --name ipGroup \

+ --resource-group Test-FW-RG \

+ --location eastus \

+$rgName = "Test-FW-RG"

+ -Name "Test-FW-VN" `

+ -ResourceGroupName "Test-FW-RG" `

+ -Location "eastus" `

+ -AllocationMethod Static `

+ -Zone 1,2,3

+ -Location "eastus" `

+|SNAT on inbound connections|In addition to DNAT, connections via the firewall public IP address (inbound) are SNATed to one of the firewall private IPs. This requirement today (also for Active/Active NVAs) to ensure symmetric routing.|To preserve the original source for HTTP/S, consider using [XFF](https://en.wikipedia.org/wiki/X-Forwarded-For) headers. For example, use a service such as [Azure Front Door](../frontdoor/front-door-http-headers-protocol.md#from-the-front-door-to-the-backend) or [Azure Application Gateway](../application-gateway/rewrite-http-headers-url.md) in front of the firewall. You can also add WAF as part of Azure Front Door and chain to the firewall.

+1. If you filter traffic to Azure SQL Database, Azure Synapse Analytics, or SQL Managed Instance, ensure the SQL connectivity mode is set to **Proxy**. To learn how to switch SQL connectivity mode, see [Azure SQL Connectivity Settings](/azure/azure-sql/database/connectivity-settings#change-the-connection-policy-via-the-azure-cli).

+1. Create a new rule collection with an application rule using SQL FQDN to allow access to a SQL server:

+ --resource-group Test-FW-RG \

+ --firewall-name Test-FW01 \

+ --collection-name sqlRuleCollection \

+ --priority 1000 \

+ --action Allow \

+ --name sqlRule \

+ --protocols mssql=1433 \

+ --source-addresses 10.0.0.0/24 \

+ --target-fqdns sql-serv1.database.windows.net

+1. If you filter traffic to Azure SQL Database, Azure Synapse Analytics, or SQL Managed Instance, ensure the SQL connectivity mode is set to **Proxy**. To learn how to switch SQL connectivity mode, see [Azure SQL Connectivity Settings](/azure/azure-sql/database/connectivity-settings#change-the-connection-policy-via-the-azure-cli).

+1. Create a new rule collection with an application rule using SQL FQDN to allow access to a SQL server:

+ $AzFw = Get-AzFirewall -Name "Test-FW01" -ResourceGroupName "Test-FW-RG"

+1. If you filter traffic to Azure SQL Database, Azure Synapse Analytics, or SQL Managed Instance, ensure the SQL connectivity mode is set to **Proxy**. To learn how to switch SQL connectivity mode, see [Azure SQL Connectivity Settings](/azure/azure-sql/database/connectivity-settings#change-the-connection-policy-via-the-azure-cli).

+1. Add the application rule with the appropriate protocol, port, and SQL FQDN and then select **Save**.

+1. Access SQL from a virtual machine in a VNet that filters the traffic through the firewall.

+1. Validate that [Azure Firewall logs](./firewall-workbook.md) show the traffic is allowed.

+ Title: DDoS protection on Azure Front Door

+Front Door is a large scaled, globally distributed service. We have many customers, including Microsoft's own large-scale cloud products that receive hundreds of thousands of requests each second. Front Door is located at the edge of Azure's network, absorbing and geographically isolating large volume attacks. This can prevent malicious traffic from going any further than the edge of the Azure network.

+[Front Door's Web Application Firewall (WAF)](../web-application-firewall/afds/afds-overview.md) can be used to mitigate many different types of attacks:

+* Using the managed rule set provides protection against many common attacks.

+If you require further protection, then you can enable [Azure DDoS Protection Standard](../ddos-protection/ddos-protection-overview.md) on the VNet where your back-ends are deployed. DDoS Protection Standard customers receive extra benefits including cost protection, SLA guarantee, and access to experts from the DDoS Rapid Response Team for immediate help during an attack.

+- Learn how to configure a [WAF policy for Azure Front Door](front-door-waf.md).

+- Learn how to [create an Azure Front Door profile](quickstart-create-front-door.md).

+- Learn [how Azure Front Door works](front-door-routing-architecture.md).

+This article outlines the protocol that Front Door supports with parts of the call path (see image). In the following sections, you'll find information about HTTP headers supported by Front Door.

+> [!IMPORTANT]

+> Front Door doesn't certify any HTTP headers that aren't documented here.

+## From client to the Front Door

+Azure Front Door accepts most headers for the incoming request without modifying them. Some reserved headers are removed from the incoming request if sent, including headers with the X-FD-* prefix.

+The debug request header, "X-Azure-DebugInfo", provides extra debugging information about the Front Door. You'll need to send "X-Azure-DebugInfo: 1" request header from the client to the AzureFront Door to receive [optional response headers](#optional-debug-response-headers) when Front Door response to the client.

+## From the Front Door to the backend

+Azure Front Door includes headers for an incoming request unless they're removed because of restrictions. Front Door also adds the following headers:

+## From the Front Door to the client

+Any headers sent to Azure Front Door from the backend are also passed through to the client. The following are headers sent from the Front Door to clients.

+| X-Cache | *X-Cache:* This header describes the caching status of the request <br/> - *X-Cache: TCP_HIT*: The first byte of the request is a cache hit in the Front Door edge. <br/> - *X-Cache: TCP_REMOTE_HIT*: The first byte of the request is a cache hit in the regional cache (origin shield layer) but a miss in the edge cache. <br/> - *X-Cache: TCP_MISS*: The first byte of the request is a cache miss, and the content is served from the origin. <br/> - *X-Cache: PRIVATE_NOSTORE*: Request can't be cached as Cache-Control response header is set to either private or no-store. <br/> - *X-Cache: CONFIG_NOCACHE*: Request is configured to not cache in the Front Door profile. |

+* Learn how to [create an Azure Front Door profile](quickstart-create-front-door.md).

+* Learn about [how Azure Front Door works](front-door-routing-architecture.md).

+description: In this article, you'll learn how to deploy the MedTech service in the Azure portal using a Quickstart template.

+ Title: Quickstart - Use Terraform to create a DPS instance

+description: Learn how to deploy an Azure IoT Device Provisioning Service (DPS) resource with Terraform in this quickstart.

+keywords: azure, devops, terraform, device provisioning service, DPS, IoT, IoT Hub DPS

+# Quickstart: Use Terraform to create an Azure IoT Device Provisioning Service

+In this quickstart, you will learn how to deploy an Azure IoT Hub Device Provisioning Service (DPS) resource with a hashed allocation policy using Terraform.

+This quickstart was tested with the following Terraform and Terraform provider versions:

+- [Terraform v1.2.8](https://releases.hashicorp.com/terraform/)

+- [AzureRM Provider v.3.20.0](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)

+[Terraform](https://www.terraform.io/) enables the definition, preview, and deployment of cloud infrastructure. Using Terraform, you create configuration files using HCL syntax. The [HCL syntax](https://www.terraform.io/language/syntax/configuration) allows you to specify the cloud provider - such as Azure - and the elements that make up your cloud infrastructure. After you create your configuration files, you create an execution plan that allows you to preview your infrastructure changes before they're deployed. Once you verify the changes, you apply the execution plan to deploy the infrastructure.

+In this article, you learn how to:

+- Create a Storage Account & Storage Container

+- Create an Event Hubs, Namespace, & Authorization Rule

+- Create an IoT Hub

+- Link IoT Hub to Storage Account endpoint & Event Hubs endpoint

+- Create an IoT Hub Shared Access Policy

+- Create a DPS Resource

+- Link DPS & IoT Hub

+## Prerequisites

+- [Install and configure Terraform](/azure/developer/terraform/quickstart-configure)

+## Implement the Terraform code

+> [!NOTE]

+> The example code in this article is located in the [Azure Terraform GitHub repo](https://github.com/Azure/terraform/tree/master/). See more [articles and sample code showing how to use Terraform to manage Azure resources](/azure/developer/terraform/)

+1. Create a directory in which to test and run the sample Terraform code and make it the current directory.

+1. Create a file named `providers.tf` and insert the following code:

+ [!code-terraform[master](~/terraform_samples/quickstart/201-iot-hub-with-device-provisioning-service/providers.tf)]

+1. Create a file named `main.tf` and insert the following code:

+ [!code-terraform[master](~/terraform_samples/quickstart/201-iot-hub-with-device-provisioning-service/main.tf)]

+1. Create a file named `variables.tf` and insert the following code:

+ [!code-terraform[master](~/terraform_samples/quickstart/201-iot-hub-with-device-provisioning-service/variables.tf)]

+1. Create a file named `outputs.tf` and insert the following code:

+ [!code-terraform[master](~/terraform_samples/quickstart/201-iot-hub-with-device-provisioning-service/outputs.tf)]

+## Initialize Terraform

+## Create a Terraform execution plan

+## Apply a Terraform execution plan

+## Verify the results

+**Azure CLI**

+Run [az iot dps show](/cli/azure/iot/dps#az-iot-dps-show) to display the Azure DPS resource.

+ ```azurecli

+ az iot dps show \

+ --name my_terraform_dps \

+ --resource-group rg

+ ```

+**Azure PowerShell**

+Run [Get-AzIoTDeviceProvisioningService](/powershell/module/az.deviceprovisioningservices/get-aziotdeviceprovisioningservice) to display the Azure DPS resource.

+ ```powershell

+ Get-AzIoTDeviceProvisioningService `

+ -ResourceGroupName "rg" `

+ -Name "my_terraform_dps"

+ ```

+The names of the resource group and the DPS instance are displayed in the terraform apply output. You can also run the [terraform output](https://www.terraform.io/cli/commands/output) command to view these output values.

+## Clean up resources

+## Troubleshoot Terraform on Azure

+[Troubleshoot common problems when using Terraform on Azure](/azure/developer/terraform/troubleshoot)

+## Next steps

+Now that you have an instance of the Device Provisioning Service, continue to the next quickstart to provision a simulated device to IoT hub:

+> [!div class="nextstepaction"]

+> [Quickstart: Provision a simulated symmetric key device](./quick-create-simulated-device-symm-key.md)

+- Azure IoT Edge for Linux on Windows 1.3.1.02092 update or higher. For more information about EFLOW release notes, see [EFLOW Releases](https://aka.ms/AzEFLOW-Releases).

+ Title: Configure Access Control in Device Update for IoT Hub | Microsoft Docs

+description: Configure Access Control in Device Update for IoT Hub.

+# Configure access control roles for Device Update resources

+In order for users to have access to Device Update, they must be granted access to the Device Update account, Instance and set the required access to the linked IoT hub.

+## Configure access control for Device Update account

+# [Azure portal](#tab/portal)

+1. In your Device Update account, select **Access control (IAM)** from the navigation menu.

+ :::image type="content" source="media/create-device-update-account/account-access-control.png" alt-text="Screenshot of access Control within Device Update account." lightbox="media/create-device-update-account/account-access-control.png":::

+2. Select **Add role assignments**.

+3. On the **Role** tab, select a Device Update role from the available options:

+ * Device Update Administrator

+ * Device Update Reader

+ * Device Update Content Administrator

+ * Device Update Content Reader

+ * Device Update Deployments Administrator

+ * Device Update Deployments Reader

+ For more information, [Learn about Role-based access control in Device Update for IoT Hub](device-update-control-access.md).

+ :::image type="content" source="media/create-device-update-account/role-assignment.png" alt-text="Screenshot of access Control role assignments within Device Update account." lightbox="media/create-device-update-account/role-assignment.png":::

+4. Select **Next**

+5. On the **Members** tab, select the users or groups that you want to assign the role to.

+ :::image type="content" source="media/create-device-update-account/role-assignment-2.png" alt-text="Screenshot of access Control member selection within Device Update account." lightbox="media/create-device-update-account/role-assignment-2.png":::

+6. Select **Review + assign**

+7. Review the new role assignments and select **Review + assign** again

+8. You're now ready to use Device Update from within your IoT Hub

+# [Azure CLI](#tab/cli)

+The following roles are available for assigning access to Device Update:

+* Device Update Administrator

+* Device Update Reader

+* Device Update Content Administrator

+* Device Update Content Reader

+* Device Update Deployments Administrator

+* Device Update Deployments Reader

+For more information, [Learn about Role-based access control in Device Update for IoT Hub](device-update-control-access.md).

+Use the [az role assignment create](/cli/azure/role/assignment#az-role-assignment-create) command to configure access control for your Device Update account.

+Replace the following placeholders with your own information:

+* *\<role>*: The Device Update role that you're assigning.

+* *\<user_group>*: The user or group that you want to assign the role to.

+* *\<account_id>*: The resource ID for the Device Update account that the user or group will get access to. You can retrieve the resource ID by using the [az iot du account show](/cli/azure/iot/du/account#az-iot-du-account-show) command and querying for the ID value: `az iot du account show -n <account_name> --query id`.

+```azurecli-interactive

+az role assignment create --role '<role>' --assignee <user_group> --scope <account_id>

+```

+## Configure access for Azure Device Update service principal in linked IoT hub

+Device Update for IoT Hub communicates with IoT Hub to manage deployments and updates and to get information about devices. To enable the access, you need to give the **Azure Device Update** service principal access with the **IoT Hub Data Contributor** role.

+# [Azure portal](#tab/portal)

+1. In the Azure portal, navigate to the IoT hub connected to your Device Update instance.

+ :::image type="content" source="media/create-device-update-account/navigate-to-iot-hub.png" alt-text="Screenshot of instance and linked IoT hub." lightbox="media/create-device-update-account/navigate-to-iot-hub.png":::

+1. Select **Access Control(IAM)** from the navigation menu. Select **Add** > **Add role assignment**.

+ :::image type="content" source="media/create-device-update-account/iot-hub-access-control.png" alt-text="Screenshot of access Control within IoT Hub." lightbox="media/create-device-update-account/iot-hub-access-control.png":::

+3. In the **Role** tab, select **IoT Hub Data Contributor**. Select **Next**.

+ :::image type="content" source="media/create-device-update-account/role-assignment-iot-hub.png" alt-text="Screenshot of access Control role assignment within IoT Hub." lightbox="media/create-device-update-account/role-assignment-iot-hub.png":::**

+4. For **Assign access to**, select **User, group, or service principal**. Select **Select Members** and search for '**Azure Device Update**'

+ :::image type="content" source="media/create-device-update-account/assign-role-to-du-service-principal.png" alt-text="Screenshot of access Control member selection for IoT Hub." lightbox="media/create-device-update-account/assign-role-to-du-service-principal.png":::

+6. Select **Next** > **Review + Assign**

+To validate that you've set permissions correctly:

+1. In the Azure portal, navigate to the IoT hub connected to your Device Update instance.

+1. Select **Access Control(IAM)** from the navigation menu.

+1. Select **Check access**.

+1. Select **User, group, or service principal** and search for '**Azure Device Update**'

+1. After clicking on **Azure Device Update**, verify that the **IoT Hub Data Contributor** role is listed under **Role assignments**

+# [Azure CLI](#tab/cli)

+Use the [az role assignment create](/cli/azure/role/assignment#az-role-assignment-create) command to create a role assignment for the Azure Device Update service principal.

+Replace *\<resource_id>* with the resource ID of your IoT hub. You can retrieve the resource ID by using the [az iot hub show](/cli/azure/iot/hub#az-iot-hub-show) command and querying for the ID value: `az iot hub show -n <hub_name> --query id`.

+```azurecli

+az role assignment create --role "IoT Hub Data Contributor" --assignee https://api.adu.microsoft.com/ --scope <resource_id>

+```

+## Next steps

+Try updating a device using one of the following quick tutorials:

+* [Update a simulated IoT Edge device](device-update-simulator.md)

+* [Update a Raspberry Pi](device-update-raspberry-pi.md)

+* [Update an Ubuntu Server 18.04 x64 Package agent](device-update-ubuntu-agent.md)

+ * **Resource ID or alias**: Enter the Resource ID of the Device Update account. You can retrieve the resource ID of a Device Update account from the Azure portal by selecting **JSON View** on the **Overview** page. Or, you can retrieve it by using the [az iot du account show](/cli/azure/iot/du/account#az-iot-du-account-show) command and querying for the ID value: `az iot du account show -n <account_name> --query id`.

+* **DEVICE_UPDATE_RESOURCE_ID**: You can retrieve the resource ID of a Device Update account from the Azure portal by selecting **JSON View** on the **Overview** page. Or, you can retrieve it by using the [az iot du account show](/cli/azure/iot/du/account#az-iot-du-account-show) command and querying for the ID value: `az iot du account show -n <account_name> --query id`.

+Use the [az iot du account private-endpoint-connection set](/cli/azure/iot/du/account/private-endpoint-connection#az-iot-du-account-private-endpoint-connection-set) command to manage private endpoint connection.

+az iot du account private-endpoint-connection set \

+An IoT hub. It's required that you use an S1 (Standard) tier or above.

+* An IoT hub. It's required that you use an S1 (Standard) tier or above.

+3. On the **Basics** tab, provide the following information for your Device Update account and instance:

+ * Check the box to assign the Device Update administrator role to yourself. You can also use the steps listed in the [Configure access control roles](configure-access-control-device-update.md) section to provide a combination of roles to users and applications for the right level of access. You need to have Owner or User Access Administrator permissions in your subscription to manage roles.

+ * **Instance Name**: A name for your instance.

+ * **IoT Hub Name**: Select the IoT Hub you want to link to your Device Update instance

+ * Check the box to grant the right access to Azure Device Update service principal in the IoT Hub to set up and operate the Device Update Service. You need to have the right permissions to add access.

+ > If you are unable to grant access to Azure Device Update service principal during resource creation, refer to [configure the access control for users and Azure Device Update service principal](configure-access-control-device-update.md) . If this access is not set you will not be able to run deployment, device management and diagnostic operations. Learn more about the [Azure Device Update service principal access](device-update-control-access.md#configuring-access-for-azure-device-update-service-principal-in-the-iot-hub).

+5. Select **Next: Diagnostics**. Enabling Microsoft diagnostics, gives Microsoft permission to collect, store, and analyze diagnostic log files from your devices when they encounter an update failure. In order to enable remote log collection for diagnostics, you need to link your Device Update instance to your Azure Blob storage account. Selecting the Azure Storage account will automatically update the storage details.

+ :::image type="content" source="media/create-device-update-account/account-diagnostics.png" alt-text="Screenshot of diagnostic details." lightbox="media/create-device-update-account/account-diagnostics.png":::

+6. On the **Networking** tab, to continue creating Device Update account and instance.

+ Choose the endpoints that devices can use to connect to your Device Update instance. Accept the default setting, Public access, for this example.

+ :::image type="content" source="media/create-device-update-account/account-networking.png" alt-text="Screenshot of networking details." lightbox="media/create-device-update-account/account-networking.png":::

+Use the [az iot du account create](/cli/azure/iot/du/account#az-iot-du-account-create) command to create a new Device Update account.

+az iot du account create --resource-group <resource_group> --account <account_name> --location <region>

+Use the [az iot du instance create](/cli/azure/iot/du/instance#az-iot-du-instance-create) command to create a Device Update instance.

+az iot du instance create --account <account_name> --instance <instance_name> --iothub-ids <iothub_id>

+Once you have created the resource, [configure the access control for users and Azure Device Update service principal](configure-access-control-device-update.md).

+* [Configure Access Control in Device Update for IoT Hub ](configure-access-control-device-update.md)

+* [Learn about Device update account and instance.](device-update-resources.md)

+* [Learn about Device update access control roles](device-update-control-access.md)

+ Title: Manage device groups in Device Update for Azure IoT Hub | Microsoft Docs

+description: Configure device groups in Device Update for Azure IoT Hub by using twin tags

+# Manage device groups in Device Update for IoT Hub