+> [!NOTE]

+> We recently changed the color of the "verified" badge from blue to gray. We will revert that change sometime in the last half of February 2022, so the "verified" badge will be blue.

+> | Python | [Sign in users](https://github.com/Azure-Samples/ms-identity-python-desktop) | MSAL Python | Resource owner password credentials |

+// You provide either a ClientSecret or a CertificateConfiguration, or a ClientAssertion. These settings are exclusive

+CERTIFICATE_THUMBPRINT=Enter_the_certificate_thumbprint_Here

+CERTIFICATE_PRIVATE_KEY=Enter_the_certificate_private_key_Here

+CLIENT_ASSERTION=Enter_the_Assertion_String_Here

+```JavaScript

+const config = {

+ auth: {

+ clientId: process.env.CLIENT_ID,

+ authority: process.env.AAD_ENDPOINT + process.env.TENANT_ID,

+ clientCertificate: {

+ thumbprint: process.env.CERTIFICATE_THUMBPRINT, // a 40-digit hexadecimal string

+ privateKey: process.env.CERTIFICATE_PRIVATE_KEY,

+ }

+ }

+};

+// Create an MSAL application object

+const cca = new msal.ConfidentialClientApplication(config);

+```

+For details, see [Use certificate credentials with MSAL Node](https://github.com/AzureAD/microsoft-authentication-library-for-js/blob/dev/lib/msal-node/docs/certificate-credentials.md).

+```JavaScript

+const clientConfig = {

+ auth: {

+ clientId: process.env.CLIENT_ID,

+ authority: process.env.AAD_ENDPOINT + process.env.TENANT_ID,

+ clientAssertion: process.env.CLIENT_ASSERTION

+ }

+};

+const cca = new msal.ConfidentialClientApplication(clientConfig);

+```

+For details, see [Initialize the ConfidentialClientApplication object](https://github.com/AzureAD/microsoft-authentication-library-for-js/blob/dev/lib/msal-node/docs/initialize-confidential-client-application.md).

+## July 2021

+### New Google sign-in integration for Azure AD B2C and B2B self-service sign-up and invited external users will stop working starting July 12, 2021

+**Type:** Plan for change

+**Service category:** B2B

+**Product capability:** B2B/B2C

+

+Previously we announced that [the exception for Embedded WebViews for Gmail authentication will expire in the second half of 2021](https://www.yammer.com/cepartners/threads/1188371962232832).

+On July 7, 2021, we learned from Google that some of these restrictions will apply starting **July 12, 2021**. Azure AD B2B and B2C customers who set up a new Google ID sign-in in their custom or line of business applications to invite external users or enable self-service sign-up will have the restrictions applied immediately. As a result, end-users will be met with an error screen that blocks their Gmail sign-in if the authentication is not moved to a system webview. See the docs linked below for details.

+Most apps use system web-view by default, and will not be impacted by this change. This only applies to customers using embedded webviews (the non-default setting.) We advise customers to move their application's authentication to system browsers instead, prior to creating any new Google integrations. To learn how to move to system browsers for Gmail authentications, read the Embedded vs System Web UI section in the [Using web browsers (MSAL.NET)](../develop/msal-net-web-browsers.md#embedded-vs-system-web-ui) documentation. All MSAL SDKs use the system web-view by default. [Learn more](../external-identities/google-federation.md#deprecation-of-web-view-sign-in-support).

+### Google sign-in on embedded web-views expiring September 30, 2021

+**Type:** Plan for change

+**Service category:** B2B

+**Product capability:** B2B/B2C

+

+About two months ago we announced that the exception for Embedded WebViews for Gmail authentication will expire in the second half of 2021.

+Recently, Google has specified the date to be **September 30, 2021**.

+Rolling out globally beginning September 30, 2021, Azure AD B2B guests signing in with their Gmail accounts will now be prompted to enter a code in a separate browser window to finish signing in on Microsoft Teams mobile and desktop clients. This applies to invited guests and guests who signed up using Self-Service Sign-Up.

+Azure AD B2C customers who have set up embedded webview Gmail authentications in their custom/line of business apps or have existing Google integrations, will no longer can let their users sign in with Gmail accounts. To mitigate this, make sure to modify your apps to use the system browser for sign-in. For more information, read the Embedded vs System Web UI section in the [Using web browsers (MSAL.NET)](../develop/msal-net-web-browsers.md#embedded-vs-system-web-ui) documentation. All MSAL SDKs use the system web-view by default.

+As the device login flow will start rolling out on September 30, 2021, it is likely that it may not be rolled out to your region yet (in which case, your end-users will be met with the error screen shown in the documentation until it gets deployed to your region.)

+For details on known impacted scenarios and what experience your users can expect, read [Add Google as an identity provider for B2B guest users](../external-identities/google-federation.md#deprecation-of-web-view-sign-in-support).

+### Bug fixes in My Apps

+**Type:** Fixed

+**Service category:** My Apps

+**Product capability:** End User Experiences

+

+- Previously, the presence of the banner recommending the use of collections caused content to scroll behind the header. This issue has been resolved.

+- Previously, there was another issue when adding apps to a collection, the order of apps in All Apps collection would get randomly reordered. This issue has also been resolved.

+For more information on My Apps, read [Sign in and start apps from the My Apps portal](https://support.microsoft.com/account-billing/sign-in-and-start-apps-from-the-my-apps-portal-2f3b1bae-0e5a-4a86-a33e-876fbd2a4510).

+### Public preview - Application authentication method policies

+**Type:** New feature

+**Service category:** MS Graph

+**Product capability:** Developer Experience

+

+Application authentication method policies in MS Graph which allow IT admins to enforce lifetime on application password secret credential or block the use of secrets altogether. Policies can be enforced for an entire tenant as a default configuration and it can be scoped to specific applications or service principals. [Learn more](/graph/api/resources/policy-overview).

+

+### Public preview - Authentication Methods registration campaign to download Microsoft Authenticator

+**Type:** New feature

+**Service category:** Microsoft Authenticator App

+**Product capability:** User Authentication

+

+The Authenticator registration campaign helps admins to move their organizations to a more secure posture by prompting users to adopt the Microsoft Authenticator app. Prior to this feature, there was no way for an admin to push their users to set up the Authenticator app.

+The registration campaign comes with the ability for an admin to scope users and groups by including and excluding them from the registration campaign to ensure a smooth adoption across the organization. [Learn more](../authentication/how-to-mfa-registration-campaign.md)

+

+### Public preview - Separation of duties check

+**Type:** New feature

+**Service category:** User Access Management

+**Product capability:** Entitlement Management

+

+In Azure AD entitlement management, an administrator can define that an access package is incompatible with another access package or with a group. Users who have the incompatible memberships will be then unable to request more access. [Learn more](../governance/entitlement-management-access-package-request-policy.md#prevent-requests-from-users-with-incompatible-access-preview).

+

+### Public preview - Identity Protection logs in Log Analytics, Storage Accounts, and Event Hubs

+**Type:** New feature

+**Service category:** Identity Protection

+**Product capability:** Identity Security & Protection

+

+You can now send the risky users and risk detections logs to Azure Monitor, Storage Accounts, or Log Analytics using the Diagnostic Settings in the Azure AD blade. [Learn more](../identity-protection/howto-export-risk-data.md).

+

+### Public preview - Application Proxy API addition for backend SSL certificate validation

+**Type:** New feature

+**Service category:** App Proxy

+**Product capability:** Access Control

+

+The onPremisesPublishing resource type now includes the property, "isBackendCertificateValidationEnabled" which indicates whether backend SSL certificate validation is enabled for the application. For all new Application Proxy apps, the property will be set to true by default. For all existing apps, the property will be set to false. For more information, read the [onPremisesPublishing resource type](/graph/api/resources/onpremisespublishing?view=graph-rest-beta&preserve-view=true) api.

+

+### General availability - Improved Authenticator setup experience for add Azure AD account in Microsoft Authenticator app by directly signing into the app.

+**Type:** New feature

+**Service category:** Microsoft Authenticator App

+**Product capability:** User Authentication

+

+Users can now use their existing authentication methods to directly sign into the Microsoft Authenticator app to set up their credential. Users don't need to scan a QR Code anymore and can use a Temporary Access Pass (TAP) or Password + SMS (or other authentication method) to configure their account in the Authenticator app.

+This improves the user credential provisioning process for the Microsoft Authenticator app and gives the end user a self-service method to provision the app. [Learn more](https://support.microsoft.com/account-billing/add-your-work-or-school-account-to-the-microsoft-authenticator-app-43a73ab5-b4e8-446d-9e54-2a4cb8e4e93c#sign-in-with-your-credentials).

+

+### General availability - Set manager as reviewer in Azure AD entitlement management access packages

+**Type:** New feature

+**Service category:** User Access Management

+**Product capability:** Entitlement Management

+

+Access packages in Azure AD entitlement management now support setting the user's manager as the reviewer for regularly occurring access reviews. [Learn more](../governance/entitlement-management-access-reviews-create.md).

+### General availability - Enable external users to self-service sign-up in Azure AD using MSA accounts

+**Type:** New feature

+**Service category:** B2B

+**Product capability:** B2B/B2C

+Users can now enable external users to self-service sign-up in Azure Active Directory using Microsoft accounts. [Learn more](../external-identities/microsoft-account.md).

+

+

+### General availability - External Identities Self-Service Sign-Up with Email One-time Passcode

+**Type:** New feature

+**Service category:** B2B

+**Product capability:** B2B/B2C

+

+Now users can enable external users to self-service sign-up in Azure Active Directory using their email and one-time passcode. [Learn more](../external-identities/one-time-passcode.md).

+

+### General availability - Anomalous token

+**Type:** New feature

+**Service category:** Identity Protection

+**Product capability:** Identity Security & Protection

+

+Anomalous token detection is now available in Identity Protection. This feature can detect that there are abnormal characteristics in the token such as time active and authentication from unfamiliar IP address. [Learn more](../identity-protection/concept-identity-protection-risks.md#sign-in-risk).

+

+### General availability - Register or join devices in Conditional Access

+**Type:** New feature

+**Service category:** Conditional Access

+**Product capability:** Identity Security & Protection

+

+The Register or join devices user action in Conditional access is now in general availability. This user action allows you to control multifactor authentication (MFA) policies for Azure AD device registration.

+Currently, this user action only allows you to enable multifactor authentication as a control when users register or join devices to Azure AD. Other controls that are dependent on or not applicable to Azure AD device registration continue to be disabled with this user action. [Learn more](../conditional-access/concept-conditional-access-cloud-apps.md#user-actions).

+### New provisioning connectors in the Azure AD Application Gallery - July 2021

+**Type:** New feature

+**Service category:** App Provisioning

+**Product capability:** 3rd Party Integration

+

+You can now automate creating, updating, and deleting user accounts for these newly integrated apps:

+- [Clebex](../saas-apps/clebex-provisioning-tutorial.md)

+- [Exium](../saas-apps/exium-provisioning-tutorial.md)

+- [SoSafe](../saas-apps/sosafe-provisioning-tutorial.md)

+- [Talentech](../saas-apps/talentech-provisioning-tutorial.md)

+- [Thrive LXP](../saas-apps/thrive-lxp-provisioning-tutorial.md)

+- [Vonage](../saas-apps/vonage-provisioning-tutorial.md)

+- [Zip](../saas-apps/zip-provisioning-tutorial.md)

+- [TimeClock 365](../saas-apps/timeclock-365-provisioning-tutorial.md)

+For more information about how to better secure your organization by using automated user account provisioning, read [Automate user provisioning to SaaS applications with Azure AD](../app-provisioning/user-provisioning.md).

+### Changes to security and Microsoft 365 group settings in Azure portal

+**Type:** Changed feature

+**Service category:** Group Management

+**Product capability:** Directory

+

+In the past, users could create security groups and Microsoft 365 groups in the Azure portal. Now users will have the ability to create groups across Azure portals, PowerShell, and API. Customers are required to verify and update the new settings have been configured for their organization. [Learn More](../enterprise-users/groups-self-service-management.md#group-settings).

+

+### "All Apps" collection has been renamed to "Apps"

+**Type:** Changed feature

+**Service category:** My Apps

+**Product capability:** End User Experiences

+

+In the My Apps portal, the collection that was called "All Apps" has been renamed to be called "Apps". As the product evolves, "Apps" is a more fitting name for this default collection. [Learn more](../manage-apps/my-apps-deployment-plan.md#plan-the-user-experience).

+

+## January 2022

+### Public preview - Custom security attributes

+**Type:** New feature

+**Service category:** Directory Management

+**Product capability:** Directory

+

+Enables you to define business-specific attributes that you can assign to Azure AD objects. These attributes can be used to store information, categorize objects, or enforce fine-grained access control. Custom security attributes can be used with Azure attribute-based access control. [Learn more](custom-security-attributes-overview.md).

+

+### Public preview - Filter groups in tokens using a substring match

+**Type:** New feature

+**Service category:** Enterprise Apps

+**Product capability:** SSO

+

+In the past, Azure AD only permitted groups to be filtered based on whether they were assigned to an application. Now, you can also use Azure AD to filter the groups included in the token. You can filter with the substring match on the display name or onPremisesSAMAccountName attributes of the group object on the token. Only groups that the user is a member of will be included in the token. This token will be recognized whether it's on the ObjectID or the on premises SAMAccountName or security identifier (SID). This feature can be used together with the setting to include only groups assigned to the application if desired to further filter the list.[Learn more](../hybrid/how-to-connect-fed-group-claims.md)

+### General availability - Continuous Access Evaluation

+**Type:** New feature

+**Service category:** Other

+**Product capability:** Access Control

+

+With Continuous access evaluation (CAE), critical security events and policies are evaluated in real time. This includes account disable, password reset, and location change. [Learn more](../conditional-access/concept-continuous-access-evaluation.md).

+

+### General Availability - User management enhancements are now available

+**Type:** New feature

+**Service category:** User Management

+**Product capability:** User Management

+

+The Azure AD portal has been updated to make it easier to find users in the All users and Deleted users pages. Changes in the preview include:

+- More visible user properties including object ID, directory sync status, creation type, and identity issuer.

+- **Search now** allows substring search and combined search of names, emails, and object IDs.

+- Enhanced filtering by user type (member, guest, and none), directory sync status, creation type, company name, and domain name.

+- New sorting capabilities on properties like name, user principal name, creation time, and deletion date.

+- A new total users count that updates with any searches or filters.

+For more information, go to [User management enhancements (preview) in Azure Active Directory](../enterprise-users/users-search-enhanced.md).

+### General Availability - My Apps customization of default Apps view

+**Type:** New feature

+**Service category:** My Apps

+**Product capability:** End User Experiences

+Customization of the default My Apps view in now in general availability. For more information on My Apps, you can go to [Sign in and start apps from the My Apps portal](https://support.microsoft.com/en-us/account-billing/sign-in-and-start-apps-from-the-my-apps-portal-2f3b1bae-0e5a-4a86-a33e-876fbd2a4510).

+

+### General Availability - Audited BitLocker Recovery

+**Type:** New feature

+**Service category:** Device Access Management

+**Product capability:** Device Lifecycle Management

+

+BitLocker keys are sensitive security items. Audited BitLocker recovery ensures that when BitLocker keys are read, an audit log is generated so that you can trace who accesses this information for given devices. [Learn more](../devices/device-management-azure-portal.md#view-or-copy-bitlocker-keys).

+### General Availability - Download a list of devices

+**Type:** New feature

+**Service category:** Device Registration and Management

+**Product capability:** Device Lifecycle Management

+Download a list of your organization's devices to a .csv file for easier reporting and management. [Learn more](../devices/device-management-azure-portal.md#download-devices).

+

+### New provisioning connectors in the Azure AD Application Gallery - January 2022

+**Type:** New feature

+**Service category:** App Provisioning

+**Product capability:** 3rd Party Integration

+

+You can now automate creating, updating, and deleting user accounts for these newly integrated apps:

+- [Autodesk SSO](../saas-apps/autodesk-sso-provisioning-tutorial.md)

+- [Evercate](../saas-apps/evercate-provisioning-tutorial.md)

+- [frankli.io](../saas-apps/frankli-io-provisioning-tutorial.md)

+- [Plandisc](../saas-apps/plandisc-provisioning-tutorial.md)

+- [Swit](../saas-apps/swit-provisioning-tutorial.md)

+- [TerraTrue](../saas-apps/terratrue-provisioning-tutorial.md)

+- [TimeClock 365 SAML](../saas-apps/timeclock-365-saml-provisioning-tutorial.md)

+For more information about how to better secure your organization by using automated user account provisioning, go to [Automate user provisioning to SaaS applications with Azure AD](../manage-apps/user-provisioning.md).

+### New Federated Apps available in Azure AD Application gallery - January 2022

+**Type:** New feature

+**Service category:** Enterprise Apps

+**Product capability:** 3rd Party Integration

+In January 2022, weΓÇÖve added the following 47 new applications in our App gallery with Federation support

+[Jooto](../saas-apps/jooto-tutorial.md), [Proprli](https://app.proprli.com/), [Pace Scheduler](https://www.pacescheduler.com/accounts/login/), [DRTrack](../saas-apps/drtrack-tutorial.md), [Dining Sidekick](../saas-apps/dining-sidekick-tutorial.md), [Cryotos](https://app.cryotos.com/oauth2/authorization/azure-client), [Emergency Management Systems](https://secure.emsystems.com.au/), [Manifestly Checklists](../saas-apps/manifestly-checklists-tutorial.md), [eLearnPOSH](../saas-apps/elearnposh-tutorial.md), [Scuba Analytics](../saas-apps/scuba-analytics-tutorial.md), [Athena Systems Login Platform](../saas-apps/athena-systems-login-platform-tutorial.md), [TimeTrack](../saas-apps/timetrack-tutorial.md), [MiHCM](../saas-apps/mihcm-tutorial.md), [Health Note](https://auth.healthnote.works/oauth), [Active Directory SSO for DoubleYou](../saas-apps/active-directory-sso-for-doubleyou-tutorial.md), [Emplifi platform](../saas-apps/emplifi-platform-tutorial.md), [Flexera One](../saas-apps/flexera-one-tutorial.md), [Hypothesis](https://web.hypothes.is/help/authorizing-hypothesis-from-the-azure-ad-app-gallery/), [Recurly](../saas-apps/recurly-tutorial.md), [XpressDox AU Cloud](https://au.xpressdox.com/Authentication/Login.aspx), [Active and Thriving - Perth Airport](../saas-apps/active-and-thriving-perth-airport-tutorial.md), [Zoom for Intune](https://zoom.us/), [UPWARD AGENT](https://app.upward.jp/login/), [Linux Foundation ID](https://openprofile.dev/), [Asset Planner](../saas-apps/asset-planner-tutorial.md), [Kiho](https://v3.kiho.fi/index/sso), [chezie](https://app.chezie.co/), [Excelity HCM](../saas-apps/excelity-hcm-tutorial.md), [yuccaHR](https://app.yuccahr.com/), [Blue Ocean Brain](../saas-apps/blue-ocean-brain-tutorial.md), [EchoSpan](../saas-apps/echospan-tutorial.md), [Archie](../saas-apps/archie-tutorial.md), [Equifax Workforce Solutions](../saas-apps/equifax-workforce-solutions-tutorial.md), [Palantir Foundry](../saas-apps/palantir-foundry-tutorial.md), [ATP SpotLight and ChronicX](../saas-apps/atp-spotlight-and-chronicx-tutorial.md), [DigiSign](https://app.digisign.org/selfcare/sso), [mConnect](https://mconnect.skooler.com/), [BrightHR](https://login.brighthr.com/), [Mural Identity](../saas-apps/mural-identity-tutorial.md), [NordPass SSO](https://app.nordpass.com/login%20use%20%22Log%20in%20to%20business%22%20option), [CloudClarity](https://portal.cloudclarity.app/dashboard), [Twic](../saas-apps/twic-tutorial.md), [Eduhouse Online](https://app.eduhouse.fi/palvelu/kirjaudu/microsoft), [Bealink](../saas-apps/bealink-tutorial.md), [Time Intelligence Bot](https://teams.microsoft.com/), [SentinelOne](https://sentinelone.com/)

+You can also find the documentation of all the applications from: https://aka.ms/AppsTutorial,

+For listing your application in the Azure AD app gallery, read the details in: https://aka.ms/AzureADAppRequest

+### Azure Ad access reviews reviewer recommendations now account for non-interactive sign-in information

+**Type:** Changed feature

+**Service category:** Access Reviews

+**Product capability:** Identity Governance

+Azure AD access reviews reviewer recommendations now account for non-interactive sign-in information, improving upon original recommendations based on interactive last sign-ins only. Reviewers can now make more accurate decisions based on the last sign-in activity of the users theyΓÇÖre reviewing. To learn more about how to create access reviews, go to [Create an access review of groups and applications in Azure AD](../governance/create-access-review.md).

+

+### Risk reason for offline Azure AD Threat Intelligence risk detection

+**Type:** Changed feature

+**Service category:** Identity Protection

+**Product capability:** Identity Security & Protection

+

+The offline Azure AD Threat Intelligence risk detection can now have a risk reason that will help customers with the risk investigation. If a risk reason is available, it will show up as **Additional Info** in the risk details of that risk event. The information can be found in the Risk detections report. It will also be available through the additionalInfo property of the riskDetections API. [Learn more](../identity-protection/howto-identity-protection-investigate-risk.md).

+

+We previously announced in April 2020, a new combined registration experience enabling users to register authentication methods for SSPR and multi-factor authentication at the same time was generally available for existing customer to opt in. Any Azure AD tenants created after August 2020 automatically have the default experience set to combined registration. Starting in 2022 Microsoft will be enabling the multi-factor authentication and SSPR combined registration experience for existing customers. [Learn more](../authentication/concept-registration-mfa-sspr-combined.md).

+To prevent accidental notification approvals, admins can now require users to enter the number displayed on the sign in screen when approving a multi-factor authentication notification in the Authenticator app. This feature adds an extra security measure to the Microsoft Authenticator app. [Learn more](../authentication/how-to-mfa-number-match.md).

+WeΓÇÖre no longer publishing sign-in logs with the following error codes because these events are pre-authentication events that occur before our service has authenticated a user. Because these events happen before authentication, our service isnΓÇÖt always able to correctly identify the user. If a user continues on to authenticate, the user sign-in will show up in your tenant Sign-in logs. These logs are no longer visible in the Azure portal UX, and querying these error codes in the Graph API will no longer return results.

+|50058| Session information isnΓÇÖt sufficient for single-sign-on.|

+|16000| Either multiple user identities are available for the current request or selected account isnΓÇÖt supported for the scenario.|

+We previously announced in April 2020, a new combined registration experience enabling users to register authentication methods for SSPR and multi-factor authentication at the same time was generally available for existing customer to opt in. Any Azure AD tenants created after August 2020 automatically have the default experience set to combined registration. Starting 2022, Microsoft will be enabling the MF).

+Previously, Conditional Access policies applied only to users when they access apps and services like SharePoint online or the Azure portal. This preview adds support for Conditional Access policies applied to service principals owned by the organization. You can block service principals from accessing resources from outside trusted-named locations or Azure Virtual Networks. [Learn more](../conditional-access/workload-identity.md).

+### Public preview - Extra attributes available as claims

+We have recently added other property to the sign-in logs called "Session Lifetime Policies Applied". This property will list all the session lifetime policies that applied to the sign-in for example, Sign-in frequency, Remember multi-factor authentication and Configurable token lifetime. [Learn more](../reports-monitoring/concept-sign-ins.md#authentication-details).

+The total number of required permissions for any single application registration mustn't exceed 400 permissions, across all APIs. The change to enforce this limit will begin rolling out mid-October 2021. Applications exceeding the limit can't increase the number of permissions theyΓÇÖre configured for. The existing limit on the number of distinct APIs for which permissions are required remains unchanged and may not exceed 50 APIs.

+If there's no trust relation between a home and resource tenant, a guest user would have previously been asked to re-register their device, which would break the previous registration. However, the user would end up in a registration loop because only home tenant device registration is supported. In this specific scenario, instead of this loop, weΓÇÖve created a new conditional access blocking page. The page tells the end user that they can't get access to conditional access protected resources as a guest user. [Learn more](../external-identities/b2b-quickstart-add-guest-users-portal.md#prerequisites).

+The "Register or join devices" user action is generally available in Conditional access. This user action allows you to control multi-factor authentication policies for Azure Active Directory (AD) device registration. Currently, this user action only allows you to enable multi-factor authentication as a control when users register or join devices to Azure AD. Other controls that are dependent on or not applicable to Azure AD device registration continue to be disabled with this user action. [Learn more](../conditional-access/concept-conditional-access-cloud-apps.md#user-actions).

+To help administrators understand that their users are blocked for multi-factor authentication as a result of fraud report, weΓÇÖve added a new audit event. This audit event is tracked when the user reports fraud. The audit log is available in addition to the existing information in the sign-in logs about fraud report. To learn how to get the audit report, see [multi-factor authentication Fraud alert](../authentication/howto-mfa-mfasettings.md#fraud-alert).

+ Title: Automate Azure AD Identity Governance tasks with Azure Automation

+description: Learn how to write PowerShell scripts in Azure Automation to interact with Azure Active Directory entitlement management and other features.

+documentationCenter: ''

+editor:

+ na

+ms.devlang: na

+# Automate Azure AD Identity Governance tasks via Azure Automation and Microsoft Graph

+[Azure Automation](/azure/automation/overview) is an Azure cloud service that allows you to automate common or repetitive systems management and processes. Microsoft Graph is the Microsoft unified API endpoint for Azure AD features that manage users, groups, access packages, access reviews, and other resources in the directory. You can manage Azure AD at scale from the PowerShell command line, using the [Microsoft Graph PowerShell SDK](/graph/powershell/get-started). You can also include the Microsoft Graph PowerShell cmdlets from a [PowerShell-based runbook in Azure Automation](/azure/automation/automation-intro), so that you can automate Azure AD tasks from a simple script.

+Azure Automation and the PowerShell Graph SDK supports certificate-based authentication and application permissions, so you can have Azure Automation runbooks authenticate to Azure AD without needing a user context.

+This article will show you how to get started using Azure Automation for Azure AD Identity Governance, by creating a simple runbook that queries entitlement management via Microsoft Graph PowerShell.

+## Create an Azure Automation account

+Azure Automation provides a cloud-hosted environment for [runbook execution](/azure/automation/automation-runbook-execution). Those runbooks can start automatically based on a schedule, or be triggered by webhooks or by Logic Apps.

+Using Azure Automation requires you to have an Azure subscription.

+**Prerequisite role**: Azure subscription or resource group owner

+1. Sign in to the Azure portal. Make sure you have access to the subscription or resource group where the Azure Automation account will be located.

+1. Select the subscription or resource group, and select **Create**. Type **Automation**, select the **Automation** Azure service from Microsoft, then select **Create**.

+1. After the Azure Automation account has been created, select **Access control (IAM)**. Then select **View** in **View access to this resource**. These users and service principals will subsequently be able to interact with the Microsoft service through the scripts to be created in that Azure Automation account.

+1. Review the users and service principals who are listed there and ensure they are authorized. Remove any users who are unauthorized.

+## Create a self-signed key pair and certificate on your computer

+So that it can operate without needing your personal credentials, the Azure Automation account you created will need to authenticate itself to Azure AD with a certificate.

+If you already have a key pair for authenticating your service to Azure AD, and a certificate that you received from a certificate authority, skip to the next section.

+To generate a self-signed certificate,

+1. Follow the instructions in [how to create a self-signed certificate](../develop/howto-create-self-signed-certificate.md), option 2, to create and export a certificate with its private key.

+1. Display the thumbprint of the certificate.

+ ```powershell

+ $cert | ft Thumbprint

+1. After you have exported the files, you can remove the certificate and key pair from your local user certificate store. In subsequent steps you will remove the `.pfx` and `.crt` files as well, once the certificate and private key have been uploaded to the Azure Automation and Azure AD services.

+## Upload the key pair to Azure Automation

+Your runbook in Azure Automation will retrieve the private key from the `.pfx` file, and use it for authenticating to Microsoft Graph.

+1. In the Azure portal for the Azure Automation account, select **Certificates** and **Add a certificate**.

+1. Upload the `.pfx` file created earlier, and type the password you provided when you created the file.

+1. After the private key is uploaded, record the certificate expiration date.

+1. You can now delete the `.pfx` file from your local computer. However, do not delete the `.crt` file yet, as you will need this file in a subsequent step.

+## Add modules for Microsoft Graph to your Azure Automation account

+By default, Azure Automation does not have any PowerShell modules preloaded for Microsoft Graph. You will need to add **Microsoft.Graph.Authentication**, and then additional modules, from the gallery to your Automation account. Note that you will need to choose whether to use the beta or v1.0 APIs through those modules, as you cannot mix both in a single runbook.

+1. In the Azure portal for the Azure Automation account, select **Modules** and then **Browse gallery**.

+1. In the Search bar, type **Microsoft.Graph.Authentication**. Select the module, select **Import**, and select **OK** to have Azure AD begin importing the module. After clicking OK, importing a module may take several minutes. Don't attempt to add more Microsoft Graph modules until the Microsoft.Graph.Authentication module import has completed, since those other modules have Microsoft.Graph.Authentication as a prerequisite.

+1. Return to the **Modules** list and select **Refresh**. Once the Status of the **Microsoft.Graph.Authentication** module has changed to **Available**, you can import the next module.

+1. If you are using the cmdlets for Azure AD identity governance features, such as entitlement management, then repeat the import process for the module **Microsoft.Graph.Identity.Governance**.

+1. Import other modules that your script may require. For example, if you are using Identity Protection, then you may wish to import the **Microsoft.Graph.Identity.SignIns** module.

+## Create an app registration and assign permissions

+Next, you will create an app registration in Azure AD, so that Azure AD will recognize your Azure Automation runbook's certificate for authentication.

+**Prerequisite role**: Global Administrator or other administrator who can consent applications to application permissions

+1. In the Azure portal, browse to **Azure Active Directory** > **App registrations**.

+1. Select **New registration**.

+1. Type a name for the application and select **Register**.

+1. Once the application registration is created, take note of the **Application (client) ID** and **Directory (tenant) ID** as you will need these items later.

+1. Select **Certificates and Secrets** and **Upload certificate**.

+1. Upload the `.crt` file created earlier.

+1. Select **API permissions** and **Add a permission**.

+1. Select **Microsoft Graph** and **Application permissions**.

+1. Select each of the permissions that your Azure Automation account will require, then select **Add permissions**.

+ * If your runbook is only performing queries for entitlement management, then it can use the **EntitlementManagement.Read.All** permission.

+ * If your runbook is making changes to entitlement management, for example to create assignments, then use the **EntitlementManagement.ReadWrite.All** permission.

+ * For other APIs, ensure that the necessary permission is added. For example, for identity protection, the **IdentityRiskyUser.Read.All** permission should be added.

+10. Select **Grant admin permissions** to give your app those permissions.

+## Create Azure Automation variables

+In this step, you will create in the Azure automation account three variables that the runbook will use to determine how to authenticate to Azure AD.

+1. In the Azure portal, return to the Azure Automation account.

+1. Select **Variables**, and **Add variable**.

+1. Create a variable named **Thumbprint**. Type, as the value of the variable, the certificate thumbprint that was generated earlier.

+1. Create a variable named **ClientId**. Type, as the value of the variable, the client ID for the application registered in Azure AD.

+1. Create a variable named **TenantId**. Type, as the value of the variable, the tenant ID of the directory where the application was registered.

+## Create an Azure Automation PowerShell runbook that can use Graph

+In this step, you will create an initial runbook. You can trigger this runbook to verify the authentication using the certificate created earlier is successful.

+1. Select **Runbooks** and **Create a runbook**.

+1. Type the name of the runbook, select **PowerShell** as the type of runbook to create, and select **Create**.

+1. Once the runbook is created, a text editing pane will appear for you to type in the PowerShell source code of the runbook.

+1. Type the following PowerShell into the text editor.

+```powershell

+Import-Module Microsoft.Graph.Authentication

+$ClientId = Get-AutomationVariable -Name 'ClientId'

+$TenantId = Get-AutomationVariable -Name 'TenantId'

+$Thumbprint = Get-AutomationVariable -Name 'Thumbprint'

+Connect-MgGraph -clientId $ClientId -tenantid $TenantId -certificatethumbprint $Thumbprint

+```

+5. Select **Test pane**, and select **Start**. Wait a few seconds for the Azure Automation processing of your runbook script to complete.

+1. If the run of your runbook is successful, then the message **Welcome to Microsoft Graph!** will appear.

+Now that you have verified that your runbook can authenticate to Microsoft Graph, extend your runbook by adding cmdlets for interacting with Azure AD features.

+## Extend the runbook to use Entitlement Management

+If the app registration for your runbook has the **EntitlementManagement.Read.All** or **EntitlementManagement.ReadWrite.All** permissions, then it can use the entitlement management APIs.

+1. For example, to get a list of Azure AD entitlement management access packages, you can update the above-created runbook, and replace the text with the following PowerShell.

+```powershell

+Import-Module Microsoft.Graph.Authentication

+$ClientId = Get-AutomationVariable -Name 'ClientId'

+$TenantId = Get-AutomationVariable -Name 'TenantId'

+$Thumbprint = Get-AutomationVariable -Name 'Thumbprint'

+$auth = Connect-MgGraph -clientId $ClientId -tenantid $TenantId -certificatethumbprint $Thumbprint

+Select-MgProfile -Name beta

+Import-Module Microsoft.Graph.Identity.Governance

+$ap = Get-MgEntitlementManagementAccessPackage -All -ErrorAction Stop

+$ap | Select-Object -Property Id,DisplayName | ConvertTo-Json

+```

+2. Select **Test pane**, and select **Start**. Wait a few seconds for the Azure Automation processing of your runbook script to complete.

+3. If the run was successful, the output instead of the welcome message will be a JSON array. The JSON array will include the ID and display name of each access package returned from the query.

+## Parse the output of an Azure Automation account in Logic Apps (optional)

+Once your runbook is published, your can create a schedule in Azure Automation, and link your runbook to that schedule to run automatically. Scheduling runbooks from Azure Automation is suitable for runbooks that do not need to interact with other Azure or Office 365 services.

+If you wish to send the output of your runbook to another service, then you may wish to consider using [Azure Logic Apps](/azure/logic-apps/logic-apps-overview) to start your Azure Automation runbook, as Logic Apps can also parse the results.

+1. In Azure Logic Apps, create a Logic App in the Logic Apps Designer starting with **Recurrence**.

+1. Add the operation **Create job** from **Azure Automation**. Authenticate to Azure AD, and select the Subscription, Resource Group, Automation Account created earlier. Select **Wait for Job**.

+1. Add the parameter **Runbook name** and type the name of the runbook to be started.

+1. Select **New step** and add the operation **Get job output**. Select the same Subscription, Resource Group, Automation Account as the previous step, and select the Dynamic value of the **Job ID** from the previous step.

+1. You can then add more operations to the Logic App, such as the [**Parse JSON** action](/azure/logic-apps/logic-apps-perform-data-operations#parse-json-action), that use the **Content** returned when the runbook completes.

+Note that in Azure Automation, a PowerShell runbook can fail to complete if it tries to write a large amount of data to the output stream at once. You can typically work around this issue by having the runbook output just the information needed by the Logic App, such as by using the `Select-Object -Property` cmdlet to exclude unneeded properties.

+## Plan to keep the certificate up to date

+If you created a self-signed certificate following the steps above for authentication, keep in mind that the certificate will have a limited lifetime before it will expire. You will need to regenerate the certificate and upload the new certificate before its expiration date.

+There are two places where you can see the expiration date in the Azure portal.

+* In Azure Automation, the **Certificates** screen displays the expiration date of the certificate.

+* In Azure AD, on the app registration, the **Certificates & secrets** screen displays the expiration date of the certificate used for the Azure Automation account.

+## Next steps

+- [Create an Automation account using the Azure portal](/azure/automation/quickstarts/create-account-portal)

+- [Manage access to resources in Active Directory entitlement management using Microsoft Graph PowerShell](/powershell/microsoftgraph/tutorial-entitlement-management?view=graph-powershell-beta)

+Additionally, it is possible for a single IP address to attempt multiple logins against multiple users. In these cases, the number of attempts per user may be under the threshold for account lockout protection in AD FS. Azure AD Connect Health now provides the ΓÇ£Risky IP reportΓÇ¥ that detects this condition and notifies administrators. The following are the key benefits for this report:

+The Risky IP report workbook is powered from data in the ADFSSignInLogs stream and can quickly visualize and analyze risky IPs. The parameters can be configured and customized for threshold counts. The workbook is also configurable based on queries, and each query can be updated and modified based on the organizationΓÇÖs needs.

+## Accessing the workbook

+To access the workbook:

+1. Sign in to the [Azure portal](https://portal.azure.com).

+2. Navigate to **Azure Active Directory** > **Monitoring** > **Workbooks**.

+3. Select the Risky IP report workbook.

+

+

+ Title: Configure F5 BIG-IP Easy Button for SSO to Oracle EBS

+description: Learn to implement SHA with header-based SSO to Oracle EBS using F5ΓÇÖs BIG-IP Easy Button guided configuration

+# Tutorial: Configure F5ΓÇÖs BIG-IP Easy Button for SSO to Oracle EBS

+In this article, you'll learn to implement Secure Hybrid Access (SHA) with header-based single sign-on (SSO) to Oracle Enterprise Business Suite (EBS) using F5ΓÇÖs BIG-IP Easy Button guided configuration.

+Enabling BIG-IP published services for Azure Active Directory (Azure AD) SSO provides many benefits, including:

+* [Improved Zero Trust governance](https://www.microsoft.com/security/blog/2020/04/02/announcing-microsoft-zero-trust-assessment-tool/) through Azure AD pre-authentication and [Conditional Access](/conditional-access/overview)

+* Full SSO between Azure AD and BIG-IP published services

+* Manage Identities and access from a single control plane, [the Azure portal](https://portal.azure.com/)

+To learn about all the benefits, see the article on [F5 BIG-IP and Azure AD integration](http://f5-aad-integration.md/) and [what is application access and single sign-on with Azure AD](/azure/active-directory/active-directory-appssoaccess-whatis).

+## Scenario description

+For this scenario, use an **Oracle EBS application using HTTP authorization headers** to manage access to protected content.

+Being legacy, the application lacks modern protocols to support a direct integration with Azure AD. The application can be modernized, but it is costly, requires careful planning, and introduces risk of potential downtime. Instead, an F5 BIG-IP Application Delivery Controller is used to bridge the gap between the legacy application and the modern ID control plane, through protocol transitioning.

+Having a BIG-IP in front of the app enables us to overlay the service with Azure AD pre-authentication and header-based SSO, significantly improving the overall security posture of the application.

+## Scenario architecture

+The secure hybrid access solution for this scenario is made up of several components including a multi-tiered Oracle architecture:

+**Oracle EBS Application:** BIG-IP published service to be protected by Azure AD SHA.

+**Azure AD:** Security Assertion Markup Language (SAML) Identity Provider (IdP) responsible for verification of user credentials, Conditional Access (CA), and SSO to the BIG-IP.

+**Oracle Internet Directory (OID):** Hosts the user database. BIG-IP checks via LDAP for authorization attributes.

+**Oracle AccessGate:** Validates authorization attributes through back channel with OID service, before issuing EBS access cookies

+**BIG-IP:** Reverse proxy and SAML service provider (SP) to the application, delegating authentication to the SAML IdP before performing header-based SSO to the Oracle service.

+SHA for this scenario supports both SP and IdP initiated flows. The following image illustrates the SP initiated flow.

+

+| Steps| Description |

+| -- |-|

+| 1| User connects to application endpoint (BIG-IP) |

+| 2| BIG-IP APM access policy redirects user to Azure AD (SAML IdP) |

+| 3| Azure AD pre-authenticates user and applies any enforced Conditional Access policies |

+| 4| User is redirected back to BIG-IP with issued token and claims |

+| 5| BIG-IP authenticates user and performs LDAP query for user Unique ID (UID) attribute |

+| 6| BIG-IP injects returned UID attribute as user_orclguid header in EBS session cookie request to Oracle AccessGate |

+| 7| Oracle AccessGate validates UID against Oracle Internet Directory (OID) service and issues EBS access cookie

+| 8| EBS user headers and cookie sent to application and returns the payload to the user |

+## Prerequisites

+Prior BIG-IP experience isnΓÇÖt necessary, but you need:

+* An Azure AD free subscription or above

+* An existing BIG-IP or [deploy a BIG-IP Virtual Edition (VE) in Azure](./f5-bigip-deployment-guide.md)

+* Any of the following F5 BIG-IP license SKUs

+ * F5 BIG-IP® Best bundle

+ * F5 BIG-IP Access Policy ManagerΓäó (APM) standalone license

+ * F5 BIG-IP Access Policy Manager™ (APM) add-on license on an existing BIG-IP F5 BIG-IP® Local Traffic Manager™ (LTM)

+ * 90-day BIG-IP full feature [trial license](https://www.f5.com/trial/big-ip-trial.php).

+* User identities [synchronized](../hybrid/how-to-connect-sync-whatis.md) from an on-premises directory to Azure AD or created directly within Azure AD and flowed back to your on-premises directory

+* An account with Azure AD application admin [permissions](/azure/active-directory/users-groups-roles/directory-assign-admin-roles#application-administrator)

+* [SSL certificate](./f5-bigip-deployment-guide.md#ssl-profile) for publishing services over HTTPS

+* An existing Oracle EBS suite including Oracle AccessGate and an LDAP enabled OID (Oracle Internet Database)

+## BIG-IP configuration methods

+There are many methods to configure BIG-IP for this scenario, including two template-based options and an advanced configuration. This tutorial covers the latest Guided Configuration 16.1 offering an Easy button template. With the Easy Button, admins no longer go back and forth between Azure AD and a BIG-IP to enable services for SHA. The deployment and policy management is handled directly between the APMΓÇÖs Guided Configuration wizard and Microsoft Graph. This rich integration between BIG-IP APM and Azure AD ensures that applications can quickly, easily support identity federation, SSO, and Azure AD Conditional Access, reducing administrative overhead.

+>[!NOTE]

+> All example strings or values referenced throughout this guide should be replaced with those for your actual environment.

+## Register Easy Button

+Before a client or service can access Microsoft Graph, it must be trusted by the [Microsoft identity platform](../develop/quickstart-register-app.md).

+A BIG-IP must also be registered as a client in Azure AD, before it is allowed to establish a trust in between each SAML SP instance of a BIG-IP published application, and Azure AD as the SAML IdP.

+1. Sign in to the [Azure AD portal](https://portal.azure.com/) with Application Administrative rights

+2. From the left navigation pane, select the **Azure Active Directory** service

+3. Under Manage, select **App registrations > New registration**

+4. Enter a display name for your application. For example, F5 BIG-IP Easy Button

+5. Specify who can use the application > **Accounts in this organizational directory only**

+6. Select **Register** to complete the initial app registration

+7. Navigate to **API permissions** and authorize the following Microsoft Graph permissions:

+ * Application.Read.All

+ * Application.ReadWrite.All

+ * Application.ReadWrite.OwnedBy

+ * Directory.Read.All

+ * Group.Read.All

+ * IdentityRiskyUser.Read.All

+ * Policy.Read.All

+ * Policy.ReadWrite.ApplicationConfiguration

+ * Policy.ReadWrite.ConditionalAccess

+ * User.Read.All

+8. Grant admin consent for your organization

+9. Go to **Certificates & Secrets**, generate a new **Client secret** and note it down

+10. Go to **Overview**, note the **Client ID** and **Tenant ID**

+## Configure Easy Button

+Initiate **Easy Button** configuration to set up a SAML Service Provider (SP) and Azure AD as an Identity Provider (IdP) for your application.

+1. Navigate to **Access > Guided Configuration > Microsoft Integration** and select **Azure AD Application**.

+ 

+2. Review the list of configuration steps and select **Next**

+ 

+3. Follow the sequence of steps required to publish your application.

+ 

+### Configuration Properties

+The **Configuration Properties** tab creates up a new application config and SSO object. Consider **Azure Service Account Details** section to be the client application you registered in your Azure AD tenant earlier. These settings allow a BIG-IP to programmatically register a SAML application directly in your tenant, along with the properties you would normally configure manually. Easy Button does this for every BIG-IP APM service being enabled for SHA.

+Some of these are global settings so can be re-used for publishing more applications, further reducing deployment time and effort.

+1. Provide a unique **Configuration Name** that enables an admin to easily distinguish between Easy Button configurations

+2. Enable **Single Sign-On (SSO) & HTTP Headers**

+3. Enter the **Tenant Id, Client ID**, and **Client Secret** you noted down from your registered application

+4. Before you select **Next**, confirm that BIG-IP can successfully connect to your tenant.

+ 

+### Service Provider

+The **Service Provider** settings define the SAML SP properties for the APM instance representing the application protected through SHA.

+1. Enter **Host**. This is the public FQDN of the application being secured. You need a corresponding DNS record for clients to resolve this address, but using a localhost record is fine during testing

+2. Enter **Entity ID**. This is the identifier Azure AD will use to identify the SAML SP requesting a token

+ 

+ Next, under optional **Security Settings** specify whether Azure AD should encrypt issued SAML assertions. Encrypting assertions between Azure AD and the BIG-IP APM provides assurance that the content tokens canΓÇÖt be intercepted, and personal or corporate data be compromised.

+3. From the **Assertion Decryption Private Key** list, select **Create New**

+ 

+4. Select **OK**. This opens the **Import SSL Certificate and Keys** dialog in a new tab

+5. Select **PKCS 12 (IIS)** to import your certificate and private key. Once provisioned close the browser tab to return to the main tab.

+ 

+6. Check **Enable Encrypted Assertion**

+7. If you have enabled encryption, select your certificate from the **Assertion Decryption Private Key** list. This is the private key for the certificate that BIG-IP APM uses to decrypt Azure AD assertions

+8. If you have enabled encryption, select your certificate from the **Assertion Decryption Certificate** list. This is the certificate that BIG-IP uploads to Azure AD for encrypting the issued SAML assertions.

+ 

+### Azure Active Directory

+This section defines all properties that you would normally use to manually configure a new BIG-IP SAML application within your Azure AD tenant. The Easy Button wizard provides a set of pre-defined application templates for Oracle PeopleSoft, Oracle E-business Suite, Oracle JD Edwards, SAP ERP as well as generic SHA template for any other apps. In this example, select **Oracle E-Business Suite > Add**. This adds the template for the Oracle E-business Suite

+

+#### Azure Configuration

+1. Enter **Display Name** of app that the BIG-IP creates in your Azure AD tenant, and the icon that the users see on [MyApps portal](https://myapplications.microsoft.com/)

+2. In the **Sign On URL (optional)** enter the public FQDN of the EBS application being secured, along with the default path for the Oracle EBS homepage

+ 

+3. Select the refresh icon next to the **Signing Key** and **Signing Certificate** to locate the certificate you imported earlier

+4. Enter the certificateΓÇÖs password in **Signing Key Passphrase**

+5. Enable **Signing Option** (optional). This ensures that BIG-IP only accepts tokens and claims that are signed by Azure AD

+ 

+6. **User and User Groups** are used to authorize access to the application. They are dynamically added from the tenant. **Add** a user or group that you can use later for testing, otherwise all access will be denied

+ 

+#### User Attributes & Claims

+When a user successfully authenticates, Azure AD issues a SAML token with a default set of claims and attributes uniquely identifying the user. The **User Attributes & Claims** tab shows the default claims to issue for the new application. It also lets you configure more claims.

+

+You can include additional Azure AD attributes if necessary, but the example PeopleSoft scenario only requires the default attributes.

+#### Additional User Attributes

+The **Additional User Attributes** tab can support a variety of distributed systems requiring attributes stored in other directories for session augmentation. Attributes fetched from an LDAP source can then be injected as additional SSO headers to further control access based on roles, Partner IDs, etc.

+1. Enable the **Advanced Settings** option

+2. Check the **LDAP Attributes** check box

+3. Select **Create New** in **Choose Authentication Server**

+4. Select **Use pool** or **Direct** server connection mode depending on your setup. This provides the **Server Address** of the target LDAP service. If using a single LDAP server, select **Direct**.

+5. Enter **Service Port** as 3060 (Default), 3161 (Secure), or any other port your Oracle LDAP service operates on

+6. Enter the **Base Search DN** (distinguished name) from which to search. This search DN is used to search groups across a whole directory.

+7. Set the **Admin DN** to the exact distinguished name for the account the APM will use to authenticate for LDAP queries, along with its password

+ 

+8. Leave all default **LDAP Schema Attributes**

+ 

+9. Under **LDAP Query Properties**, set the **Search Dn** to the base node of the LDAP server from which to search for user objects

+10. Add the name of the user object attribute that must be returned from the LDAP directory. For EBS, the default is **orclguid**

+ 

+#### Conditional Access Policy

+Conditional Access policies are enforced post Azure AD pre-authentication, to control access based on device, application, location, and risk signals.

+The **Available Policies** view, by default, will list all Conditional Access policies that do not include user-based actions.

+The **Selected Policies** view, by default, displays all policies targeting All cloud apps. These policies cannot be deselected or moved to the Available Policies list as they are enforced at a tenant level.

+To select a policy to be applied to the application being published:

+1. Select the desired policy in the **Available Policies** list

+2. Select the right arrow and move it to the **Selected Policies** list

+ The selected policies should either have an **Include** or **Exclude** option checked. If both options are checked, the policy is not enforced.

+ 

+> [!NOTE]

+> The policy list is enumerated only once when first switching to this tab. A refresh button is available to manually force the wizard to query your tenant, but this button is displayed only when the application has been deployed.

+### Virtual Server Properties

+A virtual server is a BIG-IP data plane object represented by a virtual IP address listening for client requests to the application. Any received traffic is processed and evaluated against the APM profile associated with the virtual server, before being directed according to the policy results and settings.

+1. Enter **Destination Address**. This is any available IPv4/IPv6 address that the BIG-IP can use to receive client traffic. A corresponding record should also exist in DNS, enabling clients to resolve the external URL of your BIG-IP published application to this IP.

+2. Enter **Service Port** as *443* for HTTPS

+3. Check **Enable Redirect Port** and then enter **Redirect Port**. It redirects incoming HTTP client traffic to HTTPS

+4. Select **Client SSL Profile** to enable the virtual server for HTTPS so that client connections are encrypted over TLS. Select the client SSL profile you created as part of the prerequisites or leave the default if testing

+ 

+### Pool Properties

+The **Application Pool tab** details the services behind a BIG-IP, represented as a pool containing one or more application servers.

+1. Choose from **Select a Pool**. Create a new pool or select an existing one

+2. Choose the **Load Balancing Method** as *Round Robin*

+3. Update the **Pool Servers**. Select an existing node or specify an IP and port for the servers hosting the Oracle EBS application.

+ 

+4. The **Access Gate Pool** specifies the servers Oracle EBS uses for mapping an SSO authenticated user to an Oracle E-Business Suite session. Update **Pool Servers** with the IP and port for of the Oracle application servers hosting the application

+ 

+#### Single Sign-On & HTTP Headers

+The **Easy Button wizard** supports Kerberos, OAuth Bearer, and HTTP authorization headers for SSO to published applications. As the PeopleSoft application expects headers, enable **HTTP Headers** and enter the following properties.

+* **Header Operation:** replace

+* **Header Name:** USER_NAME

+* **Header Value:** %{session.sso.token.last.username}

+

+* **Header Operation:** replace

+* **Header Name:** USER_ORCLGUID

+* **Header Value:** %{session.ldap.last.attr.orclguid}

+ 

+>[!NOTE]

+>APM session variables defined within curly brackets are CASE sensitive. If you enter OrclGUID when the Azure AD attribute name is being defined as orclguid, it will cause an attribute mapping failure.

+### Session Management

+The BIG-IPs session management settings are used to define the conditions under which user sessions are terminated or allowed to continue, limits for users and IP addresses, and corresponding user info. Consult [F5 documentation](https://support.f5.com/csp/article/K18390492) for details on these settings.

+What isnΓÇÖt covered however is Single Log-Out (SLO) functionality, which ensures all sessions between the IdP, the BIG-IP, and the user agent are terminated as users sign off. When the Easy Button deploys a SAML application to your Azure AD tenant, it also populates the Logout Url with the APMΓÇÖs SLO endpoint. That way IdP initiated sign-outs from the Azure AD MyApps portal also terminate the session between the BIG-IP and a client.

+During deployment, the SAML federation metadata for the published application is imported from your tenant, providing the APM the SAML logout endpoint for Azure AD. This helps SP initiated sign outs terminate the session between a client and Azure AD.

+## Summary

+Select **Deploy** to commit all settings and verify that the application has appeared in your tenant. This last step provides breakdown of all applied settings before theyΓÇÖre committed. Your application should now be published and accessible via SHA, either directly via its URL or through MicrosoftΓÇÖs application portals.

+## Next steps

+From a browser, connect to the **PeopleSoft applicationΓÇÖs external URL** or select the applicationΓÇÖs icon in the [Microsoft MyApps portal](https://myapps.microsoft.com/). After authenticating to Azure AD, youΓÇÖll be redirected to the BIG-IP virtual server for the application and automatically signed in through SSO.

+For increased security, organizations using this pattern could also consider blocking all direct access to the application, thereby forcing a strict path through the BIG-IP.

+## Advanced deployment

+There may be cases where the Guided Configuration templates lack the flexibility to achieve more specific requirements. For those scenarios, see [Advanced Configuration for kerberos-based SSO](./f5-big-ip-kerberos-advanced.md). Alternatively, the BIG-IP gives the option to disable **Guided ConfigurationΓÇÖs strict management mode**. This allows you to manually tweak your configurations, even though bulk of your configurations are automated through the wizard-based templates.

+You can navigate to **Access > Guided Configuration** and select the **small padlock icon** on the far right of the row for your applicationsΓÇÖ configs.

+

+At that point, changes via the wizard UI are no longer possible, but all BIG-IP objects associated with the published instance of the application will be unlocked for direct management.

+> [!NOTE]

+> Re-enabling strict mode and deploying a configuration will overwrite any settings performed outside of the Guided Configuration UI, therefore we recommend the advanced configuration method for production services.

+## Troubleshooting

+There can be many factors leading to failure to access a published application. BIG-IP logging can help quickly isolate all sorts of issues with connectivity, policy violations, or misconfigured variable mappings.

+Start troubleshooting by increasing the log verbosity level.

+1. Navigate to **Access Policy > Overview > Event Logs > Settings**

+2. Select the row for your published application then **Edit > Access System Logs**

+3. Select **Debug** from the SSO list then **OK**

+Reproduce your issue, then inspect the logs, but remember to switch this back when finished as verbose mode generates lots of data. If you see a BIG-IP branded error immediately after successful Azure AD pre-authentication, itΓÇÖs possible the issue relates to SSO from Azure AD to the BIG-IP.

+1. Navigate to **Access > Overview > Access reports**

+2. Run the report for the last hour to see logs provide any clues. The **View session** variables link for your session will also help understand if the APM is receiving the expected claims from Azure AD

+If you donΓÇÖt see a BIG-IP error page, then the issue is probably more related to the backend request or SSO from the BIG-IP to the application.

+1. In which case you should head to **Access Policy > Overview > Active Sessions** and select the link for your active session

+2. The **View Variables** link in this location may also help root cause SSO issues, particularly if the BIG-IP APM fails to obtain the right attributes

+See [BIG-IP APM variable assign examples](https://devcentral.f5.com/s/articles/apm-variable-assign-examples-1107) and [F5 BIG-IP session variables reference](https://techdocs.f5.com/en-us/bigip-15-0-0/big-ip-access-policy-manager-visual-policy-editor/session-variables.html) for more info.

+The following command from a bash shell validates the APM service account used for LDAP queries and can successfully authenticate and query a user object:

+```ldapsearch -xLLL -H 'ldap://192.168.0.58' -b "CN=oraclef5,dc=contoso,dc=lds" -s sub -D "CN=f5-apm,CN=partners,DC=contoso,DC=lds" -w 'P@55w0rd!' "(cn=testuser)" ```

+For more information, visit this F5 knowledge article [Configuring LDAP remote authentication for Active Directory](https://support.f5.com/csp/article/K11072). ThereΓÇÖs also a great BIG-IP reference table to help diagnose LDAP-related issues in this [F5 knowledge article on LDAP Query](https://techdocs.f5.com/en-us/bigip-16-1-0/big-ip-access-policy-manager-authentication-methods/ldap-query.html).

+## Deactivate a role assignment

+When a role assignment is activated, you'll see a **Deactivate** option in the PIM portal for the role assignment. When you select **Deactivate**, there's a short time lag before the role is deactivated. Also, you can't deactivate a role assignment within five minutes after activation.

+When a role is assigned, the assignment:

+- Can't be assigned for a duration of less than five minutes

+- Can't be removed within five minutes of it being assigned

+ Title: Activate Azure AD roles in PIM - Azure Active Directory | Microsoft Docs

+# Activate an Azure AD role in PIM

+1. After multifactor authentication, select **Activate before proceeding**.

+To save space we're showing only the response for one role, but all eligible role assignments that you can activate will be listed.

+If you don't require activation of a role that requires approval, you can cancel a pending request at any time.

+ When you select Cancel, the request will be canceled. To activate the role again, you'll have to submit a new request for activation.

+## Deactivate a role assignment

+When a role assignment is activated, you'll see a **Deactivate** option in the PIM portal for the role assignment. When you select **Deactivate**, there's a short time lag before the role is deactivated. Also, you can't deactivate a role assignment within five minutes after activation.

+When you activate a role in Privileged Identity Management, the activation might not instantly propagate to all portals that require the privileged role. Sometimes, even if the change is propagated, web caching in a portal may cause a delay before the change takes effect. If your activation is delayed, sign out of the portal you're trying to perform the action and then sign back in. In the Azure portal, PIM signs you out and back in automatically.

+>[!Note]

+>When a role is assigned, the assignment:

+>- Can't be asigned for a duration of less than five minutes

+>- Can't be removed within five minutes of it being assigned

+## Deactivate a role assignment

+When a role assignment is activated, you'll see a **Deactivate** option in the PIM portal for the role assignment. When you select **Deactivate**, there's a short time lag before the role is deactivated. Also, you can't deactivate a role assignment within five minutes after activation.

+>[!Note]

+>When a role is assigned, the assignment:

+>- Can't be assign for a duration of less than five minutes

+>- Can't be removed within five minutes of it being assigned

+|InternalServerError |The target app returned an unexpected error. A service issue with the target application might be preventing this from working. This attempt will automatically be retried in 40 minutes.|

+|WebExceptionProtocolError |An HTTP protocol error occurred in connecting to the target application. There is nothing to do. This attempt will automatically be retried in 40 minutes.|

+* [Graph API for provisioning logs](/graph/api/resources/provisioningobjectsummary)

+The last sign-in date is associated with the user object. The value is retained until the next sign-in of the user.

+> This Conerstone OnDemand automatic provisioning service is deprecated and support will end soon.

+ Title: 'Tutorial: Azure AD SSO integration with Kronos Workforce Dimensions'

+# Tutorial: Azure AD SSO integration with Kronos Workforce Dimensions

+> [!NOTE]

+> This integration is also available to use from Azure AD US Government Cloud environment. You can find this application in the Azure AD US Government Cloud Application Gallery and configure it in the same way as you do from public cloud.

+Once you configure Kronos Workforce Dimensions you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Defender for Cloud Apps](/cloud-app-security/proxy-deployment-aad).

+ Title: 'Tutorial: Azure AD SSO integration with Lucid (All Products)'

+# Tutorial: Azure AD SSO integration with Lucid (All Products)

+> [!NOTE]

+> This integration is also available to use from Azure AD US Government Cloud environment. You can find this application in the Azure AD US Government Cloud Application Gallery and configure it in the same way as you do from public cloud.

+* Lucid (All Products) supports **SP and IDP** initiated SSO.

+* Lucid (All Products) supports **Just In Time** user provisioning.

+## Add Lucid (All Products) from the gallery

+1. On the **Set up single sign-on with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+1. On the **Basic SAML Configuration** section, if you wish to configure the application in **IDP** initiated mode, perform the following step:

+* Click on **Test this application** in Azure portal. This will redirect to the Lucid (All Products) sign-on URL where you can initiate the login flow.

+* Go to Lucid (All Products) Sign-on URL directly and initiate the login flow from there.

+* Click on **Test this application** in Azure portal and you should be automatically signed in to the Lucid (All Products) for which you set up the SSO.

+You can also use Microsoft My Apps to test the application in any mode. When you click the Lucid (All Products) tile in the My Apps, if configured in SP mode you would be redirected to the application sign-on page for initiating the login flow and if configured in IDP mode, you should be automatically signed in to the Lucid (All Products) for which you set up the SSO. For more information about the My Apps, see [Introduction to the My Apps](https://support.microsoft.com/account-billing/sign-in-and-start-apps-from-the-my-apps-portal-2f3b1bae-0e5a-4a86-a33e-876fbd2a4510).

+Once you configure Lucid (All Products) you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Defender for Cloud Apps](/cloud-app-security/proxy-deployment-any-app).

+ Title: 'Tutorial: Azure AD SSO integration with monday.com'

+# Tutorial: Azure AD SSO integration with monday.com

+> [!NOTE]

+> This integration is also available to use from Azure AD US Government Cloud environment. You can find this application in the Azure AD US Government Cloud Application Gallery and configure it in the same way as you do from public cloud.

+* monday.com supports **SP and IDP** initiated SSO.

+* monday.com supports **Just In Time** user provisioning.

+ > If the **Identifier** and **Reply URL** values do not get populated automatically, then fill in the values manually. The **Identifier** and the **Reply URL** are the same and value is in the following pattern: `https://<YOUR_DOMAIN>.monday.com/saml/saml_callback`

+1. If you want to set up monday.com manually, open a new web browser window and sign in to monday.com as an administrator and perform the following steps:

+1. Go to the **Profile** on the top-right corner of page and click on **Admin**.

+ 

+ 

+ 

+ > For more details refer [this](https://support.monday.com/hc/articles/360000460605-SAML-Single-Sign-on?abcb=34642) article.

+Once you configure monday.com you can enforce session control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session control extends from Conditional Access. [Learn how to enforce session control with Microsoft Defender for Cloud Apps](/cloud-app-security/proxy-deployment-any-app).

+ Title: 'Tutorial: Azure AD SSO integration with Oracle Cloud Infrastructure Console'

+# Tutorial: Azure AD SSO integration with Oracle Cloud Infrastructure Console

+> [!NOTE]

+> This integration is also available to use from Azure AD US Government Cloud environment. You can find this application in the Azure AD US Government Cloud Application Gallery and configure it in the same way as you do from public cloud.

+## Add Oracle Cloud Infrastructure Console from the gallery

+ 1. **[Create an Azure AD test user](#create-an-azure-ad-test-user)** to test Azure AD single sign-on with B. Simon.

+ 1. **[Assign the Azure AD test user](#assign-the-azure-ad-test-user)** to enable B. Simon to use Azure AD single sign-on.

+1. **[Configure Oracle Cloud Infrastructure Console SSO](#configure-oracle-cloud-infrastructure-console-sso)** to configure the SSO settings on application side.

+ 1. **[Create Oracle Cloud Infrastructure Console test user](#create-oracle-cloud-infrastructure-console-test-user)** to have a counterpart of B. Simon in Oracle Cloud Infrastructure Console that is linked to the Azure AD representation of user.

+## Configure Azure AD SSO

+1. On the **Set up Single Sign-On with SAML** page, click the pencil icon for **Basic SAML Configuration** to edit the settings.

+1. On the **Basic SAML Configuration** section, perform the following steps:

+ > If the **Identifier** and **Reply URL** values do not get auto populated, then fill in the values manually according to your requirement.

+ 

+ 

+ 

+## Configure Oracle Cloud Infrastructure Console SSO

+ 

+ 

+ 

+ 

+## Test SSO

+When you select the Oracle Cloud Infrastructure Console tile in the My Apps, you will be redirected to the Oracle Cloud Infrastructure Console sign-in page. Select the **IDENTITY PROVIDER** from the drop-down menu and click **Continue** as shown below to sign in. For more information about the My Apps, see [Introduction to the My Apps](https://support.microsoft.com/account-billing/sign-in-and-start-apps-from-the-my-apps-portal-2f3b1bae-0e5a-4a86-a33e-876fbd2a4510).

+

+Once you configure the Oracle Cloud Infrastructure Console you can enforce session controls, which protect exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session controls extend from Conditional Access. [Learn how to enforce session control with Microsoft Defender for Cloud Apps](/cloud-app-security/proxy-deployment-aad).

+See a [video walkthrough](https://www.youtube.com/watch?v=8jqjHjQo-3c) of setting up the Azure AD Verifiable Credential service, including all prerequisites, like Azure AD and an Azure subscription.

+- Integrates with existing ingress solutions such as the [Azure Gateway Ingress Controller][agic], [NGINX][nginx], and [Contour][contour]. For more details on how ingress works with OSM, see [Using Ingress to manage external access to services within the cluster][osm-ingress]. For an example on integrating OSM with Contour for ingress, see [Ingress with Contour][osm-contour]. For an example on integrating OSM with ingress controllers that use the `networking.k8s.io/v1` API, such as NGINX, see [Ingress with Kubernetes Nginx Ingress Controller][osm-nginx].

+## Next steps

+After enabling the OSM add-on using the [Azure CLI][osm-azure-cli] or a [Bicep template][osm-bicep], you can:

+* [Deploy a sample application][osm-deploy-sample-app]

+* [Onboard an existing application][osm-onboard-app]

+[ip-tables-redirection]: https://release-v1-0.docs.openservicemesh.io/docs/guides/traffic_management/iptables_redirection/

+[global-exclusion]: https://release-v1-0.docs.openservicemesh.io/docs/guides/traffic_management/iptables_redirection/#global-outbound-ip-range-exclusions

+[osm-deploy-sample-app]: https://release-v1-0.docs.openservicemesh.io/docs/getting_started/install_apps/

+[osm-onboard-app]: https://release-v1-0.docs.openservicemesh.io/docs/guides/app_onboarding/

+[global-exclusion]: https://docs.openservicemesh.io/docs/guides/traffic_management/iptables_redirection/#global-outbound-ip-range-exclusions

+[agic]: ../application-gateway/ingress-controller-overview.md

+[nginx]: https://github.com/kubernetes/ingress-nginx

+[contour]: https://projectcontour.io/

+[osm-ingress]: https://release-v1-0.docs.openservicemesh.io/docs/guides/traffic_management/ingress/

+[osm-contour]: https://release-v1-0.docs.openservicemesh.io/docs/demos/ingress_contour

+[osm-nginx]: https://release-v1-0.docs.openservicemesh.io/docs/demos/ingress_k8s_nginx

+This article showed you how to install the OSM add-on on an AKS cluster and verify it is installed an running. With the the OSM add-on on your cluster you can [Deploy a sample application][osm-deploy-sample-app] or [Onboard an existing application][osm-onboard-app] to work with your OSM mesh.

+[smi]: https://smi-spec.io/

+[osm-deploy-sample-app]: https://release-v1-0.docs.openservicemesh.io/docs/getting_started/install_apps/

+[osm-onboard-app]: https://release-v1-0.docs.openservicemesh.io/docs/guides/app_onboarding/

+## Next steps

+This article showed you how to install the OSM add-on on an AKS cluster and verify it is installed an running. With the the OSM add-on on your cluster you can [Deploy a sample application][osm-deploy-sample-app] or [Onboard an existing application][osm-onboard-app] to work with your OSM mesh.

+[osm-deploy-sample-app]: https://release-v1-0.docs.openservicemesh.io/docs/getting_started/install_apps/

+[osm-onboard-app]: https://release-v1-0.docs.openservicemesh.io/docs/guides/app_onboarding/

+* **Managed Mode**: This mode offers only NMI. When installed via the AKS cluster add-on, Azure manages creation of Kubernetes primitives (AzureIdentity and AzureIdentityBinding) and identity assignment in response to CLI commands by the user. Otherwise, if installed via Helm chart, the identity needs to be manually assigned and managed by the user. For more information, see [Pod identity in managed mode](https://azure.github.io/aad-pod-identity/docs/configure/pod_identity_in_managed_mode/).

+> Running aad-pod-identity in a cluster with Kubenet is not a recommended configuration due to security concerns. Default Kubenet configuration fails to prevent ARP spoofing, which could be utilized by a pod to act as another pod and gain access to an identity it's not intended to have. Please follow the mitigation steps and configure policies before enabling aad-pod-identity in a cluster with Kubenet.

+Create an identity which will be used by the demo pod with [az identity create][az-identity-create] and set the *IDENTITY_CLIENT_ID* and *IDENTITY_RESOURCE_ID* variables.

+The managed identity that will be assigned to the pod needs to be granted permissions that align with the actions it will be taking.

+Azure will create an AzureIdentity resource in your cluster representing the identity in Azure, and an AzureIdentityBinding resource which connects the AzureIdentity to a selector. You can view these resources with

+```azurecli-interactive

+kubectl get azureidentity -n $POD_IDENTITY_NAMESPACE

+kubectl get azureidentitybinding -n $POD_IDENTITY_NAMESPACE

+```

+For a pod to use AAD pod-managed identity, the pod needs an *aadpodidbinding* label with a value that matches a selector from a *AzureIdentityBinding*. By default, the selector will match the name of the pod identity, but it can also be set using the `--binding-selector` option when calling `az aks pod-identity add`.

+To run a sample application using AAD pod-managed identity, create a `demo.yaml` file with the following contents. Replace *POD_IDENTITY_NAME*, *IDENTITY_CLIENT_ID*, and *IDENTITY_RESOURCE_GROUP* with the values from the previous steps. Replace *SUBSCRIPTION_ID* with your subscription ID.

+The On-premises data gateway provides secure data transfer between on-premises data sources and your Azure Analysis Services servers in the cloud. In addition to working with multiple Azure Analysis Services servers in the same region, the gateway also works with Azure Logic Apps, Power BI, Power Apps, and Power Automate. While the gateway you install is the same across all of these services, Azure Analysis Services and Logic Apps have some additional steps required for successful installation.

+Information provided here is specific to how Azure Analysis Services works with the On-premises data gateway. To learn more about the gateway in general and how it works with other services, see [What is an On-premises data gateway?](/data-integration/gateway/service-gateway-onprem).

+- **Connect the gateway resource to servers** - Once you have a gateway resource, you can begin connecting your servers to it. You can connect multiple servers and other resources provided they are in the same region.

+When installing for an Azure Analysis Services environment, it's important you follow the steps described in [Install and configure on-premises data gateway for Azure Analysis Services](analysis-services-gateway-install.md). This article is specific to Azure Analysis Services. It includes additional steps required to setup an On-premises data gateway resource in Azure, and connect your Azure Analysis Services server to the gateway resource.

+All three client libraries support both Azure AD interactive flow, and non-interactive authentication methods. The two non-interactive methods, Active Directory Password and Active Directory Integrated Authentication methods can be used in applications utilizing AMOMD and MSOLAP. These two methods never result in pop-up dialog boxes for sign in.

+Client applications like Excel and Power BI Desktop, and tools like SSMS and Analysis Services projects extension for Visual Studio install the latest versions of the client libraries with regular updates. Power BI Desktop, SSMS, and Analysis Services projects extension are updated monthly. Excel is [updated with Microsoft 365](https://support.microsoft.com/office/when-do-i-get-the-newest-features-for-microsoft-365-da36192c-58b9-4bc9-8d51-bb6eed468516). Microsoft 365 updates are less frequent, and some organizations use the deferred channel, meaning updates are deferred up to three months.

+Depending on the client application or tools you use, the type of authentication and how you sign in may be different. Each application may support different features for connecting to cloud services like Azure Analysis Services.

+Power BI Desktop, Visual Studio, and SSMS support Active Directory Universal Authentication, an interactive method that also supports Azure AD Multi-Factor Authentication (MFA). Azure AD MFA helps safeguard access to data and applications while providing a simple sign in process. It delivers strong authentication with several verification options (phone call, text message, smart cards with pin, or mobile app notification). Interactive MFA with Azure AD can result in a pop-up dialog box for validation. **Universal Authentication is recommended**.

+**Server administrators** are specific to an Azure Analysis Services server instance. They connect with tools like Azure portal, SSMS, and Visual Studio to perform tasks like configuring settings and managing user roles. By default, the user that creates the server is automatically added as an Analysis Services server administrator. Other administrators can be added by using Azure portal or SSMS. Server administrators must have an account in the Azure AD tenant in the same subscription. To learn more, see [Manage server administrators](analysis-services-server-admins.md).

+Once you've created an Analysis Services server resource in Azure, there may be some administration and management tasks you need to perform right away or sometime down the road. For example, run processing to the refresh data, control who can access the models on your server, or monitor your server's health. Some management tasks can only be performed in Azure portal, others in SQL Server Management Studio (SSMS), and some tasks can be done in either.

+## External open source tools

+**Tabular Editor** - An open-source tool for creating, maintaining, and managing tabular models using an intuitive, lightweight editor. A hierarchical view shows all objects in your tabular model. Objects are organized by display folders with support for multi-select property editing and DAX syntax highlighting. XMLA read-only is required for query operations. Read-write is required for metadata operations. To learn more, see [tabulareditor.github.io](https://tabulareditor.github.io/).

+**ALM Toolkit** - An open-source schema compare tool for Analysis Services tabular models and Power BI datasets, most often used for application lifecycle management (ALM) scenarios. Perform deployment across environments and retain incremental refresh historical data. Diff and merge metadata files, branches and repos. Reuse common definitions between datasets. Read-only is required for query operations. Read-write is required for metadata operations. To learn more, seeΓÇ»[alm-toolkit.com](http://alm-toolkit.com/).

+**DAX Studio** – An open-source tool for DAX authoring, diagnosis, performance tuning, and analysis. Features include object browsing, integrated tracing, query execution breakdowns with detailed statistics, DAX syntax highlighting and formatting. XMLA read-only is required for query operations. To learn more, see [daxstudio.org](https://daxstudio.org/).

+If you've deployed a model to your server, you're ready to connect to it using a client application or tool. To learn more, see [Get data from Azure Analysis Services server](analysis-services-connect.md).

+Service principals are an Azure Active Directory application resource you create within your tenant to perform unattended resource and service level operations. They're a unique type of *user identity* with an application ID and password or certificate. A service principal has only those permissions necessary to perform tasks defined by the roles and permissions for which it is assigned.

+If your data sources are accessed through a VNet, your Azure Analysis Services server must connect to those data sources as if they are on-premises, in your own environment. You must configure the **AlwaysUseGateway** server property to specify the server resource to access all data sources through an [On-premises data gateway](analysis-services-gateway.md).

+Azure SQL Managed Instance data sources run within Azure VNet with a private IP address. If public endpoint is enabled on the instance, a gateway is not required. If public endpoint is not enabled, an On-premises data gateway is required and the AlwaysUseGateway property must be set to true.

+> This property is effective only when an [On-premises data gateway](analysis-services-gateway.md) is installed and configured. The gateway can be on the VNet.

+In this tutorial, you sign in to the portal to get the server name only. Typically, users would get the server name from the server administrator.

+In order to connect to your server from Power BI Desktop, you first need the server name.

+In this tutorial, you learned how to use Power BI Desktop to connect to a data model on a server and create a basic report. If you're not familiar with how to create a data model, see the [Adventure Works Internet Sales tabular data modeling tutorial](/analysis-services/tutorial-tabular-1400/as-adventure-works-tutorial) in the SQL Server Analysis Services docs.

+az apim api versionset list --resource-group apim-hello-world-resource-group \

+az apim api versionset show --resource-group apim-hello-world-resource-group \

+ "/path2",

+ "/path3/subpath/*"

+ Title: Use the migration feature to migrate App Service Environment v2 to App Service Environment v3

+description: Learn how to migrate your App Service Environment v2 to App Service Environment v3 using the migration feature

+# Use the migration feature to migrate App Service Environment v2 to App Service Environment v3

+An App Service Environment v2 can be automatically migrated to an [App Service Environment v3](overview.md) using the migration feature. To learn more about the migration process and to see if your App Service Environment supports migration at this time, see the [Migration to App Service Environment v3 Overview](migrate.md).

+Ensure you understand how migrating to an App Service Environment v3 will affect your applications. Review the [migration process](migrate.md#overview-of-the-migration-process-using-the-migration-feature) to understand the process timeline and where and when you'll need to get involved. Also review the [FAQs](migrate.md#frequently-asked-questions), which may answer some questions you currently have.

+The recommended experience for the migration feature is using the [Azure portal](how-to-migrate.md?pivots=experience-azp). If you decide to use the Azure CLI to carry out the migration, you should follow the steps described here in order and as written since you'll be making Azure REST API calls. The recommended way for making these API calls is by using the [Azure CLI](/cli/azure/). For information about other methods, see [Getting Started with Azure REST](/rest/api/azure/).

+The following command will check whether your App Service Environment is supported for migration. If you receive an error or if your App Service Environment is in an unhealthy or suspended state, you can't migrate at this time. If your environment [won't be supported for migration](migrate.md#supported-scenarios) or you want to migrate to App Service Environment v3 without using the migration feature, see the [manual migration options](migration-alternatives.md).

+If migration is supported for your App Service Environment, there are three ways to access the migration feature. These methods include a banner at the top of the Overview page, a new item in the left-hand side menu called **Migration (preview)**, and an info box on the **Configuration** page. Select any of these methods to move on to the next step in the migration process.

+If you don't see these elements, your App Service Environment isn't supported for migration at this time or your environment is in an unhealthy or suspended state (which blocks migration). If your environment [won't be supported for migration](migrate.md#supported-scenarios) or you want to migrate to App Service Environment v3 without using the migration feature, see the [manual migration options](migration-alternatives.md).

+ Title: Migrate to App Service Environment v3 by using the migration feature

+description: Overview of the migration feature for migration to App Service Environment v3

+# Migration to App Service Environment v3 using the migration feature

+App Service can now automate migration of your App Service Environment v2 to an [App Service Environment v3](overview.md). If you want to migrate an App Service Environment v1 to an App Service Environment v3, see the [manual migration options documentation](migration-alternatives.md). App Service Environment v3 provides [advantages and feature differences](overview.md#feature-differences) over earlier versions. Make sure to review the [supported features](overview.md#feature-differences) of App Service Environment v3 before migrating to reduce the risk of an unexpected application issue.

+At this time, App Service Environment migrations to v3 using the migration feature support both [Internal Load Balancer (ILB)](create-ilb-ase.md) and [external (internet facing with public IP)](create-external-ase.md) App Service Environment v2 in the following regions:

+The migration feature doesn't plan on supporting App Service Environment v1 within a classic VNet. See the [manual migration options](migration-alternatives.md) if your App Service Environment falls into this category.

+## Overview of the migration process using the migration feature

+App Service Environment v3 requires the subnet it's in to have a single delegation of `Microsoft.Web/hostingEnvironments`. Migration will not succeed if the App Service Environment's subnet isn't delegated or it's delegated to a different resource.

+ You won't be able migrate using the migration feature at this time. If you have an unsupported environment and want to migrate immediately, see the [manual migration options](migration-alternatives.md). This doc will be updated as additional regions and supported scenarios become available.

+ Yes, you should expect about one hour of downtime during the migration step so plan accordingly. If downtime isn't an option for you, see the [manual migration options](migration-alternatives.md).

+ You won't be able migrate using the migration feature at this time. If you have an unsupported environment and want to migrate immediately, see the [manual migration options](migration-alternatives.md).

+ Zone pinned App Service Environment is currently not a supported scenario for migration using the migration feature. When supported, zone pinned App Service Environments will be migrated to zone redundant App Service Environment v3.

+ Title: Migrate to App Service Environment v3

+description: How to migrate your applications to App Service Environment v3

+# Migrate to App Service Environment v3

+> The App Service Environment v3 [migration feature](migrate.md) is now available for a set of supported environment configurations in certain regions. Consider that feature which provides an automated migration path to [App Service Environment v3](overview.md).

+If you're currently using App Service Environment v1 or v2, you have the opportunity to migrate your workloads to [App Service Environment v3](overview.md). App Service Environment v3 has [advantages and feature differences](overview.md#feature-differences) that provide enhanced support for your workloads and can reduce overall costs. Consider using the [migration feature](migrate.md) if your environment falls into one of the [supported scenarios](migrate.md#supported-scenarios). If your environment isn't currently supported by the migration feature, you can wait for support if your scenario is listed in the [upcoming supported scenarios](migrate.md#migration-feature-limitations). Otherwise, you can choose to use one of the manual migration options given in this article.

+If your App Service Environment [won't be supported for migration](migrate.md#migration-feature-limitations) with the migration feature, you must use one of the manual methods to migrate to App Service Environment v3.

+Note that multiple App Service Environments can't exist in a single subnet. If you need to use your existing subnet for your new App Service Environment v3, you'll need to delete the existing App Service Environment before you create a new one. For this scenario, the recommended migration method is to [back up your apps and then restore them](#back-up-and-restore) in the new environment after it gets created and configured. There will be application downtime during this process because of the time it takes to delete the old environment, create the new App Service Environment v3, configure any infrastructure and connected resources to work with the new environment, and deploy your apps onto the new environment.

+1. Modify **SKU and size** as needed using one of the Isolated v2 options if creating a new App Service plan. Note App Service Environment v3 uses Isolated v2 plans, which have more memory and CPU per corresponding instance size compared to the Isolated plan. For more information, see [App Service Environment v3 SKU details](overview.md#pricing).

+You can export [Azure Resource Manager (ARM) templates](../../azure-resource-manager/templates/overview.md) of your existing apps, App Service plans, and any other supported resources and deploy them in or with your new environment. To export a template for just your app, head over to your App Service and go to **Export template** under **Automation**.

+Having issues? [Let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? [Let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? [Let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+Having issues? [Let us know](https://aka.ms/PythonAppServiceQuickstartFeedback).

+## Common Azure VM management operations

+Azure Automation provides scripts for common Azure VM management operations like restart VM, stop VM, delete VM, scale up and down scenarios in Runbook gallery. The scripts can also be found in the Azure Automation [GitHub repository](https://github.com/azureautomation) You can also use these scripts as mentioned in the above steps.

+|**Azure VM management operations** | **Details**|

+| | |

+[Stop-Azure-VM-On-Alert](https://github.com/azureautomation/Stop-Azure-VM-On-Alert) | This runbook will stop an Azure Resource Manager VM in response to an Azure alert trigger. </br></br> Input is alert data with information needed to identify which VM to stop.</br></br> The runbook must be called from an Azure alert via a webhook. </br></br> Latest version of Az module should be added to the automation account. </br></br> Managed Identity should be enabled and contributor access to the automation account should be given.

+[Restart-Azure-VM-On-Alert](https://github.com/azureautomation/Restart-Azure-VM-On-Alert) | This runbook will stop an Azure Resource Manager VM in response to an Azure alert trigger. </br></br> Input is alert data with information needed to identify which VM to stop.</br></br> The runbook must be called from an Azure alert via a webhook. </br></br> Latest version of Az module should be added to the automation account. </br></br> Managed Identity should be enabled and contributor access to the automation account should be given.

+[Delete-Azure-VM-On-Alert](https://github.com/azureautomation/Delete-Azure-VM-On-Alert) | This runbook will stop an Azure Resource Manager VM in response to an Azure alert trigger. </br></br> Input is alert data with information needed to identify which VM to stop.</br></br> The runbook must be called from an Azure alert via a webhook. </br></br> Latest version of Az module should be added to the automation account. </br></br> Managed Identity should be enabled and contributor access to the automation account should be given.

+[ScaleDown-Azure-VM-On-Alert](https://github.com/azureautomation/ScaleDown-Azure-VM-On-Alert) | This runbook will stop an Azure Resource Manager VM in response to an Azure alert trigger. </br></br> Input is alert data with information needed to identify which VM to stop.</br></br> The runbook must be called from an Azure alert via a webhook. </br></br> Latest version of Az module should be added to the automation account. </br></br> Managed Identity should be enabled and contributor access to the automation account should be given.

+[ScaleUp-Azure-VM-On-Alert](https://github.com/azureautomation/ScaleUp-Azure-VM-On-Alert) | This runbook will stop an Azure Resource Manager VM in response to an Azure alert trigger. </br></br> Input is alert data with information needed to identify which VM to stop.</br></br> The runbook must be called from an Azure alert via a webhook. </br></br> Latest version of Az module should be added to the automation account. </br></br> Managed Identity should be enabled and contributor access to the automation account should be given.

+* Discover different ways to start a runbook, see [Start a runbook](./start-runbooks.md).

+* Create an activity log alert, see [Create activity log alerts](../azure-monitor/alerts/activity-log-alerts.md).

+* Learn how to create a near real-time alert, see [Create an alert rule in the Azure portal](../azure-monitor/alerts/alerts-metric.md?toc=/azure/azure-monitor/toc.json).

+ 1. Under **Availability zones**, remove all selected zones. You should not specify any zones.

+apiVersion: v1

+data:

+ password: <your base64 encoded password>

+ username: <your base64 encoded username>

+kind: Secret

+metadata:

+ name: my-login-secret

+type: Opaque

+ type: LoadBalancer

+Some common issues and troubleshooting steps for Azure Key Vault secrets provider are captured in the open source documentation [here](https://azure.github.io/secrets-store-csi-driver-provider-azure/docs/troubleshooting/) for your reference.

+### Use the GitRepository source for Helm charts

+If your Helm charts are stored in the `GitRepository` source that you configure as part of the `fluxConfigurations` resource, you can add an annotation to your HelmRelease yaml to indicate that the configured source should be used as the source of the Helm charts. The annotation is `clusterconfig.azure.com/use-managed-source: "true"`, and here is a usage example:

+```console

+apiVersion: helm.toolkit.fluxcd.io/v2beta1

+kind: HelmRelease

+metadata:

+ name: somename

+ namespace: somenamespace

+ annotations:

+ clusterconfig.azure.com/use-managed-source: "true"

+spec:

+ ...

+```

+By using this annotation, the HelmRelease that is deployed will be patched with the reference to the configured source. Note that only GitRepository source is supported for this currently.

+|maxAutoLockRenewalDuration|00:05:00|The maximum duration within which the message lock will be renewed automatically. This setting only applies for functions that receive a single message at a time.|

+|maxConcurrentCalls|16|The maximum number of concurrent calls to the callback that the should be initiate per scaled instance. By default, the Functions runtime processes multiple messages concurrently. This setting only applies for functions that receive a single message at a time.|

+|maxConcurrentSessions|8|The maximum number of sessions that can be handled concurrently per scaled instance. This setting only applies for functions that receive a single message at a time.|

+|maxMessages|1000|The maximum number of messages that will be passed to each function call. This setting only applies for functions that receive a batch of messages.|

+|sessionIdleTimeout|n/a|The maximum amount of time to wait for a message to be received for the currently active session. After this time has elapsed, the processor will close the session and attempt to process another session. This setting only applies for functions that receive a single message at a time.|

+This tutorial shows you how to configure a function app using Azure Active Directory identities instead of secrets or connection strings, where possible. Using identities helps you avoid accidentally leaking sensitive secrets and can provide better visibility into how data is accessed. To learn more about identity-based connections, see [configure an identity-based connection](functions-reference.md#configure-an-identity-based-connection).

+The identity will now be able to read secrets stored in the key vault. Later in the tutorial, you will add additional role assignments for different purposes.

+ | **Value** | Your account name | Update the name from the connection string to just your **StorageAccountName**. |

+The [`WEBSITE_CONTENTAZUREFILECONNECTIONSTRING`](functions-app-settings.md#website_contentazurefileconnectionstring) and [`WEBSITE_CONTENTSHARE`](functions-app-settings.md#website_contentshare) settings aren't supported on a Linux Consumption plan.

+1. Choose **Export** to create the API Management instance, which may take several minutes.

+>To use the OpenCensus Python extensions, you need to enable [Python worker extensions](#python-worker-extensions) in your function app by setting `PYTHON_ENABLE_WORKER_EXTENSIONS` to `1`. You also need to switch to using the Application Insights connection string by adding the [`APPLICATIONINSIGHTS_CONNECTION_STRING`](functions-app-settings.md#applicationinsights_connection_string) setting to your [application settings](functions-how-to-use-azure-function-app-settings.md#settings), if it's not already there.

+| [Azure AD Multi-Factor Authentication](../../active-directory/authentication/concept-mfa-howitworks.md) | &#x2705; | &#x2705; |

+| [Azure Backup](https://azure.microsoft.com/services/backup/) | &#x2705; | &#x2705; |

+| [Azure Database Migration Service](https://azure.microsoft.com/services/database-migration/) | &#x2705; | &#x2705; |

+| [Azure Healthcare APIs](https://azure.microsoft.com/services/healthcare-apis/) (formerly Azure API for FHIR) | &#x2705; | &#x2705; |

+| [Azure Migrate](https://azure.microsoft.com/services/azure-migrate/) | &#x2705; | &#x2705; |

+| [Azure SignalR Service](https://azure.microsoft.com/services/signalr-service/) | &#x2705; | &#x2705; |

+| [Cognitive Services Containers](../../cognitive-services/cognitive-services-container-support.md) | &#x2705; | &#x2705; |

+| [Dynamics 365 Chat (Omnichannel Engagement Hub)](/dynamics365/omnichannel/introduction-omnichannel) | &#x2705; | &#x2705; |

+| [Microsoft 365 Defender](/microsoft-365/security/defender/) (formerly Microsoft Threat Protection) | &#x2705; | &#x2705; |

+| [Azure AD Multi-Factor Authentication](../../active-directory/authentication/concept-mfa-howitworks.md) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

+| [Azure Bot Service](/azure/bot-service/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Azure DDoS Protection](https://azure.microsoft.com/services/ddos-protection/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/services/kubernetes-service/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | &#x2705; |

+| [Azure Resource Graph](../../governance/resource-graph/overview.md) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Azure Virtual Desktop](https://azure.microsoft.com/services/virtual-desktop/) (formerly Windows Virtual Desktop) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Container Instances](https://azure.microsoft.com/services/container-instances/)| &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Event Grid](https://azure.microsoft.com/services/event-grid/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+| [Microsoft Intune](/mem/intune/fundamentals/) | &#x2705; | &#x2705; | &#x2705; | &#x2705; | |

+- Specific credentials and multi-factor authentication for logical access

+JIT access works with multi-factor authentication that requires Microsoft engineers to use a smartcard to confirm their identity. All access to production systems is performed using Secure Admin Workstations (SAWs) that are consistent with published guidance on [securing privileged access](/security/compass/overview). Use of SAWs for access to production systems is required by Microsoft policy and compliance with this policy is closely monitored. These workstations use a fixed image with all software fully managed ΓÇô only select activities are allowed and users cannot accidentally circumvent the SAW design since they don't have admin privileges on these machines. Access is permitted only with a smartcard and access to each SAW is limited to specific set of users.

+### Legend

+| Symbol | Meaning |

+|--|-|

+| Γ£ô | Country is provided with detailed data. |

+| Γùæ | Country is provided with simplified data. |

+| Country is missing | Country data is not provided. |

+| Country/Region | Coverage |

+|-|:--:|

+| Anguilla | Γ£ô |

+| Antigua & Barbuda | Γ£ô |

+| Argentina | Γ£ô |

+| Aruba | Γ£ô |

+| Bahamas | Γ£ô |

+| Barbados | Γ£ô |

+| Bermuda | Γ£ô |

+| Bonaire, St Eustatius & Saba | Γ£ô |

+| Brazil | Γ£ô |

+| British Virgin Islands | Γ£ô |

+| Canada | Γ£ô |

+| Cayman Islands | Γ£ô |

+| Chile | Γ£ô |

+| Clipperton Island | Γ£ô |

+| Colombia | Γ£ô |

+| Curaçao | ✓ |

+| Dominica | Γ£ô |

+| Falkland Islands | Γ£ô |

+| Grenada | Γ£ô |

+| Guadeloupe | Γ£ô |

+| Haiti | Γ£ô |

+| Jamaica | Γ£ô |

+| Martinique | Γ£ô |

+| Mexico | Γ£ô |

+| Montserrat | Γ£ô |

+| Peru | Γ£ô |

+| Puerto Rico | Γ£ô |

+| Saint Barthélemy | ✓ |

+| Saint Kitts & Nevis | Γ£ô |

+| Saint Lucia | Γ£ô |

+| Saint Martin | Γ£ô |

+| Saint Pierre & Miquelon | Γ£ô |

+| Saint Vincent & Grenadines | Γ£ô |

+| Sint Maarten | Γ£ô |

+| South Georgia & Sandwich Islands | Γ£ô |

+| Trinidad & Tobago | Γ£ô |

+| Turks & Caicos Islands | Γ£ô |

+| U.S. Virgin Islands | Γ£ô |

+| United States | Γ£ô |

+| Uruguay | Γ£ô |

+| Venezuela | Γ£ô |

+## Asia Pacific

+| Country/Region | Coverage |

+|-|:--:|

+| Australia | Γ£ô |

+| Brunei | Γ£ô |

+| Cambodia | Γ£ô |

+| Guam | Γ£ô |

+| Hong Kong | Γ£ô |

+| India | Γ£ô |

+| Indonesia | Γ£ô |

+| Laos | Γ£ô |

+| Macao | Γ£ô |

+| Malaysia | Γ£ô |

+| Myanmar | Γ£ô |

+| New Zealand | Γ£ô |

+| Philippines | Γ£ô |

+| Singapore | Γ£ô |

+| South Korea | Γùæ |

+| Taiwan | Γ£ô |

+| Thailand | Γ£ô |

+| Vietnam | Γ£ô |

+| Country/Region | Coverage |

+|--|:--:|

+| Albania | Γ£ô |

+| Andorra | Γ£ô |

+| Austria | Γ£ô |

+| Belarus | Γ£ô |

+| Belgium | Γ£ô |

+| Bosnia-Herzegovina | Γ£ô |

+| Bulgaria | Γ£ô |

+| Croatia | Γ£ô |

+| Cyprus | Γ£ô |

+| Czech Republic | Γ£ô |

+| Denmark | Γ£ô |

+| Estonia | Γ£ô |

+| Finland | Γ£ô |

+| France | Γ£ô |

+| Germany | Γ£ô |

+| Gibraltar | Γ£ô |

+| Greece | Γ£ô |

+| Hungary | Γ£ô |

+| Iceland | Γ£ô |

+| Ireland | Γ£ô |

+| Italy | Γ£ô |

+| Latvia | Γ£ô |

+| Liechtenstein | Γ£ô |

+| Lithuania | Γ£ô |

+| Luxembourg | Γ£ô |

+| Macedonia | Γ£ô |

+| Malta | Γ£ô |

+| Moldova | Γ£ô |

+| Monaco | Γ£ô |

+| Montenegro | Γ£ô |

+| Netherlands | Γ£ô |

+| Norway | Γ£ô |

+| Poland | Γ£ô |

+| Portugal | Γ£ô |

+| Romania | Γ£ô |

+| Russian Federation | Γ£ô |

+| San Marino | Γ£ô |

+| Serbia | Γ£ô |

+| Slovakia | Γ£ô |

+| Slovenia | Γ£ô |

+| Spain | Γ£ô |

+| Sweden | Γ£ô |

+| Switzerland | Γ£ô |

+| Turkey | Γ£ô |

+| Ukraine | Γ£ô |

+| United Kingdom | Γ£ô |

+| Vatican City | Γ£ô |

+## Middle East & Africa

+| Country/Region | Coverage |

+||:--:|

+| Algeria | Γ£ô |

+| Angola | Γ£ô |

+| Bahrain | Γ£ô |

+| Benin | Γ£ô |

+| Botswana | Γ£ô |

+| Burkina Faso | Γ£ô |

+| Burundi | Γ£ô |

+| Cameroon | Γ£ô |

+| Congo | Γ£ô |

+| Democratic Republic of Congo | Γ£ô |

+| Egypt | Γ£ô |

+| Gabon | Γ£ô |

+| Ghana | Γ£ô |

+| Iraq | Γ£ô |

+| Jordan | Γ£ô |

+| Kenya | Γ£ô |

+| Kuwait | Γ£ô |

+| Lebanon | Γ£ô |

+| Lesotho | Γ£ô |

+| Malawi | Γ£ô |

+| Mali | Γ£ô |

+| Mauritania | Γ£ô |

+| Mauritius | Γ£ô |

+| Mayotte | Γ£ô |

+| Morocco | Γ£ô |

+| Mozambique | Γ£ô |

+| Namibia | Γ£ô |

+| Niger | Γ£ô |

+| Nigeria | Γ£ô |

+| Oman | Γ£ô |

+| Qatar | Γ£ô |

+| Reunion | Γ£ô |

+| Rwanda | Γ£ô |

+| Saudi Arabia | Γ£ô |

+| Senegal | Γ£ô |

+| South Africa | Γ£ô |

+| Swaziland | Γ£ô |

+| Tanzania | Γ£ô |

+| Togo | Γ£ô |

+| Tunisia | Γ£ô |

+| Uganda | Γ£ô |

+| United Arab Emirates | Γ£ô |

+| Yemen | Γ£ô |

+| Zambia | Γ£ô |

+| Zimbabwe | Γ£ô |

+## Additional information

+- See [Zoom levels and tile grid](zoom-levels-and-tile-grid.md) for more information about Azure Maps rendering.

+- [Azure Maps routing service](routing-coverage.md).

+The following tables provide information about what kind of traffic information you can request from each country or region. If a market is missing in the following tables, it is not currently supported.

+| Country/Region | Incidents | Flow |

+|-|::|:-:|

+| Argentina | Γ£ô | Γ£ô |

+| Brazil | Γ£ô | Γ£ô |

+| Canada | Γ£ô | Γ£ô |

+| Chile | Γ£ô | Γ£ô |

+| Colombia | Γ£ô | Γ£ô |

+| Guadeloupe | Γ£ô | Γ£ô |

+| Martinique | Γ£ô | Γ£ô |

+| Mexico | Γ£ô | Γ£ô |

+| Peru | Γ£ô | Γ£ô |

+| United States | Γ£ô | Γ£ô |

+| Uruguay | Γ£ô | Γ£ô |

+| Country/Region | Incidents | Flow |

+|-|::|:-:|

+| Australia | Γ£ô | Γ£ô |

+| Brunei | Γ£ô | Γ£ô |

+| Hong Kong | Γ£ô | Γ£ô |

+| India | Γ£ô | Γ£ô |

+| Indonesia | Γ£ô | Γ£ô |

+| Kazakhstan | Γ£ô | Γ£ô |

+| Macao | Γ£ô | Γ£ô |

+| Malaysia | Γ£ô | Γ£ô |

+| New Zealand | Γ£ô | Γ£ô |

+| Philippines | Γ£ô | Γ£ô |

+| Singapore | Γ£ô | Γ£ô |

+| Taiwan | Γ£ô | Γ£ô |

+| Thailand | Γ£ô | Γ£ô |

+| Vietnam | Γ£ô | Γ£ô |

+| Country/Region | Incidents | Flow |

+||::|:-:|

+| Belarus | Γ£ô | Γ£ô |

+| Belgium | Γ£ô | Γ£ô |

+| Bosnia and Herzegovina | Γ£ô | Γ£ô |

+| Bulgaria | Γ£ô | Γ£ô |

+| Croatia | Γ£ô | Γ£ô |

+| Cyprus | Γ£ô | Γ£ô |

+| Czech Republic | Γ£ô | Γ£ô |

+| Denmark | Γ£ô | Γ£ô |

+| Estonia | Γ£ô | Γ£ô |

+| Finland | Γ£ô | Γ£ô |

+| France | Γ£ô | Γ£ô |

+| Germany | Γ£ô | Γ£ô |

+| Gibraltar | Γ£ô | Γ£ô |

+| Greece | Γ£ô | Γ£ô |

+| Hungary | Γ£ô | Γ£ô |

+| Iceland | Γ£ô | Γ£ô |

+| Ireland | Γ£ô | Γ£ô |

+| Italy | Γ£ô | Γ£ô |

+| Latvia | Γ£ô | Γ£ô |

+| Liechtenstein | Γ£ô | Γ£ô |

+| Lithuania | Γ£ô | Γ£ô |

+| Luxembourg | Γ£ô | Γ£ô |

+| Malta | Γ£ô | Γ£ô |

+| Monaco | Γ£ô | Γ£ô |

+| Netherlands | Γ£ô | Γ£ô |

+| Norway | Γ£ô | Γ£ô |

+| Poland | Γ£ô | Γ£ô |

+| Portugal | Γ£ô | Γ£ô |

+| Romania | Γ£ô | Γ£ô |

+| Russian Federation | Γ£ô | Γ£ô |

+| San Marino | Γ£ô | Γ£ô |

+| Serbia | Γ£ô | Γ£ô |

+| Slovakia | Γ£ô | Γ£ô |

+| Slovenia | Γ£ô | Γ£ô |

+| Spain | Γ£ô | Γ£ô |

+| Sweden | Γ£ô | Γ£ô |

+| Switzerland | Γ£ô | Γ£ô |

+| Turkey | Γ£ô | Γ£ô |

+| Ukraine | Γ£ô | Γ£ô |

+| United Kingdom | Γ£ô | Γ£ô |

+| Country/Region | Incidents | Flow |

+|-|::|:-:|

+| Bahrain | Γ£ô | Γ£ô |

+| Egypt | Γ£ô | Γ£ô |

+| Israel | Γ£ô | Γ£ô |

+| Kenya | Γ£ô | Γ£ô |

+| Kuwait | Γ£ô | Γ£ô |

+| Lesotho | Γ£ô | Γ£ô |

+| Morocco | Γ£ô | Γ£ô |

+| Mozambique | Γ£ô | Γ£ô |

+| Nigeria | Γ£ô | Γ£ô |

+| Oman | Γ£ô | Γ£ô |

+| Qatar | Γ£ô | Γ£ô |

+| Reunion | Γ£ô | Γ£ô |

+| Saudi Arabia | Γ£ô | Γ£ô |

+| South Africa | Γ£ô | Γ£ô |

+| United Arab Emirates | Γ£ô | Γ£ô |

+## Additional information

+For more information about incorporating Azure Maps traffic data into your mapping applications, see the [Traffic](/rest/api/maps/traffic) REST API reference.

+Log Analytics Agent (MMA) does not use the system proxy settings. Hence, user has to pass proxy setting while installing MMA and these settings will be stored under MMA configuration(registry) on VM. To configure the agent to communicate to the service through a proxy server or [Log Analytics gateway](./gateway.md) after deployment, use one of the following methods to complete this task.

+| August 2021 | Fixed issue allowing Azure Monitor Metrics as the only destination | 1.1.2.0 | 1.10.9.0<sup>Hotfix</sup> |

+| September 2021 | <ul><li>Fixed issue causing data loss on restarting the agent</li><li>Fixed issue for Arc Windows servers</li></ul> | 1.1.3.2<sup>Hotfix</sup> | 1.12.2.0 <sup>1</sup> |

+| December 2021 | <ul><li>Fixed issues impacting Linux Arc-enabled servers</li><li>'Heartbeat' table > 'Category' column reports "Azure Monitor Agent" in Log Analytics for Windows</li></ul> | 1.1.4.0 | 1.14.7.0<sup>2</sup> |

+| January 2021 | <ul><li>Syslog RFC compliance for Linux</li><li>Fixed issue for Linux perf counters not flowing on restart</li><ul> | Not available yet | 1.15.2.0<sup>Hotfix</sup> |

+<sup>Hotfix</sup> Do not use AMA Linux versions v1.10.7, v1.15.1 and AMA Windows v1.1.3.1. Please use hotfixed versions listed above.

+<sup>1</sup> Known issue: No data collected from Linux Arc-enabled servers

+<sup>2</sup> Known issue: Linux performance counters data stops flowing on restarting/rebooting the machine(s)

+The **recommendation** is to enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature. Navigate to your virtual machine or scale set, select the **Extensions** tab and click on **AzureMonitorWindowsAgent** or **AzureMonitorLinuxAgent**. In the dialog that pops up, click **Enable automatic upgrade**.

+To perform a **one time update** of the agent, you must first uninstall the existing agent version and then install the new version as described above.

+The **recommendation** is to enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature, using the following PowerShell commands.

+# [Windows](#tab/PowerShellWindows)

+```powershell

+Set-AzVMExtension -ExtensionName AMAWindows -ResourceGroupName <resource-group-name> -VMName <virtual-machine-name> -Publisher Microsoft.Azure.Monitor -ExtensionType AzureMonitorWindowsAgent -TypeHandlerVersion <version-number> -Location <location> -EnableAutomaticUpgrade $true

+```

+# [Linux](#tab/PowerShellLinux)

+```powershell

+Set-AzVMExtension -ExtensionName AMALinux -ResourceGroupName <resource-group-name> -VMName <virtual-machine-name> -Publisher Microsoft.Azure.Monitor -ExtensionType AzureMonitorLinuxAgent -TypeHandlerVersion <version-number> -Location <location> -EnableAutomaticUpgrade $true

+```

+

+

+To perform a **one time update** of the agent, you must first uninstall the existing agent version and then install the new version as described above.

+The **recommendation** is to enable automatic update of the agent by enabling the [Automatic Extension Upgrade](../../virtual-machines/automatic-extension-upgrade.md) feature, using the following CLI commands.

+# [Windows](#tab/CLIWindows)

+```azurecli

+az vm extension set -name AzureMonitorWindowsAgent --publisher Microsoft.Azure.Monitor --vm-name <virtual-machine-name> --resource-group <resource-group-name> --enable-auto-upgrade true

+```

+# [Linux](#tab/CLILinux)

+```azurecli

+az vm extension set -name AzureMonitorLinuxAgent --publisher Microsoft.Azure.Monitor --vm-name <virtual-machine-name> --resource-group <resource-group-name> --enable-auto-upgrade true

+```

+- **Linux multi-homing:** Send data from Windows and Linux VMs to multiple Log Analytics workspaces (i.e. "multi-homing") and/or other [supported destinations](#data-sources-and-destinations).

+## Data sources and destinations

+The following table lists the types of data you can currently collect with the Azure Monitor agent by using data collection rules and where you can send that data. For a list of insights, solutions, and other solutions that use the Azure Monitor agent to collect other kinds of data, see [What is monitored by Azure Monitor?](../monitor-reference.md).

+The Azure Monitor agent sends data to Azure Monitor Metrics (preview) or a Log Analytics workspace supporting Azure Monitor Logs.

+| Data source | Destinations | Description |

+|:|:|:|

+| Performance | Azure Monitor Metrics (preview)<sup>1</sup> - Insights.virtualmachine namespace<br>Log Analytics workspace - [Perf](/azure/azure-monitor/reference/tables/perf) table | Numerical values measuring performance of different aspects of operating system and workloads |

+| Windows event logs | Log Analytics workspace - [Event](/azure/azure-monitor/reference/tables/Event) table | Information sent to the Windows event logging system |

+| Syslog | Log Analytics workspace - [Syslog](/azure/azure-monitor/reference/tables/syslog)<sup>2</sup> table | Information sent to the Linux event logging system |

+<sup>1</sup> [Click here](../essentials/metrics-custom-overview.md#quotas-and-limits) to review other limitations of using Azure Monitor Metrics. On Linux, using Azure Monitor Metrics as the only destination is supported in v1.10.9.0 or higher.

+<sup>2</sup> Azure Monitor Linux Agent v1.15.2 or higher supports syslog RFC formats including Cisco Meraki, Cisco ASA, Cisco FTD, Sophos XG, Juniper Networks, Corelight Zeek, CipherTrust, NXLog, McAfee and CEF (Common Event Format).

+On the **Destination** tab, add one or more destinations for the data source. You can select multiple destinations of same of different types, for instance multiple Log Analytics workspaces (i.e. "multi-homing"). Windows event and Syslog data sources can only send to Azure Monitor Logs. Performance counters can send to both Azure Monitor Metrics and Azure Monitor Logs.

+The Windows and Linux agent supports communicating either through a proxy server or Log Analytics gateway to Azure Monitor using the HTTPS protocol. Both anonymous and basic authentication (username/password) are supported. For the Windows agent connected directly to the service, the proxy configuration is specified during installation or [after deployment](../agents/agent-manage.md#update-proxy-settings) from Control Panel or with PowerShell. Log Analytics Agent (MMA) does not use the system proxy settings. Hence, user has to pass proxy setting while installing MMA and these settings will be stored under MMA configuration(registry) on VM.

+For more information about the activity log fields, see [Azure activity log event schema](../essentials/activity-log-schema.md).

+Monitoring of your Java web applications running on [Azure App Services](../../app-service/index.yml) does not require any modifications to the code. This article will walk you through enabling Azure Monitor Application Insights monitoring as well as provide preliminary guidance for automating the process for large-scale deployments.

+The recommended way to enable application monitoring for Java applications running on Azure App Services is through Azure portal.

+Turning on application monitoring in Azure portal will automatically instrument your application with Application Insights, and doesn't require any code changes.

+You can apply additional configurations, and then based on your specific scenario you [add your own custom telemetry](./java-in-process-agent.md#modify-telemetry) if needed.

+You can turn on monitoring for your Java apps running in Azure App Service just with one click, no code change required.

+Application Insights for Java is integrated with Azure App Service on Linux - both code-based and custom containers, and with App Service on Windows for code-based apps.

+The integration adds [Application Insights Java 3.x](./java-in-process-agent.md) and you will get the telemetry auto-collected.

+ :::image type="content"source="./media/azure-web-apps/change-resource.png" alt-text="Screenshot of Change your resource dropdown.":::

+3. This last step is optional. After specifying which resource to use, you can configure the Java agent. If you do not configure the Java agent, default configurations will apply.

+### Application settings definitions

+| App setting name | Definition | Value |

+|||:|

+| ApplicationInsightsAgent_EXTENSION_VERSION | Main extension, which controls runtime monitoring. | `~2` in Windows or `~3` in Linux. |

+| XDT_MicrosoftApplicationInsights_Java | Flag to control if Java agent is included. | 0 or 1 (only applicable in Windows). |

+Monitoring of your Node.js web applications running on [Azure App Services](../../app-service/index.yml) does not require any modifications to the code. This article will walk you through enabling Azure Monitor Application Insights monitoring as well as provide preliminary guidance for automating the process for large-scale deployments.

+## Enable Application Insights

+The easiest way to enable application monitoring for Node.js applications running on Azure App Services is through Azure portal.

+Turning on application monitoring in Azure portal will automatically instrument your application with Application Insights, and doesn't require any code changes.

+### Auto-instrumentation through Azure portal

+You can turn on monitoring for your Node.js apps running in Azure App Service just with one click, no code change required.

+Application Insights for Node.js is integrated with Azure App Service on Linux - both code-based and custom containers, and with App Service on Windows for code-based apps.

+The integration is in public preview. The integration adds Node.js SDK, which is in GA.

+ > [!NOTE]

+ > When you select **OK** to create the new resource you will be prompted to **Apply monitoring settings**. Selecting **Continue** will link your new Application Insights resource to your app service, doing so will also **trigger a restart of your app service**.

+3. Once you have specified which resource to use, you are all set to go.

+In order to enable telemetry collection with Application Insights, only the following Application settings need to be set:

+| App setting name | Definition | Value |

+|||:|

+| ApplicationInsightsAgent_EXTENSION_VERSION | Main extension, which controls runtime monitoring. | `~2` in Windows or `~3` in Linux. |

+| XDT_MicrosoftApplicationInsights_NodeJS | Flag to control if node.js agent is included. | 0 or 1 (only applicable in Windows). |

+> [!NOTE]

+> Profiler and snapshot debugger are not available for Node.js applications

+ If it is not running, follow the [enable Application Insights monitoring instructions](#enable-application-insights).

+For the latest updates and bug fixes, [consult the release notes](web-app-extension-release-notes.md).

+## Release notes

+See the [release notes](https://github.com/microsoft/ApplicationInsights-Java/releases) on GitHub.

+`$ ./readahead.sh show <mount-point>`

+`$ ./readahead.sh set <mount-point> [read-ahead-kb]`

+1. On the next page, click **Deploy a sample model** if you would like to deploy one of the pre-trained sample vision models. If you would like to deploy an existing [custom no-code vision solution](./tutorial-nocode-vision.md), click **Deploy a Custom Vision project**. If you do not see your Custom Vision projects, set project's domain to "General (Compact)" on [Custom Vision portal](https://www.customvision.ai/) and train a model again. Other domains are not supported currently.

+> | namespaces / eventhubs | namespace | 1-256 | Alphanumerics, periods, hyphens and underscores.<br><br>Start and end with letter or number. |

+In our examples, we're enabling Azure AD-only authentication during server or managed instance creation, with a system assigned server admin and password. This will prevent server admin access when Azure AD-only authentication is enabled, and only allows the Azure AD admin to access the resource. It's optional to add parameters to the APIs to include your own server admin and password during server creation. However, the password cannot be reset until you disable Azure AD-only authentication. An example of how to use these optional parameters to specify the server admin login name is presented in the [PowerShell](?tabs=azure-powershell#azure-sql-database) tab on this page.

+Here is an example of specifying the server admin name (instead of letting the server admin name being automatically created) at the time of logical server creation. As mentioned earlier, this login is not usable when Azure AD-only authentication is enabled.

+```powershell

+$cred = Get-Credential

+New-AzSqlServer -ResourceGroupName "<ResourceGroupName>" -Location "<Location>" -ServerName "<ServerName>" -ServerVersion "12.0" -ExternalAdminName "<AzureADAccount>" -EnableActiveDirectoryOnlyAuthentication -SqlAdministratorCredentials $cred

+```

+For read-write workloads, use `<fog-name>.zone_id.database.windows.net` as the server name. Connections will be automatically directed to the primary. This name does not change after failover. The geo-failover involves updating the DNS record, so the client connections are redirected to the new primary only after the client DNS cache is refreshed. Because the secondary instance shares the DNS zone with the primary, the client application will be able to reconnect to it using the same server-side SAN certificate. The read-write listener and read-only listener cannot be reached via [public endpoint for managed instance](../managed-instance/public-endpoint-configure.md).

+If you have logically isolated read-only workloads that are tolerant to data latency, you can run them on the geo-secondary. To connect directly to the geo-secondary, use `<fog-name>.secondary.<zone_id>.database.windows.net` as the server name. The read-write listener and read-only listener cannot be reached via [public endpoint for managed instance](../managed-instance/public-endpoint-configure.md).

+You can scale up or scale down the primary database to a different compute size (within the same service tier) without disconnecting any geo-secondaries. When scaling up, we recommend that you scale up the geo-secondary first, and then scale up the primary. When scaling down, reverse the order: scale down the primary first, and then scale down the secondary. When you scale a database to a different service tier, this recommendation is enforced.

+|Data Sync| Using a Hyperscale database as a Hub or Sync Metadata database is not supported. However, a Hyperscale database can be a member database in a Data Sync topology. |

+- Using a Hyperscale database as a Hub or Sync Metadata database is not supported. However, a Hyperscale database can be a member database in a Data Sync topology.

+This article explains how to manually failover a primary node on SQL Managed Instance General Purpose (GP) and Business Critical (BC) service tiers, and how to manually failover a secondary read-only replica node on the BC service tier only.

+> [!NOTE]

+> This article is not related with cross-region failovers on [auto-failover groups](../database/auto-failover-group-overview.md).

+>If you're sure that all backed-up items in the vault are no longer required and want to delete them at once without reviewing, [run this PowerShell script](./scripts/delete-recovery-services-vault.md). The script will delete all backup items recursively and eventually the entire vault.

+- **Step 3**: Save the PowerShell script in .ps1 format. Then, to run the script in your PowerShell console, type `./NameOfFile.ps1`. This recursively deletes all backup items and eventually the entire Recovery Services vault.

+ >[!Note]

+ >To access the PowerShell script for vault deletion, see the [PowerShell script for vault deletion](./scripts/delete-recovery-services-vault.md) article.

+To delete an individual backup items or write your own script, use the following PowerShell commands:

+- Stop protection and delete the backup data:

+ If you're using SQL in Azure VMs backup and enabled autoprotection for SQL instances, first disable the autoprotection.

+- Stop protection and delete data for all backup-protected items in cloud (for example, IaaS VM, Azure file share, and so on):

+ [Learn more](/powershell/module/az.recoveryservices/disable-azrecoveryservicesbackupprotection) about disabling protection for a Backup-protected item.

+- [Learn about Recovery Services vaults](backup-azure-recovery-services-vault-overview.md).

+- [Learn about monitoring and managing Recovery Services vaults](backup-azure-manage-windows-server.md).

+ Title: Script Sample - Delete a Recovery Services vault

+description: Learn about how to use a PowerShell script to delete a Recovery Services vault.

+# PowerShell script to delete a Recovery Services vault

+This script helps you to delete a Recovery Services vault.

+## How to execute the script?

+1. Save the script in the following section on your machine with a name of your choice and _.ps1_ extension.

+1. In the script, change the parameters (vault name, resource group name, subscription name, and subscription ID).

+1. To run it in your PowerShell environment, continue with the next steps.

+ Alternatively, you can use Cloud Shell in Azure portal for vaults with fewer backups.

+ :::image type="content" source="../media/backup-azure-delete-vault/delete-vault-using-cloud-shell-inline.png" alt-text="Screenshot showing to delete a vault using Cloud Shell." lightbox="../media/backup-azure-delete-vault/delete-vault-using-cloud-shell-expanded.png":::

+1. To upgrade to the latest version of PowerShell 7, if not done, run the following command in the PowerShell window:

+ ```azurepowershell-interactive

+ iex "& { $(irm https://aka.ms/install-powershell.ps1) } -UseMSI"

+ ```

+1. Launch PowerShell 7 as Administrator.

+1. Before you run the script for vault deletion, run the following command to upgrade the _Az module_ to the latest version:

+ ```azurepowershell-interactive

+ Uninstall-Module -Name Az.RecoveryServices

+ Set-ExecutionPolicy -ExecutionPolicy Unrestricted

+ Install-Module -Name Az.RecoveryServices -Repository PSGallery -Force -AllowClobber

+ ```

+1. In the PowerShell window, change the path to the location the file is present, and then run the file using **./NameOfFile.ps1**.

+1. Provide authentication via browser by signing into your Azure account.

+The script will continue to delete all the backup items and ultimately the entire vault recursively.

+## Script

+```azurepowershell-interactive

+Connect-AzAccount

+$VaultName = "Vault name" #enter vault name

+$Subscription = "Subscription name" #enter Subscription name

+$ResourceGroup = "Resource group name" #enter Resource group name

+$SubscriptionId = "Subscription ID" #enter Subscription ID

+Select-AzSubscription $Subscription

+$VaultToDelete = Get-AzRecoveryServicesVault -Name $VaultName -ResourceGroupName $ResourceGroup

+Set-AzRecoveryServicesAsrVaultContext -Vault $VaultToDelete

+Set-AzRecoveryServicesVaultProperty -Vault $VaultToDelete.ID -SoftDeleteFeatureState Disable #disable soft delete

+Write-Host "Soft delete disabled for the vault" $VaultName

+$containerSoftDelete = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureVM -WorkloadType AzureVM -VaultId $VaultToDelete.ID | Where-Object {$_.DeleteState -eq "ToBeDeleted"} #fetch backup items in soft delete state

+foreach ($softitem in $containerSoftDelete)

+{

+ Undo-AzRecoveryServicesBackupItemDeletion -Item $softitem -VaultId $VaultToDelete.ID -Force #undelete items in soft delete state

+}

+#Invoking API to disable enhanced security

+$azProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile

+$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azProfile)

+$accesstoken = Get-AzAccessToken

+$token = $accesstoken.Token

+$authHeader = @{

+ 'Content-Type'='application/json'

+ 'Authorization'='Bearer ' + $token

+}

+$body = @{properties=@{enhancedSecurityState= "Disabled"}}

+$restUri = 'https://management.azure.com/subscriptions/'+$SubscriptionId+'/resourcegroups/'+$ResourceGroup+'/providers/Microsoft.RecoveryServices/vaults/'+$VaultName+'/backupconfig/vaultconfig?api-version=2019-05-13' #Replace "management.azure.com" with "management.usgovcloudapi.net" if your subscription is in USGov.

+$response = Invoke-RestMethod -Uri $restUri -Headers $authHeader -Body ($body | ConvertTo-JSON -Depth 9) -Method PATCH

+#Fetch all protected items and servers

+$backupItemsVM = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureVM -WorkloadType AzureVM -VaultId $VaultToDelete.ID

+$backupItemsSQL = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureWorkload -WorkloadType MSSQL -VaultId $VaultToDelete.ID

+$backupItemsAFS = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureStorage -WorkloadType AzureFiles -VaultId $VaultToDelete.ID

+$backupItemsSAP = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureWorkload -WorkloadType SAPHanaDatabase -VaultId $VaultToDelete.ID

+$backupContainersSQL = Get-AzRecoveryServicesBackupContainer -ContainerType AzureVMAppContainer -Status Registered -VaultId $VaultToDelete.ID | Where-Object {$_.ExtendedInfo.WorkloadType -eq "SQL"}

+$protectableItemsSQL = Get-AzRecoveryServicesBackupProtectableItem -WorkloadType MSSQL -VaultId $VaultToDelete.ID | Where-Object {$_.IsAutoProtected -eq $true}

+$backupContainersSAP = Get-AzRecoveryServicesBackupContainer -ContainerType AzureVMAppContainer -Status Registered -VaultId $VaultToDelete.ID | Where-Object {$_.ExtendedInfo.WorkloadType -eq "SAPHana"}

+$StorageAccounts = Get-AzRecoveryServicesBackupContainer -ContainerType AzureStorage -Status Registered -VaultId $VaultToDelete.ID

+$backupServersMARS = Get-AzRecoveryServicesBackupContainer -ContainerType "Windows" -BackupManagementType MAB -VaultId $VaultToDelete.ID

+$backupServersMABS = Get-AzRecoveryServicesBackupManagementServer -VaultId $VaultToDelete.ID| Where-Object { $_.BackupManagementType -eq "AzureBackupServer" }

+$backupServersDPM = Get-AzRecoveryServicesBackupManagementServer -VaultId $VaultToDelete.ID | Where-Object { $_.BackupManagementType-eq "SCDPM" }

+$pvtendpoints = Get-AzPrivateEndpointConnection -PrivateLinkResourceId $VaultToDelete.ID

+foreach($item in $backupItemsVM)

+ {

+ Disable-AzRecoveryServicesBackupProtection -Item $item -VaultId $VaultToDelete.ID -RemoveRecoveryPoints -Force #stop backup and delete Azure VM backup items

+ }

+Write-Host "Disabled and deleted Azure VM backup items"

+foreach($item in $backupItemsSQL)

+ {

+ Disable-AzRecoveryServicesBackupProtection -Item $item -VaultId $VaultToDelete.ID -RemoveRecoveryPoints -Force #stop backup and delete SQL Server in Azure VM backup items

+ }

+Write-Host "Disabled and deleted SQL Server backup items"

+foreach($item in $protectableItems)

+ {

+ Disable-AzRecoveryServicesBackupAutoProtection -BackupManagementType AzureWorkload -WorkloadType MSSQL -InputItem $item -VaultId $VaultToDelete.ID #disable auto-protection for SQL

+ }

+Write-Host "Disabled auto-protection and deleted SQL protectable items"

+foreach($item in $backupContainersSQL)

+ {

+ Unregister-AzRecoveryServicesBackupContainer -Container $item -Force -VaultId $VaultToDelete.ID #unregister SQL Server in Azure VM protected server

+ }

+Write-Host "Deleted SQL Servers in Azure VM containers"

+foreach($item in $backupItemsSAP)

+ {

+ Disable-AzRecoveryServicesBackupProtection -Item $item -VaultId $VaultToDelete.ID -RemoveRecoveryPoints -Force #stop backup and delete SAP HANA in Azure VM backup items

+ }

+Write-Host "Disabled and deleted SAP HANA backup items"

+foreach($item in $backupContainersSAP)

+ {

+ Unregister-AzRecoveryServicesBackupContainer -Container $item -Force -VaultId $VaultToDelete.ID #unregister SAP HANA in Azure VM protected server

+ }

+Write-Host "Deleted SAP HANA in Azure VM containers"

+foreach($item in $backupItemsAFS)

+ {

+ Disable-AzRecoveryServicesBackupProtection -Item $item -VaultId $VaultToDelete.ID -RemoveRecoveryPoints -Force #stop backup and delete Azure File Shares backup items

+ }

+Write-Host "Disabled and deleted Azure File Share backups"

+foreach($item in $StorageAccounts)

+ {

+ Unregister-AzRecoveryServicesBackupContainer -container $item -Force -VaultId $VaultToDelete.ID #unregister storage accounts

+ }

+Write-Host "Unregistered Storage Accounts"

+foreach($item in $backupServersMARS)

+ {

+ Unregister-AzRecoveryServicesBackupContainer -Container $item -Force -VaultId $VaultToDelete.ID #unregister MARS servers and delete corresponding backup items

+ }

+Write-Host "Deleted MARS Servers"

+foreach($item in $backupServersMABS)

+ {

+ Unregister-AzRecoveryServicesBackupManagementServer -AzureRmBackupManagementServer $item -VaultId $VaultToDelete.ID #unregister MABS servers and delete corresponding backup items

+ }

+Write-Host "Deleted MAB Servers"

+foreach($item in $backupServersDPM)

+ {

+ Unregister-AzRecoveryServicesBackupManagementServer -AzureRmBackupManagementServer $item -VaultId $VaultToDelete.ID #unregister DPM servers and delete corresponding backup items

+ }

+Write-Host "Deleted DPM Servers"

+#Deletion of ASR Items

+$fabricObjects = Get-AzRecoveryServicesAsrFabric

+if ($null -ne $fabricObjects) {

+ # First DisableDR all VMs.

+ foreach ($fabricObject in $fabricObjects) {

+ $containerObjects = Get-AzRecoveryServicesAsrProtectionContainer -Fabric $fabricObject

+ foreach ($containerObject in $containerObjects) {

+ $protectedItems = Get-AzRecoveryServicesAsrReplicationProtectedItem -ProtectionContainer $containerObject

+ # DisableDR all protected items

+ foreach ($protectedItem in $protectedItems) {

+ Write-Host "Triggering DisableDR(Purge) for item:" $protectedItem.Name

+ Remove-AzRecoveryServicesAsrReplicationProtectedItem -InputObject $protectedItem -Force

+ Write-Host "DisableDR(Purge) completed"

+ }

+

+ $containerMappings = Get-AzRecoveryServicesAsrProtectionContainerMapping `

+ -ProtectionContainer $containerObject

+ # Remove all Container Mappings

+ foreach ($containerMapping in $containerMappings) {

+ Write-Host "Triggering Remove Container Mapping: " $containerMapping.Name

+ Remove-AzRecoveryServicesAsrProtectionContainerMapping -ProtectionContainerMapping $containerMapping -Force

+ Write-Host "Removed Container Mapping."

+ }

+ }

+ $NetworkObjects = Get-AzRecoveryServicesAsrNetwork -Fabric $fabricObject

+ foreach ($networkObject in $NetworkObjects)

+ {

+ #Get the PrimaryNetwork

+ $PrimaryNetwork = Get-AzRecoveryServicesAsrNetwork -Fabric $fabricObject -FriendlyName $networkObject

+ $NetworkMappings = Get-AzRecoveryServicesAsrNetworkMapping -Network $PrimaryNetwork

+ foreach ($networkMappingObject in $NetworkMappings)

+ {

+ #Get the Neetwork Mappings

+ $NetworkMapping = Get-AzRecoveryServicesAsrNetworkMapping -Name $networkMappingObject.Name -Network $PrimaryNetwork

+ Remove-AzRecoveryServicesAsrNetworkMapping -InputObject $NetworkMapping

+ }

+ }

+ # Remove Fabric

+ Write-Host "Triggering Remove Fabric:" $fabricObject.FriendlyName

+ Remove-AzRecoveryServicesAsrFabric -InputObject $fabricObject -Force

+ Write-Host "Removed Fabric."

+ }

+}

+foreach($item in $pvtendpoints)

+ {

+ $penamesplit = $item.Name.Split(".")

+ $pename = $penamesplit[0]

+ Remove-AzPrivateEndpointConnection -ResourceId $item.PrivateEndpoint.Id -Force #remove private endpoint connections

+ Remove-AzPrivateEndpoint -Name $pename -ResourceGroupName $ResourceGroup -Force #remove private endpoints

+ }

+Write-Host "Removed Private Endpoints"

+#Recheck ASR items in vault

+$fabricCount = 0

+$ASRProtectedItems = 0

+$ASRPolicyMappings = 0

+$fabricObjects = Get-AzRecoveryServicesAsrFabric

+if ($null -ne $fabricObjects) {

+ foreach ($fabricObject in $fabricObjects) {

+ $containerObjects = Get-AzRecoveryServicesAsrProtectionContainer -Fabric $fabricObject

+ foreach ($containerObject in $containerObjects) {

+ $protectedItems = Get-AzRecoveryServicesAsrReplicationProtectedItem -ProtectionContainer $containerObject

+ foreach ($protectedItem in $protectedItems) {

+ $ASRProtectedItems++

+ }

+ $containerMappings = Get-AzRecoveryServicesAsrProtectionContainerMapping `

+ -ProtectionContainer $containerObject

+ foreach ($containerMapping in $containerMappings) {

+ $ASRPolicyMappings++

+ }

+ }

+ $fabricCount++

+ }

+}

+#Recheck presence of backup items in vault

+$backupItemsVMFin = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureVM -WorkloadType AzureVM -VaultId $VaultToDelete.ID

+$backupItemsSQLFin = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureWorkload -WorkloadType MSSQL -VaultId $VaultToDelete.ID

+$backupContainersSQLFin = Get-AzRecoveryServicesBackupContainer -ContainerType AzureVMAppContainer -Status Registered -VaultId $VaultToDelete.ID | Where-Object {$_.ExtendedInfo.WorkloadType -eq "SQL"}

+$protectableItemsSQLFin = Get-AzRecoveryServicesBackupProtectableItem -WorkloadType MSSQL -VaultId $VaultToDelete.ID | Where-Object {$_.IsAutoProtected -eq $true}

+$backupItemsSAPFin = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureWorkload -WorkloadType SAPHanaDatabase -VaultId $VaultToDelete.ID

+$backupContainersSAPFin = Get-AzRecoveryServicesBackupContainer -ContainerType AzureVMAppContainer -Status Registered -VaultId $VaultToDelete.ID | Where-Object {$_.ExtendedInfo.WorkloadType -eq "SAPHana"}

+$backupItemsAFSFin = Get-AzRecoveryServicesBackupItem -BackupManagementType AzureStorage -WorkloadType AzureFiles -VaultId $VaultToDelete.ID

+$StorageAccountsFin = Get-AzRecoveryServicesBackupContainer -ContainerType AzureStorage -Status Registered -VaultId $VaultToDelete.ID

+$backupServersMARSFin = Get-AzRecoveryServicesBackupContainer -ContainerType "Windows" -BackupManagementType MAB -VaultId $VaultToDelete.ID

+$backupServersMABSFin = Get-AzRecoveryServicesBackupManagementServer -VaultId $VaultToDelete.ID| Where-Object { $_.BackupManagementType -eq "AzureBackupServer" }

+$backupServersDPMFin = Get-AzRecoveryServicesBackupManagementServer -VaultId $VaultToDelete.ID | Where-Object { $_.BackupManagementType-eq "SCDPM" }

+$pvtendpointsFin = Get-AzPrivateEndpointConnection -PrivateLinkResourceId $VaultToDelete.ID

+Write-Host "Number of backup items left in the vault and which need to be deleted:" $backupItemsVMFin.count "Azure VMs" $backupItemsSQLFin.count "SQL Server Backup Items" $backupContainersSQLFin.count "SQL Server Backup Containers" $protectableItemsSQLFin.count "SQL Server Instances" $backupItemsSAPFin.count "SAP HANA backup items" $backupContainersSAPFin.count "SAP HANA Backup Containers" $backupItemsAFSFin.count "Azure File Shares" $StorageAccountsFin.count "Storage Accounts" $backupServersMARSFin.count "MARS Servers" $backupServersMABSFin.count "MAB Servers" $backupServersDPMFin.count "DPM Servers" $pvtendpointsFin.count "Private endpoints"

+Write-Host "Number of ASR items left in the vault and which need to be deleted:" $ASRProtectedItems "ASR protected items" $ASRPolicyMappings "ASR policy mappings" $fabricCount "ASR Fabrics" $pvtendpointsFin.count "Private endpoints. Warning: This script will only remove the replication configuration from Azure Site Recovery and not from the source. Please cleanup the source manually. Visit https://go.microsoft.com/fwlink/?linkid=2182781 to learn more"

+Remove-AzRecoveryServicesVault -Vault $VaultToDelete

+#Finish

+```

+## Next steps

+[Learn more](../backup-azure-delete-vault.md) about vault deletion process.

+| Compression encodings |gzip, brotli |gzip |gzip, deflate, bzip2 |gzip, deflate, bzip2 |

+description: A common scenario for speech-to-text is transcribing large volumes of telephony data that come from various systems, such as interactive voice response (IVR). By using Speech service and the Unified speech model, a business can get high-quality transcriptions with audio capture systems.

+Telephony data that's generated through landlines, mobile phones, and radios is ordinarily of low quality. This data is also narrowband, in the range of 8&nbsp;KHz, which can create challenges when you're converting speech to text.

+The latest Speech service speech-recognition models excel at transcribing this telephony data, even when the data is difficult for a human to understand. These models are trained with large volumes of telephony data, and they have best-in-market recognition accuracy, even in noisy environments.

+A common scenario for speech-to-text is the transcription of large volumes of telephony data that comes from a variety of systems, such as interactive voice response (IVR). The audio that these systems provide can be stereo or mono, and raw, with little to no post-processing done on the signal. By using Speech service and the Unified speech model, your business can get high-quality transcriptions, whatever systems you use to capture audio.

+You can use telephony data to better understand your customers' needs, identify new marketing opportunities, or evaluate the performance of call center agents. After the data is transcribed, your business can use the output for improving telemetry, identifying key phrases, analyzing customer *sentiment*, and other purposes.

+The technologies outlined in this article are from Microsoft internally for various support-call processing services, both in real-time and batch mode.

+This article discusses some of the technology and related features that Speech service offers.

+> The Speech service Unified model is trained with diverse data and offers a single-model solution to many scenarios, from dictation to telephony analytics.

+## Azure technology for call centers

+Beyond the functional aspect of the Speech service features, their primary purpose, as applied to the call center, is to improve the customer experience in three separate domains:

+- Post-call analytics, which is essentially the batch processing of call recordings after the call.

+- Real-time analytics, which is the processing of an audio signal to extract various insights as the call is taking place (with sentiment as a prominent use case).

+- Voice assistants (bots), which either drive the dialogue between customers and the bot in an attempt to solve their issues, without agent participation, or apply AI protocols to assist the agent.

+Here is an architecture diagram showing a typical implementation of a batch scenario:

+

+## Components of speech analytics technology

+Whether the domain is post-call or real-time, Azure offers a set of mature and emerging technologies to help improve the customer experience.

+### Speech-to-text

+[Speech-to-text](speech-to-text.md) is the most sought-after feature in any call center solution. Because many of the downstream analytics processes rely on transcribed text, the word error rate (WER) metric is of utmost importance. One of the key challenges in call center transcription is the noise thatΓÇÖs prevalent in the call center (for example, other agents speaking in the background), the rich variety of language locales and dialects, and the low quality of the actual telephone signal.

+WER is highly correlated with how well the acoustic and language models are trained for a specific locale. Therefore, it's important to be able to customize the model to your locale. Our latest Unified version 4.x models are the solution to both transcription accuracy and latency. Because they're trained with tens of thousands of hours of acoustic data and billions of bits of lexical information, Unified models are the most accurate in the market for transcribing call center data.

+In the call center space, the ability to gauge whether customers have had a good experience is one of the most important areas of Speech analytics. The Microsoft [Batch Transcription API](batch-transcription.md) offers sentiment analysis per utterance. You can aggregate the set of values that are obtained as part of a call transcript to determine the sentiment of the call for both your agents and the customer.

+It's not uncommon for as much as 35 percent of a support call to be what's called *non-talk time*. Some scenarios during which non-talk occurs might include:

+* Agents taking time to look up prior case history with a customer.

+* Agents using tools that allow them to access the customer's desktop and perform certain functions.

+* Customers waiting on hold for a call transfer.

+It's important to gauge when silence is occurring in a call, because critical customer sensitivities can result from these types of scenarios and where they occur in the call.

+Some companies are experimenting with providing translated transcripts from foreign-language support calls, so that delivery managers can understand the world-wide experience of their customers. Speech service's [translation](./speech-translation.md) capabilities are excellent, featuring the audio-to-audio or audio-to-text translation for a large number of locales.

+### Text-to-speech

+[Text-to-speech](text-to-speech.md) is another important technology where bots interact with customers. The typical pathway is that a customer speaks, the voice is transcribed to text, the text is analyzed for intents, a response is synthesized based on the recognized intent, and then an asset is either surfaced to the customer or a synthesized voice response is generated. Because this entire process must occur quickly, low latency is an important component in the success of these systems.

+Speech service's end-to-end latency is considerably low for the various technologies involved, such as [speech-to-text](speech-to-text.md), [Language Understanding (LUIS)](https://azure.microsoft.com/services/cognitive-services/language-understanding-intelligent-service/), [Bot Framework](https://dev.botframework.com/), and [text-to-speech](text-to-speech.md).

+Our new synthesized voices are also nearly indistinguishable from human voices. You can use them to give your bot its unique personality.

+Another staple of analytics is to identify interactions where a specific event or experience has occurred. You would ordinarily do this with either of two approaches:

+* An ad hoc search, where users simply type a phrase and the system responds.

+* A more structured query where an analyst can create a set of logical statements that identify a scenario in a call, and then each call can be indexed against that set of queries.

+A good search example is the ubiquitous compliance statement, "This call will be recorded for quality purposes." Many companies want to make sure that their agents are providing this disclaimer to customers before the call is actually recorded. Most analytics systems have the ability to trend the behaviors found by query or search algorithms, and this reporting of trends is ultimately one of the most important functions of an analytics system. Through the [Cognitive Services directory](https://azure.microsoft.com/services/cognitive-services/directory/search/), your end-to-end solution can be significantly enhanced with indexing and search capabilities.

+### Key phrase extraction

+This area is one of the more challenging analytics applications, and one that benefits from the application of AI and machine learning. The primary scenario in this case is to infer customer intent. Why is the customer calling? What is the customer's problem? Why did the customer have a negative experience? [Cognitive Service for Language](https://azure.microsoft.com/services/cognitive-services/text-analytics/) provides a set of analytics out of the box for quickly upgrading your end-to-end solution for extracting those important keywords or phrases.

+The next sections cover batch processing and the real-time pipelines for speech recognition in a bit more detail.

+To transcribe audio in bulk, Microsoft developed the [Batch Transcription API](batch-transcription.md), which transcribes large amounts of audio data asynchronously. For transcribing call center data specifically, this solution is based on three pillars:

+- **Accuracy**: By applying fourth-generation Unified models, we offer high-quality transcription.

+- **Latency**: Bulk transcriptions must be performed quickly. The transcription jobs that are initiated via the [Batch Transcription API](batch-transcription.md) are queued immediately, and when the job starts running, it's performed faster than real-time transcription.

+- **Security**: We understand that calls might contain sensitive data, so security is our highest priority. To this end, our service has obtained (ISO), SOC, HIPAA, and PCI certifications.

+Call centers generate large volumes of audio data on a daily basis. If your business stores telephony data in a central location, such as an Azure storage account, you can use the [Batch Transcription API](batch-transcription.md) to asynchronously request and receive transcriptions.

+A typical solution uses these products and

+- **Speech service**: For transcribing speech-to-text. A standard subscription for Speech service is required to use the Batch Transcription API. Free subscriptions will not work.

+- **[Azure storage account](https://azure.microsoft.com/services/storage/)**: For storing telephony data and the transcripts that are returned by the Batch Transcription API. This storage account should use notifications, specifically for when new files are added. These notifications are used to trigger the transcription process.

+- **[Azure Functions](../../azure-functions/index.yml)**: For creating the shared access signature (SAS) URI for each recording, and triggering the HTTP POST request to start a transcription. Additionally, you use Azure Functions to create requests to retrieve and delete transcriptions by using the Batch Transcription API.

+Internally, Microsoft uses these technologies to support Microsoft customer calls in batch mode, as shown in the following diagram:

+Some businesses are required to transcribe conversations in real time. You can use real-time transcription to identify keywords and trigger searches for content and resources that are relevant to the conversation, to monitor sentiment, to improve accessibility, or to provide translations for customers and agents who aren't native speakers.

+Internally, Microsoft uses the previously mentioned technologies to analyze Microsoft customer calls in real time, as shown in the following diagram:

+

+## About interactive voice responses

+You can easily integrate Speech service into any solution by using either the [Speech SDK](speech-sdk.md) or the [REST API](./overview.md#reference-docs). However, call center transcription might require additional technologies. Ordinarily, a connection between an IVR system and Azure is required. Although we don't offer such components, the next paragraph describes what a connection to an IVR entails.

+Several IVR or telephony service products (such as Genesys or AudioCodes) offer integration capabilities that can be applied to enable an inbound and outbound audio pass-through to an Azure service. Basically, a custom Azure service might provide a specific interface to define phone call sessions (such as Call Start or Call End) and expose a WebSocket API to receive inbound stream audio that's used with Speech service. Outbound responses, such as a conversation transcription or connections with the Bot Framework, can be synthesized with the Microsoft text-to-speech service and returned to the IVR for playback.

+Another scenario is direct integration with the Session Initiation Protocol (SIP). An Azure service connects to a SIP server to get an inbound and outbound stream, which is used for the speech-to-text and text-to-speech phases. To connect to a SIP server there are commercial software offerings, such as Ozeki SDK, or the [Microsoft Graph Communications API](/graph/api/resources/communications-api-overview), that are designed to support this type of scenario for audio calls.

+The Speech service works well with built-in models. However, you might want to further customize and tune the experience for your product or environment. Customization options range from acoustic model tuning to unique voice fonts for your brand. After you've built a custom model, you can use it with any of the Speech service features in real-time or batch mode.

+| Speech-to-text | [Acoustic model](./how-to-custom-speech-train-model.md) | Create a custom acoustic model for applications, tools, or devices that are used in particular environments, such as in a car or on a factory floor, each with its own recording conditions. Examples include accented speech, background noises, or using a specific microphone for recording. |

+| | [Language model](./how-to-custom-speech-train-model.md) | Create a custom language model to improve transcription of industry-specific vocabulary and grammar, such as medical terminology or IT jargon. |

+| | [Pronunciation model](./how-to-custom-speech-train-model.md) | With a custom pronunciation model, you can define the phonetic form and display for a word or term. It's useful for handling customized terms, such as product names or acronyms. All you need to get started is a pronunciation file, which is a simple .txt file. |

+| Text-to-speech | [Voice font](./how-to-custom-voice-create-voice.md) | with custom voice fonts, you can create a recognizable, one-of-a-kind voice for your brand. It takes only a small amount of data to get started. The more data you provide, the more natural and human-like your voice font will sound. |

+Sample code is available on GitHub for each of the Speech service features. These samples cover common scenarios, such as reading audio from a file or stream, continuous and at-start recognition, and working with custom models. To view SDK and REST samples, see:

+Every locale supports the "String" parameter type, but availability of all other types differs by locale. Custom Commands uses LUIS's prebuilt entity resolution, so the availability of a parameter type in a locale depends on LUIS's prebuilt entity support in that locale. You can find [more details on LUIS's prebuilt entity support per locale](../luis/luis-reference-prebuilt-entities.md). Custom LUIS entities (such as machine learned entities) are currently not supported.

+ Title: Select an audio input device with the Speech SDK

+description: 'Learn about selecting audio input devices in the Speech SDK (C++, C#, Python, Objective-C, Java, and JavaScript) by obtaining the IDs of the audio devices connected to a system.'

+# Select an audio input device with the Speech SDK

+Version 1.3.0 of the Speech SDK introduces an API to select the audio input. This article describes how to obtain the IDs of the audio devices connected to a system. These IDs can then be used in the Speech SDK. You configure the audio device through the `AudioConfig` object:

+> Microphone use isn't available for JavaScript running in Node.js.

+## Audio device IDs on Windows for desktop applications

+Audio device [endpoint ID strings](/windows/desktop/CoreAudio/endpoint-id-strings) can be retrieved from the [`IMMDevice`](/windows/desktop/api/mmdeviceapi/nn-mmdeviceapi-immdevice) object in Windows for desktop applications.

+ // Get the pointer to endpoint number i.

+ // Initialize the container for property value.

+ // Print the endpoint friendly name and endpoint ID.

+In C#, you can use the [NAudio](https://github.com/naudio/NAudio) library to access the CoreAudio API and enumerate devices as follows:

+On the Universal Windows Platform (UWP), you can obtain audio input devices by using the `Id()` property of the corresponding [`DeviceInformation`](/uwp/api/windows.devices.enumeration.deviceinformation) object.

+The following code samples show how to do this step in C++ and C#:

+The device IDs are selected by using standard ALSA device IDs.

+Alternatively, they can be obtained by using the [ALSA C library](https://www.alsa-project.org/alsa-doc/alsa-lib/).

+Audio device selection with the Speech SDK isn't supported on iOS. Apps that use the SDK can influence audio routing through the [`AVAudioSession`](https://developer.apple.com/documentation/avfoundation/avaudiosession?language=objc) Framework.

+Enables the use of a Bluetooth headset for a speech-enabled app.

+In JavaScript, the [MediaDevices.enumerateDevices()](https://developer.mozilla.org/docs/Web/API/MediaDevices/enumerateDevices) method can be used to enumerate the media devices and find a device ID to pass to `fromMicrophone(...)`.

+> [Explore samples on GitHub](https://aka.ms/csspeech/samples)

+ Title: Specify source language for speech to text

+description: The Speech SDK allows you to specify the source language when you convert speech to text. This article describes how to use the FromConfig and SourceLanguageConfig methods to let the Speech service know the source language and provide a custom model target.

+# Specify source language for speech-to-text

+In this article, you'll learn how to specify the source language for an audio input passed to the Speech SDK for speech recognition. The example code that's provided specifies a custom speech model for improved recognition.

+## Specify source language in C#

+In the following example, the source language is provided explicitly as a parameter by using the `SpeechRecognizer` construct:

+In the following example, the source language is provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter to the `SpeechRecognizer` construct.

+In the following example, the source language and custom endpoint are provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter to the `SpeechRecognizer` construct.

+> The `SpeechRecognitionLanguage` and `EndpointId` set methods are deprecated from the `SpeechConfig` class in C#. The use of these methods is discouraged. Don't use them when you create a `SpeechRecognizer` construct.

+## Specify source language in C++

+In the following example, the source language is provided explicitly as a parameter by using the `FromConfig` method.

+In the following example, the source language is provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter to `FromConfig` when you create the `recognizer` construct.

+In the following example, the source language and custom endpoint are provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter to `FromConfig` when you create the `recognizer` construct.

+> `SetSpeechRecognitionLanguage` and `SetEndpointId` are deprecated methods from the `SpeechConfig` class in C++ and Java. The use of these methods is discouraged. Don't use them when you create a `SpeechRecognizer` construct.

+## Specify source language in Java

+In the following example, the source language is provided explicitly when you create a new `SpeechRecognizer` construct.

+In the following example, the source language is provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter when you create a new `SpeechRecognizer` construct.

+In the following example, the source language and custom endpoint are provided by using `SourceLanguageConfig`. Then, `sourceLanguageConfig` is passed as a parameter when you create a new `SpeechRecognizer` construct.

+> `setSpeechRecognitionLanguage` and `setEndpointId` are deprecated methods from the `SpeechConfig` class in C++ and Java. The use of these methods is discouraged. Don't use them when you create a `SpeechRecognizer` construct.

+## Specify source language in Python

+In the following example, the source language is provided explicitly as a parameter by using the `SpeechRecognizer` construct.

+In the following example, the source language is provided by using `SourceLanguageConfig`. Then, `SourceLanguageConfig` is passed as a parameter to the `SpeechRecognizer` construct.

+In the following example, the source language and custom endpoint are provided by using `SourceLanguageConfig`. Then, `SourceLanguageConfig` is passed as a parameter to the `SpeechRecognizer` construct.

+> The `speech_recognition_language` and `endpoint_id` properties are deprecated from the `SpeechConfig` class in Python. The use of these properties is discouraged. Don't use them when you create a `SpeechRecognizer` construct.

+## Specify source language in JavaScript

+The first step is to create a `SpeechConfig` construct:

+## Specify source language in Objective-C

+In the following example, the source language is provided explicitly as a parameter by using the `SPXSpeechRecognizer` construct.

+In the following example, the source language is provided by using `SPXSourceLanguageConfiguration`. Then, `SPXSourceLanguageConfiguration` is passed as a parameter to the `SPXSpeechRecognizer` construct.

+In the following example, the source language and custom endpoint are provided by using `SPXSourceLanguageConfiguration`. Then, `SPXSourceLanguageConfiguration` is passed as a parameter to the `SPXSpeechRecognizer` construct.

+> The `speechRecognitionLanguage` and `endpointId` properties are deprecated from the `SPXSpeechConfiguration` class in Objective-C. The use of these properties is discouraged. Don't use them when you create a `SPXSpeechRecognizer` construct.

+For a list of supported languages and locales for speech-to-text, see [Language support](language-support.md).

+See the [Speech SDK reference documentation](speech-sdk.md).

+description: An overview of the capabilities of the Speech SDK audio input stream API.

+The Speech SDK audio input stream API provides a way to stream audio into the recognizers instead of using either the microphone or the input file APIs.

+The following steps are required when you use audio input streams:

+- Identify the format of the audio stream. The format must be supported by the Speech SDK and the Azure Cognitive Services Speech service. Currently, only the following configuration is supported:

+ Audio samples are:

+ - PCM format

+ - One channel

+ - 16 bits per sample, 8,000 or 16,000 samples per second (16,000 bytes or 32,000 bytes per second)

+ - Two-block aligned (16 bit including padding for a sample)

+ The corresponding code in the SDK to create the audio format looks like this example:

+- Make sure that your code provides the RAW audio data according to these specifications. Also, make sure that 16-bit samples arrive in little-endian format. Signed samples are also supported. If your audio source data doesn't match the supported formats, the audio must be transcoded into the required format.

+- Create your own audio input stream class derived from `PullAudioInputStreamCallback`. Implement the `Read()` and `Close()` members. The exact function signature is language-dependent, but the code looks similar to this code sample:

+ // Returns audio data to the caller.

+ // E.g., return read(config.YYY, buffer, size);

+ // Close and clean up resources.

+ // Run stream through recognizer.

+description: An overview of the features, capabilities, and restrictions for keyword recognition by using the Speech Software Development Kit (SDK).

+Keyword recognition detects a word or short phrase within a stream of audio. It's also referred to as keyword spotting.

+To balance accuracy, latency, and computational complexity, keyword recognition is implemented as a multistage system. For all stages beyond the first, audio is only processed if the stage prior to it believed to have recognized the keyword of interest.

+The current system is designed with multiple stages that span the edge and cloud:

+

+* **Correct accept rate**: Measures the system's ability to recognize the keyword when it's spoken by a user. The correct accept rate is also known as the true positive rate.

+* **False accept rate**: Measures the system's ability to filter out audio that isn't the keyword spoken by a user. The false accept rate is also known as the false positive rate.

+The goal is to maximize the correct accept rate while minimizing the false accept rate. The current system is designed to detect a keyword or phrase preceded by a short amount of silence. Detecting a keyword in the middle of a sentence or utterance isn't supported.

+## Custom keyword for on-device models

+With the [Custom Keyword portal on Speech Studio](https://speech.microsoft.com/customkeyword), you can generate keyword recognition models that execute at the edge by specifying any word or short phrase. You can further personalize your keyword model by choosing the right pronunciations.

+There's no cost to use custom keyword to generate models, including both Basic and Advanced models. There's also no cost to run models on-device with the Speech SDK.

+You can use custom keyword to generate two types of on-device models for any keyword.

+| Basic | Best suited for demo or rapid prototyping purposes. Models are generated with a common base model and can take up to 15 minutes to be ready. Models might not have optimal accuracy characteristics. |

+| Advanced | Best suited for product integration purposes. Models are generated with adaptation of a common base model by using simulated training data to improve accuracy characteristics. It can take up to 48 hours for models to be ready. |

+> You can view a list of regions that support the **Advanced** model type in the [keyword recognition region support](regions.md#keyword-recognition) documentation.

+Neither model type requires you to upload training data. Custom keyword fully handles data generation and model training.

+When you create a new model, custom keyword automatically generates possible pronunciations of the provided keyword. You can listen to each pronunciation and choose all variations that closely represent the way you expect users to say the keyword. All other pronunciations shouldn't be selected.

+It's important to be deliberate about the pronunciations you select to ensure the best accuracy characteristics. For example, if you choose more pronunciations than you need, you might get higher false accept rates. If you choose too few pronunciations, where not all expected variations are covered, you might get lower correct accept rates.

+### Test models

+After on-device models are generated by custom keyword, they can be tested directly on the portal. You can use the portal to speak directly into your browser and get keyword recognition results.

+## Keyword verification

+Keyword verification is a cloud service that reduces the impact of false accepts from on-device models with robust models running on Azure. Tuning or training isn't required for keyword verification to work with your keyword. Incremental model updates are continually deployed to the service to improve accuracy and latency and are transparent to client applications.

+Keyword verification is always used in combination with speech-to-text. There's no cost to use keyword verification beyond the cost of speech-to-text.

+### Keyword verification and speech-to-text

+When keyword verification is used, it's always in combination with speech-to-text. Both services run in parallel, which means audio is sent to both services for simultaneous processing.

+

+Running keyword verification and speech-to-text in parallel yields the following benefits:

+* **No other latency on speech-to-text results**: Parallel execution means that keyword verification adds no latency. The client receives speech-to-text results as quickly. If keyword verification determines the keyword wasn't present in the audio, speech-to-text processing is terminated. This action protects against unnecessary speech-to-text processing. Network and cloud model processing increases the user-perceived latency of voice activation. For more information, see [Recommendations and guidelines](keyword-recognition-guidelines.md).

+* **Forced keyword prefix in speech-to-text results**: Speech-to-text processing ensures that the results sent to the client are prefixed with the keyword. This behavior allows for increased accuracy in the speech-to-text results for speech that follows the keyword.

+* **Increased speech-to-text timeout**: Because of the expected presence of the keyword at the beginning of audio, speech-to-text allows for a longer pause of up to five seconds after the keyword before it determines the end of speech and terminates speech-to-text processing. This behavior ensures that the user experience is correctly handled for staged commands (*\<keyword> \<pause> \<command>*) and chained commands (*\<keyword> \<command>*).

+### Keyword verification responses and latency considerations

+For each request to the service, keyword verification returns one of two responses: accepted or rejected. The processing latency varies depending on the length of the keyword and the length of the audio segment expected to contain the keyword. Processing latency doesn't include network cost between the client and Azure Speech services.

+| Keyword verification response | Description |

+| Accepted | Indicates the service believed that the keyword was present in the audio stream provided as part of the request. |

+| Rejected | Indicates the service believed that the keyword wasn't present in the audio stream provided as part of the request. |

+Rejected cases often yield higher latencies as the service processes more audio than accepted cases. By default, keyword verification processes a maximum of two seconds of audio to search for the keyword. If the keyword is determined not to be present in two seconds, the service times out and signals a rejected response to the client.

+### Use keyword verification with on-device models from custom keyword

+The Speech SDK enables seamless use of on-device models generated by using custom keyword with keyword verification and speech-to-text. It transparently handles:

+* Audio gating to keyword verification and speech recognition based on the outcome of an on-device model.

+* Communicating the keyword to keyword verification.

+* Communicating any more metadata to the cloud for orchestrating the end-to-end scenario.

+You don't need to explicitly specify any configuration parameters. All necessary information is automatically extracted from the on-device model generated by custom keyword.

+The sample and tutorials linked here show how to use the Speech SDK:

+ * [Tutorial: Create a custom commands application with simple voice commands](./how-to-develop-custom-commands-application.md)

+The Speech SDK enables easy use of personalized on-device keyword recognition models generated with custom keyword and keyword verification. To ensure that your product needs can be met, the SDK supports the following two scenarios:

+| End-to-end keyword recognition with speech-to-text | Best suited for products that will use a customized on-device keyword model from custom keyword with Azure Speech keyword verification and speech-to-text. This scenario is the most common. | <ul><li>[Voice assistant sample code](https://github.com/Azure-Samples/Cognitive-Services-Voice-Assistant)</li><li>[Tutorial: Voice enable your assistant built using Azure Bot Service with the C# Speech SDK](./tutorial-voice-enable-your-bot-speech-sdk.md)</li><li>[Tutorial: Create a custom commands application with simple voice commands](./how-to-develop-custom-commands-application.md)</li></ul> |

+| Offline keyword recognition | Best suited for products without network connectivity that will use a customized on-device keyword model from custom keyword. | <ul><li>[C# on Windows UWP sample](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/quickstart/csharp/uwp/keyword-recognizer)</li><li>[Java on Android sample](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/quickstart/java/android/keyword-recognizer)</li></ul>

+* [Read the quickstart to generate on-device keyword recognition models using custom keyword](custom-keyword-basics.md)

+* [Learn more about voice assistants](voice-assistants.md)

+* TTS Service January 2022, added 10 new languages and variants for Neural text-to-speech and new voices in preview for en-GB, fr-FR and de-DE.

+* Containers v3.0.0 released January 2022, with support for using containers in disconnected environments.

+description: This article presents Speech service phonetic alphabet and International Phonetic Alphabet (IPA) examples.

+Phonetic alphabets are used with the [Speech Synthesis Markup Language (SSML)](speech-synthesis-markup.md) to improve the pronunciation of text-to-speech voices. To learn when and how to use each alphabet, see [Use phonemes to improve pronunciation](speech-synthesis-markup.md#use-phonemes-to-improve-pronunciation).

+For some locales, Speech service defines its own phonetic alphabets, which ordinarily map to the [International Phonetic Alphabet (IPA)](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet). The seven locales that support the Microsoft Speech API (SAPI, or `sapi`) are en-US, fr-FR, de-DE, es-ES, ja-JP, zh-CN, and zh-TW.

+|Example&nbsp;1 (onset for consonant, word-initial for vowel)|Example&nbsp;2 (intervocalic for consonant, word-medial nucleus for vowel)|Example&nbsp;3 (coda for consonant, word-final for vowel)|Comments|

+| burger /b er **1** r - g ax r/ | falafel /f ax - l aa **1** - f ax l/ | guitar /g ih - t aa **1** r/ | The Speech service phone set puts stress after the vowel of the stressed syllable. |

+| inopportune /ih **2** - n aa - p ax r - t uw 1 n/ | dissimilarity /d ih - s ih **2**- m ax - l eh 1 - r ax - t iy/ | workforce /w er 1 r k - f ao **2** r s/ | The Speech service phone set puts stress after the vowel of the sub-stressed syllable. |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| eh | `ɛ` | **e**very | p**e**t | m**eh** (rare word-final) |

+| ae | `æ` | **a**ctive | c**a**t | n**ah** (rare word-final) |

+| aa | `ɑ` | **o**bstinate | p**o**ppy | r**ah** (rare word-final) |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+#### English semivowels

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+#### English nasal stops

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+The Speech service phone set puts stress after the vowel of the stressed syllable. However, the `fr-FR` Speech service phone set doesn't support the IPA substress 'ˌ'. If the IPA substress is needed, you should use the IPA directly.

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| eu | `ø` | **œu**fs | cr**eu**ser | qu**eu**e |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| ng | `┼ï` | | | park**ing**[¹](#fr-1) |

+**1** *Only for some foreign words*.

+| Example&nbsp;1 (Onset for consonant, word-initial for vowel) | Example&nbsp;2 (Intervocalic for consonant, word-medial nucleus for vowel) | Example&nbsp;3 (Coda for consonant, word-final for vowel) | Comments |

+| Allgemeinwissen /a **2** l - g ax - m ay 1 n - v ih - s n/ | Abfallentsorgungsfirma /a 1 p - f a l - ^ eh n t - z oh **2** ax r - g uh ng s - f ih ax r - m a/ | Computertomographie /k oh m - p y uw 1 - t ax r - t ow - m ow - g r a - f iy **2**/ | The Speech service phone set puts stress after the vowel of the sub-stressed syllable |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| eh: | `╔¢╦É` | **├ä**hnlichkeit | B**├ñ**r | Fasci**ae**[¹](#de-v-1) |

+| ax | `╔Ö` | 'v**e**rstauen[²](#de-v-2) | Aach**e**n | Frag**e** |

+**1** *Only in words of foreign origin, such as Fasci**ae***.<br>

+**2** *Word-initial only in words of foreign origin, such as **A**ppointment. Syllable-initial in 'v**e**rstauen*.

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| b | `b` | **B**ank | | Pu**b**[¹](#de-c-1) |

+| c | `├º` | **Ch**emie | m├╢gli**ch**st | i**ch**[²](#de-c-2) |

+| d | `d` | **d**anken | Len**d**l[³](#de-c-3) | Clau**d**e[⁴](#de-c-4) |

+| jh | `╩ñ` | **J**eff | gemana**g**t | Chan**g**e[⁵](#de-c-5) |

+| g | `g` | **g**ut | Gre**g**[⁶](#de-c-6) | |

+| ng | `┼ï` | **Ng**uyen[⁷](#de-c-7) | Schwa**nk** | R**ing** |

+| s | `s` | **S**taccato[⁸](#de-c-8) | bi**s**t | mie**s** |

+| v | `v` | **w**inken | Q**u**alle | Gr**oo**ve[⁹](#de-c-9) |

+| x | `x`[¹⁰](#de-c-10), `├º`[¹¹](#de-c-11) | Ba**ch**erach[¹²](#de-c-12) | Ma**ch**t m├╢gli**ch**st | Schma**ch** 'i**ch** |

+**1** *Only in words of foreign origin, such as Pu**b***.<br>

+**2** *Soft "ch" after "e" and "i"*.<br>

+**3** *Only in words of foreign origin, such as Len**d**l*.<br>

+**4** *Only in words of foreign origin, such as Clau**d**e*.<br>

+**5** *Only in words of foreign origin, such as Chan**g**e*.<br>

+**6** *Word-terminally only in words of foreign origin, such as Gre**g***.<br>

+**7** *Only in words of foreign origin, such as **Ng**uyen*.<br>

+**8** *Only in words of foreign origin, such as **S**taccato*.<br>

+**9** *Only in words of foreign origin, such as Gr**oo**ve*.<br>

+**10** *The IPA `x` is a hard "ch" after all non-front vowels (a, aa, oh, ow, uh, uw, and the diphthong aw)*.<br>

+**11** *The IPA `ç` is a soft "ch" after front vowels (ih, iy, eh, ae, uy, ue, oe, eu, and diphthongs ay, oy) and consonants*.<br>

+**12** *Word-initial only in words of foreign origin, such as **J**uan. Syllable-initial also in words such as Ba**ch**erach*.<br>

+| `sapi` | `ipa` | Example |

+> We need to add a [gs\] phone between two distinct vowels, except when the two vowels are a genuine diphthong. This oral consonant is a glottal stop. For more information, see [glottal stop](http://en.wikipedia.org/wiki/Glottal_stop).

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `sapi` | `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+> The `es-ES` Speech service phone set doesn't support the following Spanish IPA: `β`, `ð`, and `ɣ`. If they're needed, consider using the IPA directly.

+| Speech service tone | Bopomofo tone | Example (word) | Speech service phones | Bopomofo | Pinyin (拼音） |

+#### Examples

+#### Examples

+For the following locales, Speech service uses the [International Phonetic Alphabet (IPA)](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet).

+These locales all use the IPA stress and syllable symbols that are listed here:

+Select a tab to view the IPA phonemes that are specific to each locale.

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3|

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3|

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+#### Vowels

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+#### Vowels

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+#### Consonants

+| `ipa` | Example&nbsp;1 | Example&nbsp;2 | Example&nbsp;3 |

+[Speech Studio](https://speech.microsoft.com) is a set of UI-based tools for building and integrating features from Azure Cognitive Services Speech service in your applications. You create projects in Speech Studio by using a no-code approach, and then reference those assets in your applications by using the [Speech SDK](speech-sdk.md), the [Speech CLI](spx-overview.md), or the REST APIs.

+## Prerequisites

+Before you can begin using [Speech Studio](https://speech.microsoft.com), you need to have an Azure account and a Speech resource. If you don't already have an account and a resource, [try Speech service for free](overview.md#try-the-speech-service-for-free).

+After you've created an Azure account and a Speech service resource, do the following:

+1. Sign in to [Speech Studio](https://speech.microsoft.com) with your Azure account.

+1. In your Speech Studio subscription, select a Speech resource. You can change the resource at any time by selecting **Settings** at the top of the pane.

+In Speech Studio, the following Speech service features are available as project types:

+* **Real-time speech-to-text**: Quickly test speech-to-text by dragging audio files here without having to use any code. This is a demo tool for seeing how speech-to-text works on your audio samples. To explore the full functionality, see [What is speech-to-text?](speech-to-text.md).

+* **Custom Speech**: Create speech recognition models that are tailored to specific vocabulary sets and styles of speaking. In contrast to the base speech recognition model, Custom Speech models become part of your unique competitive advantage because they're not publicly accessible. To get started with uploading sample audio to create a Custom Speech model, see [Prepare data for Custom Speech](how-to-custom-speech-test-and-train.md).

+* **Pronunciation assessment**: Evaluate speech pronunciation and give speakers feedback on the accuracy and fluency of spoken audio. Speech Studio provides a sandbox for testing this feature quickly, without code. To use the feature with the Speech SDK in your applications, see the [Pronunciation assessment](how-to-pronunciation-assessment.md) article.

+* **Custom Voice**: Create custom, one-of-a-kind voices for text-to-speech. You supply audio files and create matching transcriptions in Speech Studio, and then use the custom voices in your applications. To create and use custom voices via endpoints, see [Create and use your voice model](how-to-custom-voice-create-voice.md).

+* **Audio Content Creation**: Build highly natural audio content for a variety of scenarios, such as audiobooks, news broadcasts, video narrations, and chat bots, with the easy-to-use [Audio Content Creation](how-to-audio-content-creation.md) tool. With Speech Studio, you can export these audio files to use in your applications.

+* **Custom Keyword**: A custom keyword is a word or short phrase that you can use to voice-activate a product. You create a custom keyword in Speech Studio, and then generate a binary file to [use with the Speech SDK](custom-keyword-basics.md) in your applications.

+* **Custom Commands**: Easily build rich, voice-command apps that are optimized for voice-first interaction experiences. Custom Commands provides a code-free authoring experience in Speech Studio, an automatic hosting model, and relatively lower complexity. The feature helps you focus on building the best solution for your voice-command scenarios. For more information, see the [Develop Custom Commands applications](how-to-develop-custom-commands-application.md) guide. Also see [Integrate with a client application by using the Speech SDK](how-to-custom-commands-setup-speech-sdk.md).

+description: With speech translation, you can add end-to-end, real-time, multi-language translation of speech to your applications, tools, and devices.

+In this article, you learn about the benefits and capabilities of the speech translation service, which enables real-time, multi-language speech-to-speech and speech-to-text translation of audio streams. By using the Speech SDK, you can give your applications, tools, and devices access to source transcriptions and translation outputs for the provided audio. Interim transcription and translation results are returned as speech is detected, and the final results can be converted into synthesized speech.

+For a list of languages that the Speech Translation API supports, see the "Speech translation" section of [Language and voice support for the Speech service](language-support.md#speech-translation).

+## Before you begin

+As your first step, see [Get started with speech translation](get-started-speech-translation.md). The speech translation service is available via the [Speech SDK](speech-sdk.md) and the [Speech CLI](spx-overview.md).

+You'll find [Speech SDK speech-to-text and translation samples](https://github.com/Azure-Samples/cognitive-services-speech-sdk) on GitHub. These samples cover common scenarios, such as reading audio from a file or stream, continuous and at-start recognition and translation, and working with custom models.

+If your applications, tools, or products are using the [Translator Speech API](./how-to-migrate-from-translator-speech-api.md), see [Migrate from the Translator Speech API to Speech service](how-to-migrate-from-translator-speech-api.md).

+* Read the [Get started with speech translation](get-started-speech-translation.md) quickstart article.

+* Get a [Speech service subscription key for free](overview.md#try-the-speech-service-for-free).

+* Get the [Speech SDK](speech-sdk.md).

+ Title: "Quickstart: The Speech CLI - Speech service"

+description: By using the Azure Speech CLI, you can interact with speech-to-text, text-to-speech, and speech translation without having to write code.

+In this article, you'll learn how to use the Azure Speech CLI (also called SPX) to access Speech services such as speech-to-text, text-to-speech, and speech translation, without having to write any code. The Speech CLI is production ready, and you can use it to automate simple workflows in the Speech service by using `.bat` or shell scripts.

+This article assumes that you have working knowledge of the Command Prompt window, terminal, or PowerShell.

+## Create a subscription configuration

+To get started, you need an Azure subscription key and region identifier (for example, `eastus`, `westus`). To learn how to get these credentials, see the [Speech service overview](overview.md#find-keys-and-locationregion) documentation.

+To configure your subscription key and region identifier, run the following commands:

+The key and region are stored for future Speech CLI commands. To view the current configuration, run the following commands:

+As needed, include the `clear` option to remove either stored value:

+To get started, you need an Azure subscription key and region identifier (for example, `eastus`, `westus`). To learn how to get these credentials, see the [Speech service overview](overview.md#find-keys-and-locationregion) documentation.

+To configure your subscription key and region identifier, run the following commands in PowerShell:

+The key and region are stored for future SPX commands. To view the current configuration, run the following commands:

+As needed, include the `clear` option to remove either stored value:

+This section shows a few basic SPX commands that are often useful for first-time testing and experimentation. Start by viewing the help that's built into the tool by running the following command:

+You can search help topics by keyword. For example, to see a list of Speech CLI usage examples, run the following command:

+To see options for the recognize command, run the following command:

+## Speech-to-text (speech recognition)

+To convert speech to text (speech recognition) by using your system's default microphone, run the following command:

+After you run the command, SPX begins listening for audio on the current active input device. It stops listening when you select **Enter**. The spoken audio is then recognized and converted to text in the console output.

+With the Speech CLI, you can also recognize speech from an audio file. Run the following command:

+> If you're using a Docker container, `--microphone` will not work.

+> If you're recognizing speech from an audio file in a Docker container, make sure that the audio file is located in the directory that you mounted previously.

+> If you get stuck or want to learn more about the Speech CLI recognition options, you can run ```spx help recognize```.

+## Text-to-speech (speech synthesis)

+The following command takes text as input and then outputs the synthesized speech to the current active output device (for example, your computer speakers).

+You can also save the synthesized output to a file. In this example, let's create a file named *my-sample.wav* in the directory where you're running the command.

+These examples presume that you're testing in English. However, Speech service supports speech synthesis in many languages. You can pull down a full list of voices either by running the following command or by visiting the [language support page](./language-support.md).

+Here's a command for using one of the voices you've discovered.

+> If you get stuck or want to learn more about the Speech CLI recognition options, you can run ```spx help synthesize```.

+## Speech-to-text translation

+With the Speech CLI, you can also do speech-to-text translation. Run the following command to capture audio from your default microphone and output the translation as text. Keep in mind that you need to supply the `source` and `target` language with the `translate` command.

+When you're translating into multiple languages, separate the language codes with a semicolon (`;`).

+> For a list of all supported languages and their corresponding locale codes, see [Language and voice support for the Speech service](language-support.md).

+> If you get stuck or want to learn more about the Speech CLI recognition options, you can run ```spx help translate```.

+* [Install GStreamer to use the Speech CLI with MP3 and other formats](./how-to-use-codec-compressed-audio-input-streams.md)

+* [Configuration options for the Speech CLI](./spx-data-store-configuration.md)

+description: In this article, you learn about the Speech CLI, a command-line tool for using Speech service without having to write any code.

+The Speech CLI is a command-line tool for using Speech service without having to write any code. The Speech CLI requires minimal setup. You can easily use it to experiment with key features of Speech service and see how it works with your use cases. Within minutes, you can run simple test workflows, such as batch speech-recognition from a directory of files or text-to-speech on a collection of strings from a file. Beyond simple workflows, the Speech CLI is production-ready, and you can scale it up to run larger processes by using automated `.bat` or shell scripts.

+Most features in the Speech SDK are available in the Speech CLI, and some advanced features and customizations are simplified in the Speech CLI. As you're deciding when to use the Speech CLI or the Speech SDK, consider the following guidance.

+* You want to experiment with Speech service features with minimal setup and without having to write code.

+* You have relatively simple requirements for a production application that uses Speech service.

+* You want to integrate Speech service functionality within a specific language or platform (for example, C#, Python, or C++).

+* You have complex requirements that might require advanced service requests.

+* You're developing custom behavior, including response streaming.

+* **Speech recognition**: Convert speech to text either from audio files or directly from a microphone, or transcribe a recorded conversation.

+* **Speech synthesis**: Convert text to speech either by using input from text files or by inputting directly from the command line. Customize speech output characteristics by using [Speech Synthesis Markup Language (SSML) configurations](speech-synthesis-markup.md), and [neural voices](speech-synthesis-markup.md#prebuilt-neural-voices-and-custom-neural-voices).

+* **Speech translation**: Translate audio in a source language to text or audio in a target language.

+* **Run on Azure compute resources**: Send Speech CLI commands to run on an Azure remote compute resource by using `spx webjob`.

+To get started with the Speech CLI, see the [quickstart](spx-basics.md). This article shows you how to run some basic commands. It also gives you slightly more advanced commands for running batch operations for speech-to-text and text-to-speech. After you've read the basics article, you should understand the syntax well enough to start writing some custom commands or automate simple Speech service operations.

+- [Get started with the Azure Speech CLI](spx-basics.md)

+- [Speech CLI configuration options](./spx-data-store-configuration.md)

+- [Speech CLI batch operations](./spx-batch-operations.md)

+description: An overview of the features, capabilities, and restrictions for voice assistants with the Speech SDK.

+By using voice assistants with the Speech service, developers can create natural, human-like, conversational interfaces for their applications and experiences.

+The voice assistant service provides fast, reliable interaction between a device and an assistant implementation that uses either [Direct Line Speech](direct-line-speech.md) (via Azure Bot Service) for adding voice capabilities to your bots or Custom Commands for voice-command scenarios.

+## Choose an assistant solution

+The first step in creating a voice assistant is to decide what you want it to do. Speech service provides multiple, complementary solutions for crafting assistant interactions. For flexibility and versatility, you can add voice in and voice out capabilities to a bot by using Azure Bot Service with the [Direct Line Speech](direct-line-speech.md) channel, or you can simply author a [Custom Commands](custom-commands.md) app for more straightforward voice-command scenarios.

+| If you want... | Consider using... | Examples |

+|Voice-command or simple task-oriented conversations with simplified authoring and hosting | [Custom Commands](custom-commands.md) | <ul><li>"Turn on the overhead light"</li><li>"Make it 5 degrees warmer"</li><li>More examples at [Speech Studio](https://speech.microsoft.com/customcommands)</li></ul>

+If you aren't yet sure what you want your assistant to do, we recommend [Direct Line Speech](direct-line-speech.md) as the best option. It offers integration with a rich set of tools and authoring aids, such as the [Virtual Assistant solution and enterprise template](/azure/bot-service/bot-builder-enterprise-template-overview) and the [QnA Maker service](../qnamaker/overview/overview.md), to build on common patterns and use your existing knowledge sources.

+If you want to keep it simpler for now, [Custom Commands](custom-commands.md) makes it easy to build rich, voice-command apps that are optimized for voice-first interaction. Custom Commands provides a unified authoring experience, an automatic hosting model, and relatively lower complexity, all of which can help you focus on building the best solution for your voice-command scenario.

+ 

+## Reference architecture for building a voice assistant by using the Speech SDK

+ 

+|[Custom keyword](./custom-keyword-basics.md) | Users can start conversations with assistants by using a custom keyword such as "Hey Contoso." An app does this with a custom keyword engine in the Speech SDK, which you can configure by going to [Get started with custom keywords](./custom-keyword-basics.md). Voice assistants can use service-side keyword verification to improve the accuracy of the keyword activation (versus using the device alone).

+|[Speech-to-text](speech-to-text.md) | Voice assistants convert real-time audio into recognized text by using [speech-to-text](speech-to-text.md) from the Speech service. This text is available, as it's transcribed, to both your assistant implementation and your client application.

+|[Text-to-speech](text-to-speech.md) | Textual responses from your assistant are synthesized through [text-to-speech](text-to-speech.md) from the Speech service. This synthesis is then made available to your client application as an audio stream. Microsoft offers the ability to build your own custom, high-quality Neural Text to Speech (Neural TTS) voice that gives a voice to your brand. To learn more, [contact us](mailto:mstts@microsoft.com).

+## Get started with voice assistants

+We offer the following quickstart articles, organized by programming language, that are designed to have you running code in less than 10 minutes:

+* [Quickstart: Create a custom voice assistant by using Direct Line Speech](quickstarts/voice-assistants.md)

+* [Quickstart: Build a voice-command app by using Custom Commands](quickstart-custom-commands-application.md)

+## Sample code and tutorials

+Sample code for creating a voice assistant is available on GitHub. The samples cover the client application for connecting to your assistant in several popular programming languages.

+* [Tutorial: Voice-enable an assistant that's built by using Azure Bot Service with the C# Speech SDK](tutorial-voice-enable-your-bot-speech-sdk.md)

+Voice assistants that you build by using Speech service can use a full range of customization options.

+> Customization options vary by language and locale. To learn more, see [Supported languages](language-support.md).

+|filter.suffix|string|False|A case-sensitive suffix string to filter documents in the source path for translation. It's most often use for file extensions.|

+|language|string|False|Language code If none is specified, we'll perform auto detect on the document.|

+|glossaries.glossaryUrl|string|True (if using glossaries)|Location of the glossary. We'll use the file extension to extract the formatting if the format parameter isn't supplied. If the translation language pair isn't present in the glossary, it won't be applied.|

+> [!NOTE]

+> In the following examples, limited access has been granted to the contents of an Azure Storage container [using a shared access signature(SAS)](/azure/storage/common/storage-sas-overview) token.

+Make sure you've specified the folder name (case sensitive) as prefix in filter.

+* Specify "storageType": "File"

+* Create source URL & SAS token for the specific blob/document.

+* Specify the target filename as part of the target URL ΓÇô though the SAS token is still for the container.

+The sample request below shows a single document translated into two target languages

+|401|Unauthorized. Check your credentials.|

+|503|Service is currently unavailable. Try again later.|

+|innerError|InnerTranslationError|New Inner Error format that conforms to Cognitive Services API Guidelines. This contains required properties: ErrorCode, message and optional properties target, details(key value pair), and inner error(this can be nested).|

+|innerError.target|string|Gets the source of the error. For example it would be "documents" or "document ID" if the document is invalid.|

+| [**dictionary/examples**](v3-0-dictionary-examples.md) | **POST** | Returns how a term is used in context. |

+Incoming video streams won't stop rendering if the user is on iOS 15.2+ and is using SDK version 1.4.1-beta.1+, the unmute/start video steps will still be required to re-start outgoing audio and video.

+description: Automate tasks that monitor, create, manage, send, and receive files for an SFTP server by using SSH and Azure Logic Apps.

+ * FileMage Gateway

+* The following SFTP-SSH actions support [chunking](../logic-apps/logic-apps-handle-large-messages.md):

+ SFTP-SSH actions that support chunking can handle files up to 1 GB, while SFTP-SSH actions that don't support chunking can handle files up to 50 MB. The default chunk size is 15 MB. However, this size can dynamically change, starting from 5 MB and gradually increasing to the 50-MB maximum. Dynamic sizing is based on factors such as network latency, server response time, and so on.

+ > [!NOTE]

+ > For logic apps in an [integration service environment (ISE)](../logic-apps/connect-virtual-network-vnet-isolated-environment-overview.md),

+ > this connector's ISE-labeled version requires chunking to use the [ISE message limits](../logic-apps/logic-apps-limits-and-config.md#message-size-limits) instead.

+ You can override this adaptive behavior when you [specify a constant chunk size](#change-chunk-size) to use instead. This size can range from 5 MB to 50 MB. For example, suppose you have a 45-MB file and a network that can that support that file size without latency. Adaptive chunking results in several calls, rather that one call. To reduce the number of calls, you can try setting a 50-MB chunk size. In different scenario, if your logic app is timing out, for example, when using 15-MB chunks, you can try reducing the size to 5 MB.

+ Chunk size is associated with a connection. This attribute means you can use the same connection for both actions that support chunking and actions that don't support chunking. In this case, the chunk size for actions that don't support chunking ranges from 5 MB to 50 MB.

+* SFTP-SSH triggers don't support message chunking. When triggers request file content, they select only files that are 15 MB or smaller. To get files larger than 15 MB, follow this pattern instead:

+* An Azure account and subscription. If you don't have an Azure subscription, [sign up for a free Azure account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+ * **Encryption algorithms**: Review [Encryption Method - SSH.NET](https://github.com/sshnet/SSH.NET#encryption-method).

+ * **Key exchange algorithms**: Review [Key Exchange Method - SSH.NET](https://github.com/sshnet/SSH.NET#key-exchange-method).

+Connection-based triggers where you need to create a connection first, such as the SFTP-SSH trigger, differ from built-in triggers that run natively in Azure Logic Apps, such as the [Recurrence trigger](../connectors/connectors-native-recurrence.md). In connection-based recurrence triggers, the schedule isn't the only driver that controls execution, and the time zone only determines the initial start time. Subsequent runs depend on the recurrence schedule, the last trigger execution, *and* other factors that might cause run times to drift or produce unexpected behavior. For example, unexpected behavior can include failure to maintain the specified schedule when daylight saving time (DST) starts and ends. To make sure that the recurrence time doesn't shift when DST takes effect, manually adjust the recurrence. That way, your workflow continues to run at the expected time. Otherwise, the start time shifts one hour forward when DST starts and one hour backward when DST ends. For more information, see [Recurrence for connection-based triggers](../connectors/apis-list.md#recurrence-for-connection-based-triggers).

+> [!IMPORTANT]

+> If you use chunking with SFTP-SSH operations that create files on your SFTP server,

+> these operations create temporary `.partial` and `.lock` files. These files help

+> the operations use chunking. Don't remove or change these files. Otherwise,

+> the file operations fail. When the operations finish, they delete the temporary files.

+ secretRef: storage-account-name

+ secretRef: storage-account-key

+ value: mycontainer

+To use this file, make sure to replace the value of `containerName` with your own value if you have changed `STORAGE_ACCOUNT_CONTAINER` variable from its original value, `mycontainer`.

+ --secrets "storage-account-name=${STORAGE_ACCOUNT},storage-account-key=${STORAGE_ACCOUNT_KEY}" \

+ --secrets "storage-account-name=${STORAGE_ACCOUNT},storage-account-key=${STORAGE_ACCOUNT_KEY}" `

+> [!NOTE]

+## Features

+## Introductory video

+> [!VIDEO https://www.youtube.com/embed/b3dopSTnSRg]

+* Unable to access a registry using `az acr login` and you receive error `CONNECTIVITY_REFRESH_TOKEN_ERROR. Access to registry was denied. Response code: 403.Unable to get admin user credentials with message: Admin user is disabled.Unable to authenticate using AAD or admin login credentials.`

+* The registry public access is disabled.Public network access rules on the registry prevent access - [solution](container-registry-troubleshoot-access.md#configure-public-access-to-registry)

+|AnalyticalTimeToLive | User-configurable | Provides the ability to delete items automatically from a container after a set time period. For details, see [Time to Live](analytical-store-introduction.md). | Yes | No | Yes | No | No |

+* **Authentication with the analytical store** is the same as the transactional store for a given database. You can use primary, secondary, or read-only keys for authentication. You can leverage linked service in Synapse Studio to prevent pasting the Azure Cosmos DB keys in the Spark notebooks. For Azure Synapse SQL serverless, you can use SQL credentials to also prevent pasting the Azure Cosmos DB keys in the SQL notebooks. The Access to these Linked Services or to these SQL credentials are available to anyone who has access to the workspace. Please note that the Cosmos DB read only key can also be used.

+### Azure Synapse Link for Azure Cosmos DB

+[Azure Synapse Link for Azure Cosmos DB](synapse-link.md) is a cloud-native hybrid transactional and analytical processing (HTAP) capability that enables near real time analytics over operational data in Azure Cosmos DB. Azure Synapse Link creates a tight seamless integration between Azure Cosmos DB and Azure Synapse Analytics.

+- Reduced analytics complexity with No ETL jobs to manage.

+- Near real-time insights into your operational data.

+- No impact on operational workloads.

+- Optimized for large-scale analytics workloads.

+- Cost effective.

+- Analytics for locally available, globally distributed, multi-region writes.

+- Native integration with Azure Synapse Analytics.

+* **Automatic and transparent sharding**: The API for MongoDB manages all of the infrastructure for you. This includes sharding and the number of shards, unlike other MongoDB offerings such as MongoDB Atlas, which require you to specify and manage sharding to horizontally scale. This gives you more time to focus on developing applications for your users.

+* **Cost efficient, granular, unlimited scalability**: Sharded collections can scale to any size, unlike other MongoDB service offerings. APIs for MongoDB users are running databases with over 600TB of storage today. Scaling is done in a cost-efficient manner, since unlike other MongoDB service offering, the Cosmos DB platform can scale in increments as small as 1/100th of a VM due to economies of scale and resource governance.

+All the APIs for MongoDB versions run on the same codebase, making upgrades a simple task that can be completed in seconds with zero downtime. Azure Cosmos DB simply flips a few feature flags to go from one version to another. The feature flags also enable continued support for older API versions such as 3.2 and 3.6. You can choose the server version that works best for you.

+* Configure near real time analytics with [Azure Synapse Link for Azure Cosmos DB](../configure-synapse-link.md)

+The latency of a synchronous logger necessarily factors into the overall latency calculation of your request-generating thread. An async logger such as [log4j2](https://logging.apache.org/log4j/log4j-2.3/manual/async.html) is recommended to decouple logging overhead from your high-performance application threads.

+pip install azure-identity

+ # Acquire a credential object for the app identity. When running in the cloud,

+ # DefaultAzureCredential uses the app's managed identity (MSI) or user-assigned service principal.

+ # When run locally, DefaultAzureCredential relies on environment variables named

+ # AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID.

+ from azure.identity import DefaultAzureCredential

+ from azure.mgmt.datalake.store.models import CreateDataLakeStoreAccountParameters

+ import adal

+ # Use these as needed for your application

+credential = DefaultAzureCredential()

+adlsAcctClient = DataLakeStoreAccountManagementClient(credential, subscription_id=subscriptionId)

+adlsAcctResult = adlsAcctClient.accounts.begin_create(

+ CreateDataLakeStoreAccountParameters(

+)

+result_list_response = adlsAcctClient.accounts.list()

+adlsAcctClient.accounts.begin_delete(resourceGroup, adlsAccountName)

+* [Open Source Big Data applications compatible with Azure Data Lake Storage Gen1](data-lake-store-compatible-oss-other-applications.md)

+[Azure Data Share](overview.md) allows you to securely share data snapshots from your Azure SQL Database and Azure Synapse Analytics resources, to other Azure subscriptions. Including Azure subscriptions outside your tenant.

+This article describes sharing data from **Azure SQL Database** and **Azure Synapse Analytics**, but Azure Data Share also allows sharing from these other kinds of resources:

+- [Azure Storage](how-to-share-from-storage.md)

+- [Azure Data Explorer](/data-explorer/data-share.md)

+This article will guide you through:

+- [What kinds of data can be shared](#whats-supported)

+- [How to prepare your environment](#prerequisites-to-share-data)

+- [How to create a share](#create-a-share)

+- [How to receive shared data](#receive-shared-data)

+>

+> - Azure Synapse Analytics (workspace) serverless SQL pool

+> - Azure SQL databases with Always Encrypted configured

+### Receive data

+- Azure Data Lake Storage Gen2

+- Azure Blob Storage

+- Azure SQL Database

+- Azure Synapse Analytics

+Shared data in **Azure SQL Database** and **Azure Synapse Analytics** is stored in tables. If the target table doesn't already exist, Azure Data Share creates the SQL table with the source schema. If a target table with the same name already exists, it will be dropped and overwritten with the latest full snapshot.

+>[!NOTE]

+When you share data from a SQL source, the following mappings are used from SQL Server data types to Azure Data Share interim data types during the snapshot process.

+>

+> 1. For data types that map to the Decimal interim type, currently snapshot supports precision up to 28. If you have data that requires precision larger than 28, consider converting to a string.

+> 1. If you are sharing data from Azure SQL database to Azure Synapse Analytics, not all data types are supported. Refer to [Table data types in dedicated SQL pool](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-tables-data-types.md) for details.

+## Prerequisites to share data

+- An Azure subscription: If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.

+- An [Azure SQL Database](../azure-sql/database/single-database-create-quickstart.md) or [Azure Synapse Analytics (formerly Azure SQL DW)](../synapse-analytics/get-started-create-workspace.md) with tables and views that you want to share.

+- [An Azure Data Share account](share-your-data-portal.md#create-a-data-share-account).

+- Your data recipient's Azure sign in e-mail address (using their e-mail alias won't work).

+- If your Azure SQL resource is in a different Azure subscription than your Azure Data Share account, register the [Microsoft.DataShare resource provider](concepts-roles-permissions.md#resource-provider-registration) in the subscription where your source Azure SQL resource is located.

+### Source-specific prerequisites

+There are also prerequisites for sharing that depend on where your data is coming from. Select your data share source and follow the steps:

+- [Azure SQL Database or Azure Synapse Analytics (formerly Azure SQL DW)](#prerequisitesforsharingazuresqlorsynapse)

+- [Azure Synapse Analytics (workspace) SQL pool](#prerequisitesforsharingazuresynapseworkspace)

+- [Azure Active Directory authentication](#azure-active-directory-authentication)

+- [SQL authentication](#sql-authentication)

+- You'll need permission to write to the databases on SQL server: *Microsoft.Sql/servers/databases/write*. This permission exists in the Azure RBAC **Contributor** role.

+- SQL Server **Azure Active Directory Admin** permissions.

+- SQL Server Firewall access:

+- Permission to write to the databases on SQL server: *Microsoft.Sql/servers/databases/write*. This permission exists in the **Contributor** role.

+- Permission for the Azure Data Share resource's managed identity to access the database:

+ 1. Connect to the Azure SQL Database/Data Warehouse using the [Query Editor](../azure-sql/database/connect-query-portal.md#connect-using-azure-active-directory) or SQL Server Management Studio with Azure Active Directory authentication.

+ 1. Execute the following script to add the Data Share resource-Managed Identity as a db_datareader. Connect using Active Directory and not SQL Server authentication.

+ ```

+ > The *<share_acc_name>* is the name of your Data Share resource.

+- An Azure SQL Database User with **'db_datareader'** access to navigate and select the tables or views you wish to share.

+- SQL Server Firewall access:

+ 1. Select **Save**.

+- Permission to write to the SQL pool in Synapse workspace: *Microsoft.Synapse/workspaces/sqlPools/write*. This permission exists in the **Contributor** role.

+- Permission for the Data Share resource's managed identity to access Synapse workspace SQL pool:

+ 1. Select **Develop** from the left navigation in the Synapse Studio. Execute the following script in SQL pool to add the Data Share resource-Managed Identity as a db_datareader.

+ ```

+- Synapse workspace Firewall access:

+ 1. Select **Save**.

+ 

+1. Select **Create**.

+1. Fill out the details for your share. Specify a name, share type, description of share contents, and terms of use (optional).

+ 

+1. To add Datasets to your share, select **Add Datasets**.

+1. Select the dataset type that you would like to add. There will be a different list of dataset types depending on the share type (snapshot or in-place) you selected in the previous step.

+ 

+1. Select your SQL server or Synapse workspace. If you're using Azure Active Directory authentication and the checkbox **Allow Data Share to run the above 'create user' SQL script on my behalf** appears, check the checkbox. If you're using SQL authentication, provide credentials, and be sure you've followed the prerequisites so that you have permissions.

+ Select **Next** to navigate to the object you would like to share and select 'Add Datasets'. You can select tables and views from Azure SQL Database and Azure Synapse Analytics (formerly Azure SQL DW), or tables from Azure Synapse Analytics (workspace) dedicated SQL pool.

+ 

+1. In the Recipients tab, enter in the email addresses of your Data Consumer by selecting '+ Add Recipient'. The email address needs to be recipient's Azure sign in email.

+ 

+1. If you have selected snapshot share type, you can configure snapshot schedule to provide updates of your data to your data consumer.

+ 

+1. Select a start time and recurrence interval.

+Your Azure Data Share has now been created and the recipient of your Data Share can now accept your invitation.

+- Azure Subscription: If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.

+- A Data Share invitation: An invitation from Microsoft Azure with a subject titled "Azure Data Share invitation from **<yourdataprovider@domain.com>**".

+- Register the [Microsoft.DataShare resource provider](concepts-roles-permissions.md#resource-provider-registration) in the Azure subscription where you'll create a Data Share resource and the Azure subscription where your target Azure data stores are located.

+- You'll need a resource in Azure to store the shared data. You can use these kinds of resources:

+ - [Azure Storage](../storage/common/storage-account-create.md)

+ - [Azure SQL Database](../azure-sql/database/single-database-create-quickstart.md)

+ - [Azure Synapse Analytics (formerly Azure SQL DW)](../synapse-analytics/get-started-create-workspace.md)

+ - [Azure Synapse Analytics (workspace) dedicated SQL pool](../synapse-analytics/get-started-analyze-sql-pool.md)

+There are also prerequisites for the resource where the received data will be stored.

+- [Azure Storage prerequisites](#prerequisites-for-target-storage-account)

+- [Azure SQL Database or Azure Synapse Analytics (formerly Azure SQL DW) prerequisites](#prerequisitesforreceivingtoazuresqlorsynapse)

+- [Azure Synapse Analytics (workspace) SQL pool prerequisites](#prerequisitesforreceivingtoazuresynapseworkspacepool)

+- An [Azure Storage account](../storage/common/storage-account-create.md).

+- Permission to write to the storage account: *Microsoft.Storage/storageAccounts/write*. This permission exists in the Azure RBAC **Contributor** role.

+- Permission to add role assignment of the Data Share resource's managed identity to the storage account: which is present in *Microsoft.Authorization/role assignments/write*. This permission exists in the Azure RBAC **Owner** role.

+### Prerequisites for receiving data into Azure SQL Database or Azure Synapse Analytics (formerly Azure SQL DW)

+- An [Azure SQL Database](../azure-sql/database/single-database-create-quickstart.md) or [Azure Synapse Analytics (formerly Azure SQL DW)](../synapse-analytics/get-started-create-workspace.md).

+- Permission to write to the databases on SQL server: *Microsoft.Sql/servers/databases/write*. This permission exists in the Azure RBAC **Contributor** role.

+- SQL Server Firewall access:

+ 1. Select **Save**.

+For a SQL server where you're **not** the **Azure Active Directory admin**, complete these prerequisites before accepting a data share:

+- An [Azure SQL Database](../azure-sql/database/single-database-create-quickstart.md) or [Azure Synapse Analytics (formerly Azure SQL DW)](../synapse-analytics/get-started-create-workspace.md).

+- Permission to write to databases on the SQL server: *Microsoft.Sql/servers/databases/write*. This permission exists in the Azure RBAC **Contributor** role.

+- Permission for the Data Share resource's managed identity to access the Azure SQL Database or Azure Synapse Analytics:

+ 1. Connect to the Azure SQL Database/Data Warehouse using the [Query Editor](../azure-sql/database/connect-query-portal.md#connect-using-azure-active-directory) or SQL Server Management Studio with Azure Active Directory authentication.

+ ```

+ > The *<share_acc_name>* is the name of your Data Share resource.

+- SQL Server Firewall access:

+ 1. Select **Save**.

+- An Azure Synapse Analytics (workspace) dedicated SQL pool. Receiving data into serverless SQL pool isn't currently supported.

+- Permission to write to the SQL pool in Synapse workspace: *Microsoft.Synapse/workspaces/sqlPools/write*. This permission exists in the Azure RBAC **Contributor** role.

+- Permission for the Data Share resource's managed identity to access the Synapse workspace SQL pool:

+ 1. In the [Azure portal](https://portal.azure.com/), navigate to Synapse workspace.

+ 1. In Synapse Studio, select **Develop** from the left navigation. Execute the following script in SQL pool to add the Data Share resource-Managed Identity as a 'db_datareader, db_datawriter, db_ddladmin'.

+ ```

+- Synapse workspace Firewall access:

+ 1. Select **+Add client IP**. Client IP address is subject to change. This process might need to be repeated the next time you're sharing SQL data from Azure portal.

+ 1. Select **Save**.

+You can open invitation from email or directly from the [Azure portal](https://portal.azure.com/).

+To open an invitation from email, check your inbox for an invitation from your data provider. The invitation is from Microsoft Azure, titled **Azure Data Share invitation from <yourdataprovider@domain.com>**. Select **View invitation** to see your invitation in Azure.

+

+Then, select the share you would like to view.

+1. Make sure all fields are reviewed, including the **Terms of Use**. If you agree to the terms of use, you'll be required to check the box to indicate you agree.

+ 

+1. Under *Target Data Share Account*, select the Subscription and Resource Group that you'll be deploying your Data Share into.

+1. For the **Data Share Account** field, select **Create new** if you don't have an existing Data Share account. Otherwise, select an existing Data Share account that you'd like to accept your data share into.

+1. For the **Received Share Name** field, you may leave the default specified by the data provide, or specify a new name for the received share.

+1. Once you've agreed to the terms of use and specified a Data Share account to manage your received share, Select **Accept and configure**. A share subscription will be created.

+ 

+If you don't want to accept the invitation, Select *Reject*.

+1. Select **Datasets** tab. Check the box next to the dataset you'd like to assign a destination to. Select **+ Map to target** to choose a target data store.

+ 

+ 

+1. For snapshot-based sharing, if the data provider has created a snapshot schedule to provide regular updates to the data, you can also enable snapshot schedule by selecting the **Snapshot Schedule** tab. Check the box next to the snapshot schedule and select **+ Enable**.

+1. You can trigger a snapshot by selecting **Details** tab followed by **Trigger snapshot**. Here, you can trigger a full snapshot of your data. If it's your first time receiving data from your data provider, select full copy. When a snapshot is executing, the next snapshots won't start until the previous one is complete.

+ 

+1. When the last run status is *successful*, go to target data store to view the received data. Select **Datasets**, and select the link in the Target Path.

+ 

+This step only applies to snapshot-based sharing. To view history of your snapshots, select **History** tab. Here you'll find history of all snapshots that were generated for the past 30 days.

+SQL snapshot performance is impacted by many factors. It's always recommended to conduct your own performance testing. Below are some example factors impacting performance.

+- Source or destination data store input/output operations per second (IOPS) and bandwidth.

+- Hardware configuration (For example: vCores, memory, DWU) of the source and target SQL data store.

+- Concurrent access to the source and target data stores. If you're sharing multiple tables and views from the same SQL data store, or receive multiple tables and views into the same SQL data store, performance will be impacted.

+- Network bandwidth between the source and destination data stores, and location of source and target data stores.

+- Size of the tables and views being shared. SQL snapshot sharing does a full copy of the entire table. If the size of the table grows over time, snapshot will take longer.

+The most common cause of snapshot failure is that Data Share doesn't have permission to the source or target data store. In order to grant Data Share permission to the source or target Azure SQL Database or Azure Synapse Analytics (formerly Azure SQL DW), you must run the provided SQL script when connecting to the SQL database using Azure Active Directory authentication. To troubleshoot other SQL snapshot failures, refer to [Troubleshoot snapshot failure](data-share-troubleshoot.md#snapshots).

+You've learned how to share and receive data from SQL sources using Azure Data Share service. To learn more about sharing from other data sources, continue to [supported data stores](supported-data-stores.md).

+[Azure Data Share](overview.md) allows you to securely share data snapshots from your Azure storage resources to other Azure subscriptions. Including Azure subscriptions outside your tenant.

+This article describes sharing data from **Azure Blob Storage**, **Azure Data Lake Storage Gen1**, and **Azure Data Lake Storage Gen2**.

+However, Azure Data Share also allows sharing from these other kinds of resources:

+- [Azure SQL Database and Azure Synapse Analytics](how-to-share-from-sql.md)

+- [Azure Data Explorer](/data-explorer/data-share.md)

+This article will guide you through:

+- [What kinds of data can be shared](#whats-supported)

+- [How to prepare your environment](#prerequisites-to-share-data)

+- [How to create a share](#create-a-share)

+- [How to receive shared data](#receive-shared-data)

+You can use the table of contents to jump to the section you need, or continue with this article to follow the process from start to finish.

+## What's supported

+Azure Data Share supports sharing data from Azure Data Lake Gen1, Azure Data Lake Gen2, and Azure storage.

+|Resource type | Sharable resource |

+|-|--

+|Azure Data Lake Gen1 and Gen2 |Files |

+||Folders|

+||File systems|

+|Azure Storage |*Blobs |

+||Folders|

+||Containers|

+>[!NOTE]

+> *Block, append, and page blobs are all supported. However, when they are shared they will be received as **block blobs**.

+Data shared from these sources can be received by Azure Data Lake Gen2 or Azure Blob Storage.

+### Share behavior

+For file systems, containers, or folders, you can choose to make full or incremental snapshots of your data.

+A **full snapshot** copies all specified files and folders at every snapshot.

+An **incremental snapshot** copies only new or updated files, based on the last modified time of the files.

+Existing files that have the same name are overwritten during a snapshot. A file that is deleted from the source isn't deleted on the target. Empty subfolders at the source aren't copied over to the target.

+## Prerequisites to share data

+- If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.

+- [An Azure Data Share account](share-your-data-portal.md#create-a-data-share-account).

+- Your data recipient's Azure sign in e-mail address (using their e-mail alias won't work).

+- If your Azure SQL resource is in a different Azure subscription than your Azure Data Share account, register the [Microsoft.DataShare resource provider](concepts-roles-permissions.md#resource-provider-registration) in the subscription where your source Azure SQL resource is located.

+### Prerequisites for the source storage account

+- An Azure Storage account. If you don't already have an account, [create one](../storage/common/storage-account-create.md).

+- Permission to write to the storage account. Write permission is in *Microsoft.Storage/storageAccounts/write*. It's part of the Contributor role.

+- Permission to add role assignment to the storage account. This permission is in *Microsoft.Authorization/role assignments/write*. It's part of the Owner role.

+1. Sign in to the [Azure portal](https://portal.azure.com/).

+1. Select **Create**.

+1. Provide the details for your share. Specify a name, share type, description of share contents, and terms of use (optional).

+ 

+1. To add datasets to your share, select **Add Datasets**.

+1. Select a dataset type to add. The list of dataset types depends on whether you selected snapshot-based sharing or in-place sharing in the previous step.

+ 

+1. Go to the object you want to share. Then select **Add Datasets**.

+ 

+1. On the **Recipients** tab, add the email address of your data consumer by selecting **Add Recipient**.

+ 

+1. If you selected a snapshot share type, you can set up the snapshot schedule to update your data for the data consumer.

+ 

+1. Select a start time and recurrence interval.

+You've now created your Azure data share. The recipient of your data share can accept your invitation.

+## Prerequisites to receive data

+Before you accept a data share invitation, make sure you have the following prerequisites:

+- An Azure subscription. If you don't have a subscription, create a [free account](https://azure.microsoft.com/free/).

+- An invitation from Azure. The email subject should be "Azure Data Share invitation from *\<yourdataprovider\@domain.com>*".

+- A registered [Microsoft.DataShare resource provider](concepts-roles-permissions.md#resource-provider-registration) in:

+ - The Azure subscription where you'll create a Data Share resource.

+ - The Azure subscription where your target Azure data stores are located.

+- An Azure Storage account. If you don't already have one, [create an account](../storage/common/storage-account-create.md).

+- Permission to write to the storage account. This permission is in *Microsoft.Storage/storageAccounts/write*. It's part of the Contributor role.

+- Permission to add role assignment to the storage account. This assignment is in *Microsoft.Authorization/role assignments/write*. It's part of the Owner role.

+## Receive shared data

+You can open an invitation from email or directly from the [Azure portal](https://portal.azure.com/).

+1. To open an invitation from email, check your inbox for an invitation from your data provider. The invitation from Microsoft Azure is titled "Azure Data Share invitation from *\<yourdataprovider\@domain.com>*". Select **View invitation** to see your invitation in Azure.

+ If you're a guest user of a tenant, you'll be asked to verify your email address for the tenant prior to viewing Data Share invitation for the first time. Once verified, it's valid for 12 months.

+ 

+1. Select the share you want to view.

+1. Review all of the fields, including the **Terms of use**. If you agree to the terms, select the check box.

+ 

+ - In the **Data share account** field, select **Create new** if you don't have a Data Share account. Otherwise, select an existing Data Share account that will accept your data share.

+ - In the **Received share name** field, either leave the default that the data provider specified or specify a new name for the received share.

+1. Select **Accept and configure**. A share subscription is created.

+ 

+ The received share appears in your Data Share account.

+ If you don't want to accept the invitation, select **Reject**.

+1. On the **Datasets** tab, select the check box next to the dataset where you want to assign a destination. Select **Map to target** to choose a target data store.

+ 

+1. Select a target data store for the data. Files in the target data store that have the same path and name as files in the received data will be overwritten.

+ 

+1. For snapshot-based sharing, if the data provider uses a snapshot schedule to regularly update the data, you can enable the schedule from the **Snapshot Schedule** tab. Select the box next to the snapshot schedule. Then select **Enable**. The first scheduled snapshot will start within one minute of the schedule time and subsequent snapshots will start within seconds of the scheduled time.

+1. You can trigger a snapshot from the **Details** tab. On the tab, select **Trigger snapshot**. You can choose to trigger a full snapshot or incremental snapshot of your data. If you're receiving data from your data provider for the first time, select **Full copy**. When a snapshot is executing, subsequent snapshots won't start until the previous one complete.

+ 

+1. When the last run status is *successful*, go to the target data store to view the received data. Select **Datasets**, and then select the target path link.

+ 

+You can view the history of your snapshots only in snapshot-based sharing. To view the history, open the **History** tab. Here you see the history of all of the snapshots that were generated in the past 30 days.

+Storage snapshot performance is impacted by many factors in addition to number of files and size of the shared data. It's always recommended to conduct your own performance testing. Below are some example factors impacting performance.

+- Concurrent access to the source and target data stores.

+- Location of source and target data stores.

+- For incremental snapshot, the number of files in the shared dataset can impact the time it takes to find the list of files with last modified time after the last successful snapshot.

+You've learned how to share and receive data from a storage account by using the Azure Data Share service. To learn about sharing from other data sources, see the [supported data stores](supported-data-stores.md).

+1. **Source SQL Server**: SQL Server instance on-premises, private cloud, or any public cloud virtual machine. All editions of SQL Server 2016 and above are supported.

+- SQL Server 2014 and below are not supported.

+ - Owner or Contributor role for the Azure subscription (required if creating a new DMS service).

+ - Owner or Contributor role for the Azure subscription (required if creating a new DMS service).

+Use the Azure SQL Migration extension in Azure Data Studio to migrate the databases from a SQL Server instance (SQL Server 2016 and above) to a [SQL Server on Azure Virtual Machine](../azure-sql/virtual-machines/windows/sql-server-on-azure-vm-iaas-what-is-overview.md) with minimal downtime. For methods that may require some manual effort, see the article [SQL Server instance migration to SQL Server on Azure Virtual Machine](../azure-sql/migration-guides/virtual-machines/sql-server-to-sql-on-azure-vm-migration-overview.md).

+Use the Azure SQL Migration extension in Azure Data Studio to migrate the databases from a SQL Server instance (SQL Server 2016 and above) to a [SQL Server on Azure Virtual Machine](../azure-sql/virtual-machines/windows/sql-server-on-azure-vm-iaas-what-is-overview.md) with minimal downtime. For methods that may require some manual effort, see the article [SQL Server instance migration to SQL Server on Azure Virtual Machine](../azure-sql/migration-guides/virtual-machines/sql-server-to-sql-on-azure-vm-migration-overview.md).

+The Azure DNS private zones auto registration feature manages DNS records for virtual machines deployed in a virtual network. When you [link a virtual network](./private-dns-virtual-network-links.md) with a private DNS zone with this setting enabled, a DNS record gets created for each virtual machine deployed in the virtual network.

+* For common questions and answers about private zones in Azure DNS, including specific behavior you can expect for certain kinds of operations, see [Private DNS FAQ](./dns-faq-private.yml).

+> - The destination storage (Azure Storage or Azure Data Lake Storage) account must be in the same subscription as the event hub.

+> - Event Hubs doesn't support capturing events in a **premium** storage account.

+You can enable or disable emitting empty files when no events occur during the Capture window.

+> - The destination storage (Azure Storage or Azure Data Lake Storage) account must be in the same subscription as the event hub.

+> - Event Hubs doesn't support capturing events in a **premium** storage account.

+The Event Hubs Premium (premium tier) is designed for high-end streaming scenarios that require elastic, superior performance with predictable latency. The performance is achieved by providing reserved compute, memory, and storage resources, which minimize cross-tenant interference in a managed multi-tenant PaaS environment.

+It replicates events to three replicas, distributed across Azure availability zones where available. All replicas are synchronously flushed to the underlying fast storage before the send operation is reported as completed. Events that aren't read immediately or that need to be re-read later can be retained up to 90 days, transparently held in an availability-zone redundant storage tier.

+In addition to these storage-related features and all capabilities and protocol support of the standard tier, the isolation model of the premium tier enables features like [dynamic partition scale-up](dynamically-add-partitions.md). You also get far more generous [quota allocations](event-hubs-quotas.md). Event Hubs Capture is included at no extra cost.

+> Event Hubs Premium supports TLS 1.2 or greater .

+## Why premium?

+The premium tier offers three compelling benefits for customers who require better isolation in a multitenant environment with low latency and high throughput data ingestion needs.

+### Superior performance with the new two-tier storage engine

+The premium tier uses a new two-tier log storage engine that drastically improves the data ingress performance with substantially reduced overall latency without compromising the durability guarantees.

+### Better isolation and predictability

+The premium tier offers an isolated compute and memory capacity to achieve more predictable latency and far reduced *noisy neighbor* impact risk in a multi-tenant deployment.

+It implements a *cluster in cluster* model in its multitenant clusters to provide predictability and performance while retaining all the benefits of a managed multitenant PaaS environment.

+### Cost savings and scalability

+As the premium tier is a multitenant offering, it can dynamically scale more flexibly and very quickly. Capacity is allocated in processing units (PUs) that allocate isolated pods of CPU/memory inside the cluster. The number of those pods can be scaled up/down per namespace. Therefore, the premium tier is a low-cost option for messaging scenarios with the overall throughput range that is less than 120 MB/s but higher than what you can achieve with the standard SKU.

+## Premium vs. dedicated tiers

+In comparison to the dedicated offering, the premium tier provides the following benefits:

+- Isolation inside a very large multi-tenant environment that can shift resources quickly

+- Scale far more elastically and quicker

+- PUs can be dynamically adjusted

+Therefore, the premium tier is often a more cost effective option for mid-range (<120MB/sec) throughput requirements, especially with changing loads throughout the day or week, when compared to the dedicated tier.

+For the extra robustness gained by availability-zone support, the minimal deployment scale for the dedicated tier is 8 capacity units (CU), but you'll have availability zone support in the premium tier from the first PU in all availability zone regions.

+You can purchase 1, 2, 4, 8 and 16 processing units for each namespace. As the premium tier is a capacity-based offering, the achievable throughput isn't set by a throttle as it's' in the standard tier, but depends on the work you ask Event Hubs to do, similar to the dedicated tier. The effective ingest and stream throughput per PU will depend on various factors, including:

+* Number of producers and consumers

+* Payload size

+* Partition count

+* Egress request rate

+* Usage of Event Hubs Capture, Schema Registry, and other advanced features

+For more information, see [comparison between Event Hubs SKUs](event-hubs-quotas.md).

+## Encryption of events

+Azure Event Hubs provides encryption of data at rest with Azure Storage Service Encryption (Azure SSE). The Event Hubs service uses Azure Storage to store the data. All the data that's stored with Azure Storage is encrypted using Microsoft-managed keys. If you use your own key (also referred to as Bring Your Own Key (BYOK) or customer-managed key), the data is still encrypted using the Microsoft-managed key, but in addition the Microsoft-managed key will be encrypted using the customer-managed key. This feature enables you to create, rotate, disable, and revoke access to customer-managed keys that are used for encrypting Microsoft-managed keys. Enabling the BYOK feature is a one time setup process on your namespace. For more information, see [Configure customer-managed keys for encrypting Azure Event Hubs data at rest](configure-customer-managed-key.md).

+> [!NOTE]

+> All Event Hubs namespaces are enabled for the Apache Kafka RPC protocol by default can be used by your existing Kafka based applications. Having Kafka enabled on your cluster does not affect your non-Kafka use cases; there is no option or need to disable Kafka on a cluster.

+## Pricing

+The Premium offering is billed by [Processing Units (PUs)](event-hubs-scalability.md#processing-units) which correspond to a share of isolated resources (CPU, Memory, and Storage) in the underlying infrastructure.

+See the following articles:

+- [Create an event hub](event-hubs-create.md). Select **Premium** for **Pricing tier**.

+- [Event Hubs Premium pricing](https://azure.microsoft.com/pricing/details/event-hubs/) for more details on pricing

+- [Event Hubs FAQ](event-hubs-faq.yml) to find answers to some frequently asked questions about Event Hubs.

+The solution is illustrated in the following diagram. As illustrated, you can architect the scenario either using more specific route (Option 1) or AS-path prepend (Option 2) to influence VNet path selection. To influence on-premises network route selection for Azure bound traffic, you need configure the interconnection between the on-premises location as less preferable. How you configure the interconnection link as preferable depends on the routing protocol used within the on-premises network. You can use local preference with iBGP or metric with IGP (OSPF or IS-IS).

+## Next steps

+Advance to the next article to learn how to add a custom domain to your Front Door.

+> [!div class="nextstepaction"]

+> [Add a custom domain](how-to-add-custom-domain.md)

+## HDInsight 3.6 to 4.0 Migration Guides

+- [Migrate Apache Spark 2.1 and 2.2 workloads to 2.3 and 2.4](spark/migrate-versions.md).

+- [Migrate Azure HDInsight 3.6 Hive workloads to HDInsight 4.0](interactive-query/apache-hive-migrate-workloads.md).

+- [Migrate Apache Kafka workloads to Azure HDInsight 4.0](kafk).

+- [Migrate an Apache HBase cluster to a new version](hbase/apache-hbase-migrate-new-version.md).

+HDInsight 3.6 will continue to run on Ubuntu 16.04. It will reach the end of standard support by 30 June 2021, and will change to Basic support starting on 1 July 2021. For more information about dates and support options, see [Azure HDInsight versions](./hdinsight-component-versioning.md). Ubuntu 18.04 won't be supported for HDInsight 3.6. If youΓÇÖd like to use Ubuntu 18.04, youΓÇÖll need to migrate your clusters to HDInsight 4.0. Spark 3.0 with HDInsight 4.0 is available only on Ubuntu 16.04. Spark 3.1 with HDInsight 4.0 will be shipping soon and will be available on Ubuntu 18.04.

+| patch (conditional) | Yes | Yes | Support for [JSON Patch](https://www.hl7.org/fhir/http.html#patch) only. We have included a workaround to use JSON Patch in a bundle in [this PR](https://github.com/microsoft/fhir-server/pull/2143).

+| [$export](../../healthcare-apis/data-transformation/export-data.md) | Yes | Yes | Supports system, group, and patient. |

+* [**Request Units (RUs)**](../../cosmos-db/concepts-limits.md) - You can configure up to 10,000 RUs in the portal for Azure API for FHIR. You will need a minimum of 400 RUs or 40 RUs/GB, whichever is larger. If you need more than 10,000 RUs, you can put in a support ticket to have the RUs increased. The maximum available is 1,000,000. In addition, we support [autoscaling of RUs](autoscale-azure-api-fhir.md).

+* **Subscription Limit** - By default, each subscription is limited to a maximum of 10 FHIR server instances. If you need more instances per subscription, open a support ticket and provide details about your needs.

+PUT {{FHIR_URL}}/SearchParameter/{SearchParameter ID}

+The autoscale feature adjusts computing resources automatically to optimize the overall service scalability. It requires no action from customers.

+When transaction workloads are high, the autoscale feature increases computing resources automatically. When transaction workloads are low, it decreases computing resources accordingly. Whether you are performing read requests that include simple queries like getting patient information using a patient ID, and advanced queries like getting all DiagnosticReport resources for patients whose name is "Sarah", or you're creating or updating FHIR resources, the autoscale feature manages the dynamics and complexity of resource allocation to ensure high scalability.

+PUT {{FHIR_URL}}/SearchParameter/{SearchParameter ID}

+Workspace names can be re-used in the same Azure subscription, but not in a different Azure subscription, after deletion. However, when the move operation is supported and enabled, workspaces and its child resources can be moved from one subscription to another subscription if certain requirements are met. One requirement is that the two subscriptions must be part of the same Azure Active Directory (Azure AD) tenant. Another requirement is that the Private Link configuration is not enabled. Names for FHIR services, DICOM services and IoT connectors can be re-used in the same or different subscription after deletion if there is no collision with the URLs of any existing services.

+>[Deploy workspace in the Azure portal](healthcare-apis-quickstart.md)

+> The quotas and limits described in this article apply to the new multiple IoT hub architecture. Currently, there are a few legacy IoT Central applications that were created before April 2021 that haven't yet been migrated to the multiple IoT hub architecture. Use the `az iot central device manual-failover` command in the [Azure CLI](/cli/azure/?view=azure-cli-latest&preserve-view=true) to check if your application still uses a single IoT hub. This triggers an IoT hub failover if your application uses the multiple IoT hub architecture. It returns an error if your application uses the older architecture.

+| Number of capabilities in a device template | 300 | For performance reasons, you shouldn't exceed this limit. |

+> [!TIP]

+> This is a personal preference that only applies to you.

+You can set an organization as the default organization to use in your application as a personal preference. The default organization becomes the default option whenever you choose an organization, such as when you add a new user or add a device to your IoT Central application.

+The following limits apply to organizations:

+In the example described in the following sections, the downstream device sends JSON data in the following format to the IoT Edge gateway device:

+```json

+{

+ "device": {

+ "deviceId": "<downstream-deviceid>"

+ },

+ "measurements": {

+ "temp": <temperature>,

+ "pressure": <pressure>,

+ "humidity": <humidity>,

+ "scale": "celsius",

+ }

+}

+You want to use an IoT Edge module to transform the data and convert the temperature value from `Celsius` to `Fahrenheit` before sending it to IoT Central:

+ "scale": "fahrenheit"

+1. Select the **Bash** shell.

+ Make a note of the `username` and `password` values, you use them later. You only need one of the passwords shown in the command output.

+1. In the [Azure Cloud Shell](https://shell.azure.com/), create a new folder and navigate to it by running the following commands:

+ ```azurecli

+ mkdir yournewfolder

+ cd yournewfolder

+ ```

+1. Select **IoT Edge gateway device** and select **Create a device**. Enter *IoT Edge gateway device* as the device name, enter *gateway-01* as the device ID, make sure **IoT Edge gateway device** is selected as the device template and **No** is selected as **Simulate this device?**. Select **Create**.

+1. Don't select a device template. Select **+ New**. Enter *Downstream 01* as the device name, enter *downstream-01* as the device ID, make sure that the device template is **Unassigned** and **No** is selected as **Simulate this device?**. Select **Create**.

+To check that the IoT Edge gateway device is running correctly:

+1. Save the changes and run the following command to verify that the *config.yaml* file is correct:

+ ```bash

+ sudo iotedge check

+ ```

+1. Restart the IoT Edge runtime:

+ During `sudo apt install nodejs npm node-typescript` commands, you could be asked to allow installations: press `Y` if prompted.

+The temperature is sent in Fahrenheit. Because the IoT Edge device is transforming the data from the downstream device, the telemetry is associated with the gateway device in IoT Central. To visualize the transformed telemetry, create a view in the **IoT Edge gateway device** template and republish it.

+To manage IoT Edge devices, [create and edit deployment manifests](concepts-iot-edge.md#iot-edge-deployment-manifests-and-iot-central-device-templates) and deploy them onto the device directly from IoT Central. You can also run commands on modules from within IoT Central.

+description: Azure IoT Central is an IoT application platform that simplifies the creation of IoT solutions. It helps to reduce the burden and cost of IoT management operations, and development. This article provides an overview of the features of Azure IoT Central.

+IoT Central is an IoT application platform that reduces the burden and cost of developing, managing, and maintaining enterprise-grade IoT solutions. If you choose to build with IoT Central, you'll have the opportunity to focus time, money, and energy on transforming your business with IoT data, rather than just maintaining and updating a complex and continually evolving IoT infrastructure.

+This article provides an overview of IoT Central and describes its core functionality.

+## Create an IoT Central application

+- [Healthcare](../healthcare/tutorial-continuous-patient-monitoring.md)

+After you create your application, the next step is to create and connect devices. Every device connected to IoT Central uses a _device template_. A device template is the blueprint that defines the characteristics and behavior of a type of device such as the:

+- Properties that an operator sets, that determine the behavior of the device. For example, a target temperature for the device.

+- Commands that an operator can call, that run on a device. For example, a command to remotely reboot a device.

+Before you begin, you should complete the previous quickstart [Connect your first device](./quick-deploy-iot-central.md). It shows you how to create an Azure IoT Central application and connect the **IoT Plug and Play** smartphone app to it.

+1. In the **Conditions** section, you define what triggers your rule. Use the following information to define a single condition based on accelerometer z-axis telemetry. This rule uses aggregation, so you receive a maximum of one email for each device every five minutes:

+1. On the **Cloud Properties** tab, change the **Acidity (pH) threshold** value to **9** and select **Save**.

+You can use the IoT Central in-store analytics condition monitoring application template to build an end-to-end solution. The application template lets you digitally connect to and monitor a retail store environment using different kinds of sensor devices. These sensor devices generate telemetry that you can convert into business insights to help the retailer reduce operating costs and create a great experience for their customers.

+1. Set of IoT sensors sending telemetry data to a gateway device.

+1. Gateway devices sending telemetry and aggregated insights to IoT Central.

+1. Continuous data export to the desired Azure service for manipulation.

+1. Data can be structured in the desired format and sent to a storage service.

+1. Business applications can query data and generate insights that power retail operations.

+1. Select **Administration > Your Application**.

+You can create device templates that enable you and the application operators to configure and manage devices. You can create a template by building a custom one, by importing an existing template file, or by importing a template from the Azure IoT device catalog. After you create and customize a device template, use it to connect real devices to your application. Optionally, use a device template to generate simulated devices for testing.

+1. Select **Next: Review**.

+1. Scroll in the list of capabilities and find the `RelativeHumidity` telemetry type. It's the row item with the editable **Display name** value of *RelativeHumidity*.

+In the following steps, you customize the `RelativeHumidity` telemetry type for the RuuviTag sensors. Optionally, customize some of the other telemetry types.

+For the `RelativeHumidity` telemetry type, make the following changes:

+1. Update the **Display Name** value from *RelativeHumidity* to a custom value such as *Humidity*.

+1. Change the **Semantic Type** option from *Relative humidity* to *Humidity*. Optionally, set schema values for the humidity telemetry type in the expanded schema view. Schema settings allow you to create detailed validation requirements for the data that your sensors track. For example, you could set minimum and maximum operating range values for a given interface.

+1. Choose the RuuviTag device template in **Target devices**. The rule you define will apply to all sensors based on that template. Optionally, you could create a filter that would apply the rule only to a defined subset of the sensors.

+1. Choose `Humidity` as the **Telemetry**. It's the device capability that you customized in a previous step.

+* Download the [Azure_RTOS_6.1_ATSAME54-XPRO_IAR_Samples_2020_10_10.zip](https://github.com/azure-rtos/samples/releases/download/v6.1_rel/Azure_RTOS_6.1_ATSAME54-XPRO_IAR_Samples_2021_11_03.zip) file and extract it to a working directory. Choose a directory with a short path to avoid compiler errors when you build.

+* Download the [Azure_RTOS_6.1_ATSAME54-XPRO_MPLab_Samples_2020_10_10.zip](https://github.com/azure-rtos/samples/releases/download/v6.1_rel/Azure_RTOS_6.1_ATSAME54-XPRO_MPLab_Samples_2021_11_03.zip) file and unzip it to a working directory. Choose a directory with a short path to avoid compiler errors when you build.

+>There is a third auto-generated certificate that the IoT Edge security manager creates, the **IoT Edge hub server certificate**. This certificate always has a 30 day lifetime, but is automatically renewed before expiring. The auto-generated CA lifetime value set in the config file doesn't affect this certificate.

+> [Azure IoT Hub Device Streams Video](/shows/Internet-of-Things-Show/Azure-IoT-Hub-Device-Streams)

+For each offer, you define the access that users in your organization will have to work on resources in the customer tenant. This is done through a manifest that specifies the Azure Active Directory (Azure AD) users, groups, and service principals that will have access to customer resources, along with [roles](tenants-users-roles.md#role-support-for-azure-lighthouse) that define their level of access.

+## Public and private plans

+If you want to limit your offer to specific customers, you can publish a private plan. When you do so, the plan can only be purchased for the specific subscription IDs that you provide. For more info, see [Private plans](../../marketplace/private-plans.md).

+> Private plans are not supported with subscriptions established through a reseller of the Cloud Solution Provider (CSP) program.

+In this article, you'll learn how to publish a public or private Managed Service offer to [Azure Marketplace](https://azuremarketplace.microsoft.com) using the [commercial marketplace](../../marketplace/overview.md) program in Partner Center. Customers who purchase the offer will then delegate subscriptions or resource groups, allowing you to manage them through [Azure Lighthouse](../overview.md).

+You need to have a valid [account in Partner Center](../../marketplace/create-account.md) to create and publish offers. If you don't have an account already, the [sign-up process](https://aka.ms/joinmarketplace) will lead you through the steps of creating an account in Partner Center and enrolling in the commercial marketplace program.

+|Can limit offer to specific customers |Yes (only with private plans, which can't be used with subscriptions established through a reseller of the Cloud Solution Provider (CSP) program) |Yes |

+To learn about the general publishing process, review the [commercial marketplace documentation](../../marketplace/overview.md). You should also review the [commercial marketplace certification policies](/legal/marketplace/certification-policies), particularly the [Managed Services](/legal/marketplace/certification-policies#700-managed-services) section.

+Once you've completed all of the sections, your next step is to publish the offer. After you initiate the publishing process, your offer will go through several validation and publishing steps. For more information, see [Review and publish an offer to the commercial marketplace](../../marketplace/review-publish-offer.md)

+You can [publish an updated version of your offer](../../marketplace/update-existing-offer.md) at any time. For example, you may want to add a new role definition to a previously-published offer. When you do so, customers who have already added the offer will see an icon in the [**Service providers**](view-manage-service-providers.md) page in the Azure portal that lets them know an update is available. Each customer will be able to [review the changes and update to the new version](view-manage-service-providers.md#update-service-provider-offers).

+After a customer adds your offer, they can [delegate one or more specific subscriptions or resource groups](view-manage-service-providers.md#delegate-resources), which will be onboarded to Azure Lighthouse. If a customer has accepted an offer but has not yet delegated any resources, they'll see a note at the top of the **Provider offers** section of the [**Service providers**](view-manage-service-providers.md) page in the Azure portal.

+Once the customer delegates a subscription (or one or more resource groups within a subscription), the **Microsoft.ManagedServices** resource provider will be registered for that subscription, and users in your tenant will be able to access the delegated resources according to the authorizations that you defined in your offer.

+- Learn about the [commercial marketplace](../../marketplace/overview.md).

+- HTTP probes do not support probing on the following ports due to security concerns: 19, 21, 25, 70, 110, 119, 143, 220, 993.

+| Validation | Addition of validation for HA ports | A validation was added to ensure that HA port rules and non HA port rules are only configurable when Floating IP is enabled. Previously, this configuration would go through, but not work as intended. No change to functionality was made. You can learn more [here](load-balancer-ha-ports-overview.md#limitations)| June 2020 |

+| numberOfProbes, "Unhealthy threshold" | Health probe configuration property numberOfProbes, otherwise known as "Unhealthy threshold" in Portal, is not respected. Load Balancer health probes will probe up/down immediately after 1 probe regardless of the property's configured value | To reflect the current behavior, please set the value of numberOfProbes ("Unhealthy threshold" in Portal) as 1 |

+* When a workflow is disabled, you can still resubmit runs.

+* When a workflow is disabled, you can still resubmit runs.

+ If you're new to Azure Logic Apps, try this [quickstart](../logic-apps/quickstart-create-first-logic-app-workflow.md), which creates your first logic apps in the Azure portal and focuses more on the basic concepts.

+* If your logic app needs to communicate through a firewall that limits traffic to specific IP addresses, that firewall needs to allow access for *both* the [inbound](logic-apps-limits-and-config.md#inbound) and [outbound](logic-apps-limits-and-config.md#outbound) IP addresses used by Azure Logic Apps or runtime in the Azure region where your logic app exists. If your logic app also uses [managed connectors](../connectors/managed.md), such as the Office 365 Outlook connector or SQL connector, or uses [custom connectors](/connectors/custom-connectors/), the firewall also needs to allow access for *all* the [managed connector outbound IP addresses](logic-apps-limits-and-config.md#outbound) in your logic app's Azure region.

+* Azure Logic Apps continues all in-progress and pending runs until they finish. Based on the volume or backlog, this process might take time to complete.

+* Azure Logic Apps doesn't create or run new workflow instances.

+* The trigger won't fire the next time that its conditions are met.

+* The trigger state remembers the point at which the logic app was stopped. So, if you reactivate the logic app, the trigger fires for all the unprocessed items since the last run.

+ To stop the trigger from firing on unprocessed items since the last run, clear the trigger's state before you reactivate the logic app:

+ 1. In the logic app, edit any part of the workflow's trigger.

+ 1. Save your changes. This step resets your trigger's current state.

+ 1. Reactivate your logic app.

+* When a workflow is disabled, you can still resubmit runs.

+* Azure Logic Apps makes a best effort to cancel any in-progress and pending runs.

+* Azure Logic Apps doesn't create or run new workflow instances.

+ - To learn more about the components of SmartNoise, check out the GitHub repositories for [SmartNoise Core](https://github.com/opendifferentialprivacy/smartnoise-core), [SmartNoise SDK](https://github.com/opendifferentialprivacy/smartnoise-sdk), and [SmartNoise samples](https://github.com/opendifferentialprivacy/smartnoise-samples).

+1. The compute resource receives the job and begins training. The compute resource uses information stored in key vault to access storage accounts to download training files and upload output.

+## Microsoft Sentinel

+Microsoft Sentinel is a security solution that can integrate with Azure Machine Learning. For example, using Jupyter notebooks provided through Azure Machine Learning. For more information, see [Use Jupyter notebooks to hunt for security threats](/azure/sentinel/notebooks).

+### Public access

+Microsoft Sentinel can automatically create a workspace for you if you are OK with a public endpoint. In this configuration, the security operations center (SOC) analysts and system administrators connect to notebooks in your workspace through Sentinel.

+For information on this process, see [Create an Azure ML workspace from Microsoft Sentinel](/azure/sentinel/notebooks?tabs=public-endpoint#create-an-azure-ml-workspace-from-microsoft-sentinel)

+### Private endpoint

+If you want to secure your workspace and associated resources in a VNet, you must create the Azure Machine Learning workspace first. You must also create a virtual machine 'jump box' in the same VNet as your workspace, and enable Azure Bastion connectivity to it. Similar to the public configuration, SOC analysts and administrators can connect using Microsoft Sentinel, but some operations must be performed using Azure Bastion to connect to the VM.

+For more information on this configuration, see [Create an Azure ML workspace from Microsoft Sentinel](/azure/sentinel/notebooks?tabs=private-endpoint#create-an-azure-ml-workspace-from-microsoft-sentinel)

+The service principal can also be used to authenticate to the Azure Machine Learning [REST API](/rest/api/azureml/) (preview). You use the Azure Active Directory [client credentials grant flow](../active-directory/azuread-dev/v1-oauth2-client-creds-grant-flow.md), which allow service-to-service calls for headless authentication in automated workflows.

+> [!IMPORTANT]

+> If you are currently using Azure Active Directory Authentication Library (ADAL) to get credentials, we recommend that you [Migrate to the Microsoft Authentication Library (MSAL)](/azure/active-directory/develop/msal-migration). ADAL support is scheduled to end on June 30, 2022.

+For information and samples on authenticating with MSAL, see the following articles:

+* JavaScript - [How to migrate a Javascript app from ADAL.js to MSAL.js](/azure/active-directory/develop/msal-compare-msal-js-and-adal-js).

+* Node.js - [How to migrate a Node.js app from ADAL to MSAL](/azure/active-directory/develop/msal-node-migration).

+* Python - [ADAL to MSAL migration guide for Python](/azure/active-directory/develop/migrate-python-adal-msal).

+This section creates a manifest with authorization information for Azure Active Directory (Azure AD) user accounts. This information is required in order to enable access to the customer's resources through [Azure Lighthouse](../lighthouse/overview.md).

+2. Enter your **Tenant ID**. This is a GUID associated with the Azure AD tenant ID of your organization; that is, the managing tenant from which you will access your customers' resources. If you don't have this handy, you can find it by hovering over your account name on the upper right-hand side of the Azure portal, or by selecting **Switch directory**.

+Managed Services are Azure Marketplace offers that enable cross-tenant and multi-tenant management with Azure Lighthouse. To learn more, see [What is Azure Lighthouse?](../lighthouse/overview.md) When a customer purchases a Managed Service offer, theyΓÇÖre able to delegate one or more subscription or resource group. You can then work on those resources by using [Azure Lighthouse](../lighthouse/overview.md).

+## Categories

+Select up to three **[Categories](./categories.md)** for grouping your offer into the appropriate marketplace search areas. This table shows the categories that are available for Power BI Visuals.

+| Category | Description |

+| | - |

+| All | All the different types of visuals that are certified for use within your organization. |

+| Change over time | These visuals are used to display the changing trend of measures over time. |

+| Comparison | These visuals are used to compare categories by their measures. |

+| Correlation | These visuals show the degree to which two or more variables are correlated. |

+| Distribution | These visuals show how the values of a variable are distributed. |

+| Flow | These visuals show the dynamic relationships, or flow between variables. |

+| Infographics | These visuals present information graphically, so it's easier to understand. |

+| Maps | Visualize your data in map form. |

+| Part-to-Whole | These visuals are used to display the parts of a variable in relation to the whole. |

+| R visuals | These visuals require R script to run. |

+| KPI | These visuals are used to display key performance indicators. |

+| Filters | Narrow down the data within a report by using filters. |

+| Narratives | Use narratives to tell a story with text and data. |

+| Other | More specialized visuals to discover. |

+|||

+## Industries

+Select up to two **Industries** industries which will be used to display your offer when customers filter their search on industries in the online store. This table shows the industries available for Power BI Visuals.

+| Industry |

+| |

+| Automotive |

+| Defense & Intelligence |

+| Distribution |

+| Education |

+| Energy |

+| Financial Services |

+| Government |

+| Healthcare |

+| Hospitality & Travel |

+| Manufacturing |

+| Media & Communications |

+| Nonprofit & IGO |

+| Professional services |

+| Retail |

+|||

+- [**Offer listing**](power-bi-visual-offer-listing.md)

+1. To validate the end-to-end purchase flow, purchase plans using the _preview URL_ generated during the _Publisher Sign off_ phase of publishing. Note that the customer account used for the purchase will be billed and invoiced. Publisher Payout will occur when the [criteria](/partner-center/payment-thresholds-methods-timeframes) are met and will be paid out per the [payout schedule](/partner-center/payout-policy-details) with the agency fee deducted from the purchase price.

+| **`arosvc.azurecr.io`** | **HTTPS:443** | Global Internal Private registry for ARO Operators. Required if you do not allow the service-endpoints Microsoft.ContainerRegistry on your subnets. |

+| **`arosvc.$REGION.data.azurecr.io`** | **HTTPS:443** | Regional Internal Private registry for ARO Operators. Required if you do not allow the service-endpoints Microsoft.ContainerRegistry on your subnets. |

+ Title: Azure Payment HSM certification and compliance

+description: Information on Azure Payment HSM certification and compliance

+tags: azure-resource-manager

+# Certification and compliance

+Thales payShield 10K HSMs are certified to FIPS 140-2 Level 3 and PCI HSM v3.

+The Azure Payment HSM service is currently undergoing PCI DSS and PCI 3DS audit assessment.

+The Azure Payment HSM can be deployed as part of a validated PCI P2PE and PCI PIN component or solution, Microsoft can provide evidence of proof for customer to meet their P2PE and PIN certification requirements.

+## Next steps

+- Learn more about [Azure Payment HSM](overview.md)

+- See some common [deployment scenarios](deployment-scenarios.md)

+- Read the [frequently asked questions](faq.yml)

+

+ Title: Azure Payment HSM deployment scenarios

+description: Azure HSM deployment scenarios for high availability deployment and disaster recovery deployment

+tags: azure-resource-manager

+# Deployment scenarios

+Microsoft deploys payment hardware security modules (HSM) in stamps within a region and multi-region to enable high availability (HA) and disaster recovery. In a region, HSMs are deployed across different stamps to prevent single rack failure, and customers must provision two devices in a region from two separate stamps in order to achieve high availability. For disaster recovery, customer must provision HSM devices in an alternative region.

+Thales doesn't provide PayShield SDK to customers, which supports HA over a cluster (a collection of HSMs initialized with same LMK). However, the customers usage scenario of the Thales PayShield devices is like a Stateless Server. Thus, no synchronization is required between HSMs during application runtime. Customers handle the HA using their custom client. One implementation would be to load balance between healthy HSMs connected to the application. Customers are responsible for implementing high availability by provisioning multiple devices, load balancing them, and using any kind of available backup mechanism to back up keys.

+## Recommended high availability deployment

+For High Availability, customer must allocate HSM between stamp 1 and stamp 2 (in other words, no two HSMs from same stamp)

+## Recommended disaster recovery deployment

+This scenario caters to regional-level failure. The usual strategy is to completely switch the application stack (and its HSMs), rather than trying to reach an HSM in Region 2 from application in Region 1 due to latency.

+## Next steps

+- Learn more about [Azure Payment HSM](overview.md)

+- Find out how to [get started with Azure Payment HSM](getting-started.md)

+- Learn about [Certification and compliance](certification-compliance.md)

+- Read the [frequently asked questions](faq.yml)

+ Title: Getting started with Azure Payment HSM

+description: Information to begin using Azure Payment HSM

+tags: azure-resource-manager

+# Getting started with Azure Payment HSM

+To get started with Azure Payment HSM (preview), contact your Microsoft sales representative and request access [via email](mailto:paymentHSMRequest@microsoft.com). Upon approval, you'll be provided with onboarding instructions.

+## Availability

+The Azure Public Preview is currently available in **East US** and **North Europe**.

+## Prerequisites

+Azure Payment HSM customers must have:

+- Access to the Thales Customer Portal (Customer ID)

+- Thales smart cards and card reader for payShield Manager

+## Cost

+The HSM devices will be charged based on the service pricing page. All other Azure resources for networking and virtual machines will incur regular Azure costs too.

+## payShield customization considerations

+If you are using payShield on-premise today with a custom firmware, a porting exercise is required to update the firmware to a version compatible with the Azure deployment. Please contact your Thales account manager to request a quote.

+Ensure that the following information is provided:

+- Customization hardware platform (e.g., payShield 9000 or payShield 10K)

+- Customization firmware number

+## Support

+There is no service-level agreement (SLA) for this public preview. Use of this service for production workloads isn't supported

+The HSM base firmware installed in public preview is Thales payShield10K base software version 1.4a 1.8.3.

+Microsoft will provide support for hardware issues, networking issues, and provisioning issues. Support tickets can be created from the Azure portal. Select **Dedicated HSM** as the Service Type, and mention "payment HSM" in the summary field, with a severity case of B or C.

+Support through engineering escalation is only available during business hours: Monday - Friday, 9 AM - 5 PM PST.

+Thales provides application-level support, such as client software, HSM configuration, and backup.

+Customers are responsible for applying payShield security patches and upgrading payShield firmware for their provisioned HSMs. Thales payShield10K versions prior to 1.4a 1.8.3. aren't supported

+Microsoft will apply payShield security patches to unallocated HSMs.

+## Next steps

+- Learn more about [Azure Payment HSM](overview.md)

+- See some common [deployment scenarios](deployment-scenarios.md)

+- Learn about [Certification and compliance](certification-compliance.md)

+- Read the [frequently asked questions](faq.yml)

+

+ Title: What is Azure Payment HSM?

+description: Learn how Azure Payment HSM is an Azure service that provide cryptographic key operations for real-time, critical payment transactions

+tags: azure-resource-manager

+# What is Azure Payment HSM?

+Azure Payment HSM Service is a "BareMetal" service delivered using [Thales payShield 10K payment hardware security modules (HSM)](https://cpl.thalesgroup.com/encryption/hardware-security-modules/payment-hsms/payshield-10k) to provide cryptographic key operations for real-time, critical payment transactions in the Azure cloud. Azure Payment HSM is designed specifically to help a service provider and an individual financial institution accelerate their payment system's digital transformation strategy and adopt the public cloud. It meets the most stringent security, audit compliance, low latency, and high-performance requirements by the Payment Card Industry (PCI).

+Payment HSMs are provisioned and connected directly to users' virtual network, and HSMs are under users' sole administration control. HSMs can be easily provisioned as a pair of devices and configured for high availability. Users of the service utilize [Thales payShield Manager](https://cpl.thalesgroup.com/encryption/hardware-security-modules/payment-hsms/payshield-manager) for secure remote access to the HSMs as part of their Azure-based subscription. Multiple subscription options are available to satisfy a broad range of performance and multiple application requirements that can be upgraded quickly in line with end-user business growth. Azure payment HSM service offers highest performance level 2500 CPS.

+Azure Payment HSM a highly specialized service. Therefore, we recommend that you fully understand the key concepts, including [pricing](https://azure.microsoft.com/services/azure-payment-hsm/) and [support](getting-started.md#support).

+## Why use Azure Payment HSM?

+Momentum is building as financial institutions move some or all of their payment applications to the cloud. This entails a migration from the legacy on-premises (on-prem) applications and HSMs to a cloud-based infrastructure that isn't generally under their direct control. Often it means a subscription service rather than perpetual ownership of physical equipment and software. Corporate initiatives for efficiency and a scaled-down physical presence are the drivers for this. Conversely, with cloud-native organizations, the adoption of cloud-first without any on-premise presence is their fundamental business model. Whatever the reason, end users of a cloud-based payment infrastructure expect reduced IT complexity, streamlined security compliance, and flexibility to scale their solution seamlessly as their business grows.

+The cloud offers significant benefits, but challenges when migrating a legacy on-premise payment application (involving payment HSMs) to the cloud must be addressed. Some of these are:

+- Shared responsibility and trust ΓÇô what potential loss of control in some areas is acceptable?

+- Latency ΓÇô how can an efficient, high-performance link between the application and HSM be achieved?

+- Performing everything remotely ΓÇô what existing processes and procedures may need to be adapted?

+- Security certifications and audit compliance ΓÇô how will current stringent requirements be fulfilled?

+Azure Payment HSM addresses these challenges and delivers a compelling value proposition to users of the service through the following features.

+### Enhanced security and compliance

+End users of the service can leverage Microsoft security and compliance investments to increase their security posture. Microsoft maintains PCI DSS and PCI 3DS compliant Azure data centers, including those which house Azure Payment HSM solutions. The Azure Payment HSM solution can be deployed as part of a validated PCI P2PE / PCI PIN component or solution, helping to simplify ongoing security audit compliance. Thales payShield 10K HSMs deployed in the security infrastructure are certified to FIPS 140-2 Level 3 and PCI HSM v3.

+

+### Customer-managed HSM in Azure

+The Azure Payment HSM is a part of a subscription service that offers single-tenant HSMs for the service customer to have complete administrative control and exclusive access to the HSM. The customer could be a payment service provider acting on behalf of multiple financial institutions or a financial institution that wishes to directly access the Azure Payment HSM service. Once the HSM is allocated to a customer, Microsoft has no access to customer data. Likewise, when the HSM is no longer required, customer data is zeroized and erased as soon as the HSM is released to ensure complete privacy and security is maintained. The customer is responsible for ensuring sufficient HSM subscriptions are active to meet their requirements for backup, disaster recovery, and resilience to achieve the same performance available on their on-premise HSMs.

+### Accelerate digital transformation and innovation in cloud

+For existing Thales payShield customers wishing to add a cloud option, the Azure Payment HSM solution offers native access to a payment HSM in Azure for "lift and shift" while still experiencing the low latency they're accustomed to via their on-premise payShield HSMs. The solution also offers high-performance transactions for mission-critical payment applications. Consequently, customers can continue their digital transformation strategy by leveraging technology innovation in the cloud. Existing Thales payShield customers can utilize their existing remote management solutions (payShield Manager and payShield TMD together with associated smart card readers and smart cards as appropriate) to work with the Azure Payment HSM service. Customers new to payShield can source the hardware accessories from Thales or one of its partners before deploying their HSM as part of the subscription service.

+## Typical use cases

+With benefits including low latency and the ability to quickly add more HSM capacity as required, the cloud service is a perfect fit for a broad range of use cases, including:

+Payment processing

+- Card & mobile payment authorization

+- PIN & EMV cryptogram validation

+- 3D-Secure authentication

+Payment credential issuing

+- Cards

+- Mobile secure elements

+- Wearables

+- Connected devices

+- Host card emulation (HCE) applications

+Securing keys & authentication data

+- POS, mPOS & SPOC key management

+- Remote key loading (for ATM & POS/mPOS devices)

+- PIN generation & printing

+- PIN routing

+Sensitive data protection

+- Point-to-point encryption (P2PE)

+- Security tokenization (for PCI DSS compliance)

+- EMV payment tokenization

+## Suitable for both existing and new payment HSM users

+The solution provides clear benefits for both Payment HSM users with a legacy on-premise HSM footprint and those new payment ecosystem entrants with no legacy infrastructure to support and who may choose a cloud-native approach from the outset.

+Benefits for existing on-premise HSM users

+- Requires no modifications to payment applications or HSM software to migrate existing applications to the Azure solution

+- Enables more flexibility and efficiency in HSM utilization

+- Simplifies HSM sharing between multiple teams, geographically dispersed

+- Reduces physical HSM footprint in their legacy data centers

+- Improves cash flow for new projects

+Benefits for new payment participants

+- Avoids introduction of on-premise HSM infrastructure

+- Lowers upfront investment via the Azure subscription model

+- Offers access to latest certified hardware and software on-demand

+## Glossary

+| Term | Definition |

+|||

+| 3DS | 3D Secure |

+| ATM | Automated Teller Machine |

+| EMV | Europay Mastercard Visa |

+| FIPS | Federal Information Processing Standards |

+| HCE | Host Card Emulation |

+| HSM | Hardware Security Module |

+| mPOS | Mobile Point of Sale |

+| P2PE | Point-to-Point Encryption |

+| PCI | Payment Card Industry |

+| PIN | Personal Identification Number |

+| POS | Point of Sale |

+| SPOC | Software-based PIN Entry on Commercial off the Shelf (COTS) Solutions |

+| TMD | payShield Trusted Management Device |

+## Next steps

+- Learn more about [Azure Payment HSM](overview.md)

+- Find out how to [get started with Azure Payment HSM](getting-started.md)

+- See some common [deployment scenarios](deployment-scenarios.md)

+- Learn about [Certification and compliance](certification-compliance.md)

+- Read the [frequently asked questions](faq.yml)

+ Title: Azure Purview supported data sources and file types

+description: This article provides details about supported data sources, file types, and functionalities in Azure Purview.

+# Supported data sources and file types

+This article discusses currently supported data sources, file types, and scanning concepts in Azure Purview.

+## Azure Purview data sources

+The table below shows the supported capabilities for each data source. Select the data source, or the feature, to learn more.

+|**Category**| **Data Store** |**Technical metadata** |**Classification** |**Lineage** | **Access Policy** |

+|||||||

+| Azure | [Azure Blob Storage](register-scan-azure-blob-storage-source.md)| [Yes](register-scan-azure-blob-storage-source.md#register) | [Yes](register-scan-azure-blob-storage-source.md#scan)| Limited* | [Yes](how-to-access-policies-storage.md) |

+|| [Azure Cosmos DB](register-scan-azure-cosmos-database.md)| [Yes](register-scan-azure-cosmos-database.md#register) | [Yes](register-scan-azure-cosmos-database.md#scan)|No*|No|

+|| [Azure Data Explorer](register-scan-azure-data-explorer.md)| [Yes](register-scan-azure-data-explorer.md#register) | [Yes](register-scan-azure-data-explorer.md#scan)| No* | No |

+|| [Azure Data Lake Storage Gen1](register-scan-adls-gen1.md)| [Yes](register-scan-adls-gen1.md#register) | [Yes](register-scan-adls-gen1.md#scan)| Limited* | No |

+|| [Azure Data Lake Storage Gen2](register-scan-adls-gen2.md)| [Yes](register-scan-adls-gen2.md#register) | [Yes](register-scan-adls-gen2.md#scan)| Limited* | [Yes](how-to-access-policies-storage.md) |

+|| [Azure Database for MySQL](register-scan-azure-mysql-database.md) | [Yes](register-scan-azure-mysql-database.md#register) | [Yes](register-scan-azure-mysql-database.md#scan) | No* | No |

+|| [Azure Database for PostgreSQL](register-scan-azure-postgresql.md) | [Yes](register-scan-azure-postgresql.md#register) | [Yes](register-scan-azure-postgresql.md#scan) | No* | No |

+|| [Azure Dedicated SQL pool (formerly SQL DW)](register-scan-azure-synapse-analytics.md)| [Yes](register-scan-azure-synapse-analytics.md#register) | [Yes](register-scan-azure-synapse-analytics.md#scan)| No* | No |

+|| [Azure Files](register-scan-azure-files-storage-source.md)|[Yes](register-scan-azure-files-storage-source.md#register) | [Yes](register-scan-azure-files-storage-source.md#scan) | Limited* | No |

+|| [Azure SQL Database](register-scan-azure-sql-database.md)| [Yes](register-scan-azure-sql-database.md#register) |[Yes](register-scan-azure-sql-database.md#scan)| No* | No |

+|| [Azure SQL Database Managed Instance](register-scan-azure-sql-database-managed-instance.md)| [Yes](register-scan-azure-sql-database-managed-instance.md#scan) | [Yes](register-scan-azure-sql-database-managed-instance.md#scan) | No* | No |

+|| [Azure Synapse Analytics (Workspace)](register-scan-synapse-workspace.md)| [Yes](register-scan-synapse-workspace.md#register) | [Yes](register-scan-synapse-workspace.md#scan)| [Yes - Synapse pipelines](how-to-lineage-azure-synapse-analytics.md)| No|

+|Database| [Amazon RDS](register-scan-amazon-rds.md) | [Yes](register-scan-amazon-rds.md#register-an-amazon-rds-data-source) | [Yes](register-scan-amazon-rds.md#scan-an-amazon-rds-database) | No | No |

+|| [Cassandra](register-scan-cassandra-source.md)|[Yes](register-scan-cassandra-source.md#register) | No | [Yes](register-scan-cassandra-source.md#lineage)| No|

+|| [Db2](register-scan-db2.md) | [Yes](register-scan-db2.md#register) | No | [Yes](register-scan-db2.md#lineage) | No |

+|| [Google BigQuery](register-scan-google-bigquery-source.md)| [Yes](register-scan-google-bigquery-source.md#register)| No | [Yes](register-scan-google-bigquery-source.md#lineage)| No|

+|| [Hive Metastore Database](register-scan-hive-metastore-source.md) | [Yes](register-scan-hive-metastore-source.md#register) | No | [Yes*](register-scan-hive-metastore-source.md#lineage) | No|

+|| [MySQL](register-scan-mysql.md) | [Yes](register-scan-mysql.md#register) | No | [Yes](register-scan-mysql.md#scan) | No |

+|| [Oracle](register-scan-oracle-source.md) | [Yes](register-scan-oracle-source.md#register)| No | [Yes*](register-scan-oracle-source.md#lineage) | No|

+|| [PostgreSQL](register-scan-postgresql.md) | [Yes](register-scan-postgresql.md#register) | No | [Yes](register-scan-postgresql.md#lineage) | No |

+|| [SAP HANA](register-scan-sap-hana.md) | [Yes](register-scan-sap-hana.md#register) | No | No | No |

+|| [Snowflake](register-scan-snowflake.md) | [Yes](register-scan-snowflake.md#register) | No | [Yes](register-scan-snowflake.md#lineage) | No |

+|| [SQL Server](register-scan-on-premises-sql-server.md)| [Yes](register-scan-on-premises-sql-server.md#register) |[Yes](register-scan-on-premises-sql-server.md#scan) | No* | No|

+|| [Teradata](register-scan-teradata-source.md)| [Yes](register-scan-teradata-source.md#register)| No | [Yes*](register-scan-teradata-source.md#lineage) | No|

+|File|[Amazon S3](register-scan-amazon-s3.md)|[Yes](register-scan-amazon-s3.md)| [Yes](register-scan-amazon-s3.md)| Limited* | No|

+|Services and apps| [Erwin](register-scan-erwin-source.md)| [Yes](register-scan-erwin-source.md#register)| No | [Yes](register-scan-erwin-source.md#lineage)| No|

+|| [Looker](register-scan-looker-source.md)| [Yes](register-scan-looker-source.md#register)| No | [Yes](register-scan-looker-source.md#lineage)| No|

+|| [Power BI](register-scan-power-bi-tenant.md)| [Yes](register-scan-power-bi-tenant.md#register)| No | [Yes](how-to-lineage-powerbi.md)| No|

+|| [Salesforce](register-scan-salesforce.md) | [Yes](register-scan-salesforce.md#register) | No | No | No |

+|| [SAP ECC](register-scan-sapecc-source.md)| [Yes](register-scan-sapecc-source.md#register) | No | [Yes*](register-scan-sapecc-source.md#lineage) | No|

+|| [SAP S/4HANA](register-scan-saps4hana-source.md) | [Yes](register-scan-saps4hana-source.md#register)| No | [Yes*](register-scan-saps4hana-source.md#lineage) | No|

+\* Besides the lineage on assets within the data source, lineage is also supported if dataset is used as a source/sink in [Data Factory](how-to-link-azure-data-factory.md) or [Synapse pipeline](how-to-lineage-azure-synapse-analytics.md).

+> [!NOTE]

+> Currently, Azure Purview can't scan an asset that has `/`, `\`, or `#` in its name. To scope your scan and avoid scanning assets that have those characters in the asset name, use the example in [Register and scan an Azure SQL Database](register-scan-azure-sql-database.md#creating-the-scan).

+## Scan regions

+The following is a list of all the Azure data source (data center) regions where the Azure Purview scanner runs. If your Azure data source is in a region outside of this list, the scanner will run in the region of your Azure Purview instance.

+### Azure Purview scanner regions

+- Australia East

+- Australia Southeast

+- Brazil South

+- Canada Central

+- Central India

+- Central US

+- East Asia

+- East US

+- East US 2

+- France Central

+- Japan East

+- Korea Central

+- North Central US

+- North Europe

+- South Africa North

+- South Central US

+- Southeast Asia

+- UAE North

+- UK South

+- West Central US

+- West Europe

+- West US

+- West US 2

+## File types supported for scanning

+The following file types are supported for scanning, for schema extraction, and classification where applicable:

+- Structured file formats supported by extension: AVRO, ORC, PARQUET, CSV, JSON, PSV, SSV, TSV, TXT, XML, GZIP

+ > [!Note]

+ > * Azure Purview scanner only supports schema extraction for the structured file types listed above.

+ > * For AVRO, ORC, and PARQUET file types, Azure Purview scanner does not support schema extraction for files that contain complex data types (for example, MAP, LIST, STRUCT).

+ > * Azure Purview scanner supports scanning snappy compressed PARQUET types for schema extraction and classification.

+ > * For GZIP file types, the GZIP must be mapped to a single csv file within.

+ > Gzip files are subject to System and Custom Classification rules. We currently don't support scanning a gzip file mapped to multiple files within, or any file type other than csv.

+ > * For delimited file types(CSV, PSV, SSV, TSV, TXT), we do not support data type detection. The data type will be listed as "string" for all columns.

+- Document file formats supported by extension: DOC, DOCM, DOCX, DOT, ODP, ODS, ODT, PDF, POT, PPS, PPSX, PPT, PPTM, PPTX, XLC, XLS, XLSB, XLSM, XLSX, XLT

+- Azure Purview also supports custom file extensions and custom parsers.

+## Nested data

+Currently, nested data is only supported for JSON content.

+For all [system supported file types](#file-types-supported-for-scanning), if there is nested JSON content in a column, then the scanner parses the nested JSON data and surfaces it within the schema tab of the asset.

+Nested data, or nested schema parsing, is not supported in SQL. A column with nested data will be reported and classified as is, and subdata will not be parsed.

+## Sampling within a file

+In Azure Purview terminology,

+- L1 scan: Extracts basic information and meta data like file name, size and fully qualified name

+- L2 scan: Extracts schema for structured file types and database tables

+- L3 scan: Extracts schema where applicable and subjects the sampled file to system and custom classification rules

+For all structured file formats, Azure Purview scanner samples files in the following way:

+- For structured file types, it samples the top 128 rows in each column or the first 1 MB, whichever is lower.

+- For document file formats, it samples the first 20 MB of each file.

+ - If a document file is larger than 20 MB, then it is not subject to a deep scan (subject to classification). In that case, Azure Purview captures only basic meta data like file name and fully qualified name.

+- For **tabular data sources(SQL, CosmosDB)**, it samples the top 128 rows.

+## Resource set file sampling

+A folder or group of partition files is detected as a *resource set* in Azure Purview, if it matches with a system resource set policy or a customer defined resource set policy. If a resource set is detected, then Azure Purview will sample each folder that it contains. Learn more about resource sets [here](concept-resource-sets.md).

+File sampling for resource sets by file types:

+- **Delimited files (CSV, PSV, SSV, TSV)** - 1 in 100 files are sampled (L3 scan) within a folder or group of partition files that are considered a 'Resource set'

+- **Data Lake file types (Parquet, Avro, Orc)** - 1 in 18446744073709551615 (long max) files are sampled (L3 scan) within a folder or group of partition files that are considered a *resource set*

+- **Other structured file types (JSON, XML, TXT)** - 1 in 100 files are sampled (L3 scan) within a folder or group of partition files that are considered a 'Resource set'

+- **SQL objects and CosmosDB entities** - Each file is L3 scanned.

+- **Document file types** - Each file is L3 scanned. Resource set patterns don't apply to these file types.

+## Classification

+All 206 system classification rules apply to structured file formats. Only the MCE classification rules apply to document file types (Not the data scan native regex patterns, bloom filter-based detection). For more information on supported classifications, see [Supported classifications in Azure Purview](supported-classifications.md).

+## Next steps

+- [Register and scan Azure Blob storage source](register-scan-azure-blob-storage-source.md)

+- [Scans and ingestion in Azure Purview](concept-scans-and-ingestion.md)

+- [Manage data sources in Azure Purview](manage-data-sources.md)

+- When multi-factor authentication is enabled, to sign in to Azure Purview Studio, you must perform multi-factor authentication.

+1. In the **Conditional Access-Policies** menu, select **New policy**, provide a name, and then select **Configure rules**.

+1. Under **Assignments**, select **Users and groups**, check **Select users and groups**, and then select the user or group for Conditional Access. Select **Select**, and then select **Done** to accept your selection.

+1. Select **Cloud apps**, select **Select apps**. You see all apps available for Conditional Access. Select **Azure Purview**, at the bottom select **Select**, and then select **Done**.

+

+1. Select **Access controls**, select **Grant**, and then check the policy you want to apply. For this example, we select **Require multi-factor authentication**.

+1. Set **Enable policy** to **On** and select **Create**.

+- [Use Azure Purview Studio](use-azure-purview-studio.md)

+description: This article gives an overview permission, access control, and collections in Azure Purview. Role-based access control (RBAC) is managed within Azure Purview itself, so this guide will cover the basics to secure your information.

+A collection is a tool Azure Purview uses to group assets, sources, and other artifacts into a hierarchy for discoverability and to manage access control. All accesses to Azure Purview's resources are managed from collections in the Azure Purview account itself.

+You can assign Azure Purview roles to users, security groups and service principals from your Azure Active Directory that is associated with your purview account's subscription.

+- [Azure Purview supported data sources](azure-purview-connector-overview.md)

+* For automatic assignment, see [Supported data stores in Azure Purview](./azure-purview-connector-overview.md).

+Before applying these recommendations to your environment, you should consult your security team as some may not be applicable to your security requirements.

+## Network security

+Azure Purview is a Platform as a Service (PaaS) solution in Azure. You can enable the following network security capabilities for your Azure Purview accounts:

+- Enable [end-to-end network isolation](catalog-private-link-end-to-end.md) using Private Link Service.

+- Deploy [Network Security Group (NSG) rules](#use-network-security-groups) for subnets where Azure data sources private endpoints, Azure Purview private endpoints and self-hosted runtime VMs are deployed.

+Azure Purview can scan data sources in Azure or an on-premises environment by using ingestion private endpoints.

+- To scan data sources, you must use a self-hosted integration runtime.

+The following NSG rules are required on for **Azure Purview account, portal and ingestion private endpoints**:

+For more information, see [Self-hosted integration runtime networking requirements](manage-integration-runtimes.md#networking-requirements).

+## Access management

+Related to roles and access management in Azure Purview, you can apply the following security best practices:

+- Assign roles to Azure Active Directory groups instead of assigning roles to individual users.

+### Manage an Azure Purview account in control plane and data plane

+Data plane refers to all operations, related to interacting with Azure Purview inside Data Map and Data Catalog.

+You can assign control plane and data plane roles to users, security groups and service principals from your Azure Active Directory tenant that is associated to Azure Purview instance's Azure subscription.

+|Set up a Private Endpoint for Azure Purview | Control plane | Contributor  | Azure RBAC roles |

+We use Azure Active Directory to provide authentication and authorization mechanisms for Azure Purview inside Collections. You can assign Azure Purview roles to the following security principals from your Azure Active Directory tenant that is associated with Azure subscription where your Azure Purview instance is hosted:

+Azure Purview is a data governance solution in cloud. You can register and scan different data sources from various data systems from your on-premises, Azure, or multi-cloud environments into Azure Purview. While data source is registered and scanned in Azure Purview, the actual data and data sources stay in their original locations, only metadata is extracted from data sources and stored in Azure Purview Data Map, which means you do not need to move data out of the region or their original location to extract the metadata into Azure Purview.

+When an Azure Purview account is deployed, in addition, a managed resource group is also deployed in your Azure subscription. A managed Azure Storage Account and a Managed Event Hubs are deployed inside this resource group. The managed storage account is used to ingest metadata from data sources during the scan. Since these resources are consumed by the Azure Purview they cannot be accessed by any other users or principals, except the Azure Purview account. This is because an Azure role-based access control (RBAC) deny assignment is added automatically for all principals to this resource group at the time of Azure Purview account deployment, preventing any CRUD operations on these resources if they are not initiated from Azure Purview.

+To extract metadata from a data source system into Azure Purview Data Map, it is required to register and scan the data source systems in Azure Purview Data Map. To automate this process, we have made available [connectors](azure-purview-connector-overview.md) for different data source systems in Azure Purview to simplify the registration and scanning process.

+Use [this guide](azure-purview-connector-overview.md) to read more about each connector and their supported authentication options.

+## Other recommendations

+- Limit the amount of software in the self-hosted integration runtime VMs. Although it is not a mandatory requirement to have a dedicated VM for a self-hosted runtime for Azure Purview, we highly suggest using dedicated VMs especially for production environments.

+- Monitor the VMs using [Azure Monitor for VMs](../azure-monitor/vm/vminsights-overview.md). By using Log analytics agent, you can capture content such as performance metrics to adjust required capacity for your VMs.

+- By integrating virtual machines with Microsoft Defender for Cloud, you can you prevent, detect, and respond to threats.

+- Use multiple machines for greater resilience and availability. You can deploy and register multiple self-hosted integration runtimes to distribute the scans across multiple self-hosted integration runtime machines or deploy the self-hosted integration runtime on a Virtual Machine Scale Set for higher redundancy and scalability.

+- Optionally, you can plan to enable Azure backup from your self-hosted integration runtime VMs to increase the recovery time of a self-hosted integration runtime VM if there is a VM level disaster.

+* [Use the Azure Purview Studio](use-azure-purview-studio.md)

+* [Use the Azure Purview Studio](use-azure-purview-studio.md)

+ Title: 'Quickstart: Create Azure Purview Account using .NET SDK'

+description: Create an Azure Purview Account using .NET SDK.

+ms.devlang: csharp

+# Quickstart: Create an Azure Purview account using .NET SDK

+In this quickstart, you'll use the [.NET SDK](/dotnet/api/overview/azure/purviewresourceprovider) to create an Azure Purview account.

+Azure Purview is a data governance service that helps you manage and govern your data landscape. By connecting to data across your on-premises, multi-cloud, and software-as-a-service (SaaS) sources, Azure Purview creates an up-to-date map of your information. It identifies and classifies sensitive data, and provides end-to-end linage. Data consumers are able to discover data across your organization, and data administrators are able to audit, secure, and ensure right use of your data.

+For more information about Azure Purview, [see our overview page](overview.md). For more information about deploying Azure Purview across your organization, [see our deployment best practices](deployment-best-practices.md).

+### Visual Studio

+The walkthrough in this article uses Visual Studio 2019. The procedures for Visual Studio 2013, 2015, or 2017 may differ slightly.

+### Azure .NET SDK

+Download and install [Azure .NET SDK](https://azure.microsoft.com/downloads/) on your machine.

+## Create an application in Azure Active Directory

+1. In [Create an Azure Active Directory application](../active-directory/develop/howto-create-service-principal-portal.md#register-an-application-with-azure-ad-and-create-a-service-principal), create an application that represents the .NET application you are creating in this tutorial. For the sign-on URL, you can provide a dummy URL as shown in the article (`https://contoso.org/exampleapp`).

+1. In [Get values for signing in](../active-directory/develop/howto-create-service-principal-portal.md#get-tenant-and-app-id-values-for-signing-in), get the **application ID** and **tenant ID**, and note down these values that you use later in this tutorial.

+1. In [Certificates and secrets](../active-directory/develop/howto-create-service-principal-portal.md#authentication-two-options), get the **authentication key**, and note down this value that you use later in this tutorial.

+1. In [Assign the application to a role](../active-directory/develop/howto-create-service-principal-portal.md#assign-a-role-to-the-application), assign the application to the **Contributor** role at the subscription level so that the application can create data factories in the subscription.

+## Create a Visual Studio project

+Next, create a C# .NET console application in Visual Studio:

+1. Launch **Visual Studio**.

+2. In the Start window, select **Create a new project** > **Console App (.NET Framework)**. .NET version 4.5.2 or above is required.

+3. In **Project name**, enter **PurviewQuickStart**.

+4. Select **Create** to create the project.

+## Install NuGet packages

+1. Select **Tools** > **NuGet Package Manager** > **Package Manager Console**.

+2. In the **Package Manager Console** pane, run the following commands to install packages. For more information, see the [Microsoft.Azure.Management.Purview NuGet package](https://www.nuget.org/packages/Microsoft.Azure.Management.Purview/).

+ ```powershell

+ Install-Package Microsoft.Azure.Management.Purview

+ Install-Package Microsoft.Azure.Management.ResourceManager -IncludePrerelease

+ Install-Package Microsoft.IdentityModel.Clients.ActiveDirectory

+ ```

+## Create an Azure Purview client

+1. Open **Program.cs**, include the following statements to add references to namespaces.

+ ```csharp

+ using System;

+ using System.Collections.Generic;

+ using System.Linq;

+ using Microsoft.Rest;

+ using Microsoft.Rest.Serialization;

+ using Microsoft.Azure.Management.ResourceManager;

+ using Microsoft.Azure.Management.Purview;

+ using Microsoft.Azure.Management.Purview.Models;

+ using Microsoft.IdentityModel.Clients.ActiveDirectory;

+ ```

+2. Add the following code to the **Main** method that sets the variables. Replace the placeholders with your own values. For a list of Azure regions in which Azure Purview is currently available, search on **Azure Purview** and select the regions that interest you on the following page: [Products available by region](https://azure.microsoft.com/global-infrastructure/services/).

+ ```csharp

+ // Set variables

+ string tenantID = "<your tenant ID>";

+ string applicationId = "<your application ID>";

+ string authenticationKey = "<your authentication key for the application>";

+ string subscriptionId = "<your subscription ID where the data factory resides>";

+ string resourceGroup = "<your resource group where the data factory resides>";

+ string region = "<the location of your resource group>";

+ string purviewAccountName =

+ "<specify the name of purview account to create. It must be globally unique.>";

+ ```

+3. Add the following code to the **Main** method that creates an instance of **PurviewManagementClient** class. You use this object to create an Azure Purview Account.

+ ```csharp

+ // Authenticate and create a purview management client

+ var context = new AuthenticationContext("https://login.windows.net/" + tenantID);

+ ClientCredential cc = new ClientCredential(applicationId, authenticationKey);

+ AuthenticationResult result = context.AcquireTokenAsync(

+ "https://management.azure.com/", cc).Result;

+ ServiceClientCredentials cred = new TokenCredentials(result.AccessToken);

+ var client = new PurviewManagementClient(cred)

+ {

+ SubscriptionId = subscriptionId

+ };

+ ```

+## Create an Azure Purview account

+Add the following code to the **Main** method that creates a **Azure Purview Account**.

+```csharp

+// Create a purview Account

+Console.WriteLine("Creating Azure Purview Account " + purviewAccountName + "...");

+Account account = new Account()

+{

+Location = region,

+Identity = new Identity(type: "SystemAssigned"),

+Sku = new AccountSku(name: "Standard", capacity: 4)

+};

+try

+{

+ client.Accounts.CreateOrUpdate(resourceGroup, purviewAccountName, account);

+ Console.WriteLine(client.Accounts.Get(resourceGroup, purviewAccountName).ProvisioningState);

+}

+catch (ErrorResponseModelException purviewException)

+{

+Console.WriteLine(purviewException.StackTrace);

+ }

+ Console.WriteLine(

+ SafeJsonConvert.SerializeObject(account, client.SerializationSettings));

+ while (client.Accounts.Get(resourceGroup, purviewAccountName).ProvisioningState ==

+ "PendingCreation")

+ {

+ System.Threading.Thread.Sleep(1000);

+ }

+Console.WriteLine("\nPress any key to exit...");

+Console.ReadKey();

+```

+## Run the code

+Build and start the application, then verify the execution.

+The console prints the progress of creating Azure Purview Account.

+### Sample output

+```json

+Creating Azure Purview Account testpurview...

+Succeeded

+{

+ "sku": {

+ "capacity": 4,

+ "name": "Standard"

+ },

+ "identity": {

+ "type": "SystemAssigned"

+ },

+ "location": "southcentralus"

+}

+Press any key to exit...

+```

+## Verify the output

+Go to the **Azure Purview accounts** page in the [Azure portal](https://portal.azure.com) and verify the account created using the above code.

+## Delete Azure Purview account

+To programmatically delete an Azure Purview Account, add the following lines of code to the program:

+```csharp

+Console.WriteLine("Deleting the Azure Purview Account");

+client.Accounts.Delete(resourceGroup, purviewAccountName);

+```

+## Check if Azure Purview account name is available

+To check availability of a purview account, use the following code:

+```csharp

+CheckNameAvailabilityRequest checkNameAvailabilityRequest = newCheckNameAvailabilityRequest()

+{

+ Name = purviewAccountName,

+ Type = "Microsoft.Purview/accounts"

+};

+Console.WriteLine("Check Azure Purview account name");

+Console.WriteLine(client.Accounts.CheckNameAvailability(checkNameAvailabilityRequest).NameAvailable);

+```

+The above code with print 'True' if the name is available and 'False' if the name is not available.

+## Next steps

+The code in this tutorial creates a purview account, deletes a purview account and checks for name availability of purview account. You can now download the .NET SDK and learn about other resource provider actions you can perform for an Azure Purview account.

+Follow these next articles to learn how to navigate the Azure Purview Studio, create a collection, and grant access to Azure Purview.

+* [How to use the Azure Purview Studio](use-azure-purview-studio.md)

+* [Create a collection](quickstart-create-collection.md)

+* [Add users to your Azure Purview account](catalog-permissions.md)

+ Title: Create an Azure Policy exception for Azure Purview

+description: This article describes how to create an Azure Policy exception for Azure Purview while leaving existing Policies in place to maintain security.

+# Create an Azure Policy exception for Azure Purview

+Many subscriptions have [Azure Policies](../governance/policy/overview.md) in place that restrict the creation of some resources. This is to maintain subscription security and cleanliness. However, Azure Purview accounts deploy two other Azure resources when they are created: an Azure Storage account, and an Event Hubs namespace. When you [create Azure Purview Account](create-catalog-portal.md), these resources will be deployed. They will be managed by Azure, so you don't need to maintain them, but you will need to deploy them.

+To maintain your policies in your subscription, but still allow the creation of these managed resources, you can create a policy exception.

+## Create a policy exception for Azure Purview

+1. Navigate to the [Azure portal](https://portal.azure.com) and search for **Policy**

+ :::image type="content" source="media/create-purview-portal-faq/search-for-policy.png" alt-text="Screenshot showing the Azure portal search bar, searching for Policy keyword.":::

+1. Follow [Create a custom policy definition](../governance/policy/tutorials/create-custom-policy-definition.md) or modify existing policy to add two exceptions with `not` operator and `resourceBypass` tag:

+ ```json

+ {

+ "mode": "All",

+ "policyRule": {

+ "if": {

+ "anyOf": [

+ {

+ "allOf": [

+ {

+ "field": "type",

+ "equals": "Microsoft.Storage/storageAccounts"

+ },

+ {

+ "not": {

+ "field": "tags['<resourceBypass>']",

+ "exists": true

+ }

+ }]

+ },

+ {

+ "allOf": [

+ {

+ "field": "type",

+ "equals": "Microsoft.EventHub/namespaces"

+ },

+ {

+ "not": {

+ "field": "tags['<resourceBypass>']",

+ "exists": true

+ }

+ }]

+ }]

+ },

+ "then": {

+ "effect": "deny"

+ }

+ },

+ "parameters": {}

+ }

+ ```

+

+ > [!Note]

+ > The tag could be anything beside `resourceBypass` and it's up to you to define value when creating Azure Purview in later steps as long as the policy can detect the tag.

+ :::image type="content" source="media/create-catalog-portal/policy-definition.png" alt-text="Screenshot showing how to create policy definition.":::

+1. [Create a policy assignment](../governance/policy/assign-policy-portal.md) using the custom policy created.

+ :::image type="content" source="media/create-catalog-portal/policy-assignment.png" alt-text="Screenshot showing how to create policy assignment" lightbox="./media/create-catalog-portal/policy-assignment.png":::

+> [!Note]

+> If you have **Azure Policy** and need to add exception as in **Prerequisites**, you need to add the correct tag. For example, you can add `resourceBypass` tag:

+> :::image type="content" source="media/create-catalog-portal/add-purview-tag.png" alt-text="Add tag to Azure Purview account.":::

+## Next steps

+To set up Azure Purview by using Private Link, see [Use private endpoints for your Azure Purview account](./catalog-private-link.md).

+ Title: 'Quickstart: Create an Azure Purview account using Python'

+description: Create an Azure Purview account using Python.

+ms.devlang: python

+# Quickstart: Create an Azure Purview account using Python

+In this quickstart, youΓÇÖll create an Azure Purview account programatically using Python. [Python reference for Azure Purview](/python/api/azure-mgmt-purview/) is available, but this article will take you through all the steps needed to create an account with Python.

+Azure Purview is a data governance service that helps you manage and govern your data landscape. By connecting to data across your on-premises, multi-cloud, and software-as-a-service (SaaS) sources, Azure Purview creates an up-to-date map of your information. It identifies and classifies sensitive data, and provides end-to-end linage. Data consumers are able to discover data across your organization, and data administrators are able to audit, secure, and ensure right use of your data.

+For more information about Azure Purview, [see our overview page](overview.md). For more information about deploying Azure Purview across your organization, [see our deployment best practices](deployment-best-practices.md).

+## Install the Python package

+1. Open a terminal or command prompt with administrator privileges.

+2. First, install the Python package for Azure management resources:

+ ```python

+ pip install azure-mgmt-resource

+ ```

+3. To install the Python package for Azure Purview, run the following command:

+ ```python

+ pip install azure-mgmt-purview

+ ```

+ The [Python SDK for Azure Purview](https://github.com/Azure/azure-sdk-for-python) supports Python 2.7, 3.3, 3.4, 3.5, 3.6 and 3.7.

+4. To install the Python package for Azure Identity authentication, run the following command:

+ ```python

+ pip install azure-identity

+ ```

+ > [!NOTE]

+ > The "azure-identity" package might have conflicts with "azure-cli" on some common dependencies. If you meet any authentication issue, remove "azure-cli" and its dependencies, or use a clean machine without installing "azure-cli" package.

+## Create a purview client

+1. Create a file named **purview.py**. Add the following statements to add references to namespaces.

+ ```python

+ from azure.identity import ClientSecretCredential

+ from azure.mgmt.resource import ResourceManagementClient

+ from azure.mgmt.purview import PurviewManagementClient

+ from azure.mgmt.purview.models import *

+ from datetime import datetime, timedelta

+ import time

+ ```

+2. Add the following code to the **Main** method that creates an instance of PurviewManagementClient class. You'll use this object to create a purview account, delete purview accounts, check name availability, and other resource provider operations.

+ ```python

+ def main():

+

+ # Azure subscription ID

+ subscription_id = '<subscription ID>'

+

+ # This program creates this resource group. If it's an existing resource group, comment out the code that creates the resource group

+ rg_name = '<resource group>'

+ # The purview name. It must be globally unique.

+ purview_name = '<purview account name>'

+ # Location name, where Azure Purview account must be created.

+ location = '<location name>'

+ # Specify your Active Directory client ID, client secret, and tenant ID

+ credentials = ClientSecretCredential(client_id='<service principal ID>', client_secret='<service principal key>', tenant_id='<tenant ID>')

+ # resource_client = ResourceManagementClient(credentials, subscription_id)

+ purview_client = PurviewManagementClient(credentials, subscription_id)

+ ```

+## Create a purview account

+1. Add the following code to the **Main** method that creates a **purview account**. If your resource group already exists, comment out the first `create_or_update` statement.

+ ```python

+ # create the resource group

+ # comment out if the resource group already exits

+ resource_client.resource_groups.create_or_update(rg_name, rg_params)

+ #Create a purview

+ identity = Identity(type= "SystemAssigned")

+ sku = AccountSku(name= 'Standard', capacity= 4)

+ purview_resource = Account(identity=identity,sku=sku,location =location )

+

+ try:

+ pa = (purview_client.accounts.begin_create_or_update(rg_name, purview_name, purview_resource)).result()

+ print("location:", pa.location, " Azure Purview Account Name: ", pa.name, " Id: " , pa.id ," tags: " , pa.tags)

+ except:

+ print("Error")

+ print_item(pa)

+

+ while (getattr(pa,'provisioning_state')) != "Succeeded" :

+ pa = (purview_client.accounts.get(rg_name, purview_name))

+ print(getattr(pa,'provisioning_state'))

+ if getattr(pa,'provisioning_state') != "Failed" :

+ print("Error in creating Azure Purview account")

+ break

+ time.sleep(30)

+ ```

+2. Now, add the following statement to invoke the **main** method when the program is run:

+ ```python

+ # Start the main method

+ main()

+ ```

+## Full script

+HereΓÇÖs the full Python code:

+```python

+

+ from azure.identity import ClientSecretCredential

+ from azure.mgmt.resource import ResourceManagementClient

+ from azure.mgmt.purview import PurviewManagementClient

+ from azure.mgmt.purview.models import *

+ from datetime import datetime, timedelta

+ import time

+

+ # Azure subscription ID

+ subscription_id = '<subscription ID>'

+

+ # This program creates this resource group. If it's an existing resource group, comment out the code that creates the resource group

+ rg_name = '<resource group>'

+ # The purview name. It must be globally unique.

+ purview_name = '<purview account name>'

+ # Specify your Active Directory client ID, client secret, and tenant ID

+ credentials = ClientSecretCredential(client_id='<service principal ID>', client_secret='<service principal key>', tenant_id='<tenant ID>')

+ # resource_client = ResourceManagementClient(credentials, subscription_id)

+ purview_client = PurviewManagementClient(credentials, subscription_id)

+

+ # create the resource group

+ # comment out if the resource group already exits

+ resource_client.resource_groups.create_or_update(rg_name, rg_params)

+ #Create a purview

+ identity = Identity(type= "SystemAssigned")

+ sku = AccountSku(name= 'Standard', capacity= 4)

+ purview_resource = Account(identity=identity,sku=sku,location ="southcentralus" )

+

+ try:

+ pa = (purview_client.accounts.begin_create_or_update(rg_name, purview_name, purview_resource)).result()

+ print("location:", pa.location, " Azure Purview Account Name: ", purview_name, " Id: " , pa.id ," tags: " , pa.tags)

+ except:

+ print("Error in submitting job to create account")

+ print_item(pa)

+

+ while (getattr(pa,'provisioning_state')) != "Succeeded" :

+ pa = (purview_client.accounts.get(rg_name, purview_name))

+ print(getattr(pa,'provisioning_state'))

+ if getattr(pa,'provisioning_state') != "Failed" :

+ print("Error in creating Azure Purview account")

+ break

+ time.sleep(30)

+# Start the main method

+main()

+```

+## Run the code

+Build and start the application. The console prints the progress of Azure Purview account creation. Wait until itΓÇÖs completed.

+HereΓÇÖs the sample output:

+```console

+location: southcentralus Azure Purview Account Name: purviewpython7 Id: /subscriptions/8c2c7b23-848d-40fe-b817-690d79ad9dfd/resourceGroups/Demo_Catalog/providers/Microsoft.Purview/accounts/purviewpython7 tags: None

+Creating

+Creating

+Succeeded

+```

+## Verify the output

+Go to the **Azure Purview accounts** page in the Azure portal and verify the account created using the above code.

+## Delete Azure Purview account

+To delete purview account, add the following code to the program, then run:

+```python

+pa = purview_client.accounts.begin_delete(rg_name, purview_name).result()

+```

+## Next steps

+The code in this tutorial creates a purview account and deletes a purview account. You can now download the python SDK and learn about other resource provider actions you can perform for an Azure Purview account.

+Follow these next articles to learn how to navigate the Azure Purview Studio, create a collection, and grant access to Azure Purview.

+* [How to use the Azure Purview Studio](use-azure-purview-studio.md)

+* [Create a collection](quickstart-create-collection.md)

+* [Add users to your Azure Purview account](catalog-permissions.md)

+Azure Purview is a data governance service that helps you manage and govern your data landscape. By connecting to data across your on-premises, multi-cloud, and software-as-a-service (SaaS) sources, Azure Purview creates an up-to-date map of your information. It identifies and classifies sensitive data, and provides end-to-end linage. Data consumers are able to discover data across your organization, and data administrators are able to audit, secure, and ensure right use of your data.

+To create a UAMI, follow our [guide to create a user-assigned managed identity](manage-credentials.md#create-a-user-assigned-managed-identity).

+* [Using the Azure Purview Studio](use-azure-purview-studio.md)

+Azure Purview is a data governance service that helps you manage and govern your data landscape. By connecting to data across your on-premises, multi-cloud, and software-as-a-service (SaaS) sources, Azure Purview creates an up-to-date map of your information. It identifies and classifies sensitive data, and provides end-to-end linage. Data consumers are able to discover data across your organization, and data administrators are able to audit, secure, and ensure right use of your data.

+* [How to use the Azure Purview Studio](use-azure-purview-studio.md)

+1. To create a collection, you'll need to be in the collection admin list under role assignments. If you created the Azure Purview resource, you should be listed as a collection admin under the root collection already. If not, you'll need to contact the collection admin to grant your permission.

+ :::image type="content" source="./media/how-to-create-and-manage-collections/select-add-a-collection.png" alt-text="Screenshot of Azure Purview studio window, showing the new collection window, with the 'Add a collection' button highlighted." border="true":::

+Since permissions are managed through collections in Azure Purview, it is important to understand the roles and what permissions they will give your users. A user granted permissions on a collection will have access to sources and assets associated with that collection, and inherit permissions to subcollections. Inheritance [can be restricted](#restrict-inheritance), but is allowed by default.

+ :::image type="content" source="./media/how-to-create-and-manage-collections/search-user-permissions.png" alt-text="Screenshot of Azure Purview studio collection admin window with the search bar highlighted." border="true":::

+1. Select **Register** or register icon on collection node to register a data source. Only a data source admin can register sources.

+1. Fill in the data source name, and other source information. It lists all the collections where you have scan permission on the bottom of the form. You can select one collection. All assets under this source will belong to the collection you select.

+The collections listed here are restricted to subcollections of the data source collection.

+ 1. Check the collection-based permission model by following the [add roles and restricting access on collections guide above](#add-roles-and-restrict-access-through-collections).

+ 1. If you don't have read permission on a collection, the assets under that collection will not be listed in search results. If you get the direct URL of one asset and open it, you will see the no access page. Contact your Azure Purview admin to grant you the access. You can select the **Refresh** button to check the permission again.

+1. In the right side panel, choose the target collection you want move to. You can only see the collections where you have write permissions. The asset can also only be added to the subcollections of the data source collection.

+1. On the next page, the search results of the assets under selected collection will be shown. You can narrow the results by selecting the facet filters. Or you can see the assets under other collections by selecting the sub/related collection names.

+* [Supported data sources](azure-purview-connector-overview.md)

+* For more information about glossary terms, see the [glossary reference](reference-azure-purview-glossary.md)

+- Scanning some data sources requires additional setup on the self-hosted integration runtime machine. For example, JDK, Visual C++ Redistributable, or specific driver. Refer to [each source article](azure-purview-connector-overview.md) for prerequisite details.

+In order to create and manage collections in Azure Purview, you will need to be a **Collection Admin** within Azure Purview. We can check these permissions in the [Azure Purview Studio](use-azure-purview-studio.md). You can find the studio by going to your Azure Purview resource in the [Azure portal](https://portal.azure.com), and selecting the **Open Azure Purview Studio** tile on the overview page.

+To create your collection, we'll start in the [Azure Purview Studio](use-azure-purview-studio.md). You can find the studio by going to your Azure Purview resource in the Azure portal and selecting the **Open Azure Purview Studio** tile on the overview page.

+ Title: Azure Purview product glossary

+description: A glossary defining the terminology used throughout Azure Purview

+# Azure Purview product glossary

+Below is a glossary of terminology used throughout Azure Purview.

+## Annotation

+Information that is associated with data assets in Azure Purview, for example, glossary terms and classifications. After they are applied, annotations can be used within Search to aid in the discovery of the data assets.

+## Approved

+The state given to any request that has been accepted as satisfactory by the designated individual or group who has authority to change the state of the request.

+## Asset

+Any single object that is stored within an Azure Purview data catalog.

+> [!NOTE]

+> A single object in the catalog could potentially represent many objects in storage, for example, a resource set is an asset but it's made up of many partition files in storage.

+## Azure Information Protection

+A cloud solution that supports labeling of documents and emails to classify and protect information. Labeled items can be protected by encryption, marked with a watermark, or restricted to specific actions or users, and is bound to the item. This cloud-based solution relies on Azure Rights Management Service (RMS) for enforcing restrictions.

+## Business glossary

+A searchable list of specialized terms that an organization uses to describe key business words and their definitions. Using a business glossary can provide consistent data usage across the organization.

+## Classification report

+A report that shows key classification details about the scanned data.

+## Classification

+A type of annotation used to identify an attribute of an asset or a column such as "Age", ΓÇ£Email Address", and "Street Address". These attributes can be assigned during scans or added manually.

+## Classification rule

+A classification rule is a set of conditions that determine how scanned data should be classified when content matches the specified pattern.

+## Classified asset

+An asset where Azure Purview extracts schema and applies classifications during an automated scan. The scan rule set determines which assets get classified. If the asset is considered a candidate for classification and no classifications are applied during scan time, an asset is still considered a classified asset.

+## Column pattern

+A regular expression included in a classification rule that represents the column names that you want to match.

+## Contact

+An individual who is associated with an entity in the data catalog.

+## Control plane operation

+Operations that manage resources in your subscription, such as role-based access control and Azure Policy, that are sent to the Azure Resource Manager end point.

+## Credential

+A verification of identity or tool used in an access control system. Credentials can be used to authenticate an individual or group to grant access to a data asset.

+## Data catalog

+Azure Purview features that enable customers to view and manage the metadata for assets in your data estate.

+## Data map

+Azure Purview features that enable customers to manage their data estate, such as scanning, lineage, and movement.

+## Data pattern

+A regular expression that represents the data that is stored in a data field. For example, a data pattern for employee ID could be Employee{GUID}.

+## Data plane operation

+An operation within a specific Azure Purview instance, such as editing an asset or creating a glossary term. Each instance has predefined roles, such as "data reader" and "data curator" that control which data plane operations a user can perform.

+## Discovered asset

+An asset that Azure Purview identifies in a data source during the scanning process. The number of discovered assets includes all files or tables before resource set grouping.

+## Distinct match threshold

+The total number of distinct data values that need to be found in a column before the scanner runs the data pattern on it. For example, a distinct match threshold of eight for employee ID requires that there are at least eight unique data values among the sampled values in the column that match the data pattern set for employee ID.

+## Expert

+An individual within an organization who understands the full context of a data asset or glossary term.

+## Full scan

+A scan that processes all assets within a selected scope of a data source.

+## Fully Qualified Name (FQN)

+A path that defines the location of an asset within its data source.

+## Glossary term

+An entry in the Business glossary that defines a concept specific to an organization. Glossary terms can contain information on synonyms, acronyms, and related terms.

+## Incremental scan

+A scan that detects and processes assets that have been created, modified, or deleted since the previous successful scan. To run an incremental scan, at least one full scan must be completed on the source.

+## Ingested asset

+An asset that has been scanned, classified (when applicable), and added to the Azure Purview data map. Ingested assets are discoverable and consumable within the data catalog through automated scanning or external connections, such as Azure Data Factory and Azure Synapse.

+## Insights

+An area within Azure Purview where you can view reports that summarize information about your data.

+## Integration runtime

+The compute infrastructure used to scan in a data source.

+## Lineage

+How data transforms and flows as it moves from its origin to its destination. Understanding this flow across the data estate helps organizations see the history of their data, and aid in troubleshooting or impact analysis.

+## Management Center

+An area within Azure Purview where you can manage connections, users, roles, and credentials.

+## Minimum match threshold

+The minimum percentage of matches among the distinct data values in a column that must be found by the scanner for a classification to be applied.

+For example, a minimum match threshold of 60% for employee ID requires that 60% of all distinct values among the sampled data in a column match the data pattern set for employee ID. If the scanner samples 128 values in a column and finds 60 distinct values in that column, then at least 36 of the distinct values (60%) must match the employee ID data pattern for the classification to be applied.

+## On-premises data

+Data that is in a data center controlled by a customer, for example, not in the cloud or software as a service (SaaS).

+## Owner

+An individual or group in charge of managing a data asset.

+## Pattern rule

+A configuration that overrides how Azure Purview groups assets as resource sets and displays them within the catalog.

+## Azure Purview instance

+A single Azure Purview resource.

+## Registered source

+A source that has been added to an Azure Purview instance and is now managed as a part of the Data catalog.

+## Related terms

+Glossary terms that are linked to other terms within the organization.

+## Resource set

+A single asset that represents many partitioned files or objects in storage. For example, Azure Purview stores partitioned Apache Spark output as a single resource set instead of unique assets for each individual file.

+## Role

+Permissions assigned to a user within an Azure Purview instance. Roles, such as Azure Purview Data Curator or Azure Purview Data Reader, determine what can be done within the product.

+## Scan

+An Azure Purview process that examines a source or set of sources and ingests its metadata into the data catalog. Scans can be run manually or on a schedule using a scan trigger.

+## Scan ruleset

+A set of rules that define which data types and classifications a scan ingests into a catalog.

+## Scan trigger

+A schedule that determines the recurrence of when a scan runs.

+## Search

+A data discovery feature of Azure Purview that returns a list of assets that match to a keyword.

+## Search relevance

+The scoring of data assets that determine the order search results are returned. Multiple factors determine an asset's relevance score.

+## Self-hosted integration runtime

+An integration runtime installed on an on-premises machine or virtual machine inside a private network that is used to connect to data on-premises or in a private network.

+## Sensitivity label

+Annotations that classify and protect an organizationΓÇÖs data. Azure Purview integrates with Microsoft Information Protection for creation of sensitivity labels.

+## Sensitivity label report

+A summary of which sensitivity labels are applied across the data estate.

+## Service

+A product that provides standalone functionality and is available to customers by subscription or license.

+## Source

+A system where data is stored. Sources can be hosted in various places such as a cloud or on-premises. You register and scan sources so that you can manage them in Azure Purview.

+## Source type

+A categorization of the registered sources used in an Azure Purview instance, for example, Azure SQL Database, Azure Blob Storage, Amazon S3, or SAP ECC.

+## Steward

+An individual who defines the standards for a glossary term. They are responsible for maintaining quality standards, nomenclature, and rules for the assigned entity.

+## Term template

+A definition of attributes included in a glossary term. Users can either use the system-defined term template or create their own to include custom attributes.

+## Next steps

+To get started with Azure Purview, see [Quickstart: Create an Azure Purview account](create-catalog-portal.md).

+ Title: Learn about Azure Purview open-source tools and utilities

+description: This tutorial lists various tools and utilities available in Azure Purview and discusses their usage.

+# Customer Intent: As an Azure Purview administrator, I want to kickstart and be up and running with Azure Purview service in a matter of minutes; additionally, I want to perform and set up automations, batch-mode API executions and scripts that help me run Azure Purview smoothly and effectively for the long-term on a regular basis.

+# Azure Purview open-source tools and utilities

+This article lists several open-source tools and utilities (command-line, python, and PowerShell interfaces) that help you get started quickly on Azure Purview service in a matter of minutes! These tools have been authored & developed by collective effort of the Azure Purview Product Group and the open-source community. The objective of such tools is to make learning, starting up, regular usage, and long-term adoption of Azure Purview breezy and super fast.

+### Intended audience

+- Azure Purview community including customers, developers, ISVs, partners, evangelists, and enthusiasts.

+- Azure Purview catalog is based on [Apache Atlas](https://atlas.apache.org/) and extends full support for Apache Atlas APIs. We welcome Apache Atlas community, enthusiasts, and developers to wholeheartedly build on and evangelize Azure Purview.

+### Azure Purview customer journey stages

+- *Azure Purview Learners*: Learners who are starting fresh with Azure Purview service and are keen to understand and explore how a multi-cloud unified data governance solution works. A section of learners includes users who want to compare and contrast Azure Purview with other competing solutions in the data governance market and try it before adopting for long-term usage.

+- *Azure Purview Innovators*: Innovators who are keen to understand existing and latest features, ideate, and conceptualize features upcoming on Azure Purview. They are adept at building and developing solutions for customers, and have futuristic forward-looking ideas for the next-gen cutting-edge data governance product.

+- *Azure Purview Enthusiasts/Evangelists*: Enthusiasts who are a combination of Learners and Innovators. They have developed solid understanding and knowledge of Azure Purview, hence, are upbeat about adoption of Azure Purview. They can help evangelize Azure Purview as a service and educate several other Azure Purview users and probable customers across the globe.

+- *Azure Purview Adopters*: Adopters who have migrated from starting up and exploring Azure Purview and are smoothly using Azure Purview for more than a few months.

+- *Azure Purview Long-Term Regular Users*: Long-term users who have been using Azure Purview for more than one year and are now confident and comfortable using most advanced Azure Purview use cases on the Azure portal and Azure Purview Studio; furthermore they have near perfect knowledge and awareness of the Azure Purview REST APIs and the other use cases supported via Azure Purview APIs.

+## Azure Purview open-source tools and utilities list

+1. [Purview-API-via-PowerShell](https://github.com/Azure/Azure-Purview-API-PowerShell/blob/main/README.md)

+ - **Recommended customer journey stages**: *Learners, Innovators, Enthusiasts, Adopters, Long-Term Regular Users*

+ - **Description**: This utility is based on and covers the entire set of [Azure Purview REST API Reference](/rest/api/purview/) Microsoft Docs. [Download & Install from PowerShell Gallery](https://aka.ms/purview-api-ps). It helps you execute all the documented Azure Purview REST APIs through a breezy fast and easy to use PowerShell interface. Use and automate Azure Purview APIs for regular and long-term usage via command-line and scripted methods. This is an alternative for customers looking to do bulk tasks in automated manner, batch-mode, or scheduled cron jobs; as against the GUI method of using the Azure portal and Azure Purview Studio. Detailed documentation, sample usage guide, self-help, and examples are available on [GitHub:Azure-Purview-API-PowerShell](https://github.com/Azure/Azure-Purview-API-PowerShell).

+1. [Purview-Starter-Kit](https://aka.ms/PurviewKickstart)

+ - **Recommended customer journey stages**: *Learners, Innovators, Enthusiasts*

+ - **Description**: PowerShell script to perform initial setup of Azure Purview account. Useful for anyone looking to set up several fresh new Azure Purview account(s) in less than 5 minutes!

+1. [Azure Purview Lab](https://aka.ms/purviewlab)

+ - **Recommended customer journey stages**: *Learners, Innovators, Enthusiasts*

+ - **Description**: A hands-on-lab introducing the myriad features of Azure Purview and helping you learn the concepts in a practical and hands-on approach where you execute each step on your own by hand to develop the best possible understanding of Azure Purview.

+1. [Azure Purview CLI](https://aka.ms/purviewcli)

+ - **Recommended customer journey stages**: *Innovators, Enthusiasts, Adopters, Long-Term Regular Users*

+ - **Description**: Python-based tool to execute the Azure Purview APIs similar to [Purview-API-via-PowerShell](https://aka.ms/purview-api-ps) but has limited/lesser functionality than the PowerShell-based framework.

+1. [Azure Purview Demo](https://aka.ms/pvdemo)

+ - **Recommended customer journey stages**: *Learners, Innovators, Enthusiasts*

+ - **Description**: An Azure Resource Manager (ARM) template-based tool to automatically set up and deploy fresh new Azure Purview account quickly and securely at the issue of just one command. It is similar to [Purview-Starter-Kit](https://aka.ms/PurviewKickstart), the extra feature being it deploys a few more pre-configured data sources - Azure SQL Database, Azure Data Lake Storage Gen2 Account, Azure Data Factory, Azure Synapse Analytics Workspace

+1. [PyApacheAtlas: Interface between Azure Purview and Apache Atlas](https://github.com/wjohnson/pyapacheatlas) using Atlas APIs

+ - **Recommended customer journey stages**: *Innovators, Enthusiasts, Adopters, Long-Term Regular Users*

+ - **Description**: A python package to work with Azure Purview and Apache Atlas API. Supports bulk loading, custom lineage, and more from a Pythonic set of classes and Excel templates. The package supports programmatic interaction and an Excel template for low-code uploads.

+1. [Azure Purview Event Hubs Notifications Reader](https://github.com/Azure/Azure-Purview-API-PowerShell/blob/main/purview_atlas_eventhub_sample.py)

+ - **Recommended customer journey stages**: *Innovators, Enthusiasts, Adopters, Long-Term Regular Users*

+ - **Description**: This tool demonstrates how to read Azure Purview's Event Hubs and catch real-time Kafka notifications from the Event Hubs in Atlas Notifications (https://atlas.apache.org/2.0.0/Notifications.html) format. Further, it generates an excel sheet CSV of the entities and assets on the fly that are discovered live during a scan, and any other notifications of interest that Azure Purview generates.

+## Feedback and disclaimer

+None of the tools come with an express warranty from Microsoft verifying their efficacy or guarantees of functionality. They are certified to be free of any malicious activity or viruses, and guaranteed to not collect any private or sensitive data.

+For feedback or questions about efficacy and functionality during usage, contact the respective tool owners and authors on the contact details mentioned in the respective GitHub repo.

+## Next steps

+> [!div class="nextstepaction"]

+> [Purview-API-PowerShell](https://aka.ms/purview-api-ps)

+To create and manage collections in Azure Purview, you'll need to be a **Collection Admin** within Azure Purview. We can check these permissions in the [Azure Purview Studio](use-azure-purview-studio.md).

+ Title: Use the Azure Purview Studio

+description: This article describes how to use Azure Purview Studio.

+# Use Azure Purview Studio

+This article gives an overview of some of the main features of Azure Purview.

+## Prerequisites

+* An Active Azure Purview account is already created in Azure portal and the user has permissions to access [Azure Purview Studio](https://web.purview.azure.com/resource/).

+## Launch Azure Purview account

+* To launch your Azure Purview account, go to Azure Purview accounts in Azure portal, select the account you want to launch and launch the account.

+ :::image type="content" source="./media/use-purview-studio/open-purview-studio.png" alt-text="Screenshot of Azure Purview window in Azure portal, with Azure Purview Studio button highlighted." border="true":::

+* Another way to launch Azure Purview account is to go to `https://web.purview.azure.com`, select **Azure Active Directory** and an account name to launch the account.

+## Home page

+**Home** is the starting page for the Azure Purview client.

+The following list summarizes the main features of **Home page**. Each number in the list corresponds to a highlighted number in the preceding screenshot.

+1. Friendly name of the catalog. You can set catalog name in **Management** > **Account information**.

+2. Catalog analytics shows the number of:

+ * Data sources

+ * Assets

+ * Glossary terms

+3. The search box allows you to search for data assets across the data catalog.

+4. The quick access buttons give access to frequently used functions of the application. The buttons that are presented, depend on the role assigned to your user account at the root collection.

+ * For *collection admin*, the available button is **Knowledge center**.

+ * For *data curator*, the buttons are **Browse assets**, **Manage glossary**, and **Knowledge center**.

+ * For *data reader*, the buttons are **Browse assets**, **View glossary**, and **Knowledge center**.

+ * For *data source admin* + *data curator*, the buttons are **Browse assets**, **Manage glossary**, and **Knowledge center**.

+ * For *data source admin* + *data reader*, the buttons are **Browse assets**, **View glossary**, and **Knowledge center**.

+

+ > [!NOTE]

+ > For more information about Azure Purview roles, see [Access control in Azure Purview](catalog-permissions.md).

+5. The left navigation bar helps you locate the main pages of the application.

+6. The **Recently accessed** tab shows a list of recently accessed data assets. For information about accessing assets, see [Search the Data Catalog](how-to-search-catalog.md) and [Browse by asset type](how-to-browse-catalog.md). **My items** tab is a list of data assets owned by the logged-on user.

+7. **Links** contains links to region status, documentation, pricing, overview, and Azure Purview status

+8. The top navigation bar contains information about release notes/updates, change purview account, notifications, help, and feedback sections.

+## Knowledge center

+Knowledge center is where you can find all the videos and tutorials related to Azure Purview.

+## Guided tours

+Each UX in Azure Purview Studio will have guided tours to give overview of the page. To start the guided tour, select **help** on the top bar and select **guided tours**.

+## Next steps

+> [!div class="nextstepaction"]

+> [Add a security principal](tutorial-scan-data.md)

+> Effective July 1, 2022, the Microsoft Security Code Analysis (MSCA) extension will be retired. Existing MSCA customers will retain their access to MSCA through July 1, 2022. Please refer to the [OWASP Source Code Analysis Tools](https://owasp.org/www-community/Source_Code_Analysis_Tools) for alternative options in Azure DevOps. For customers planning to migrate to GitHub, you can check out [GitHub Advanced Security](https://docs.github.com/github/getting-started-with-github/about-github-advanced-security).

+> Effective July 1, 2022, the Microsoft Security Code Analysis (MSCA) extension will be retired. Existing MSCA customers will retain their access to MSCA through July 1, 2022. Please refer to the [OWASP Source Code Analysis Tools](https://owasp.org/www-community/Source_Code_Analysis_Tools) for alternative options in Azure DevOps. For customers planning to migrate to GitHub, you can check out [GitHub Advanced Security](https://docs.github.com/github/getting-started-with-github/about-github-advanced-security).

+> Effective July 1, 2022, the Microsoft Security Code Analysis (MSCA) extension will be retired. Existing MSCA customers will retain their access to MSCA through July 1, 2022. Please refer to the [OWASP Source Code Analysis Tools](https://owasp.org/www-community/Source_Code_Analysis_Tools) for alternative options in Azure DevOps. For customers planning to migrate to GitHub, you can check out [GitHub Advanced Security](https://docs.github.com/github/getting-started-with-github/about-github-advanced-security).

+> Effective July 1, 2022, the Microsoft Security Code Analysis (MSCA) extension will be retired. Existing MSCA customers will retain their access to MSCA through July 1, 2022. Please refer to the [OWASP Source Code Analysis Tools](https://owasp.org/www-community/Source_Code_Analysis_Tools) for alternative options in Azure DevOps. For customers planning to migrate to GitHub, you can check out [GitHub Advanced Security](https://docs.github.com/github/getting-started-with-github/about-github-advanced-security).

+> Effective July 1, 2022, the Microsoft Security Code Analysis (MSCA) extension will be retired. Existing MSCA customers will retain their access to MSCA through July 1, 2022. Please refer to the [OWASP Source Code Analysis Tools](https://owasp.org/www-community/Source_Code_Analysis_Tools) for alternative options in Azure DevOps. For customers planning to migrate to GitHub, you can check out [GitHub Advanced Security](https://docs.github.com/github/getting-started-with-github/about-github-advanced-security).

+| - [Microsoft 365 Defender incident integration](../../sentinel/microsoft-365-defender-sentinel-integration.md#incident-integration) |Public Preview |Public Preview|

+ Title: Overview of Key Management in Azure

+description: This article provides an overview of Key Management in Azure.

+documentationcenter: na

+ na

+# Key management in Azure

+In Azure, encryption keys can be either platform managed or customer managed.

+Platform-managed keys (PMKs) are encryption keys that are generated, stored, and managed entirely by Azure. Customers do not interact with PMKs. The keys used for [Azure Data Encryption-at-Rest](encryption-atrest.md), for instance, are PMKs by default.

+Customer-managed keys (CMK), on the other hand, are those that can be read, created, deleted, updated, and/or administered by one or more customers. Keys stored in a customer-owned key vault or hardware security module (HSM) are CMKs. Bring Your Own Key (BYOK) is a CMK scenario in which a customer imports (brings) keys from an outside storage location into an Azure key management service (see the [Azure Key Vault: Bring your own key specification](../../key-vault/keys/byok-specification.md)).

+A specific kind of customer-managed key is the "key encryption key" (KEK). A KEK is a master key, that controls access to one or more encryption keys that are themselves encrypted.

+Customer-managed keys can be stored on-premise or, more commonly, in a cloud key management service.

+## Azure key management services

+Azure offers several options for storing and managing your keys in the cloud, including Azure Key Vault, Azure Managed HSM, Dedicated HSM, and Payments HSM. These options differ in terms of their FIPS compliance level, management overhead, and intended applications.

+**Azure Key Vault (Standard Tier)**: A FIPS 140-2 Level 1 validated multi-tenant cloud key management service that can also be used to store secrets and certificates. Keys stored in Azure Key Vault are software-protected and can be used for encryption-at-rest and custom applications. Key Vault provides a modern API and the widest breadth of regional deployments and integrations with Azure Services. For more information, see [About Azure Key Vault](../../key-vault/general/overview.md).

+**Azure Key Vault (Premium Tier)**: A FIPS 140-2 Level 2 validated multi-tenant HSM offering that can be used to store keys in a secure hardware boundary. Microsoft manages and operates the underlying HSM, and keys stored in Azure Key Vault Premium can be used for encryption-at-rest and custom applications. Key Vault Premium also provides a modern API and the widest breadth of regional deployments and integrations with Azure Services. For more information, see [About Azure Key Vault](../../key-vault/general/overview.md).

+**Azure Managed HSM**: A FIPS 140-2 Level 3 validated single-tenant HSM offering that gives customers full control of an HSM for encryption-at-rest, Keyless SSL, and custom applications. Customers receive a pool of three HSM partitionsΓÇötogether acting as one logical, highly available HSM appliance--fronted by a service that exposes crypto functionality through the Key Vault API. Microsoft handles the provisioning, patching, maintenance, and hardware failover of the HSMs, but does not have access to the keys themselves, because the service executes within Azure's Confidential Compute Infrastructure. Managed HSM is integrated with the Azure SQL, Azure Storage, and Azure Information Protection PaaS services and offers support for Keyless TLS with F5 and Nginx. For more information, see [What is Azure Key Vault Managed HSM?](../../key-vault/managed-hsm/overview.md)

+**Azure Dedicated HSM**: A FIPS 140-2 Level 3 validated bare metal HSM offering, that lets customers lease a general-purpose HSM appliance that resides in Microsoft datacenters. The customer has complete and total ownership over the HSM device and is responsible for patching and updating the firmware when required. Microsoft has no permissions on the device or access to the key material, and Dedicated HSM is not integrated with any Azure PaaS offerings. Customers can interact with the HSM using the PKCS#11, JCE/JCA, and KSP/CNG APIs. This offering is most useful for legacy lift-and-shift workloads, PKI, SSL Offloading and Keyless TLS (supported integrations include F5, Nginx, Apache, Palo Alto, IBM GW and more), OpenSSL applications, Oracle TDE, and Azure SQL TDE IaaS. For more information, see [What is Azure Key Vault Managed HSM?](../../dedicated-hsm/overview.md)

+**Azure Payments HSM**: A FIPS 140-2 Level 3, PCI HSM v3, validated bare metal offering that lets customers lease a payment HSM appliance in Microsoft datacenters for payments operations, including payment processing, payment credential issuing, securing keys and authentication data, and sensitive data protection. The service is currently undergoing PCI DSS and PCI 3DS audits. Azure Payment HSM offers single-tenant HSMs for customers to have complete administrative control and exclusive access to the HSM. Once the HSM is allocated to a customer, Microsoft has no access to customer data. Likewise, when the HSM is no longer required, customer data is zeroized and erased as soon as the HSM is released, to ensure complete privacy and security is maintained. This offering is currently in public preview. For more information, see [About Azure Key Vault](../../payment-hsm/overview.md).

+### Pricing

+The Azure Key Vault Standard and Premium tiers are billed on a transactional basis, with an additional monthly per-key charge for premium hardware-backed keys. Managed HSM, Dedicated HSM, and Payments HSM do not charge on a transactional basis; instead they are always-in-use devices that are billed at a fixed hourly rate. For detailed pricing information, see [Key Vault pricing](https://azure.microsoft.com/pricing/details/key-vault), [Dedicated HSM pricing](https://azure.microsoft.com/pricing/details/azure-dedicated-hsm), and [Payment HSM pricing](https://azure.microsoft.com/pricing/details/payment-hsm).

+### Service Limits

+Managed HSM, Dedicated HSM, and Payments HSM offer dedicated capacity. Key Vault Standard and Premium are multi-tenant offerings and have throttling limits. For service limits, see [Key Vault service limits](../../key-vault/general/service-limits.md).

+### Encryption-At-Rest

+Azure Key Vault and Azure Key Vault Managed HSM have integrations with Azure Services and Microsoft 365 for Customer Managed Keys, meaning customers may use their own keys in Azure Key Vault and Azure Key Managed HSM for encryption-at-rest of data stored in these services. Dedicated HSM and Payments HSM are Infrastructure-as-Service offerings and do not offer integrations with Azure Services. For an overview of encryption-at-rest with Azure Key Vault and Managed HSM, see [Azure Data Encryption-at-Rest](encryption-atrest.md).

+### APIs

+Dedicated HSM and Payments HSM support the PKCS#11, JCE/JCA, and KSP/CNG APIs, but Azure Key Vault and Managed HSM do not. Azure Key Vault and Managed HSM use the Azure Key Vault REST API and offer SDK support. For more information on the Azure Key Vault API, see [Azure Key Vault REST API Reference](/rest/api/keyvault/).