-Custom policies are a set of XML files that you upload to your Azure AD B2C tenant to define user journeys. We provide starter packs that have several pre-built policies, including sign-up and sign-in, password reset, and profile editing policies. For more information, see [Get started with custom policies in Azure AD B2C](tutorial-create-user-flows.md?pivots=b2c-custom-policy).

-* You need [Domain Services Contributor](/azure/role-based-access-control/built-in-roles#domain-services-contributor) Azure role to create the required Azure AD DS resources.

-* You need [Domain Services Contributor](/azure/role-based-access-control/built-in-roles#domain-services-contributor) Azure role to create the required Azure AD DS resources.

-As an application developer, you can use the System for Cross-Domain Identity Management (SCIM) user management API to enable automatic provisioning of users and groups between your application and Azure AD (AAD). This article describes how to build a SCIM endpoint and integrate with the AAD provisioning service. The SCIM specification provides a common user schema for provisioning. When used in conjunction with federation standards like SAML or OpenID Connect, SCIM gives administrators an end-to-end, standards-based solution for access management.

- Identify the application's objects and attributes to determine how they map to the user and group schema supported by the AAD SCIM implementation.

-1. Understand the AAD SCIM implementation

- Understand how the AAD SCIM client is implemented to model your SCIM protocol request handling and responses.

- An endpoint must be SCIM 2.0-compatible to integrate with the AAD provisioning service. As an option, use Microsoft Common Language Infrastructure (CLI) libraries and code samples to build your endpoint. These samples are for reference and testing only; we recommend against using them as dependencies in your production app.

-1. Integrate your SCIM endpoint with the AAD SCIM client

- If your organization uses a third-party application to implement a profile of SCIM 2.0 that AAD supports, you can quickly automate both provisioning and deprovisioning of users and groups.

-1. Publish your application to the AAD application gallery

-## Understand the AAD SCIM implementation

-To support a SCIM 2.0 user management API, this section describes how the AAD SCIM client is implemented and shows how to model your SCIM protocol request handling and responses.

-|Accept a single bearer token for authentication and authorization of AAD to your application.||

-Use the general guidelines when implementing a SCIM endpoint to ensure compatibility with AAD:

-* Values sent should be stored in the same format as what the were sent in. Invalid values should be rejected with a descriptive, actionable error message. Transformations of data should not happen between data being sent by Azure AD and data being stored in the SCIM application. (e.g. A phone number sent as 55555555555 should not be saved/returned as +5 (555) 555-5555)

-* Don't require a case-sensitive match on structural elements in SCIM, in particular **PATCH** `op` operation values, as defined in [section 3.5.2](https://tools.ietf.org/html/rfc7644#section-3.5.2). AAD emits the values of `op` as **Add**, **Replace**, and **Remove**.

-* Microsoft AAD makes requests to fetch a random user and group to ensure that the endpoint and the credentials are valid. It's also done as a part of the **Test Connection** flow in the [Azure portal](https://portal.azure.com).

-* Custom complex and multivalued attributes are supported but AAD does not have many complex data structures to pull data from in these cases. Simple paired name/value type complex attributes can be mapped to easily, but flowing data to complex attributes with three or more subattributes are not well supported at this time.

-* The "type" sub-attribute values of multivalued complex attributes must be unique. For example, there can not be two different email addresses with the "work" sub-type.

-* Microsoft AAD only uses the following operators: `eq`, `and`

-* The /schemas request will be made by the Azure AD SCIM client every time someone saves the provisioning configuration in the Azure Portal or every time a user lands on the edit provisioning page in the Azure Portal. Any additional attributes discovered will be surfaced to customers in the attribute mappings under the target attribute list. Schema discovery only leads to additional target attributes being added. It will not result in attributes being removed.

-The following illustration shows the messages that AAD sends to a SCIM service to manage the lifecycle of a user in your application's identity store.

-Group provisioning and deprovisioning are optional. When implemented and enabled, the following illustration shows the messages that AAD sends to a SCIM service to manage the lifecycle of a group in your application's identity store. Those messages differ from the messages about users in two ways:

-This section provides example SCIM requests emitted by the AAD SCIM client and example expected responses. For best results, you should code your app to handle these requests in this format and emit the expected responses.

-> To understand how and when the AAD user provisioning service emits the operations described below, see the section [Provisioning cycles: Initial and incremental](how-provisioning-works.md#provisioning-cycles-initial-and-incremental) in [How provisioning works](how-provisioning-works.md).

-Azure AD also supports an agent based solution to provide connectivity to applications in private networks (on-premises, hosted in Azure, hosted in AWS, etc.). Customers can deploy a lightweight agent, which provides connectivity to Azure AD without opening an inbound ports, on a server in their private network. Learn more [here](./on-premises-scim-provisioning.md).

-Azure Active Directory (AAD) queries the service for a user with an `externalId` attribute value matching the mailNickname attribute value of a user in AAD. The query is expressed as a Hypertext Transfer Protocol (HTTP) request such as this example, wherein jyoung is a sample of a mailNickname of a user in Azure Active Directory.

-> This is an example only. Not all users will have a mailNickname attribute, and the value a user has may not be unique in the directory. Also, the attribute used for matching (which in this case is `externalId`) is configurable in the [AAD attribute mappings](customize-application-attributes.md).

-If the response to a query to the web service for a user with an `externalId` attribute value that matches the mailNickname attribute value of a user doesn't return any users, then AAD requests that the service provision a user corresponding to the one in AAD. Here is an example of such a request:

-To deprovision a user from an identity store fronted by an SCIM service, AAD sends a request such as:

-## Integrate your SCIM endpoint with the AAD SCIM client

-Azure AD can be configured to automatically provision assigned users and groups to applications that implement a specific profile of the [SCIM 2.0 protocol](https://tools.ietf.org/html/rfc7644). The specifics of the profile are documented in [Understand the Azure AD SCIM implementation](#understand-the-aad-scim-implementation).

-1. Sign in to the [AAD portal](https://aad.portal.azure.com). Note that you can get access a free trial for Azure Active Directory with P2 licenses by signing up for the [developer program](https://developer.microsoft.com/office/dev-program)

-## Publish your application to the AAD application gallery

-> * Support a [SCIM 2.0](#understand-the-aad-scim-implementation) user and group endpoint (Only one is required but both are recommended)

-> It's not recommended to leave the token field blank in the AAD provisioning configuration custom app UI. The token generated is primarily available for testing purposes.

-> While it's not possible to setup OAuth on the non-gallery applications, you can manually generate an access token from your authorization server and input it as the secret token to a non-gallery application. This allows you to verify compatibility of your SCIM server with the AAD SCIM client before onboarding to the app gallery, which does support the OAuth code grant.

-[Learn more about passwordless authentication](concept-authentication-passwordless.md)

-Microsoft works with researchers, law enforcement, various security teams at Microsoft, and other trusted sources to find leaked username and password pairs. Organizations with Azure AD Premium P2 licenses can create Conditional Access policies incorporating [Azure AD Identity Protection user risk detections](../identity-protection/concept-identity-protection-risks.md#user-linked-detections).

-1. Under **Conditions** > **User risk**, set **Configure** to **Yes**. Under **Configure user risk levels needed for policy to be enforced** select **High**, then select **Done**.

-1. Under **Access controls** > **Grant**, select **Grant access**, **Require password change**, and select **Select**.

-1. Confirm your settings and set **Enable policy** to **Report-only**.

-## Enable through Identity Protection

-1. Sign in to the **Azure portal**.

-1. Select **All services**, then browse to **Azure AD Identity Protection**.

-1. Select **User risk policy**.

-1. Under **Assignments**, select **Users**.

- 1. Under **Include**, select **All users**.

- 1. Under **Exclude**, select **Select excluded users**, choose your organization's emergency access or break-glass accounts, and select **Select**.

- 1. Select **Done**.

-1. Under **Conditions**, select **User risk**, then choose **High**.

- 1. Select **Select**, then **Done**.

-1. Under **Controls** > **Access**, choose **Allow access**, and then select **Require password change**.

- 1. Select **Select**.

-1. Set **Enforce Policy** to **On**.

-1. Select **Save**.

-1. Under **Conditions** > **Sign-in risk**, set **Configure** to **Yes**. Under **Select the sign-in risk level this policy will apply to**

-1. Under **Access controls** > **Grant**, select **Grant access**, **Require multi-factor authentication**, and select **Select**.

-## Enable through Identity Protection

-1. Sign in to the **Azure portal**.

-1. Select **All services**, then browse to **Azure AD Identity Protection**.

-1. Select **Sign-in risk policy**.

-1. Under **Assignments**, select **Users**.

- 1. Under **Include**, select **All users**.

- 1. Under **Exclude**, select **Select excluded users**, choose your organization's emergency access or break-glass accounts, and select **Select**.

- 1. Select **Done**.

-1. Under **Conditions**, select **Sign-in risk**, then choose **Medium and above**.

- 1. Select **Select**, then **Done**.

-1. Under **Controls** > **Access**, choose **Allow access**, and then select **Require multi-factor authentication**.

- 1. Select **Select**.

-1. Set **Enforce Policy** to **On**.

-1. Select **Save**.

-[Azure Active Directory documentation](../index.yml)

-Azure Active Directory (Azure AD) supports applying sensitivity labels published by the [Microsoft 365 compliance center](https://sip.protection.office.com/homepage) to Microsoft 365 groups. Sensitivity labels apply to group across services like Outlook, Microsoft Teams, and SharePoint. For more information about Microsoft 365 apps support, see [Microsoft 365 support for sensitivity labels](/microsoft-365/compliance/sensitivity-labels-teams-groups-sites#support-for-the-sensitivity-labels).

-1. Labels are published in the Microsoft 365 Compliance Center for this Azure AD organization.

-When you make changes to group settings for a published label in [Microsoft 365 compliance center](https://sip.protection.office.com/homepage), those policy changes aren't automatically applied on the labeled groups. Once the sensitivity label is published and applied to groups, Microsoft recommend that you not change the group settings for the label in Microsoft 365 Compliance Center.

-As an Azure or Microsoft 365 subscriber, you can purchase the Azure Active Directory Premium editions online. For detailed steps, see [Buy or remove licenses](https://docs.microsoft.com/microsoft-365/commerce/licenses/buy-licenses?view=o365-worldwide).

-Now that you have Azure AD Premium, you can [customize your domain](add-custom-domain.md), add your [corporate branding](customize-branding.md), [create a tenant](active-directory-access-create-new-tenant.md), and [add groups](active-directory-groups-create-azure-portal.md) and [users](add-users-azure-active-directory.md).

-To learn the differences between Azure Active Directory and Azure Active Directory, see [Compare Active Directory to Azure Active Directory](active-directory-compare-azure-ad-to-ad.md). You can also refer [Microsoft Cloud for Enterprise Architects Series](/microsoft-365/solutions/cloud-architecture-models) posters to better understand the core identity services in Azure like Azure AD and Microsoft-365.

-If you are using cloud Azure MFA, for multi factor authentication, with federated users, we highly recommend enabling additional security protection. This security protection prevents bypassing of cloud Azure MFA when federated with Azure AD. When enabled, for a federated domain in your Azure AD tenant, it ensures that a bad actor cannot bypass Azure MFA by imitating that a multi factor authentication has already been performed by the identity provider. The protection can be enabled via new security setting, `federatedIdpMfaBehavior`.For additional information see [Best practices for securing Active Directory Federation Services](https://docs.microsoft.com/windows-server/identity/ad-fs/deployment/best-practices-securing-ad-fs#enable-protection-to-prevent-by-passing-of-cloud-azure-mfa-when-federated-with-azure-ad)

-* [Manage and customize Active Directory Federation Services using Azure AD Connect](how-to-connect-fed-management.md)

-To learn more about the differences between Azure AD Connect sync and Azure AD Connect cloud provisioning, see the article [What is Azure AD Connect cloud provisioning?](../cloud-sync/what-is-cloud-sync.md)

-Learn more about how to [integrate your on-premises identities with Azure AD](whatis-hybrid-identity.md).

-Risk can be detected at the **User** and **Sign-in** level and two types of detection or calculation **Real-time** and **Offline**.

-> Our system may detect that the risk event that contributed to the risk user risk score was a false positives or the user risk was remediated with policy enforcement such as completing an MFA prompt or secure password change. Therefore our system will dismiss the risk state and a risk detail of ΓÇ£AI confirmed sign-in safeΓÇ¥ will surface and it will no longer contribute to the userΓÇÖs risk.

-### User-linked detections

-Risky activity can be detected for a user that is not linked to a specific malicious sign-in but to the user itself.

-These risks are calculated offline using Microsoft's internal and external threat intelligence sources including security researchers, law enforcement professionals, security teams at Microsoft, and other trusted sources.

-| Risk detection | Description |

-| | |

-| Leaked credentials | This risk detection type indicates that the user's valid credentials have been leaked. When cybercriminals compromise valid passwords of legitimate users, they often share those credentials. This sharing is typically done by posting publicly on the dark web, paste sites, or by trading and selling the credentials on the black market. When the Microsoft leaked credentials service acquires user credentials from the dark web, paste sites, or other sources, they are checked against Azure AD users' current valid credentials to find valid matches. For more information about leaked credentials, see [Common questions](#common-questions). |

-| Azure AD threat intelligence | This risk detection type indicates user activity that is unusual for the given user or is consistent with known attack patterns based on Microsoft's internal and external threat intelligence sources. |

-| Possible attempt to access Primary Refresh Token (PRT)| This risk detection type is detected by Microsoft Defender for Endpoint (MDE). A Primary Refresh Token (PRT) is a key artifact of Azure AD authentication on Windows 10, Windows Server 2016, and later versions, iOS, and Android devices. It is a JSON Web Token (JWT) that's specially issued to Microsoft first-party token brokers to enable single sign-on (SSO) across the applications used on those devices. Attackers can attempt to access this resource to move laterally into an organization or perform credential theft. This detection will move users to high risk and will only fire in organizations that have deployed MDE. This is a low-volume detection that will be infrequently seen by most organizations. However, when it does occur it is high risk and users should be remediated.|

-A sign-in risk represents the probability that a given authentication request is not authorized by the identity owner.

-These risks can be calculated in real-time or calculated offline using Microsoft's internal and external threat intelligence sources including security researchers, law enforcement professionals, security teams at Microsoft, and other trusted sources.

-| Anonymous IP address | Real-time | This risk detection type indicates sign-ins from an anonymous IP address (for example, Tor browser or anonymous VPN). These IP addresses are typically used by actors who want to hide their login telemetry (IP address, location, device, and so on) for potentially malicious intent. |

-| Anomalous Token | Offline | This detection indicates that there are abnormal characteristics in the token such as an unusual token lifetime or a token that is played from an unfamiliar location. This detection covers Session Tokens and Refresh Tokens. ***NOTE:** Anomalous token is tuned to incur more noise than other detections at the same risk level. This tradeoff is chosen to increase the likelihood of detecting replayed tokens that may otherwise go unnoticed. Because this is a high noise detection, there is a higher than normal chance that some of the sessions flagged by this detection are false positives. We recommend investigating the sessions flagged by this detection in the context of other sign-ins from the user. If the location, application, IP address, User Agent, or other characteristics are unexpected for the user, the tenant admin should consider this as an indicator of potential token replay*. |

-| Unfamiliar sign-in properties | Real-time | This risk detection type considers past sign-in history (IP, Latitude / Longitude and ASN) to look for anomalous sign-ins. The system stores information about previous locations used by a user, and considers these "familiar" locations. The risk detection is triggered when the sign-in occurs from a location that is not already in the list of familiar locations. Newly created users will be in "learning mode" for a while where unfamiliar sign-in properties risk detections will be turned off while our algorithms learn the user's behavior. The learning mode duration is dynamic and depends on how much time it takes the algorithm to gather enough information about the user's sign-in patterns. The minimum duration is five days. A user can go back into learning mode after a long period of inactivity and after a secure password reset. The system also ignores sign-ins from familiar devices, and locations that are geographically close to a familiar location. <br><br> We also run this detection for basic authentication (or legacy protocols). Because these protocols do not have modern properties such as client ID, there is limited telemetry to reduce false positives. We recommend our customers to move to modern authentication. <br><br> Unfamiliar sign-in properties can be detected on both interactive and non-interactive sign-ins. When this detection is detected on non-interactive sign-ins, it deserves increased scrutiny due to the risk of token replay attacks. |

-| Admin confirmed user compromised | Offline | This detection indicates an admin has selected 'Confirm user compromised' in the Risky users UI or using riskyUsers API. To see which admin has confirmed this user compromised, check the user's risk history (via UI or API). |

-| Azure AD threat intelligence | Offline | This risk detection type indicates sign-in activity that is unusual for the given user or is consistent with known attack patterns based on Microsoft's internal and external threat intelligence sources. |

-### Other risk detections

-| Risk detection | Detection type | Description |

-| Additional risk detected | Real-time or Offline | This detection indicates that one of the above premium detections was detected. Since the premium detections are visible only to Azure AD Premium P2 customers, they are titled "additional risk detected" for customers without Azure AD Premium P2 licenses. |

-Identity Protection categorizes risk into three tiers: low, medium, and high. When configuring [custom Identity protection policies](./concept-identity-protection-policies.md#custom-conditional-access-policy), you can also configure it to trigger upon **No risk** level. No Risk means there is no active indication that the user's identity has been compromised.

-While Microsoft does not provide specific details about how risk is calculated, we will say that each level brings higher confidence that the user or sign-in is compromised. For example, something like one instance of unfamiliar sign-in properties for a user might not be as threatening as leaked credentials for another user.

-Leaked credentials are processed anytime Microsoft finds a new, publicly available batch. Because of the sensitive nature, the leaked credentials are deleted shortly after processing. Only new leaked credentials found after you enable password hash synchronization (PHS) will be processed against your tenant. Verifying against previously found credential pairs is not done.

-If you have not seen any leaked credential risk events, it is because of the following reasons:

-Organizations must decide the level of risk they are willing to accept balancing user experience and security posture.

- - Azure AD MFA can be triggered, allowing to user to prove it is them by using one of their registered authentication methods, resetting the sign-in risk.

-Policies allow for excluding users such as your [emergency access or break-glass administrator accounts](../roles/security-emergency-access.md). Organizations may need to exclude other accounts from specific policies based on the way the accounts are used. Exclusions should be reviewed regularly to see if they are still applicable.

-There are two locations where these policies may be configured, Conditional Access and Identity Protection. Configuration using Conditional Access policies is the preferred method, providing more context including:

-Before enabling remediation policies, organizations may want to [investigate](howto-identity-protection-investigate-risk.md) and [remediate](howto-identity-protection-remediate-unblock.md) any active risks.

-| Azure Purview | [Credentials for source authentication in Azure Purview](../../purview/manage-credentials.md) |

-> | Create Azure AD Domain Services instance | [Application Administrator](../roles/permissions-reference.md#application-administrator)<br>[Groups Administrator](../roles/permissions-reference.md#groups-administrator)<br> [Domain Services Contributor](/azure/role-based-access-control/built-in-roles#domain-services-contributor)| |

-Require Azure AD Multi-Factor Authentication (MFA) at sign-in for all individual users who are permanently assigned to one or more of the Azure AD administrator roles: Global Administrator, Privileged Role Administrator, Exchange Administrator, and SharePoint Administrator. Use the guide to enable [Multi-factor Authentication (MFA) for your administrator accounts](../authentication/howto-mfa-userstates.md) and ensure that all those users have registered at [https://aka.ms/mfasetup](https://aka.ms/mfasetup). More information can be found under step 2 and step 3 of the guide [Protect access to data and services in Microsoft 365](https://support.office.com/article/Protect-access-to-data-and-services-in-Office-365-a6ef28a4-2447-4b43-aae2-f5af6d53c68e).

-## Adding Cisco Webex Meetings from the gallery

-

- * **[Create Cisco Webex Meetings test user](#create-cisco-webex-meetings-test-user)** - to have a counterpart of B.Simon in Cisco Webex Meetings that is linked to the Azure AD representation of user.

-

- 1. On the **Basic SAML Configuration** section, click the edit/pen icon.

- 

- 

- 1. select **SAML 2.0** as **Federation Protocol**.

- 1. In the **AuthContextClassRef** textbox, type one of the following values:

- * `urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified`

- * `urn:oasis:names:tc:SAML:2.0:ac:classes:Password`

-

- To enable the MFA by using Azure AD, enter the two values like this:

- `urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport;urn:oasis:names:tc:SAML:2.0:ac:classes:X509`

-Once you configure Cisco Webex Meetings you can enforce Session Control, which protects exfiltration and infiltration of your organizationΓÇÖs sensitive data in real time. Session Control extends from Conditional Access. [Learn how to enforce session control with Microsoft Defender for Cloud Apps](/cloud-app-security/proxy-deployment-aad).

-To learn about all the benefits, see the article on [F5 BIG-IP and Azure AD integration](/azure/active-directory/manage-apps/f5-aad-integration) and [what is application access and single sign-on with Azure AD](/azure/active-directory/active-directory-appssoaccess-whatis).

-* An [SSL Web certificate](/azure/active-directory/manage-apps/f5-bigip-deployment-guide#ssl-profile) for publishing services over HTTPS, or use default BIG-IP certs while testing

-If making a change to the app is a no go, then consider having the BIG-IP listen for the application's sign-out call, and upon detecting the request have it trigger SLO. Refer to our [Oracle PeopleSoft SLO guidance](/azure/active-directory/manage-apps/f5-big-ip-oracle-peoplesoft-easy-button#peoplesoft-single-logout) for using BIG-IP irules to achieve this. More details on using BIG-IP iRules to achieve this is available in the F5 knowledge article [Configuring automatic session termination (logout) based on a URI-referenced file name](https://support.f5.com/csp/article/K42052145) and [Overview of the Logout URI Include option](https://support.f5.com/csp/article/K12056).

-There may be cases where the Guided Configuration templates lack the flexibility to achieve more specific requirements. For those scenarios, see [Advanced Configuration for headers-based SSO](/azure/active-directory/manage-apps/f5-big-ip-header-advanced). Alternatively, the BIG-IP gives the option to disable **Guided ConfigurationΓÇÖs strict management mode**. This allows you to manually tweak your configurations, even though bulk of your configurations are automated through the wizard-based templates.

-For more information, visit this F5 knowledge article [Configuring LDAP remote authentication for Active Directory](https://support.f5.com/csp/article/K11072). ThereΓÇÖs also a great BIG-IP reference table to help diagnose LDAP-related issues in this [F5 knowledge article on LDAP Query](https://techdocs.f5.com/en-us/bigip-16-1-0/big-ip-access-policy-manager-authentication-methods/ldap-query.html).

-To learn about all the benefits, see the article on [F5 BIG-IP and Azure AD integration](/azure/active-directory/manage-apps/f5-aad-integration) and [what is application access and single sign-on with Azure AD](/azure/active-directory/active-directory-appssoaccess-whatis).

-* An existing BIG-IP or [deploy a BIG-IP Virtual Edition (VE) in Azure](/azure/active-directory/manage-apps/f5-bigip-deployment-guide)

-* An [SSL Web certificate](/azure/active-directory/manage-apps/f5-bigip-deployment-guide#ssl-profile) for publishing services over HTTPS, or use default BIG-IP certs while testing

-If making a change to the app is a no go, then consider having the BIG-IP listen for the application's sign-out call, and upon detecting the request have it trigger SLO. Refer to our [Oracle PeopleSoft SLO guidance](/azure/active-directory/manage-apps/f5-big-ip-oracle-peoplesoft-easy-button#peoplesoft-single-logout) for using BIG-IP irules to achieve this. More details on using BIG-IP iRules to achieve this is available in the F5 knowledge article [Configuring automatic session termination (logout) based on a URI-referenced file name](https://support.f5.com/csp/article/K42052145) and [Overview of the Logout URI Include option](https://support.f5.com/csp/article/K12056).

-There may be cases where the Guided Configuration templates lack the flexibility to achieve more specific requirements. For those scenarios, see [Advanced Configuration for headers-based SSO](/azure/active-directory/manage-apps/f5-big-ip-header-advanced). Alternatively, the BIG-IP gives the option to disable **Guided ConfigurationΓÇÖs strict management mode**. This allows you to manually tweak your configurations, even though bulk of your configurations are automated through the wizard-based templates.

-See [BIG-IP APM variable assign examples](https://devcentral.f5.com/s/articles/apm-variable-assign-examples-1107) and [F5 BIG-IP session variables reference](https://techdocs.f5.com/en-us/bigip-15-0-0/big-ip-access-policy-manager-visual-policy-editor/session-variables.html) for more info.

-To learn about all the benefits, see the article on [F5 BIG-IP and Azure AD integration](/azure/active-directory/manage-apps/f5-aad-integration) and [what is application access and single sign-on with Azure AD](/azure/active-directory/active-directory-appssoaccess-whatis).

-* An existing BIG-IP or [deploy a BIG-IP Virtual Edition (VE) in Azure](/azure/active-directory/manage-apps/f5-bigip-deployment-guide)

-* An [SSL Web certificate](/azure/active-directory/manage-apps/f5-bigip-deployment-guide#ssl-profile) for publishing services over HTTPS, or use default BIG-IP certs while testing

-As our example AD infrastructure is based on a .com domain suffix used both, internally and externally, we donΓÇÖt require any additional attributes to achieve a functional KCD SSO implementation. See the [advanced tutorial](/azure/active-directory/manage-apps/f5-big-ip-kerberos-advanced) for cases where you have multiple domains or userΓÇÖs log-in using an alternate suffix.

-There may be cases where the Guided Configuration templates lack the flexibility to achieve more specific requirements. For those scenarios, see [Advanced Configuration for kerberos-based SSO](/azure/active-directory/manage-apps/f5-big-ip-kerberos-advanced).

-See [BIG-IP APM variable assign examples]( https://devcentral.f5.com/s/articles/apm-variable-assign-examples-1107) and [F5 BIG-IP session variables reference]( https://techdocs.f5.com/en-us/bigip-15-0-0/big-ip-access-policy-manager-visual-policy-editor/session-variables.html) for more info.

- a. In the **Identifier (Entity ID)** text box, type a URL using the following pattern:

- `https://<TENANTHOSTNAME>.edge.prod.fortisase.com/remote/saml/metadata`

- b. In the **Reply URL** text box, type a URL using the following pattern:

- `https://<TENANTHOSTNAME>.edge.prod.fortisase.com/remote/saml/login`

- c. In the **Sign on URL** text box, type a URL using the following pattern:

- `https://<TENANTHOSTNAME>.edge.prod.fortisase.com/remote/login`

- > These values are not real. Update these values with the actual Identifier, Reply URL and Sign on URL. Contact [FortiSASE Client support team](mailto:fgc@fortinet.com) to get these values. You can also refer to the patterns shown in the **Basic SAML Configuration** section in the Azure portal.

-To configure single sign-on on **FortiSASE** side, you need to send the downloaded **Certificate (Base64)** and appropriate copied URLs from Azure portal to [FortiSASE support team](mailto:fgc@fortinet.com). They set this setting to have the SAML SSO connection set properly on both sides.

-In this section, a user called Britta Simon is created in FortiSASE. FortiSASE supports just-in-time user provisioning, which is enabled by default. There is no action item for you in this section. If a user doesn't already exist in FortiSASE, a new one is created after authentication.

-Learn more about [Virtual machine - LowUsageVmV2 (Right-size or shutdown underutilized virtual machines)](/azure/advisor/advisor-cost-recommendations#optimize-virtual-machine-spend-by-resizing-or-shutting-down-underutilized-instances).

-Learn more about [Cost Optimization - Microsoft Azure Well Architected Framework](/azure/architecture/framework/cost/overview)

-Learn more about [Batch account - RecreatePool (Delete and recreate your pool to remove a deprecated internal component)](/azure/batch/best-practices#pool-lifetime-and-billing).

-Learn more about [Batch account - EolImage (Recreate your pool with a new image)](/azure/batch/batch-pool-vm-sizes#supported-vm-images).

-Learn more about [Kubernetes service - MonitoringAddonWorkspaceIsDeleted (Monitoring addon workspace is deleted)](/azure/azure-monitor/containers/container-insights-optout#azure-cli).

-Learn more about [Alert Rule - ScheduledQueryRulesLogAlert (Repair your log alert rule)](/azure/azure-monitor/alerts/alerts-troubleshoot-log#query-used-in-a-log-alert-is-not-valid).

-Learn more about [Alert Rule - ScheduledQueryRulesRp (Log alert rule was disabled)](/azure/azure-monitor/alerts/alerts-troubleshoot-log#query-used-in-a-log-alert-is-not-valid).

-Learn more about [Storage Account - StorageAccountScaleTarget (Prevent hitting subscription limit for maximum storage accounts)](/azure/storage/blobs/storage-performance-checklist#what-to-do-when-approaching-a-scalability-target).

-Learn more about [Operational Excellence - Microsoft Azure Well Architected Framework](/azure/architecture/framework/devops/overview)

-Learn more about [Redis Cache Server - RedisCacheNetworkBandwidth (Improve your Cache and application performance when running with high network bandwidth)](/azure/azure-cache-for-redis/cache-troubleshoot-server#server-side-bandwidth-limitation).

-Learn more about [Redis Cache Server - RedisCacheServerLoad (Improve your Cache and application performance when running with high server load)](/azure/azure-cache-for-redis/cache-troubleshoot-client#high-client-cpu-usage).

-Learn more about [Redis Cache Server - RedisCacheUsedMemory (Improve your Cache and application performance when running with high memory pressure)](/azure/azure-cache-for-redis/cache-troubleshoot-client#memory-pressure-on-redis-client).

-Learn more about [Traffic Manager profile - FastFailOverTTL (Configure DNS Time to Live to 20 seconds)](/azure/traffic-manager/traffic-manager-monitoring#endpoint-failover-and-recovery).

-Learn more about [SQL data warehouse - DataSkewSqlDW (Remove data skew to increase query performance)](/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-tables-distribute#how-to-tell-if-your-distribution-column-is-a-good-choice).

-Learn more about [SQL data warehouse - SqlDwReduceTempdbContention (Scale up or update resource class to reduce tempdb contention with SQL Data Warehouse)](/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-manage-monitor#monitor-tempdb).

-Learn more about [SQL data warehouse - FileSplittingGuidance (Split staged files in the storage account to increase load performance)](/azure/synapse-analytics/sql/data-loading-best-practices#preparing-data-in-azure-storage).

-Learn more about [SQL data warehouse - LoadBatchSizeGuidance (Increase batch size when loading to maximize load throughput, data compression, and query performance)](/azure/synapse-analytics/sql/data-loading-best-practices#increase-batch-size-when-using-sqlbulkcopy-api-or-bcp).

-Learn more about [SQL data warehouse - ColocateStorageAccount (Co-locate the storage account within the same region to minimize latency when loading)](/azure/synapse-analytics/sql/data-loading-best-practices#preparing-data-in-azure-storage).

-Learn more about [App service - AppServiceOutboundConnections (Check outbound connections from your App Service resource)](/azure/app-service/app-service-best-practices#socketresources).

-Learn more about [Performance Efficiency - Microsoft Azure Well Architected Framework](/azure/architecture/framework/scalability/overview)

-Learn more about [Redis Cache Server - RedisCacheMemoryFragmentation (Availability may be impacted from high memory fragmentation. Increase fragmentation memory reservation to avoid potential impact.)](/azure/azure-cache-for-redis/cache-configure#memory-policies).

-Learn more about [Virtual machine - MigrateStandardStorageAccountToPremium (Upgrade the standard disks attached to your premium-capable VM to premium disks)](/azure/virtual-machines/disks-types#premium-ssd).

-Learn more about [Virtual machine - ASRUpdateOutboundConnectivityProtocolToServiceTags (Update your outbound connectivity protocol to Service Tags for Azure Site Recovery)](/azure/site-recovery/azure-to-azure-about-networking#outbound-connectivity-using-service-tags).

-Learn more about [Cosmos DB account - CosmosDBMongoServerSideRetries (Enable Server Side Retry (SSR) on your Azure Cosmos DB's API for MongoDB account)](/azure/cosmos-db/cassandra/prevent-rate-limiting-errors).

-Learn more about [Virtual network gateway - BasicVPNGateway (Move to production gateway SKUs from Basic gateways)](/azure/vpn-gateway/vpn-gateway-about-vpn-gateway-settings#gwsku).

-Learn more about [Application gateway - AppGatewayHostOverride (Avoid hostname override to ensure site integrity)](/azure/application-gateway/troubleshoot-app-service-redirection-app-service-url#alternate-solution-use-a-custom-domain-name).

-Learn more about [Storage Account - StorageSoftDelete (Enable Soft Delete to protect your blob data)](https://aka.ms/softdelete).

-Learn more about [Storage Account - StoragePremiumBlobQuotaLimit (Use Managed Disks for storage accounts reaching capacity limit)](/azure/storage/common/scalability-targets-standard-account#premium-performance-page-blob-storage).

-Learn more about [App service - AppServiceCPUExhaustion (Consider scaling out your App Service Plan to avoid CPU exhaustion)](/azure/app-service/app-service-best-practices#CPUresources).

-Learn more about [App service - AppServiceFixBackupDatabaseSettings (Fix the backup database settings of your App Service resource)](/azure/app-service/app-service-best-practices#appbackup.).

-Learn more about [App service - AppServiceMemoryExhaustion (Consider scaling up your App Service Plan SKU to avoid memory exhaustion)](/azure/app-service/app-service-best-practices#memoryresources).

-Learn more about [App service - AppServiceFixBackupStorageSettings (Fix the backup storage settings of your App Service resource)](/azure/app-service/app-service-best-practices#appbackup.).

-With *kubenet*, only the nodes receive an IP address in the virtual network subnet. Pods can't communicate directly with each other. Instead, User Defined Routing (UDR) and IP forwarding is used for connectivity between pods across nodes. By default, UDRs and IP forwarding configuration is created and maintained by the AKS service, but you have to the option to [bring your own route table for custom route management][byo-subnet-route-table]. You could also deploy pods behind a service that receives an assigned IP address and load balances traffic for the application. The following diagram shows how the AKS nodes receive an IP address in the virtual network subnet, but not the pods:

-[user-assigned managed identity]: use-managed-identity.md#bring-your-own-control-plane-mi

-Sign in to the [Azure portal](https://portal.azure.com/), and then select the [Cloud Shell](/azure/cloud-shell/overview) button in the upper-right corner.

-Sign in to the [Azure portal](https://portal.azure.com/), and then select the [Cloud Shell](/azure/cloud-shell/overview) button in the upper-right corner.

- --resource-group myresourcegroup \

- --outbound-type managedNATGateway \

-* Azure Resource Manager [template](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.apimanagement/api-management-create-with-internal-vnet-publicip) (API version 2021-01-01-preview )

-* Azure Resource Manager [template](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.apimanagement/api-management-create-with-external-vnet-publicip) (API version 2021-01-01-preview)

-* Learn more about [configuring policies](/azure/api-management/api-management-howto-policies) in API Management.

-* Learn about API Management [capacity](api-management-capacity.md).

-* Websockets APIs support the following valid buffer types for messages: Close, BinaryFragment, BinayrMessage, UTF8Fragment, and UTF8Message.

-> [Using an App Service Environment v3](using.md)

-The deployment process to Azure App Service will use your Azure credentials from the Azure CLI automatically. If the Azure CLI isn't installed locally, then the Maven plugin will authenticate with Oauth or device sign in. For more information, see [authentication with Maven plugins](https://github.com/microsoft/azure-maven-plugins/wiki/Authentication).

-The following types of redirection are supported:

-Application Gateway redirection support offers the following capabilities:

- Redirects from one listener to another listener on the gateway. This enables HTTP to HTTPS redirection on a site.

- When configuring redirects with a multi-site target listener, it is required that all the host names (with or without wildcard characters) defined as part of the source listener are also part of the destination listener. This ensures that no traffic is dropped due to missing host names on the destination listener while setting up HTTP to HTTPS redirection.

- This type of redirection enables HTTP to HTTPS redirection only on a specific site area, for example a shopping cart area denoted by /cart/*.

-* The file size must be less than 50 MB.

-* The file size must be less than 50 MB.

- > Custom neural models models are only available in a few regions. If you plan on training a neural model, please select or create a resource in one of [these supported regions](/azure/applied-ai-services/form-recognizer/concept-custom-neural#l).

- endpoint=endpoint, credential=AzureKeyCredential(key)

- )

- document_analysis_client = DocumentAnalysisClient(

- endpoint=endpoint, credential=AzureKeyCredential(key)

- )

- document_analysis_client = DocumentAnalysisClient(

- endpoint=endpoint, credential=AzureKeyCredential(key)

- )

-# Audit logs for Azure Attestation

-Audit logs are secure, immutable, timestamped records of discrete events that happened over time. These logs capture important events that may affect the functionality of your attestation instance.

-Azure Attestation manages attestation instances and the policies associated with them. Actions associated with instance management and policy changes are audited and logged.

-This article contains information on the events that are logged, the information collected, and the location of these logs.

-## About Audit logs

-Azure Attestation uses code to produce audit logs for events that affect the way attestation is performed. This typically boils down to how or when policy changes are made to your attestation instance as well as some admin actions.

-### Auditable Events

-Here are some of the audit logs we collect:

-| Event/API | Event Description |

-|--|--|

-| Create Instance | Creates a new instance of an attestation service. |

-| Destroy Instance | Destroys an instance of an attestation service. |

-| Add Policy Certificate | Addition of a certificate to the current set of policy management certificates. |

-| Remove Policy Certificate | Remove a certificate from the current set of policy management certificates. |

-| Set Current Policy | Sets the attestation policy for a given TEE type. |

-| Reset Attestation Policy | Resets the attestation policy for a given TEE type. |

-| Prepare to Update Policy | Prepare to update attestation policy for a given TEE type. |

-| Rehydrate Tenants After Disaster | Re-seals all of the attestation tenants on this instance of the attestation service. This can only be performed by Attestation Service admins. |

-### Collected information

-For each of these events, Azure Attestation collects the following information:

- - Azure AD UPN

- - Object ID

- - Certificate

- - Azure AD Tenant ID

- - Environment

- - Service Region

- - Service Role

- - Service Role Instance

- - Resource ID

- - Resource Region

-### Sample Audit log

-Audit logs are provided in JSON format. Here is an example of what an audit log may look like.

-```json

-{

- "operationName": "SetCurrentPolicy",

- "resultType": "Success",

- "resultDescription": null,

- "auditEventCategory": [

- "ApplicationManagement"

- ],

- "nCloud": null,

- "requestId": null,

- "callerIpAddress": null,

- "callerDisplayName": null,

- "callerIdentities": [

- {

- "callerIdentityType": "ObjectID",

- "callerIdentity": "<some object ID>"

- },

- {

- "callerIdentityType": "TenantId",

- "callerIdentity": "<some tenant ID>"

- }

- ],

- "targetResources": [

- {

- "targetResourceType": "Environment",

- "targetResourceName": "PublicCloud"

- },

- {

- "targetResourceType": "ServiceRegion",

- "targetResourceName": "EastUS2"

- },

- {

- "targetResourceType": "ServiceRole",

- "targetResourceName": "AttestationRpType"

- },

- {

- "targetResourceType": "ServiceRoleInstance",

- "targetResourceName": "<some service role instance>"

- },

- {

- "targetResourceType": "ResourceId",

- "targetResourceName": "/subscriptions/<some subscription ID>/resourceGroups/<some resource group name>/providers/Microsoft.Attestation/attestationProviders/<some instance name>"

- },

- {

- "targetResourceType": "ResourceRegion",

- "targetResourceName": "EastUS2"

- }

- ],

- "ifxAuditFormat": "Json",

- "env_ver": "2.1",

- "env_name": "#Ifx.AuditSchema",

- "env_time": "2020-11-23T18:23:29.9427158Z",

- "env_epoch": "MKZ6G",

- "env_seqNum": 1277,

- "env_popSample": 0.0,

- "env_iKey": null,

- "env_flags": 257,

- "env_cv": "##00000000-0000-0000-0000-000000000000_00000000-0000-0000-0000-000000000000_00000000-0000-0000-0000-000000000000",

- "env_os": null,

- "env_osVer": null,

- "env_appId": null,

- "env_appVer": null,

- "env_cloud_ver": "1.0",

- "env_cloud_name": null,

- "env_cloud_role": null,

- "env_cloud_roleVer": null,

- "env_cloud_roleInstance": null,

- "env_cloud_environment": null,

- "env_cloud_location": null,

- "env_cloud_deploymentUnit": null

-}

-```

-### Azure Confidential VM attestation

-[Windows Server: Azure Edition (WSAE)](https://aka.ms/wsae) is a new edition of Windows Server focused on innovation and efficiency. Featuring an annual release cadence and optimized to run on Azure properties, WSAE brings new functionality to Windows Server users faster than the traditional Long-Term Servicing Channel (LTSC) editions of Windows Server (2016,2019,2022, etc.) the first version of this new variant is Windows Server 2022 Datacenter: Azure Edition, announced at Microsoft Ignite in November 2021.

- - Use customized views and search queries to visualize your runbook results, runbook job status, and other related key indicators or metrics through an [Azure dashboard](/azure/azure-portal/azure-portal-dashboards).

-Using Azure Monitor logs, you can consolidate logs from different resources in the same workspace where it can be analyzed with [queries](/azure/azure-monitor/logs/log-query-overview) to quickly retrieve, consolidate, and analyze the collected data. You can create and test queries using [Log Analytics](/azure/azure-monitor/logs/log-query-overview) in the Azure portal and then either directly analyze the data using these tools or save queries for use with [visualization](/azure/azure-monitor/best-practices-analysis) or [alert rules](/azure/azure-monitor/alerts/alerts-overview).

-Azure Monitor uses a version of the [Kusto query language (KQL)](/azure/kusto/query/) used by Azure Data Explorer that is suitable for simple log queries. It also includes advanced functionality such as aggregations, joins, and smart analytics. You can quickly learn the query language using [multiple lessons](/azure/azure-monitor/logs/get-started-queries).

- - **Log Analytics** : Enter the Subscription ID and workspace name. If you don't have a workspace, you must [create one before proceeding](/azure/azure-monitor/logs/quick-create-workspace).

- - Event Hub namespace: [Create Event Hub](/azure/event-hubs/event-hubs-create) if you don't have one yet.

- - Event Hub name (optional): If you don't specify a name, an event hub is created for each log category. If you are sending multiple categories, specify a name to limit the number of Event Hubs created. See [Azure Event Hubs quotas and limits](/azure/event-hubs/event-hubs-quotas) for details.

- - Event Hub policy (optional): A policy defines the permissions that the streaming mechanism has. See [Event Hubs feature](/azure/event-hubs/event-hubs-features#publisher-policy).

- - **Partner integration**: You must first install a partner integration into your subscription. Configuration options will vary by partner. For more information, see [Azure Monitor integration](/azure/partner-solutions/overview).

-After a few moments, the new setting appears in your list of settings for this resource, and logs are streamed to the specified destinations as new event data is generated. There can be 15 minutes time difference between the event emitted and its appearance in [Log Analytics workspace](/azure/azure-monitor/logs/data-ingestion-time).

- 1. To open the **Create alert rule** screen, click **+New alert rule** on the top of the page. For more information on the options to configure the alerts, see [Log alerts in Azure](/azure/azure-monitor/alerts/alerts-log#create-a-log-alert-rule-in-the-azure-portal)

-Activity log is aΓÇ»[platform log](/azure/azure-monitor/essentials/platform-logs-overview)in Azure that provides insight into subscription-level events. The activity log for Automation account includes information about when an automation resource is modified or created or deleted. However, it does not capture the name or ID of the resource.

-* To learn how to use Azure role-based access control (Azure RBAC), see [Manage role permissions and security in Azure Automation](/azure/automation/automation-role-based-access-control).

-* For information on how Azure protects your privacy and secures your data, see [Azure Automation data security](/azure/automation/automation-managing-data).

-* To learn about configuring the Automation account to use encryption, see [Encryption of secure assets in Azure Automation](/azure/automation/automation-secure-asset-encryption).

-Azure Resource Manager provides a language to develop repeatable and consistent deployment templates for Azure resources. The template is a JavaScript Object Notation (JSON) file that defines the infrastructure and configuration for your project. It uses declarative syntax, which lets you state what you intend to deploy without having to write the sequence of programming commands to create it. In the template, you specify the resources to deploy and the properties for those resources. [Learn more](/azure/azure-resource-manager/templates/overview).

-We've introduced a new language named [Bicep](/azure/azure-resource-manager/bicep/overview) that offers the same capabilities as ARM templates but with a syntax that's easier to use. Each Bicep file is automatically converted to an ARM template during deployment. If you're considering infrastructure as code options, we recommend Bicep. For more information, see [What is Bicep?](/azure/azure-resource-manager/bicep/overview)

- Azure Blueprints (Preview) define a repeatable set of Azure resources that implements and adheres to an organization's standards, patterns, and requirements. Blueprints are a declarative way to orchestrate the deployment of various resource templates and other artifacts such as, Role assignments, Policy assignments, ARM templates and Resource groups. [Learn more](/azure/governance/blueprints/overview).

-### [Azure Automation](/azure/automation/overview)

-Azure Automation orchestrates repetitive processes using graphical, PowerShell, and Python runbooks in the cloud or hybrid environments. It provides a persistent shared assets including variables, connections, objects that allow orchestration of complex jobs. [Learn more](/azure/automation/automation-runbook-gallery).

- | Allows to write an [Automation PowerShell runbook](/azure/automation/learn/powershell-runbook-managed-identity) that deploys an Azure resource by using an [Azure Resource Manager template](/azure/azure-resource-manager/templates/quickstart-create-templates-use-the-portal).</br> </br> Schedule tasks, for example ΓÇô Stop dev/test VMs or services at night and turn on during the day. </br> </br> Response to alerts such as system alerts, service alerts, high CPU/memory alerts, create ServiceNow tickets, and so on. </br> </br> Hybrid automation where you can manage to automate on-premises servers such as SQL Server, Active Directory and so on. </br> </br> Azure resource life-cycle management and governance include resource provisioning, de-provisioning, adding correct tags, locks, NSGs and so on. | IT administrators, System administrators, IT operations administrators who are skilled at using PowerShell or Python based scripting. </br> </br> Infrastructure administrators manage the on-premises infrastructure using scripts or executing long-running jobs such as month-end operations on servers running on-premises.

-**Configuration management** : Collects inventory and tracks changes in your environment. [Learn more](/azure/automation/change-tracking/overview).

-You can configure desired the state of your machines to discover and correct configuration drift. [Learn more](/azure/automation/automation-dsc-overview).

-**Update management** : Assess compliance of servers and can schedule update installation on your machines. [Learn more](/azure/automation/update-management/overview).

-Replaces repetitive, day-to-day operational tasks with an exception-only management model, where a healthy, steady-state of VM is equal to hands-free management. [Learn more](/azure/automanage/automanage-virtual-machines).

-Azure Policy based Guest configuration is the next iteration of Azure Automation State configuration. [Learn more](/azure/governance/policy/concepts/guest-configuration-policy-effects).

- - The next iteration of [Azure Automation State Configuration](/azure/automation/automation-dsc-overview).

- | Obtain compliance data that may include: The configuration of the operating system ΓÇô files, registry, and services, Application configuration or presence, Check environment settings. </br> </br> Audit or deploy settings to all machines (Set) in scope either reactively to existing machines or proactively to new machines as they are deployed. </br> </br> Respond to policy events to provide [remediation on demand or continuous remediation.](/azure/governance/policy/concepts/guest-configuration-policy-effects#remediation-on-demand-applyandmonitor) | The Central IT, Infrastructure Administrators, Auditors (Cloud custodians) are working towards the regulatory requirements at scale and ensuring that servers' end state looks as desired. </br> </br> The application teams validate compliance before releasing change.

-Orchestrates repetitive processes using graphical, PowerShell, and Python runbooks in the cloud or hybrid environment. [Learn more](/azure/automation/automation-runbook-types?).

- - You can invoke a runbook on the basis of [Azure Monitor alert](/azure/automation/automation-create-alert-triggered-runbook) or through a [webhook](/azure/automation/automation-webhooks).

-Provides a serverless event-driven compute platform for automation that allows you to write code to react to critical events from various sources, third-party services, and on-premises systems. For example, an HTTP trigger without worrying about the underlying platform. [Learn more](/azure/azure-functions/functions-overview).

- - You can orchestrate complex workflows through [durable functions](/azure/azure-functions/durable/durable-functions-overview?tabs=csharp).

- - You should avoid large, and long-running functions that can cause unexpected timeout issues. [Learn more](/azure/azure-functions/functions-best-practices?tabs=csharp#write-robust-functions).

- - When you write Powershell scripts within the Function Apps, you must tweak the scripts to define how the function behaves such as - how it's triggered, its input and output parameters. [Learn more](/azure/azure-functions/functions-reference-powershell?tabs=portal).

-Logic Apps is a platform for creating and running complex orchestration workflows that integrate your apps, data, services, and systems. [Learn more](/azure/logic-apps/logic-apps-overview).

-Orchestrates repetitive processes using graphical, PowerShell, and Python runbooks in the cloud or hybrid environment. It provides persistent shared assets, including variables, connections, objects, that allows orchestration of complex jobs. [Learn more](/azure/automation/overview).

-Provides a serverless event-driven compute platform for automation that allows you to write code to react to critical events from various sources, third-party services, and on-premises systems. For example, an HTTP trigger without worrying about the underlying platform [Learn more](/azure/azure-functions/functions-overview).

- - You can orchestrate complex workflows through [durable functions](/azure/azure-functions/durable/durable-functions-overview?tabs=csharp).

- - You should avoid large, and long-running functions that can cause unexpected timeout issues. [Learn more](/azure/azure-functions/functions-best-practices?tabs=csharp#write-robust-functions).

- - When you write Powershell scripts within the Function Apps, you must tweak the scripts to define how the function behaves such as - how it's triggered, its input and output parameters. [Learn more](/azure/azure-functions/functions-reference-powershell?tabs=portal).

-> When you use the webhook with PowerShell 7 runbook, it auto-converts the webhook input parameter to an invalid JSON. For more information, see [Known issues - 7.1 (preview)](/azure/automation/automation-runbook-types#known-issues71-preview). We recommend that you use the webhook with PowerShell 5 runbook.

-* To trigger a runbook from an alert, see [Use an alert to trigger an Azure Automation runbook](automation-create-alert-triggered-runbook.md).

-> | Azure Purview |

- | Content type | Optional. Indicate if the file you're importing is a Key Vault reference or a JSON file. For more information about Key Vault references, go to [Use Key Vault references in an ASP.NET Core app](/azure/azure-app-configuration/use-key-vault-references-dotnet-core). | JSON (application/json) |

-Use the Azure CLI as explained below to import App Configuration data. If you don't have the Azure CLI installed locally, you can optionally use [Azure Cloud Shell](/azure/cloud-shell/overview). Specify the source of the data: `appconfig`, `appservice` or `file`. Optionally specify a source label with `--src-label` and a label to apply with `--label`.

-Use the Azure CLI as explained below to export configurations from App Configuration to another place. If you don't have the Azure CLI installed locally, you can optionally use [Azure Cloud Shell](/azure/cloud-shell/overview). Specify the destination of the data: `appconfig`, `appservice` or `file`. Specify a label for the data you want to export with `--label` or export data with no label by not entering a label.

-> [Create an ASP.NET Core web app](./quickstart-aspnet-core-app.md)

-Importantly, if the Azure Arc-enabled data services are directly connected to Azure, then users can use [Azure Resource Manager APIs](/rest/api/resources/), the Azure CLI, and the Azure portal to operate the Azure Arc data services. The experience in directly connected mode is much like how you would use any other Azure service with provisioning/de-provisioning, scaling, configuring, and so on all in the Azure portal. If the Azure Arc-enabled data services are indirectly connected to Azure, then the Azure portal is a read-only view. You can see the inventory of SQL managed instances and Postgres Hyperscale instances that you have deployed and the details about them, but you cannot take action on them in the Azure portal. In the indirectly connected mode, all actions must be taken locally using Azure Data Studio, the appropriate CLI, or Kubernetes native tools like kubectl.

-After you upload your [metrics, logs](upload-metrics-and-logs-to-azure-monitor.md), or [usage](view-billing-data-in-azure.md), you can view your Azure Arc-enabled SQL managed instances or Azure Arc-enabled Postgres Hyperscale server groups in the Azure portal. To view your resource in the [Azure portal](https://portal.azure.com), follow these steps:

-After you complete your first [metrics or logs upload to Azure](upload-metrics-and-logs-to-azure-monitor.md) or [usage data upload](view-billing-data-in-azure.md), you can see the Azure Arc data controller and any Azure Arc-enabled SQL managed instances or Azure Arc-enabled Postgres Hyperscale server resources in the [Azure portal](https://portal.azure.com).

-Sometimes, dependent modules fail to download successfully when adding the extensions `connectedk8s` and `k8s-configuration` through Azure CLI or Azure Powershell. To fix this problem, manually remove and then add the extensions in the local environment.

-### 1. Check OSM Controller **Deployment**

-### 2. Check the OSM Controller **Pod**

-Column `READY` with `0/1` indicates the control plane container is crashing - we need to get logs. See `Get OSM Controller Logs from Azure Support Center` section below.

-### 3. Check OSM Controller **Service**

-> Note: The `CLUSTER-IP` would be different. The service `NAME` and `PORT(S)` must be the same as seen in the output.

-### 4. Check OSM Controller **Endpoints**:

-### 5. Check OSM Injector **Deployment**

-### 6. Check OSM Injector **Pod**

-### 7. Check OSM Injector **Service**

-### 8. Check OSM Injector **Endpoints**

-### 9. Check **Validating** and **Mutating** webhooks:

-NAME WEBHOOKS AGE

-arc-osm-webhook-osm 1 81m

-kubectl get MutatingWebhookConfiguration --selector app=osm-controller

-NAME WEBHOOKS AGE

-arc-osm-webhook-osm 1 102m

-Check for the service and the CA bundle of the **Validating** webhook:

-```

-kubectl get ValidatingWebhookConfiguration arc-osm-webhook-osm -o json | jq '.webhooks[0].clientConfig.service'

- "path": "/validate-webhook",

-Check for the service and the CA bundle of the **Mutating** webhook:

-kubectl get ValidatingWebhookConfiguration arc-osm-webhook-osm -o json | jq -r '.webhooks[0].clientConfig.caBundle' | wc -c

-The number in the output indicates the number of bytes, or the size of the CA Bundle. If this is empty, 0, or some number under a 1000, it would indicate that the CA Bundle is not correctly provisioned. Without a correct CA Bundle, the ValidatingWebhook would throw an error and prohibit you from making changes to the `osm-config` ConfigMap in the `arc-osm-system` namespace.

-Let's look at a sample error when the CA Bundle is incorrect:

- ```bash

- kubectl patch ConfigMap osm-config -n arc-osm-system --type merge --patch '{"data":{"config_resync_interval":"2m"}}'

- ```

- ```bash

- Error from server (InternalError): Internal error occurred: failed calling webhook "osm-config-webhook.k8s.io": Post https://osm-config-validator.arc-osm-system.svc:9093/validate-webhook?timeout=30s: x509: certificate signed by unknown authority

- ```

-Use one of the following workarounds when the **Validating** Webhook Configuration has a bad certificate:

- ```bash

- kubectl rollout restart deployment -n arc-osm-system osm-controller

- ```

- ```bash

- kubectl delete ValidatingWebhookConfiguration arc-osm-webhook-osm

- ```

- ```bash

- kubectl delete ValidatingWebhookConfiguration arc-osm-webhook-osm; kubectl patch ConfigMap osm-config -n arc-osm-system --type merge --patch '{"data":{"config_resync_interval":"15s"}}'

- ```

-### 10. Check the `osm-config` **ConfigMap**

->[!Note]

->The OSM Controller does not require `osm-config` ConfigMap to be present in the `arc-osm-system` namespace. The controller has reasonable default values for the config and can operate without it.

-Check for the existence:

-```bash

-kubectl get ConfigMap -n arc-osm-system osm-config

-Check the content of the `osm-config` ConfigMap:

-```bash

-kubectl get ConfigMap -n arc-osm-system osm-config -o json | jq '.data'

-```

-You will get the following output:

-```json

-{

- "egress": "false",

- "enable_debug_server": "false",

- "enable_privileged_init_container": "false",

- "envoy_log_level": "error",

- "permissive_traffic_policy_mode": "true",

- "prometheus_scraping": "true",

- "service_cert_validity_duration": "24h",

- "tracing_enable": "false",

- "use_https_ingress": "false",

-}

-Refer [OSM ConfigMap documentation](https://release-v0-8.docs.openservicemesh.io/docs/osm_config_map/) to understand `osm-config` ConfigMap values.

-### 11. Check Namespaces

-kc get namespace bookbuyer -o json | jq '.metadata.annotations'

-kc get namespace bookbuyer -o json | jq '.metadata.labels'

->After `osm namespace add` is called, only **new** pods will be injected with an Envoy sidecar. Existing pods must be restarted with `kubectl rollout restard deployment` command.

-### 12. Verify the SMI CRDs

-Ensure that the CRDs correspond to the same OSM upstream version. E.g. if you are using v0.8.4, ensure that the CRDs match the ones that are available in the release branch v0.8.4 of [OSM OSS project](https://docs.openservicemesh.io/). Refer [OSM release notes](https://github.com/openservicemesh/osm/releases).

-If CRDs are missing, use the following commands to install them on the cluster. Ensure that you replace the version in the command.

-kubectl apply -f https://raw.githubusercontent.com/openservicemesh/osm/release-v0.8.2/charts/osm/crds/access.yaml

-kubectl apply -f https://raw.githubusercontent.com/openservicemesh/osm/release-v0.8.2/charts/osm/crds/specs.yaml

-kubectl apply -f https://raw.githubusercontent.com/openservicemesh/osm/release-v0.8.2/charts/osm/crds/split.yaml

-### 13. Troubleshoot Certificate Management

-### 14. Upgrade Envoy

-When a new pod is created in a namespace monitored by the add-on, OSM will inject an [envoy proxy sidecar](https://docs.openservicemesh.io/docs/guides/app_onboarding/sidecar_injection/) in that pod. If the envoy version needs to be updated, steps to do so can be found in the [Upgrade Guide](https://release-v0-11.docs.openservicemesh.io/docs/getting_started/upgrade/#envoy) on the OSM docs site.

-# Azure Arc-enabled Open Service Mesh (Preview)

-## Basic installation of Azure Arc-enabled OSM

-export VERSION=<osm-arc-version>

-While Azure Arc-enabled Open Service Mesh is in preview, the `az k8s-extension create` command only accepts `pilot` for the `--release-train` flag. `--auto-upgrade-minor-version` is always set to `false` and a version must be provided. If you are using an OpenShift cluster, use the steps in the [section](#install-osm-on-an-openshift-cluster).

-az k8s-extension create --cluster-name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --release-train pilot --name osm --version $VERSION

-You should see output similar to the output shown below. It may take 3-5 minutes for the actual OSM helm chart to get deployed to the cluster. Until this deployment happens, you will continue to see installState as Pending.

- "autoUpgradeMinorVersion": false,

- "releaseTrain": "pilot",

- "version": "x.x.x"

-## Custom installations of Azure Arc-enabled OSM

-The following sections describe certain custom installations of Azure Arc-enabled OSM. Custom installations require setting

-

-

-To ensure that the privileged init container setting is not reverted to the default, pass in the "osm.osm.enablePrivilegedInitContainer" : "true" configuration setting to all subsequent az k8s-extension create commands.

-HPA automatically scales up or down control plane pods based on the average target CPU utilization (%) and average target

-append to your existing JSON settings file as below, repeating the key/value pairs for each control plane pod

-(`osmController`, `injector`) that you want to enable HPA on.

-In order to prevent disruptions during planned outages, control plane pods `osm-controller` and `osm-injector` have a PDB

-To enable PDB, create or append to your existing JSON settings file as follows for each desired control plane pod

-### Install OSM with cert-manager for Certificate Management

-the need for storing private keys in Kubernetes. Refer to OSM's [cert-manager documentation](https://release-v0-11.docs.openservicemesh.io/docs/guides/certificates/)

-> Use the commands provided in the OSM GitHub documentation with caution. Ensure that you use the correct namespace name `arc-osm-system`.

-To install OSM with cert-manager as the certificate provider, create or append to your existing JSON settings file the `certificateProvider.kind`

-### Install OSM with Contour for Ingress

-> Use the commands provided in the OSM GitHub documentation with caution. Ensure that you use the correct namespace name `arc-osm-system`.

-Once you have created a JSON file with applicable values as described in above custom installation sections, set the

-Run the `az k8s-extension create` command to create the OSM extension, passing in the settings file using the

-`--configuration-settings` flag:

- az k8s-extension create --cluster-name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --release-train pilot --name osm --version $VERSION --configuration-settings-file $SETTINGS_FILE

- "defaultValue": "Pilot",

- "autoUpgradeMinorVersion": false,

- "releaseTrain": "pilot",

- "version": "x.x.x"

-Certificate:

- Enable Egress Policy: true

- Enable Multicluster Mode: false

- Enable WASM Stats: true

- Address: jaeger.osm-system.svc.cluster.local

- Enable: false

- Endpoint: /api/v2/spans

- Port: 9411

- Envoy Image: mcr.microsoft.com/oss/envoyproxy/envoy:v1.18.3

- Init Container Image: mcr.microsoft.com/oss/openservicemesh/init:v0.9.1

- Max Data Plane Connections: 0

- Use HTTPS Ingress: false

-Refer to the [Config API reference](https://docs.openservicemesh.io/docs/api_reference/config/v1alpha1/) for more information. Notice that **spec.traffic.enablePermissiveTrafficPolicyMode** is set to **true**. Permissive traffic policy mode in OSM is a mode where the [SMI](https://smi-spec.io/) traffic policy enforcement is bypassed. In this mode, OSM automatically discovers services that are a part of the service mesh and programs traffic policy rules on each Envoy proxy sidecar to be able to communicate with these services.

-## OSM controller configuration (version v0.8.4)

-Currently you can access and configure the OSM controller configuration via the ConfigMap. To view the OSM controller configuration settings, query the `osm-config` ConfigMap via `kubectl` to view its configuration settings.

-```azurecli-interactive

-kubectl get configmap osm-config -n arc-osm-system -o json

-```

-Output:

-```json

-{

- "egress": "false",

- "enable_debug_server": "false",

- "enable_privileged_init_container": "false",

- "envoy_log_level": "error",

- "permissive_traffic_policy_mode": "true",

- "prometheus_scraping": "true",

- "service_cert_validity_duration": "24h",

- "tracing_enable": "false",

- "use_https_ingress": "false"

-}

-```

-Read [OSM ConfigMap documentation](https://release-v0-8.docs.openservicemesh.io/docs/osm_config_map/) to understand each of the available configurations.

-To make changes to the OSM ConfigMap for version v0.8.4, use the following guidance:

-1. Copy and save the changes you wish to make in a JSON file. In this example, we are going to change the permissive_traffic_policy_mode from true to false. Each time you make a change to `osm-config`, you will have to provide the full list of changes (compared to the default `osm-config`) in a JSON file.

- ```json

- {

- "osm.osm.enablePermissiveTrafficPolicy" : "false"

- }

- ```

-

- Set the file path as an environment variable:

-

- ```azurecli-interactive

- export SETTINGS_FILE=<json-file-path>

- ```

-

-2. Run the same `az k8s-extension create` command used to create the extension, but now pass in the configuration settings file:

- ```azurecli-interactive

- az k8s-extension create --cluster-name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --release-train pilot --name osm --version $VERSION --configuration-settings-file $SETTINGS_FILE

- ```

-

- > [!NOTE]

- > To ensure that the ConfigMap changes are not reverted to the default, pass in the same configuration settings to all subsequent az k8s-extension create commands.

-You can start with a [demo application](https://release-v0-11.docs.openservicemesh.io/docs/getting_started/quickstart/manual_demo/#deploy-applications) or use your test environment to try out SMI policies.

-> [!NOTE]

-> Ensure that the version of the bookstore application you run matches the version of the OSM extension installed on your cluster. Ex: if you are using v0.8.4 of the OSM extension, use the bookstore demo from release-v0.8 branch of OSM upstream repository.

-The OSM extension does not install add-ons like [Flagger](https://docs.flagger.app/), [Jaeger](https://www.jaegertracing.io/docs/getting-started/), [Prometheus](https://prometheus.io/docs/prometheus/latest/installation/) and [Grafana](https://grafana.com/docs/grafana/latest/installation/) so that users can integrate OSM with their own running instances of those tools instead. To integrate with your own instances, check the following documentation:

-> Use the commands provided in the OSM GitHub documentation with caution. Ensure that you use the correct namespace name 'arc-osm-system' when making changes to `osm-mesh-config`.

-Both Azure Monitor and Azure Application Insights helps you maximize the availability and performance of your applications and services by delivering a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments.

-Azure Arc-enabled Open Service Mesh will have deep integrations into both of these Azure services, and provide a seemless Azure experience for viewing and responding to critical KPIs provided by OSM metrics. Follow the steps below to allow Azure Monitor to scrape prometheus endpoints for collecting application metrics.

-1. Ensure that the application namespaces that you wish to be monitored are onboarded to the mesh. Follow the guidance [available here](#onboard-namespaces-to-the-service-mesh).

-2. Expose the prometheus endpoints for application namespaces.

- For v0.8.4, ensure that `prometheus_scraping` is set to `true` in the `osm-config` ConfigMap.

-4. Add the namespaces you want to monitor in container-azm-ms-osmconfig ConfigMap. Download the ConfigMap from [here](https://github.com/microsoft/Docker-Provider/blob/ci_prod/kubernetes/container-azm-ms-osmconfig.yaml).

- ```azurecli-interactive

- monitor_namespaces = ["namespace1", "namespace2"]

- ```

-Read more about integration with Azure Monitor [here](https://github.com/microsoft/Docker-Provider/blob/ci_dev/Documentation/OSMPrivatePreview/ReadMe.md).

-### Navigating the OSM dashboard

-1. Access your Azure Arc-connected Kubernetes cluster using this [link](https://aka.ms/azmon/osmarcux).

-2. Go to Azure Monitor and navigate to the Reports tab to access the OSM workbook.

-3. Select the time-range & namespace to scope your services.

-

-## Upgrade the OSM extension instance to a specific version

-### Supported Upgrades

-The OSM extension can be upgraded up to the next minor version. Downgrades and major version upgrades are not supported at this time.

-### CRD Upgrades

-The OSM extension cannot be upgraded to a new version if that version contains CRD version updates without deleting the existing CRDs first. You can check if an OSM upgrade also includes CRD version updates by checking the CRD Updates section of the [OSM release notes](https://github.com/openservicemesh/osm/releases).

-Make sure to back up your Custom Resources prior to deleting the CRDs so that they can be easily recreated after upgrading. Afterwards, follow the upgrade instructions captured below.

-> [!NOTE]

-> Upgrading the CRDs will affect the data plane as the SMI policies won't exist between the time they're deleted and the time they're created again.

-### Upgrade instructions

-1. Delete the old CRDs and custom resources (Run from the root of the [OSM repo](https://github.com/openservicemesh/osm)). Ensure that the tag of the [OSM CRDs](https://github.com/openservicemesh/osm/tree/main/cmd/osm-bootstrap/crds) corresponds to the new version of the chart.

- ```azurecli-interactive

- kubectl delete --ignore-not-found --recursive -f ./charts/osm/crds/

-2. Install the updated CRDs.

- ```azurecli-interactive

- kubectl apply -f charts/osm/crds/

- ```

-3. Set the new chart version as an environment variable:

- ```azurecli-interactive

- export VERSION=<chart version>

- ```

-

-4. Run az k8s-extension create with the new chart version

- ```azurecli-interactive

- az k8s-extension create --cluster-name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --release-train pilot --name osm --version $VERSION --configuration-settings-file $SETTINGS_FILE

- ```

-5. Recreate Custom Resources using new CRDs

-This output should not include OSM. If you don't have any other extensions installed on your cluster, it will just be an empty array.

-> [!NOTE]

-> Get started quickly with an [Azure Arc Jumpstart](https://aka.ms/arc-jumpstart-osm) scenario using Cluster API.

-* Virtual machines (preview): Provision, resize, delete and manage virtual machines based on [VMware vSphere](/azure/azure-arc/vmware-vsphere/overview) or [Azure Stack HCI](/azure-stack/hci/manage/azure-arc-enabled-virtual-machines) and enable VM self-service through role-based access.

-* Perform virtual machine lifecycle and management operations for [VMware vSphere](/azure/azure-arc/vmware-vsphere/overview) and [Azure Stack HCI](/azure-stack/hci/manage/azure-arc-enabled-virtual-machines) environments.

-* Experience Azure Arc-enabled services by exploring the [Jumpstart proof of concept](https://azurearcjumpstart.io/azure_arc_jumpstart/).

- - Delete the default connectivity endpoint: ```az rest --method delete --uri https://management.azure.com/subscriptions/<subscription>/resourceGroups/<resourcegroup>/providers/Microsoft.HybridCompute/machines/<arc enabled server name>/providers/Microsoft.HybridConnectivity/endpoints/default?api-version=2021-10-06-preview```

-For any issues encountered with the Azure Arc resource bridge, you can collect logs for further investigation. To collect the logs, use the Azure CLI [`Az arcappliance log`](https://docs.microsoft.com/cli/azure/arcappliance/logs?#az-arcappliance-logs-vmware) command.

-[Troubleshoot common issues related to resource bridge](../resource-bridge/troubleshoot-resource-bridge.md)

-To specify a maintenance window, check the days you want and specify the maintenance window start hour for each day. Then, select **OK**. The maintenance window time is in UTC.

-Consider choosing a geo-redundant storage account to ensure high availability of persisted data. For more information, see [Azure Storage redundancy](/azure/storage/common/storage-redundancy?toc=/azure/storage/blobs/toc.json).

-Consider choosing a geo-redundant storage account to ensure high availability of your exported data. For more information, see [Azure Storage redundancy](/azure/storage/common/storage-redundancy?toc=/azure/storage/blobs/toc.json).

-For more information on how to set it up, see [Configure geo-replication for Premium Azure Cache for Redis instances](/azure/azure-cache-for-redis/cache-how-to-geo-replication).

-Once the affected region is restored, your unavailable Azure Cache for Redis is automatically restored and available for use again. For more strategies for moving your cache to a different region, see [Move Azure Cache for Redis instances to different regions](/azure/azure-cache-for-redis/cache-moving-resources).

-param($[string] $input, $TriggerMetadata)

-description: Learn how to use IntelliJ to create a simple HTTP-triggered Java function, which you then publish to run in a serverless environment in Azure.

-This article shows you:

-## Set up your development environment

-To create and publish Java functions to Azure using IntelliJ, install the following software:

-+ An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs&utm_campaign=visualstudio).

-+ An [Azure supported Java Development Kit (JDK)](/azure/developer/java/fundamentals/java-support-on-azure) for Java 8

-+ An [IntelliJ IDEA](https://www.jetbrains.com/idea/download/) Ultimate Edition or Community Edition installed

-+ [Maven 3.5.0+](https://maven.apache.org/download.cgi)

-+ Latest [Function Core Tools](https://github.com/Azure/azure-functions-core-tools)

-## Installation and sign in

-1. In IntelliJ IDEA's Settings/Preferences dialog (Ctrl+Alt+S), select **Plugins**. Then, find the **Azure Toolkit for IntelliJ** in the **Marketplace** and click **Install**. After installed, click **Restart** to activate the plugin.

- ![Azure Toolkit for IntelliJ plugin in Marketplace][marketplace]

-2. To sign in to your Azure account, open sidebar **Azure Explorer**, and then click the **Azure Sign In** icon in the bar on top (or from IDEA menu **Tools/Azure/Azure Sign in**).

- ![The IntelliJ Azure Sign In command][intellij-azure-login]

-3. In the **Azure Sign In** window, select **Device Login**, and then click **Sign in** ([other sign in options](/azure/developer/java/toolkit-for-intellij/sign-in-instructions)).

- ![The Azure Sign In window with device login selected][intellij-azure-popup]

-4. Click **Copy&Open** in **Azure Device Login** dialog .

- ![The Azure Login Dialog window][intellij-azure-copycode]

-5. In the browser, paste your device code (which has been copied when you click **Copy&Open** in last step) and then click **Next**.

- ![The device login browser][intellij-azure-link-ms-account]

-6. In the **Select Subscriptions** dialog box, select the subscriptions that you want to use, and then click **Select**.

- ![The Select Subscriptions dialog box][intellij-azure-login-select-subs]

-

-In this section, you use Azure Toolkit for IntelliJ to create a local Azure Functions project. Later in this article, you'll publish your function code to Azure.

-1. Open IntelliJ Welcome dialog, select *Create New Project* to open a new Project wizard, select *Azure Functions*.

- 

-1. Select *Http Trigger*, then click *Next* and follow the wizard to go through all the configurations in the following pages; confirm your project location then click *Finish*; Intellj IDEA will then open your new project.

- 

-1. Navigate to `src/main/java/org/example/functions/HttpTriggerFunction.java` to see the code generated. Beside the line *17*, you will notice that there is a green *Run* button, click it and select *Run 'azure-function-exam...'*, you will see that your function app is running locally with a few logs.

- 

- 

-1. You can try the function by accessing the printed endpoint from browser, like `http://localhost:7071/api/HttpTrigger-Java?name=Azure`.

- 

-1. The log is also printed out in your IDEA, now, stop the function app by clicking the *stop* button.

- 

-1. To debug the function code in your project locally, select the *Debug* button in the toolbar. If you don't see the toolbar, enable it by choosing **View** > **Appearance** > **Toolbar**.

- 

-1. Click on line *20* of the file `src/main/java/org/example/functions/HttpTriggerFunction.java` to add a breakpoint, access the endpoint `http://localhost:7071/api/HttpTrigger-Java?name=Azure` again , you will find the breakpoint is hit, you can try more debug features like *step*, *watch*, *evaluation*. Stop the debug session by click the stop button.

- 

-1. Right click your project in IntelliJ Project explorer, select *Azure -> Deploy to Azure Functions*

- 

-1. If you don't have any Function App yet, click *+* in the *Function* line. Type in the function app name and choose proper platform, here we can simply accept default. Click *OK* and the new function app you just created will be automatically selected. Click *Run* to deploy your functions.

- 

- 

-1. You can manage your function apps with *Azure Explorer* in your IDEA, click on *Function App*, you will see all your function apps here.

- 

-1. Click to select on one of your function apps, and right click, select *Show Properties* to open the detail page.

- 

-1. Right click on your *HttpTrigger-Java* function app, and select *Trigger Function*, you will see that the browser is opened with the trigger URL.

- 

-1. Right click on the package *org.example.functions* and select *New -> Azure Function Class*.

- 

-1. Fill in the class name *HttpTest* and select *HttpTrigger* in the create function class wizard, click *OK* to create, in this way, you can create new functions as you want.

- 

-

- 

-1. Deleting functions in Azure Explorer

-

- 

-

-You've created a Java project with an HTTP triggered function, run it on your local machine, and deployed it to Azure. Now, extend your function by...

-[marketplace]:./media/functions-create-first-java-intellij/marketplace.png

-[intellij-azure-login]: media/functions-create-first-java-intellij/intellij-azure-login.png

-[intellij-azure-popup]: media/functions-create-first-java-intellij/intellij-azure-login-popup.png

-[intellij-azure-copycode]: media/functions-create-first-java-intellij/intellij-azure-login-copyopen.png

-[intellij-azure-link-ms-account]: media/functions-create-first-java-intellij/intellij-azure-login-linkms-account.png

-[intellij-azure-login-select-subs]: media/functions-create-first-java-intellij/intellij-azure-login-selectsubs.png

-Function apps run in, and are maintained by, the Azure App Service platform. As such, your function apps have access to most of the features of Azure's core web hosting platform. The left pane is where you access the many features of the App Service platform that you can use in your function apps.

-> [!NOTE]

-> Not all App Service features are available when a function app runs on the Consumption hosting plan.

-The rest of this article focuses on the following App Service features in the Azure portal that are useful for Functions:

-|Australia East| 100 | 20 |

-|East US 2| 100 | 20 |

-|South Central US| 100 | 20 |

-2. In the left pane, select **New proxy**.

-**Permissions required:** Since MO is a tenant level resource, the scope of the permission would be higher than a subscription scope. Therefore, an Azure tenant admin may be needed to perform this step. [Follow these steps to elevate Azure AD Tenant Admin as Azure Tenant Admin](/azure/role-based-access-control/elevate-access-global-admin). It will give the Azure AD admin 'owner' permissions at the root scope.

-Take this [quick survey](https://forms.microsoft.com/r/CBhWuT1rmM) or share your feedback/questions regarding the preview on the [Azure Monitor Agent User Community](https://teams.microsoft.com/l/team/19%3af3f168b782f64561b52abe75e59e83bc%40thread.tacv2/conversations?groupId=770d6aa5-c2f7-4794-98a0-84fd6ae7f193&tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47).

-System.Diagnostics.Tracing has an [Autoflush feature](https://docs.microsoft.com/dotnet/api/system.diagnostics.trace.autoflush). This causes SDK to flush with every telemetry item, which is undesirable, and can cause logging adapter issues like delayed telemetry, overloading network, inefficient transmission, etc.

-[start]: ./app-insights-overview.md

-description: Troubleshoot auto-instrumentation in Azure Application Insights

-# Troubleshooting Azure Application Insights auto-instrumentation

-This article will help you troubleshoot problems with auto-instrumentation in Azure Application Insights.

-> [!NOTE]

-> Auto-instrumentation used to be known as "codeless attach" before October 2021.

-## Telemetry data isn't reported after enabling auto-instrumentation

-Review these common scenarios if you've enabled Azure Application Insights auto-instrumentation for your app service but don't see telemetry data reported.

-### The Application Insights SDK was previously installed

-Auto-instrumentation will fail when .NET and .NET Core apps were already instrumented with the SDK.

-Remove the Application Insights SDK if you would like to auto-instrument your app.

-### An app was published using an unsupported version of .NET or .NET Core

-Verify a supported version of .NET or .NET Core was used to build and publish applications.

-Refer to the .NET or .NET core documentation to determine if your version is supported.

-### A diagnostics library was detected

-Auto-instrumentation will fail if it detects the following libraries.

-These libraries will need to be removed for auto-instrumentation to succeed.

-## More help

-If you have questions about Azure Application Insights auto-instrumentation, you can post a question in our [Microsoft Q&A question page](/answers/topics/azure-monitor.html).

-> To follow these instructions, you must use the [dedicated plan](https://docs.microsoft.com/azure/azure-functions/dedicated-plan) to allow editing code in App Service Editor.

-This may happen because of adaptive sampling. Adaptive sampling is enabled by default in all the latest versions of the Application Insights ASP.NET and ASP.NET Core Software Development Kits (SDKs). See the [Sampling in Application Insights](/azure/azure-monitor/app/sampling) for more details.

-* [.NET trace logs in Application Insights](./asp-net-trace-logs.md)

-Run the following command in the [Azure Command-Line Interface (CLI)](https://docs.microsoft.com/cli/azure/account?view=azure-cli-latest#az-account-list-locations) to list available regions.

-* [Python](./opencensus-python.md)

-<br>Snapshot Collector used via SDK is not supported when Interop feature is enabled. [See more not supported scenarios.](https://docs.microsoft.com/azure/azure-monitor/app/snapshot-debugger-troubleshoot#not-supported-scenarios)

-description: Troubleshooting guide for Application Insights portal connectivity issues

-# "Error retrieving data" message on Application Insights portal

-This is a troubleshooting guide for the Application Insights portal when encountering connectivity errors similar to `Error retrieving data` or `Missing localization resource`.

-

-The source of the issue is likely third-party browser plugins that interfere with the portal's connectivity.

-To confirm that this is the source of the issue and to identify which plugin is interfering:

-| [Autoscale behaviors for VMs and AppService](/azure/azure-monitor/autoscale/autoscale-overview) | [microsoft.insights/autoscalesettings](/azure/azure-monitor/platform/metrics-supported#microsoftinsightsautoscalesettings) |

-| Log Analytics agent gathered data for the [Metric alerts on logs](/azure/azure-monitor/alerts/alerts-metric-logs#metrics-and-dimensions-supported-for-logs) feature | [Microsoft.OperationalInsights/workspaces](/azure/azure-monitor/platform/metrics-supported##microsoftoperationalinsightsworkspaces)

-| [Application Insights availability tests](/azure/azure-monitor/app/availability-overview) | [Microsoft.Insights/Components](/azure/azure-monitor/essentials/metrics-supported#microsoftinsightscomponents)

-| [Autoscale for VMs and AppService](/azure/azure-monitor/autoscale/autoscale-overview) | [Microsoft.insights/autoscalesettings](/azure/azure-monitor/essentials/resource-logs-categories#microsoftinsightsautoscalesettings)|

-| [Application Insights availability tests](/azure/azure-monitor/app/availability-overview) | [Microsoft.insights/Components](/azure/azure-monitor/essentials/resource-logs-categories#microsoftinsightscomponents) |

-| [Autoscale for VMs and AppService](/azure/azure-monitor/autoscale/autoscale-overview) | [Autoscale Tables](/azure/azure-monitor/reference/tables/tables-resourcetype#azure-monitor-autoscale-settings) |

-For a partial list of entires that the Azure Monitor services writes to the activity log, see [Azure resource provider operations](/azure/role-based-access-control/resource-provider-operations#monitor). There may be other entires not listed here.

-For more information on the schema of Activity Log entries, see [Activity Log schema](/azure/azure-monitor/essentials/activity-log-schema).

-1. Disable stdout logs across all namespaces in the cluster by modifying the following in the ConfigMap file:

-The only requirement to enable Activity log insights is to [configure the Activity log to export to a Log Analytics workspace](activity-log.md#send-to-log-analytics-workspace). Pre-built [workbooks](/azure/azure-monitor/visualize/workbooks-overview) curate this data, which is stored in the [AzureActivity](/azure/azure-monitor/reference/tables/azureactivity) table in the workspace.

- * **Azure Activity Logs Entries** shows the count of Activity log records in each [activity log category](/azure/azure-monitor/essentials/activity-log-schema#categories).

-* [Creating a diagnostic setting to send Activity logs to other destinations](./diagnostic-settings.md)

-|Average_% Available Memory|Yes|% Available Memory|Count|Average|Average_% Available Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Available Swap Space|Yes|% Available Swap Space|Count|Average|Average_% Available Swap Space. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Committed Bytes In Use|Yes|% Committed Bytes In Use|Count|Average|Average_% Committed Bytes In Use. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% DPC Time|Yes|% DPC Time|Count|Average|Average_% DPC Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Free Inodes|Yes|% Free Inodes|Count|Average|Average_% Free Inodes. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Free Space|Yes|% Free Space|Count|Average|Average_% Free Space. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Idle Time|Yes|% Idle Time|Count|Average|Average_% Idle Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Interrupt Time|Yes|% Interrupt Time|Count|Average|Average_% Interrupt Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% IO Wait Time|Yes|% IO Wait Time|Count|Average|Average_% IO Wait Time. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Nice Time|Yes|% Nice Time|Count|Average|Average_% Nice Time. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Privileged Time|Yes|% Privileged Time|Count|Average|Average_% Privileged Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Processor Time|Yes|% Processor Time|Count|Average|Average_% Processor Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Used Inodes|Yes|% Used Inodes|Count|Average|Average_% Used Inodes. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Used Memory|Yes|% Used Memory|Count|Average|Average_% Used Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Used Space|Yes|% Used Space|Count|Average|Average_% Used Space. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% Used Swap Space|Yes|% Used Swap Space|Count|Average|Average_% Used Swap Space. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_% User Time|Yes|% User Time|Count|Average|Average_% User Time. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Available MBytes|Yes|Available MBytes|Count|Average|Average_Available MBytes. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Available MBytes Memory|Yes|Available MBytes Memory|Count|Average|Average_Available MBytes Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Available MBytes Swap|Yes|Available MBytes Swap|Count|Average|Average_Available MBytes Swap. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Avg. Disk sec/Read|Yes|Avg. Disk sec/Read|Count|Average|Average_Avg. Disk sec/Read. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Avg. Disk sec/Transfer|Yes|Avg. Disk sec/Transfer|Count|Average|Average_Avg. Disk sec/Transfer. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Avg. Disk sec/Write|Yes|Avg. Disk sec/Write|Count|Average|Average_Avg. Disk sec/Write. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric). |Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Bytes Received/sec|Yes|Bytes Received/sec|Count|Average|Average_Bytes Received/sec. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Bytes Sent/sec|Yes|Bytes Sent/sec|Count|Average|Average_Bytes Sent/sec. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Bytes Total/sec|Yes|Bytes Total/sec|Count|Average|Average_Bytes Total/sec. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Current Disk Queue Length|Yes|Current Disk Queue Length|Count|Average|Average_Current Disk Queue Length. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Disk Read Bytes/sec|Yes|Disk Read Bytes/sec|Count|Average|Average_Disk Read Bytes/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Disk Reads/sec|Yes|Disk Reads/sec|Count|Average|Average_Disk Reads/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Disk Transfers/sec|Yes|Disk Transfers/sec|Count|Average|Average_Disk Transfers/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Disk Write Bytes/sec|Yes|Disk Write Bytes/sec|Count|Average|Average_Disk Write Bytes/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Disk Writes/sec|Yes|Disk Writes/sec|Count|Average|Average_Disk Writes/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Free Megabytes|Yes|Free Megabytes|Count|Average|Average_Free Megabytes. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Free Physical Memory|Yes|Free Physical Memory|Count|Average|Average_Free Physical Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric). |Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Free Space in Paging Files|Yes|Free Space in Paging Files|Count|Average|Average_Free Space in Paging Files. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Free Virtual Memory|Yes|Free Virtual Memory|Count|Average|Average_Free Virtual Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Logical Disk Bytes/sec|Yes|Logical Disk Bytes/sec|Count|Average|Average_Logical Disk Bytes/sec. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Page Reads/sec|Yes|Page Reads/sec|Count|Average|Average_Page Reads/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Page Writes/sec|Yes|Page Writes/sec|Count|Average|Average_Page Writes/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Pages/sec|Yes|Pages/sec|Count|Average|Average_Pages/sec. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Pct Privileged Time|Yes|Pct Privileged Time|Count|Average|Average_Pct Privileged Time. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Pct User Time|Yes|Pct User Time|Count|Average|Average_Pct User Time. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Physical Disk Bytes/sec|Yes|Physical Disk Bytes/sec|Count|Average|Average_Physical Disk Bytes/sec. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Processes|Yes|Processes|Count|Average|Average_Processes. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Processor Queue Length|Yes|Processor Queue Length|Count|Average|Average_Processor Queue Length. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric). |Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Size Stored In Paging Files|Yes|Size Stored In Paging Files|Count|Average|Average_Size Stored In Paging Files. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Bytes|Yes|Total Bytes|Count|Average|Average_Total Bytes. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Bytes Received|Yes|Total Bytes Received|Count|Average|Average_Total Bytes Received. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Bytes Transmitted|Yes|Total Bytes Transmitted|Count|Average|Average_Total Bytes Transmitted. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Collisions|Yes|Total Collisions|Count|Average|Average_Total Collisions. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Packets Received|Yes|Total Packets Received|Count|Average|Average_Total Packets Received. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Packets Transmitted|Yes|Total Packets Transmitted|Count|Average|Average_Total Packets Transmitted. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Rx Errors|Yes|Total Rx Errors|Count|Average|Average_Total Rx Errors. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Total Tx Errors|Yes|Total Tx Errors|Count|Average|Average_Total Tx Errors. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Uptime|Yes|Uptime|Count|Average|Average_Uptime. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Used MBytes Swap Space|Yes|Used MBytes Swap Space|Count|Average|. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Used Memory kBytes|Yes|Used Memory kBytes|Count|Average|Average_Used Memory kBytes. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Used Memory MBytes|Yes|Used Memory MBytes|Count|Average|Average_Used Memory MBytes. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Users|Yes|Users|Count|Average|Average_Users. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Average_Virtual Shared Memory|Yes|Virtual Shared Memory|Count|Average|Average_Virtual Shared Memory. Supported for: Linux. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, ObjectName, InstanceName, CounterPath, SourceSystem|

-|Event|Yes|Event|Count|Average|Event. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Source, EventLog, Computer, EventCategory, EventLevel, EventLevelName, EventID|

-|Heartbeat|Yes|Heartbeat|Count|Total|Heartbeat. Supported for: Linux, Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, OSType, Version, SourceComputerId|

-|Update|Yes|Update|Count|Average|Update. Supported for: Windows. Part of [metric alerts for logs feature](https://aka.ms/am-log-to-metric).|Computer, Product, Classification, UpdateState, Optional, Approved|

-* [Azure NAT Gateway](/azure/virtual-network/nat-gateway/troubleshoot-nat)

-In addition to the Pay-As-You-Go model, Log Analytics has **Commitment Tiers**, which can save you as much as 30 percent compared to the Pay-As-You-Go price. With commitment tier pricing, you can commit to buy data ingestion starting at 100 GB/day at a lower price than Pay-As-You-Go pricing. Any usage above the commitment level (overage) is billed at that same price per GB as provided by the current commitment tier. The commitment tiers have a 31-day commitment period from the time a commitment tier is selected.

-Billing for the commitment tiers is done on a daily basis. See [Azure Monitor pricing](https://azure.microsoft.com/pricing/details/monitor/) for a detailed listing of the commitment tiers and their prices.

-> [!NOTE]

-> Starting June 2, 2021, **Capacity Reservations** were renamed to **Commitment Tiers**. Data collected above your commitment tier level (overage) is now billed at the same price-per-GB as the current commitment tier level, lowering costs compared to the old method of billing at the Pay-As-You-Go rate, and reducing the need for users with large data volumes to fine-tune their commitment level. Three new commitment tiers were also added: 1000, 2000, and 5000 GB/day.

-This article describes the monitoring data generated by Azure Monitor. Azure Monitor uses [itself](/azure/azure-monitor/overview) to monitor certain parts of its own functionality. You can monitor:

- If you're unfamiliar with the features of Azure Monitor common to all Azure services that use it, read [Monitoring Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource).

-Azure Monitor collects the same kinds of monitoring data as other Azure resources that are described in [Monitoring data from Azure resources](/azure/azure-monitor/essentials/monitor-azure-resource#monitoring-data-from-Azure-resources).

-You can analyze metrics for *Azure Monitor* with metrics from other Azure services using metrics explorer by opening **Metrics** from the **Azure Monitor** menu. See [Getting started with Azure Metrics Explorer](/azure/azure-monitor/essentials/metrics-getting-started) for details on using this tool.

-For reference, you can see a list of [all resource metrics supported in Azure Monitor](/azure/azure-monitor/essentials/metrics-supported).

-All resource logs in Azure Monitor have the same fields followed by service-specific fields. The common schema is outlined in [Azure Monitor resource log schema](/azure/azure-monitor/essentials/resource-logs-schema) The schemas for autoscale resource logs are found in the [Azure Monitor Data Reference](azure-monitor-monitoring-reference.md#resource-logs)

-The [Activity log](/azure/azure-monitor/essentials/activity-log) is a type of platform log in Azure that provides insight into subscription-level events. You can view it independently or route it to Azure Monitor Logs, where you can do much more complex queries using Log Analytics.

-Azure Monitor alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues in your system before your customers notice them. You can set alerts on [metrics](/azure/azure-monitor/alerts/alerts-metric-overview), [logs](/azure/azure-monitor/alerts/alerts-unified-log), and the [activity log](/azure/azure-monitor/alerts/activity-log-alerts). Different types of alerts have benefits and drawbacks.

-For an in-depth discussion of using alerts with autoscale, see [Troubleshoot Azure autoscale](/azure/azure-monitor/autoscale/autoscale-troubleshoot).

- | [Azure Purview](../purview/index.yml) | Microsoft.Purview/accounts | [**Yes**](./essentials/metrics-supported.md#microsoftpurviewaccounts) | [**Yes**](./essentials/resource-logs-categories.md#microsoftpurviewaccounts) | | |

-**New articles**

-## Azure Purview limits

-The latest values for Azure Purview quotas can be found in the [Azure Purview quota page](../../purview/how-to-manage-quotas.md).

-You can use [Azure AD application](/azure/active-directory/develop/app-objects-and-service-principals) to connect to SignalR service. As long as the application has the right permission to access SignalR service, no access key is needed.

-You can also use [managed identity](/azure/active-directory/managed-identities-azure-resources/overview) to authenticate with SignalR service.

-But in some applications there may be an additional component in front of SignalR service and all client connections need to go through that component first (to gain additional benefits like network security, [Azure Application Gateway](/azure/application-gateway/overview) is a common service that provides such functionality).

-In production environment, you can use other Azure services to manage config/secrets like Azure [Key Vault](/azure/key-vault/general/overview) and [App Configuration](/azure/azure-app-configuration/overview). See their documentation to learn how to set up config provider for those services.

-5 WebSocket connections is the default value that can be changed in [configuration](https://github.com/Azure/azure-signalr/blob/dev/docs/run-asp-net-core.md#connectioncount).

-From SDK version 1.5.0, we're enabling dynamic scale ServiceEndpoints for ASP.NET Core version first. So you don't have to restart app server when you need to add/remove a ServiceEndpoint. As ASP.NET Core is supporting default configuration like `appsettings.json` with `reloadOnChange: true`, you don't need to change a code and it's supported by nature. And if you'd like to add some customized configuration and work with hot-reload, please refer to [this](https://docs.microsoft.com/aspnet/core/fundamentals/configuration/?view=aspnetcore-3.1).

-> [Setup SignalR Service for disaster recovery and high availability](./signalr-concept-disaster-recovery.md)

-For more information about Multi-Factor Authentication support for SQL tools, see [Using multi-factor Azure Active Directory authentication](/azure/azure-sql/database/authentication-mfa-ssms-overview).

-> [System.Data.SqlClient](/dotnet/api/system.data.sqlclient) uses the Azure Active Directory Authentication Library (ADAL), which will be deprecated. If you're using the [System.Data.SqlClient](/dotnet/api/system.data.sqlclient) namespace for Azure Active Directory authentication, migrate applications to [Microsoft.Data.SqlClient](/sql/connect/ado-net/introduction-microsoft-data-sqlclient-namespace) and the [Microsoft Authentication Library (MSAL)](/azure/active-directory/develop/msal-migration). For more information about using Azure AD authentication with SqlClient, see [Using Azure Active Directory authentication with SqlClient](/sql/connect/ado-net/sql/azure-active-directory-authentication).

-[Azure Storage Explorer](/azure/vs-azure-tools-storage-manage-with-storage-explorer) is a standalone application that simplifies working with event file targets stored in blobs in Azure Storage. You can use Storage Explorer to:

-> [System.Data.SqlClient](/dotnet/api/system.data.sqlclient) uses the Azure Active Directory Authentication Library (ADAL), which will be deprecated. If you're using the [System.Data.SqlClient](/dotnet/api/system.data.sqlclient) namespace for Azure Active Directory authentication, migrate applications to [Microsoft.Data.SqlClient](/sql/connect/ado-net/introduction-microsoft-data-sqlclient-namespace) and the [Microsoft Authentication Library (MSAL)](/azure/active-directory/develop/msal-migration). For more information about using Azure AD authentication with SqlClient, see [Using Azure Active Directory authentication with SqlClient](/sql/connect/ado-net/sql/azure-active-directory-authentication).

-[13]: ./media/authentication-aad-configure/13connect-to-db2.png

-**Question**: Will [Azure Purview](../../purview/overview.md) replace SQL Data Discovery & Classification or will SQL Data Discovery & Classification be retired soon?

-**Answer**: We continue to support SQL Data Discovery & Classification and encourage you to adopt [Azure Purview](../../purview/overview.md) which has richer capabilities to drive advanced classification capabilities and data governance. If we decide to retire any service, feature, API or SKU, you will receive advance notice including a migration or transition path. Learn more about Microsoft Lifecycle policies here.

-| UAE North | Yes | | |

-There are two ways to capture traces in SQL Server; Extended Events (XEvents) and Profiler Traces. However, [SQL Server Profiler](/sql/tools/sql-server-profiler/sql-server-profiler) is deprecated trace technology not supported for Azure SQL Database. [Extended Events](/sql/relational-databases/extended-events/extended-events) is the newer tracing technology that allows more versatility and less impact to the observed system, and its interface is integrated into SQL Server Management Studio (SSMS). For more information on querying extended events in Azure SQL Database, see [Extended events in Azure SQL Database](/azure/azure-sql/database/xevent-db-diff-from-svr).

-As customers modernize their infrastructure, application, and data tiers, they also modernize their identity management capabilities by shifting to Azure AD. Azure SQL offers multiple [Azure AD Authentication](/azure/azure-sql/database/authentication-aad-overview) options:

-If you have not yet enabled Windows authentication for Azure AD principals against your managed instance, you may run a trace against a managed instance using an [Azure AD Authentication](/azure/azure-sql/database/authentication-aad-overview) option, including:

-|Clients must be joined to AD. The domain must have a functional level of Windows Server 2012 or higher. | You can determine if the client is joined to AD by running the [dsregcmd command](/azure/active-directory/devices/troubleshoot-device-dsregcmd): `dsregcmd.exe /status` |

- If this is the first time you're configuring Azure AD Kerberos settings, the [Get-AzureAdKerberosServer cmdlet](/azure/active-directory/authentication/howto-authentication-passwordless-security-key-on-premises#view-and-verify-the-azure-ad-kerberos-server) will display empty information, as in the following sample output:

- Run the [Set-AzureAdKerberosServer PowerShell cmdlet](/azure/active-directory/authentication/howto-authentication-passwordless-security-key-on-premises#create-a-kerberos-server-object) to add the Trusted Domain Object. Be sure to include `-SetupCloudTrust` parameter. If there is no Azure AD service account, this command will create a new Azure AD service account. If there is an Azure AD service account already, this command will only create the requested Trusted Domain object.

-1. Identify your [Azure AD tenant ID](/azure/active-directory/fundamentals/active-directory-how-to-find-tenant).

-|Clients must be joined to Azure AD or Hybrid Azure AD. | You can determine if this prerequisite is met by running the [dsregcmd command](/azure/active-directory/devices/troubleshoot-device-dsregcmd): `dsregcmd.exe /status` |

-Customers should first implement [Azure AD Connect](/azure/active-directory/hybrid/whatis-azure-ad-connect) to integrate on-premises directories with Azure AD.

-|Clients must be joined to Azure AD or Hybrid Azure AD. | You can determine if this prerequisite is met by running the [dsregcmd command](/azure/active-directory/devices/troubleshoot-device-dsregcmd): `dsregcmd.exe /status` |

-|Clients must be joined to AD. The domain must have a functional level of Windows Server 2012 or higher. | You can determine if the client is joined to AD by running the [dsregcmd command](/azure/active-directory/devices/troubleshoot-device-dsregcmd): `dsregcmd.exe /status` |

- - From the jump-box VM, verify you have access to [vCenter and NSX-T portals](/azure/azure-vmware/tutorial-configure-networking).

-For Network planning and setup, use the [Network planning checklist - Azure VMware Solution | Microsoft Docs](/azure/azure-vmware/tutorial-network-checklist)

-Our application performance monitoring and troubleshooting partners have industry-leading solutions in VMware-based environments that assure the availability, reliability, and responsiveness of applications and services. Our customers have adopted many of these solutions integrated with VMware NSX-T for their on-premises deployments. As one of our key principles, we want to enable them to continue to use their investments and VMware solutions running on Azure. Many of these Independent Software Vendors (ISV) have validated their solutions with Azure VMware Solution.

-Additionally to the supported operating systems by VMware on vSphere we have worked with Red Hat, SUSE and Canonical to extend the support model currently in place for Azure Virtual Machines to the workloads running on Azure VMware Solution, given that it is a first-party Azure service. You can check the following sites of vendors for more information about the benefits of running their operating system on Azure.

-Our security partners have industry-leading solutions in VMware-based environments that cover many aspects of the security ecosystem like threat protection and security scanning. Our customers have adopted many of these solutions integrated with VMware NSX-T for their on-premises deployments. As one of our key principles, we want to enable them to continue to use their investments and VMware solutions running on Azure. Many of these Independent Software Vendors (ISV) have validated their solutions with Azure VMware Solution.

-By default, Azure Bastion is automatically enabled to allow copy and paste for all sessions connected through the bastion resource. You don't need to configure anything additional. This applies to both the Basic and the Standard SKU tier. If you want to disable the copy and paste feature, the Standard SKU is required.

-You can copy and paste text between your local device and the remote session. Only text copy/paste is supported. By default, this feature is enabled. If you want to disable this feature, you can change the setting on the configuration page for your bastion host. To disable, your bastion host must be configured with the Standard SKU tier.

->If you have a Certificate Authority Authorization (CAA) record with your DNS provider, it must include DigiCert as a valid CA. A CAA record allows domain owners to specify with their DNS providers which CAs are authorized to issue certificates for their domain. If a CA receives an order for a certificate for a domain that has a CAA record and that CA is not listed as an authorized issuer, it is prohibited from issuing the certificate to that domain or subdomain. For information about managing CAA records, see [Manage CAA records](https://support.dnsimple.com/articles/manage-caa-record/). For a CAA record tool, see [CAA Record Helper](https://sslmate.com/caa/).

- 

- 

-1. From the **CDN profile** page, select **Manage**.

-

- 

-

- The CDN management portal opens.

-2. Select the **HTTP Large** tab, then select **Rules Engine**.

-

- The options for a new rule are displayed.

-

- 

-3. Enter a name in the **Name / Description** textbox.

-4. Identify the type of requests the rule applies to. Use the default match condition, **Always**.

- 

- > For a detailed list of conditional expressions, see [Rules engine conditional expressions](cdn-verizon-premium-rules-engine-reference-match-conditions.md).

-5. To add a new feature, select the **+** button next to **Features**. In the dropdown on the left, select **Force Internal Max-Age**. In the textbox that appears, enter **300**. Do not change the remaining default values.

- 

-6. Select **Add** to save the new rule. The new rule is now awaiting approval. After it has been approved, the status changes from **Pending XML** to **Active XML**.

-

- > [!IMPORTANT]

- > Rules changes can take up to 10 minutes to propagate through Azure CDN.

- >

- >

-* The file size must be less than 50 MB (6 MB for the free tier) and dimensions at least 50 x 50 pixels and at most 10000 x 10000 pixels.

-| Entity Type | EN-US | ZH-CN | DE | FR | ES | IT | PT-BR | JP | KO | NL | TR |

-|::|:--:|:--:|:--:|:--:|:--:|:--:|:|:|:|:|:|

-| Calendar | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Communication | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Email | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| HomeAutomation | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Notes | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Places | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| RestaurantReservation | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| ToDo | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Utilities | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Weather | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-| Web | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô | Γ£ô |

-The **grouping** syntax, with parentheses, can be nested two levels. For example: `(({Entity1.RoleName1} | {Entity1.RoleName2} ) | {Entity2} )`. This feature allows any of the three entities to be matched.

-Understand how [sentiment](luis-reference-prebuilt-sentiment.md) is returned in the .json response.

-> Try the [Azure Video Analyzer for Media](/azure/azure-video-analyzer/video-analyzer-for-media-docs/video-indexer-overview) as a demonstration of how you can get captions for videos that you upload.

-* [Get speech recognition results](get-speech-recognition-results.md)

-* [Question answering quickstart](/azure/cognitive-services/language-service/question-answering/quickstart/sdk?pivots=studio)

-* [Update Sources API reference](/rest/api/cognitiveservices/questionanswering/question-answering-projects/update-sources)

-During a Teams meeting, all chat messages sent by Teams users or Communication Services users are stored in the geographic region associated with the Microsoft 365 organization hosting the meeting. For more information, review the article [Location of data in Microsoft Teams](/microsoftteams/location-of-data-in-teams). For each Communication Services user in the meetings, there is also a copy of the most recently sent message that is stored in the geographic region associated with the Communication Services resource used to develop the Communication Services application. For more information, review the article [Region availability and data residency](/azure/communication-services/concepts/privacy).

-## Outbound virtual video device

-## Overview

-Once an outbound virtual video device is created, use DeviceManager to make a new virtual video device that behaves just like any other webcam connected to your computer or mobile phone.

-The following is an overview of the steps required to create an outbound virtual video device.

-1. Create a `VirtualDeviceIdentification` with basic identification information for the new outbound virtual video device.

- ```java

- VirtualDeviceIdentification deviceId = new VirtualDeviceIdentification();

- deviceId.setId("QuickStartVirtualVideoDevice");

- deviceId.setName("My First Virtual Video Device");

- ```

-2. Create an array of `VideoFormat` with the video formats supported by the app. It is fine to have only one video format supported, but at least one of the provided video formats must be of the `MediaFrameKind::VideoSoftware` type. When multiple formats are provided, the order of the format in the list does not influence or prioritize which one will be used. The selected format is based on external factors like network bandwidth.

- format.setMediaFrameKind(MediaFrameKind.VIDEO_SOFTWARE);

-3. Create `OutboundVirtualVideoDeviceOptions` and set `DeviceIdentification` and `VideoFormats` with the previously created objects.

- OutboundVirtualVideoDeviceOptions m_options = new OutboundVirtualVideoDeviceOptions();

- // ...

- m_options.setDeviceIdentification(deviceId);

- m_options.setVideoFormats(videoFormats);

-4. Make sure the `OutboundVirtualVideoDeviceOptions::OnFlowChanged` delegate is defined. This delegate will inform its listener about events requiring the app to start or stop producing video frames. In this quick start, `m_mediaFrameSender` is used as trigger to let the app know when it's time to start generating frames. Feel free to use any mechanism in your app as a trigger.

- private MediaFrameSender m_mediaFrameSender;

- // ...

- m_options.addOnFlowChangedListener(virtualDeviceFlowControlArgs -> {

- if (virtualDeviceFlowControlArgs.getMediaFrameSender().getRunningState() == VirtualDeviceRunningState.STARTED) {

- // Tell the app's frame generator to start producing frames.

- m_mediaFrameSender = virtualDeviceFlowControlArgs.getMediaFrameSender();

- } else {

- // Tell the app's frame generator to stop producing frames.

- m_mediaFrameSender = null;

- }

-5. Use `Device

- private OutboundVirtualVideoDevice m_outboundVirtualVideoDevice;

- // ...

- m_outboundVirtualVideoDevice = m_deviceManager.createOutboundVirtualVideoDevice(m_options).get();

-6. Tell device manager to use the recently created virtual camera on calls.

- private LocalVideoStream m_localVideoStream;

- // ...

- for (VideoDeviceInfo videoDeviceInfo : m_deviceManager.getCameras())

- {

- String deviceId = videoDeviceInfo.getId();

- if (deviceId.equalsIgnoreCase("QuickStartVirtualVideoDevice")) // Same id used in step 1.

- {

- m_localVideoStream = LocalVideoStream(videoDeviceInfo, getApplicationContext());

- }

- }

-7. In a non-UI thread or loop in the app, cast the `MediaFrameSender` to the appropriate type defined by the `MediaFrameKind` property of `VideoFormat`. For example, cast it to `SoftwareBasedVideoFrame` and then call the `send` method according to the number of planes defined by the MediaFormat.

- java.nio.ByteBuffer plane1 = null;

- Random rand = new Random();

- byte greyValue = 0;

- // ...

- java.nio.ByteBuffer plane1 = null;

- Random rand = new Random();

- while (m_outboundVirtualVideoDevice != null) {

- while (m_mediaFrameSender != null) {

- if (m_mediaFrameSender.getMediaFrameKind() == MediaFrameKind.VIDEO_SOFTWARE) {

- SoftwareBasedVideoFrame sender = (SoftwareBasedVideoFrame) m_mediaFrameSender;

- // Gets the timestamp for when the video frame has been created.

- // This allows better synchronization with audio.

- int timeStamp = sender.getTimestamp();

- // Adjusts frame dimensions to the video format that network conditions can manage.

- if (plane1 == null || videoFormat.getStride1() * videoFormat.getHeight() != plane1.capacity()) {

- plane1 = ByteBuffer.allocateDirect(videoFormat.getStride1() * videoFormat.getHeight());

- plane1.order(ByteOrder.nativeOrder());

- // Generates random gray scaled bands as video frame.

- int bandsCount = rand.nextInt(15) + 1;

- byte greyValue = (byte)rand.nextInt(254);

- java.util.Arrays.fill(plane1.array(), bandBegin, bandBegin + bandThickness, greyValue);

- // Sends video frame to the other participants in the call.

- FrameConfirmation fr = sender.sendFrame(plane1, timeStamp).get();

- // Waits before generating the next video frame.

- // Video format defines how many frames per second app must generate.

- // Virtual camera hasn't been created yet.

- // Let's wait a little bit before checking again.

- // This is for demo only purposes.

- // Feel free to use a better synchronization mechanism.

- Thread.sleep(100);

-10) Leave the default value of 80 for the port, and then select **Enter** to complete the workflow.

-description: Learn how to identify the restore time and restore a live or deleted Azure Cosmos DB account. It shows how to use the event feed to identify the restore time and restore the account using Azure portal, PowerShell, CLI, or a Resource Manager template.

-This article describes how to identify the restore time and restore a live or deleted Azure Cosmos DB account. It shows restore the account using [Azure portal](#restore-account-portal), [PowerShell](#restore-account-powershell), [CLI](#restore-account-cli), or a [Resource Manager template](#restore-arm-template).

-To restore Azure Cosmos DB live accounts that are not deleted, it is a best practice to always identify the [latest restorable timestamp](get-latest-restore-timestamp.md) for the container. You can then use this timestamp to restore the account to it's latest version.

-1. The **resources** > **properties** > **restoreParameters** object contains the restore details. The **restoreTimestampInUtc** gives you the time at which the account was restored and the **databasesToRestore** shows the specific database and container from which the account was restored.

-### <a id="trigger-restore-ps"></a>Trigger a restore operation

-#### List all the versions of mongodb collections of a database in a live database account

-#### List all the resources of a mongodb database account that are available to restore at a given timestamp and region

- * Install the latest version of [Azure CLI](/cli/azure/install-azure-cli) or version higher than 2.26.0

- * Sign into your Azure account with `az login` command.

-### <a id="trigger-restore-cli"></a>Trigger a restore operation with CLI

-1. Create a new Azure Cosmos DB account by restoring from an existing account.

-2. Create a new Azure Cosmos DB account by restoring only selected databases and containers from an existing database account.

-Run the following command to get the restore details. The `az cosmosdb show` command output shows the value of `createMode` property. If the value is set to **Restore**. it indicates that the account was restored from another account. The `restoreParameters` property has further details such as `restoreSource`, which has the source account ID. The last GUID in the `restoreSource` parameter is the instanceId of the source account. And the restoreTimestamp will be under the restoreParameters object:

-### <a id="enumerate-sql-api"></a>Enumerate restorable resources for SQL API

-Run the following CLI command to list all the accounts that can be restored in the current subscription

-The response includes all the database accounts (both live and deleted) that can be restored and the regions that they can be restored from:

- "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/7133a59a-d1c0-4645-a699-6e296d6ac865",

- "name": "7133a59a-d1c0-4645-a699-6e296d6ac865",

-Run the following CLI command to list all the versions of databases. This command only works with live accounts. The `instance-id` and the `location` parameters are obtained from the `name` and `location` properties in the response of `az cosmosdb restorable-database-account list` command. The instanceId attribute is also a property of source database account that is being restored:

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

- "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/7133a59a-d1c0-4645-a699-6e296d6ac865/restorableSqlDatabases/40e93dbd-2abe-4356-a31a-35567b777220",

- "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/7133a59a-d1c0-4645-a699-6e296d6ac865/restorableSqlDatabases/243c38cb-5c41-4931-8cfb-5948881a40ea",

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

-### <a id="enumerate-mongodb-api"></a>Enumerate restorable resources for MongoDB API account

-#### List all the versions of mongodb databases in a live database account

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

-#### List all the versions of mongodb collections of a database in a live database account

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

- --instance-id "7133a59a-d1c0-4645-a699-6e296d6ac865" \

-## <a id="restore-arm-template"></a>Restore using the Resource Manager template

-You can also restore an account using Resource Manager template. When defining the template include the following parameters:

-* Set the `createMode` parameter to *Restore*

-* Define the `restoreParameters`, notice that the `restoreSource` value is extracted from the output of the `az cosmosdb restorable-database-account list` command for your source account. The Instance ID attribute for your account name is used to do the restore.

-* Set the `restoreMode` parameter to *PointInTime* and configure the `restoreTimestampInUtc` value.

-Next deploy the template by using Azure PowerShell or CLI. The following example shows how to deploy the template with a CLI command:

-| <input type="checkbox"/> | SDK Logging | Use SDK logging to capture additional diagnostics information and troubleshoot latency issues. Log the [CosmosDiagnostics](/java/api/com.azure.cosmos.cosmosdiagnostics?view=azure-java-stable&preserve-view=true) in Java SDK for more detailed cosmos diagnostic information for the current request to the service. As an example use case, capture Diagnostics on any exception and on completed operations if the `CosmosDiagnostics#getDuration()` is greater than a designated threshold value (i.e. if you have an SLA of 10 seconds, then capture diagnostics when `getDuration()` > 10 seconds). It's advised to only use these diagnostics during performance testing. For more information, follow [capture diagnostics on Java SDK](/azure/cosmos-db/sql/troubleshoot-java-sdk-v4-sql#capture-the-diagnostics) |

-* If you know typical request rates for your current database workload, read about [estimating request units using Azure Cosmos DB capacity planner](estimate-ru-with-capacity-planner.md)

-To learn more about Azure Cosmos DB, go to [Azure Cosmos DB](/azure/cosmos-db/introduction).

-> [Import data into Azure Cosmos DB for the SQL API](../import-data.md)

-Do not verify a Database and/or Container exists by calling `Create...IfNotExistsAsync` and/or `Read...Async` in the hot path and/or before doing an item operation. The validation should only be done on application startup when it is necessary, if you expect them to be deleted (otherwise it's not needed). These metadata operations will generate extra end-to-end latency, have no SLA, and their own separate [limitations](/azure/cosmos-db/sql/troubleshoot-request-rate-too-large#rate-limiting-on-metadata-requests) that do not scale like data operations.

-Do not verify a Database and/or Collection exists by calling `Create...IfNotExistsAsync` and/or `Read...Async` in the hot path and/or before doing an item operation. The validation should only be done on application startup when it is necessary, if you expect them to be deleted (otherwise it's not needed). These metadata operations will generate extra end-to-end latency, have no SLA, and their own separate [limitations](/azure/cosmos-db/sql/troubleshoot-request-rate-too-large#rate-limiting-on-metadata-requests) that do not scale like data operations.

-If you need to verify that a database or container exists, don't do so by calling `Create...IfNotExistsAsync` or `Read...Async` before doing an item operation. The validation should only be done on application startup when it's necessary, if you expect them to be deleted. These metadata operations generate extra latency, have no service-level agreement (SLA), and have their own separate [limitations](/azure/cosmos-db/sql/troubleshoot-request-rate-too-large#rate-limiting-on-metadata-requests). They don't scale like data operations.

-Data plane requests can come to Cosmos DB via the following 3 paths.

-When a data plane request is blocked with 403 Forbidden, the error message will specify via which of the above 3 paths the request came to Cosmos DB.

-This scenario happens when non-data [operations are disallowed in the account](../how-to-setup-rbac.md#permission-model). On this scenario, it's common to see errors like the ones below:

-Perform the operation through Azure Resource Manager, Azure portal, Azure CLI, or Azure PowerShell. Or reallow execution of non-data operations.

-In the sample app, this object is passed to the `UpdateEntity` method in the `TableService` class. This method first loads the existing entity from the Table API using the [GetEntity](/dotnet/api/azure.data.tables.tableclient.getentity) method on the [TableClient](/dotnet/api/azure.data.tables.tableclient). It then updates that entity object and uses the `UpdateEntity` method save the updates to the database. Note how the [UpdateEntity](/dotnet/api/azure.data.tables.tableclient.updateentity) method takes the current Etag of the object to insure the object has not changed since it was initially loaded. If you want to update the entity regardless, you may pass a value of `Etag.Any` to the `UpdateEntity` method.

-description: Store structured data in the cloud using the Azure Tables client library for Python.

-# Get started with Azure Tables client library using Python

-The Azure Table storage and the Azure Cosmos DB are services that store structured NoSQL data in the cloud, providing a key/attribute store with a schemaless design. Because Table storage and Azure Cosmos DB are schemaless, it's easy to adapt your data as the needs of your application evolve. Access to the table storage and table API data is fast and cost-effective for many types of applications, and is typically lower in cost than traditional SQL for similar volumes of data.

-You can use the Table storage or the Azure Cosmos DB to store flexible datasets like user data for web applications, address books, device information, or other types of metadata your service requires. You can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account.

-### About this sample

-This sample shows you how to use the [Azure Data Tables SDK for Python](https://pypi.org/project/azure-data-tables/) in common Azure Table storage scenarios. The name of the SDK indicates it is for use with Azure Tables storage, but it works with both Azure Cosmos DB and Azure Tables storage, each service just has a unique endpoint. These scenarios are explored using Python examples that illustrate how to:

-* Create and delete tables

-* Insert and query entities

-* Modify entities

-While working through the scenarios in this sample, you may want to refer to the [Azure Data Tables SDK for Python API reference](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/tables/azure-data-tables).

-## Prerequisites

-You need the following to complete this sample successfully:

-* [Python](https://www.python.org/downloads/) 2.7 or 3.6+.

-* [Azure Data Tables SDK for Python](https://pypi.python.org/pypi/azure-data-tables/). This SDK connects with both Azure Table storage and the Azure Cosmos DB Table API.

-* [Azure Storage account](../../storage/common/storage-account-create.md) or [Azure Cosmos DB account](https://azure.microsoft.com/try/cosmosdb/).

-## Create an Azure service account

-**Create an Azure storage account**

-**Create an Azure Cosmos DB Table API account**

-## Install the Azure Data Tables SDK for Python

-After you've created a Storage account, your next step is to install the [Microsoft Azure Data Tables SDK for Python](https://pypi.python.org/pypi/azure-data-tables/). For details on installing the SDK, refer to the [README.md](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/tables/azure-data-tables/README.md) file in the Data Tables SDK for Python repository on GitHub.

-## Import the TableServiceClient and TableEntity classes

-To work with entities in the Azure Data Tables service in Python, you use the `TableServiceClient` and `TableEntity` classes. Add this code near the top your Python file to import both:

-```python

-from azure.data.tables import TableServiceClient

-from azure.data.tables import TableEntity

-## Connect to Azure Table service

-You can either connect to the Azure Storage account or the Azure Cosmos DB Table API account. Get the shared key or connection string based on the type of account you are using.

-### Creating the Table service client from a shared key

-Create a `TableServiceClient` object, and pass in your Cosmos DB or Storage account name, account key and table endpoint. Replace `myaccount`, `mykey` and `mytableendpoint` with your Cosmos DB or Storage account name, key and table endpoint.

-```python

-from azure.core.credentials import AzureNamedKeyCredential

-credential = AzureNamedKeyCredential("myaccount", "mykey")

-table_service = TableServiceClient(endpoint="mytableendpoint", credential=credential)

-### Creating the Table service client from a connection string

-Copy your Cosmos DB or Storage account connection string from the Azure portal, and create a `TableServiceClient` object using your copied connection string:

-```python

-table_service = TableServiceClient.from_connection_string(conn_str='DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;TableEndpoint=mytableendpoint;')

-## Create a table

-Call `create_table` to create the table.

-```python

-table_service.create_table('tasktable')

-## Add an entity to a table

-Create a table in your account and get a `TableClient` to perform operations on the newly created table. To add an entity, you first create an object that represents your entity, then pass the object to the `TableClient.create_entity` method. The entity object can be a dictionary or an object of type `TableEntity`, and defines your entity's property names and values. Every entity must include the required [PartitionKey and RowKey](#partitionkey-and-rowkey) properties, in addition to any other properties you define for the entity.

-This example creates a dictionary object representing an entity, then passes it to the `create_entity` method to add it to the table:

-```python

-table_client = table_service.get_table_client(table_name="tasktable")

-task = {u'PartitionKey': u'tasksSeattle', u'RowKey': u'001',

- u'description': u'Take out the trash', u'priority': 200}

-table_client.create_entity(entity=task)

-This example creates an `TableEntity` object, then passes it to the `create_entity` method to add it to the table:

-```python

-task = TableEntity()

-task[u'PartitionKey'] = u'tasksSeattle'

-task[u'RowKey'] = u'002'

-task[u'description'] = u'Wash the car'

-task[u'priority'] = 100

-table_client.create_entity(task)

-### PartitionKey and RowKey

-You must specify both a **PartitionKey** and a **RowKey** property for every entity. These are the unique identifiers of your entities, as together they form the primary key of an entity. You can query using these values much faster than you can query any other entity properties because only these properties are indexed.

-The Table service uses **PartitionKey** to intelligently distribute table entities across storage nodes. Entities that have the same **PartitionKey** are stored on the same node. **RowKey** is the unique ID of the entity within the partition it belongs to.

-## Update an entity

-To update all of an entity's property values, call the `update_entity` method. This example shows how to replace an existing entity with an updated version:

-task = {u'PartitionKey': u'tasksSeattle', u'RowKey': u'001',

- u'description': u'Take out the garbage', u'priority': 250}

-table_client.update_entity(task)

-If the entity that is being updated doesn't already exist, then the update operation will fail. If you want to store an entity whether it exists or not, use `upsert_entity`. In the following example, the first call will replace the existing entity. The second call will insert a new entity, since no entity with the specified PartitionKey and RowKey exists in the table.

-# Replace the entity created earlier

-task = {u'PartitionKey': u'tasksSeattle', u'RowKey': u'001',

- u'description': u'Take out the garbage again', u'priority': 250}

-table_client.upsert_entity(task)

-# Insert a new entity

-task = {u'PartitionKey': u'tasksSeattle', u'RowKey': u'003',

- u'description': u'Buy detergent', u'priority': 300}

-table_client.upsert_entity(task)

-> [!TIP]

-> The **mode=UpdateMode.REPLACE** parameter in `update_entity` method replaces all properties and values of an existing entity, which you can also use to remove properties from an existing entity. The **mode=UpdateMode.MERGE** parameter is used by default to update an existing entity with new or modified property values without completely replacing the entity.

-## Modify multiple entities

-To ensure the atomic processing of a request by the Table service, you can submit multiple operations together in a batch. First, add multiple operations to a list. Next, call `Table_client.submit_transaction` to submit the operations in an atomic operation. All entities to be modified in batch must be in the same partition.

-This example adds two entities together in a batch:

-task004 = {u'PartitionKey': u'tasksSeattle', u'RowKey': '004',

- 'description': u'Go grocery shopping', u'priority': 400}

-task005 = {u'PartitionKey': u'tasksSeattle', u'RowKey': '005',

- u'description': u'Clean the bathroom', u'priority': 100}

-operations = [("create", task004), ("create", task005)]

-table_client.submit_transaction(operations)

-## Query for an entity

-To query for an entity in a table, pass its PartitionKey and RowKey to the `Table_client.get_entity` method.

-task = table_client.get_entity('tasksSeattle', '001')

-print(task['description'])

-print(task['priority'])

-## Query a set of entities

-You can query for a set of entities by supplying a filter string with the **query_filter** parameter. This example finds all tasks in Seattle by applying a filter on PartitionKey:

-tasks = table_client.query_entities(query_filter="PartitionKey eq 'tasksSeattle'")

-for task in tasks:

- print(task['description'])

- print(task['priority'])

-## Query a subset of entity properties

-You can also restrict which properties are returned for each entity in a query. This technique, called *projection*, reduces bandwidth and can improve query performance, especially for large entities or result sets. Use the **select** parameter and pass the names of the properties you want returned to the client.

-The query in the following code returns only the descriptions of entities in the table.

-> [!NOTE]

-> The following snippet works only against the Azure Storage. It is not supported by the Storage Emulator.

-tasks = table_client.query_entities(

- query_filter="PartitionKey eq 'tasksSeattle'", select='description')

-for task in tasks:

- print(task['description'])

-## Query for an entity without partition and row keys

-You can also list entities within a table without using the partition and row keys. Use the `table_client.list_entities` method as show in the following example:

-print("Get the first item from the table")

-tasks = table_client.list_entities()

-lst = list(tasks)

-print(lst[0])

-## Delete an entity

-Delete an entity by passing its **PartitionKey** and **RowKey** to the `delete_entity` method.

-table_client.delete_entity('tasksSeattle', '001')

-## Delete a table

-If you no longer need a table or any of the entities within it, call the `delete_table` method to permanently delete the table from Azure Storage.

-```python

-table_service.delete_table('tasktable')

-* [FAQ - Develop with the Table API](table-api-faq.yml)

-* [Azure Data Tables SDK for Python API reference](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/tables/azure-data-tables)

-* [Python Developer Center](https://azure.microsoft.com/develop/python/)

-* [Microsoft Azure Storage Explorer](../../vs-azure-tools-storage-manage-with-storage-explorer.md): A free, cross-platform application for working visually with Azure Storage data on Windows, macOS, and Linux.

-* [Working with Python in Visual Studio (Windows)](/visualstudio/python/overview-of-python-tools-for-visual-studio)

-You must have an account admin role for a subscription to download its invoice. Users with owner, contributor, or reader roles can download its invoice, if the account admin has given them permission. For more information, see [Allow users to download invoices](../manage/manage-billing-access.md#opt-in).

-description: Steps to migrate from Azure Data Catalog to Microsoft's unified data governance service--Azure Purview.

-#Customer intent: As an Azure Data Catalog user, I want to know why and how to migrate to Azure Purview so that I can use the best tools to manage my data.

-# Migrate from Azure Data Catalog to Azure Purview

-Microsoft launched a unified data governance service to help manage and govern your on-premises, multi-cloud, and software-as-a-service (SaaS) data. Azure Purview creates a map of your data landscape with automated data discovery, sensitive data classification, and end-to-end data lineage. Azure Purview enables data curators to manage and secure their data estate and empowers data consumers to find valuable, trustworthy data.

-The document shows you how to do the migration from Azure Data Catalog to Azure Purview.

-To migrate from Azure Data Catalog to Azure Purview, we recommend the following approach:

-:heavy_check_mark: Step 3: [Migrate to Azure Purview](#migrate-to-azure-purview)

-:heavy_check_mark: Step 4: [Cutover from Azure Data Catalog to Azure Purview](#cutover-from-azure-data-catalog-to-azure-purview)

-> Azure Data Catalog and Azure Purview are different services, so there is no in-place upgrade experience. Intentional migration effort required.

-Look at [Azure Purview](https://azure.microsoft.com/services/purview/) and understand key differences of Azure Data Catalog and Azure Purview.

-||Azure Data Catalog |Azure Purview |

-Compare [Azure Data Catalog supported sources](data-catalog-dsr.md) and [Azure Purview supported sources](../purview/purview-connector-overview.md), to confirm you can support your data landscape.

- Take this opportunity to identify logical and business connections between your data sources and assets. Azure Purview will allow you to create a map of your data landscape that reflects how your data is used and discovered in your organization.

-1. Review [Azure Purview best practices for deployment and architecture](../purview/deployment-best-practices.md) to develop a deployment strategy for Azure Purview.

-## Migrate to Azure Purview

-Manually migrate your data from Azure Data Catalog to Azure Purview.

-[Create an Azure Purview account](../purview/create-catalog-portal.md), [create collections](../purview/create-catalog-portal.md) in your data map, set up [permissions for your users](../purview/catalog-permissions.md), and onboard your data sources.

-We suggest you review the Azure Purview best practices documentation before deploying your Azure Purview account, so you can deploy the best environment for your data landscape.

-## Cutover from Azure Data Catalog to Azure Purview

-After the business has begun to use Azure Purview, cutover from Azure Data Catalog by deleting the Azure Data Catalog.

-For general use cases, it is recommended to include global parameters in the ARM template. This integrates natively with the solution outlined in [the CI/CD doc](continuous-integration-delivery.md). In case of automatic publishing and Azure Purview connection, **PowerShell script** method is required. You can find more about PowerShell script method later. Global parameters will be added as an ARM template parameter by default as they often change from environment to environment. You can enable the inclusion of global parameters in the ARM template from the **Manage** hub.

-> The **Include in ARM template** configuration is only available in "Git mode". Currently it is disabled in "live mode" or "Data Factory" mode. In case of automatic publishing or Azure Purview connection, do not use Include global parameters method; use PowerShell script method.

-What we have observed is that the token was obtained from the original tenant, but the service is in guest tenant and trying to use the token to visit DevOps in guest tenant. This is not the expected behavior.

-You have detached the Git configuration and set it up again with the "Import resources" flag selected, which sets the service as "in sync". This means no change during publication..

-You can delete the SSIS-IR and Shared IRs to allow the move operation. If you do not want to delete the integration runtimes, then the best way is to follow the copy and clone document to do the copy and after it's done, delete the old data factory.

-### Cannot publish because of 4 MB ARM template limit

-You cannot deploy because you hit Azure Resource Manager limit of 4 MB total template size. You need a solution to deploy after crossing the limit.

-Azure Resource Manager restricts template size to be 4-MB. Limit the size of your template to 4-MB, and each parameter file to 64 KB. The 4 MB limit applies to the final state of the template after it has been expanded with iterative resource definitions, and values for variables and parameters. But, you have crossed the limit.

-You cannot connect to GIT Enterprise because of permission issues. You can see error like **422 - Unprocessable Entity.**

-* You have not configured Oauth for the service.

-It is possible to recover the instance only if source control was configured for it with DevOps or Git. This action will bring all the latest published resources, but **will not** restore any unpublished pipelines, datasets, or linked services. If there is no Source control, recovering a deleted instance from the Azure backend is not possible because once the service receives the delete command, the instance is permanently deleted without any backup.

-### Cannot deploy to different stage using automatic publish method

-### Git Repository or Azure Purview Connection Disconnected

-During development and deployment cycles, you may want to unit test your pipeline before you manually or automatically publish your pipeline. Test automation allows you to run more tests, in less time, with guaranteed repeatability. Automatically re-testing all your pipelines before deployment gives you some protection against regression faults. Automated testing is a key component of CI/CD software development approaches: inclusion of automated tests in CI/CD deployment pipelines can significantly improve quality. In long run, tested pipeline artifacts are reused saving you cost and time.

-When you have 1000s of old temporary ARM json files in PartialTemplates folder, publish may fail.

-* [Twitter information about Data Factory](https://twitter.com/hashtag/DataFactory)

-description: Learn about how to connect a Data Factory to Azure Purview

-# Connect Data Factory to Azure Purview (Preview)

-[Azure Purview](../purview/overview.md) is a unified data governance service that helps you manage and govern your on-premises, multi-cloud, and software-as-a-service (SaaS) data. You can connect your data factory to Azure Purview. That connection allows you to use Azure Purview for capturing lineage data, and to discover and explore Azure Purview assets.

-## Connect Data Factory to Azure Purview

-You have two options to connect data factory to Azure Purview:

-### Connect to Azure Purview account in Data Factory

-You need to have **Owner** or **Contributor** role on your data factory to connect to an Azure Purview account.

-1. In the ADF authoring UI, go to **Manage** -> **Azure Purview**, and select **Connect to an Azure Purview account**.

- :::image type="content" source="./media/data-factory-purview/register-purview-account.png" alt-text="Screenshot for registering an Azure Purview account.":::

-3. Once connected, you can see the name of the Azure Purview account in the tab **Azure Purview account**.

-If your Azure Purview account is protected by firewall, create the managed private endpoints for Azure Purview. Learn more about how to let Data Factory [access a secured Azure Purview account](how-to-access-secured-purview-account.md). You can either do it during the initial connection or edit an existing connection later.

-The Azure Purview connection information is stored in the data factory resource like the following. To establish the connection programmatically, you can update the data factory and add the `purviewConfiguration` settings. When you want to push lineage from SSIS activities, also add `catalogUri` tag additionally.

-### Register Data Factory in Azure Purview

-For how to register Data Factory in Azure Purview, see [How to connect Azure Data Factory and Azure Purview](../purview/how-to-link-azure-data-factory.md).

-Data factory's managed identity is used to authenticate lineage push operations from data factory to Azure Purview.

-Grant the data factory's managed identity **Data Curator** role on your Azure Purview **root collection**. Learn more about [Access control in Azure Purview](../purview/catalog-permissions.md) and [Add roles and restrict access through collections](../purview/how-to-create-and-manage-collections.md#add-roles-and-restrict-access-through-collections).

-When connecting data factory to Azure Purview on authoring UI, ADF tries to add such role assignment automatically. If you have **Collection admins** role on the Azure Purview root collection and have access to Azure Purview account from your network, this operation is done successfully.

-## Monitor Azure Purview connection

-Once you connect the data factory to an Azure Purview account, you see the following page with details on the enabled integration capabilities.

- - Cannot reach the Azure Purview account from your current network because the account is protected by firewall. You can launch the ADF UI from a private network that has connectivity to your Azure Purview account instead.

- - You don't have permission to check role assignments on the Azure Purview account. You can contact the Azure Purview account admin to check the role assignments for you. Learn about the needed Azure Purview role from [Set up authentication](#set-up-authentication) section.

-## Report lineage data to Azure Purview

-Once you connect the data factory to an Azure Purview account, when you execute pipelines, Data Factory push lineage information to the Azure Purview account. For detailed supported capabilities, see [Supported Azure Data Factory activities](../purview/how-to-link-azure-data-factory.md#supported-azure-data-factory-activities). For an end to end walkthrough, refer to [Tutorial: Push Data Factory lineage data to Azure Purview](tutorial-push-lineage-to-purview.md).

-## Discover and explore data using Azure Purview

-Once you connect the data factory to an Azure Purview account, you can use the search bar at the top center of Data Factory authoring UI to search for data and perform actions. Learn more from [Discover and explore data in ADF using Azure Purview](how-to-discover-explore-purview-data.md).

-[Tutorial: Push Data Factory lineage data to Azure Purview](tutorial-push-lineage-to-purview.md)

-[Discover and explore data in ADF using Azure Purview](how-to-discover-explore-purview-data.md)

-[Access a secured Azure Purview account](how-to-access-secured-purview-account.md)

->- When the service imports a schema in the authoring UI, it infers the schema. It does so by sampling the top rows from the Dynamics query result to initialize the source column list. In that case, columns with no values in the top rows are omitted. The same behavior applies to copy executions if there is no explicit mapping. You can review and add more columns into the mapping, which are honored during copy runtime.

-If you want to restrict access for data factory resources in your subscriptions by private link, please follow [Use portal to create private link for managing Azure resources](https://docs.microsoft.com/azure/azure-resource-manager/management/create-private-link-access-portal?source=docs)

-[Azure Purview](turorial-push-lineage-to-purview.md)

-description: Learn about how to access a firewall protected Azure Purview account through private endpoints from Azure Data Factory

-# Access a secured Azure Purview account from Azure Data Factory

-This article describes how to access a secured Azure Purview account from Azure Data Factory for different integration scenarios.

-## Azure Purview private endpoint deployment scenarios

-You can use [Azure private endpoints](../private-link/private-endpoint-overview.md) for your Azure Purview accounts to allow secure access from a virtual network (VNet) to the catalog over a Private Link. Azure Purview provides different types of private points for various access need: *account* private endpoint, *portal* private endpoint, and *ingestion* private endpoints. Learn more from [Azure Purview private endpoints conceptual overview](../purview/catalog-private-link.md#conceptual-overview).

-If your Azure Purview account is protected by firewall and denies public access, make sure you follow below checklist to set up the private endpoints so Data Factory can successfully connect to Azure Purview.

-| Scenario | Required Azure Purview private endpoints |

-| [Run pipeline and report lineage to Azure Purview](tutorial-push-lineage-to-purview.md) | For Data Factory pipeline to push lineage to Azure Purview, Azure Purview ***account*** and ***ingestion*** private endpoints are required. <br>- When using **Azure Integration Runtime**, follow the steps in [Managed private endpoints for Azure Purview](#managed-private-endpoints-for-azure-purview) section to create managed private endpoints in the Data Factory managed virtual network.<br>- When using **Self-hosted Integration Runtime**, follow the steps in [this section](../purview/catalog-private-link-end-to-end.md#option-2enable-account-portal-and-ingestion-private-endpoint-on-existing-azure-purview-accounts) to create the *account* and *ingestion* private endpoints in your integration runtime's virtual network. |

-| [Discover and explore data using Azure Purview on ADF UI](how-to-discover-explore-purview-data.md) | To use the search bar at the top center of Data Factory authoring UI to search for Azure Purview data and perform actions, you need to create Azure Purview ***account*** and ***portal*** private endpoints in the virtual network that you launch the Data Factory Studio. Follow the steps in [Enable *account* and *portal* private endpoint](../purview/catalog-private-link-account-portal.md#option-2enable-account-and-portal-private-endpoint-on-existing-azure-purview-accounts). |

-## Managed private endpoints for Azure Purview

-[Managed private endpoints](managed-virtual-network-private-endpoint.md#managed-private-endpoints) are private endpoints created in the Azure Data Factory Managed Virtual Network establishing a private link to Azure resources. When you run pipeline and report lineage to a firewall protected Azure Purview account, create an Azure Integration Runtime with "Virtual network configuration" option enabled, then create the Azure Purview ***account*** and ***ingestion*** managed private endpoints as follows.

-To create managed private endpoints for Azure Purview on Data Factory authoring UI:

-1. Go to **Manage** -> **Azure Purview**, and click **Edit** to edit your existing connected Azure Purview account or click **Connect to an Azure Purview account** to connect to a new Azure Purview account.

-3. Click **+ Create all** button to batch create the needed Azure Purview private endpoints, including the ***account*** private endpoint and the ***ingestion*** private endpoints for the Azure Purview managed resources - Blob storage, Queue storage, and Event Hubs namespace. You need to have at least **Reader** role on your Azure Purview account for Data Factory to retrieve the Azure Purview managed resources' information.

- :::image type="content" source="./media/how-to-access-secured-purview-account/purview-create-all-managed-private-endpoints.png" alt-text="Create managed private endpoint for your connected Azure Purview account.":::

- :::image type="content" source="./media/how-to-access-secured-purview-account/name-purview-private-endpoints.png" alt-text="Name the managed private endpoints for your connected Azure Purview account.":::

-5. Click **Create** to create the private endpoints. After creation, 4 private endpoint requests will be generated that must [get approved by an owner of Azure Purview](#approve-private-endpoint-connections).

-Such batch managed private endpoint creation is provided on the Azure Purview UI only. If you want to create the managed private endpoints programmatically, you need to create those PEs individually. You can find Azure Purview managed resources' information from Azure portal -> your Azure Purview account -> Managed resources.

-After you create the managed private endpoints for Azure Purview, you see "Pending" state first. The Azure Purview owner need to approve the private endpoint connections for each resource.

-If you have permission to approve the Azure Purview private endpoint connection, from Data Factory UI:

-1. Go to **Manage** -> **Azure Purview** -> **Edit**

-If you don't have permission to approve the Azure Purview private endpoint connection, ask the Azure Purview account owner to do as follows.

-You can monitor the created managed private endpoints for Azure Purview at two places:

-description: Learn how to discover, explore data in Azure Data Factory using Azure Purview

-# Discover and explore data in ADF using Azure Purview

-In this article, you will register an Azure Purview Account to a Data Factory. That connection allows you to discover Azure Purview assets and interact with them through ADF capabilities.

-## Using Azure Purview in Data Factory

-The use Azure Purview in Data Factory requires you to have access to that Azure Purview account. Data Factory passes-through your Azure Purview permission. As an example, if you have a curator permission role, you will be able to edit metadata scanned by Azure Purview.

-To discover data registered and scanned by Azure Purview, you can use the Search bar at the top center of Data Factory portal. Make sure that you select Azure Purview to search for all of your organization data.

-You can directly create Linked Service, Dataset, or dataflow over the data you search by Azure Purview.

-[Tutorial: Push Data Factory lineage data to Azure Purview](turorial-push-lineage-to-purview.md)

-[Connect an Azure Purview Account into Data Factory](connect-data-factory-to-azure-purview.md)

-[How to Search Data in Azure Purview Data Catalog](../purview/how-to-search-catalog.md)

-description: Learn about how to push Data Factory lineage data to Azure Purview

-# Push Data Factory lineage data to Azure Purview

-In this tutorial, you'll use the Data Factory user interface (UI) to create a pipeline that run activities and report lineage data to Azure Purview account. Then you can view all the lineage information in your Azure Purview account.

-* **Azure Purview account**. The Azure Purview account captures all lineage data generated by data factory. If you don't have an Azure Purview account, see [Create an Azure Purview](../purview/create-catalog-portal.md).

-## Run pipeline and push lineage data to Azure Purview

-### Step 1: Connect Data Factory to your Azure Purview account

-You can establish the connection between Data Factory and Azure Purview account by following the steps in [Connect Data Factory to Azure Purview](connect-data-factory-to-azure-purview.md).

-### Step 4: View lineage information in your Azure Purview account

-On Azure Purview UI, you can browse assets and choose type "Azure Data Factory". You can also search the Data Catalog using keywords.

- :::image type="content" source="./media/data-factory-purview/copy-lineage.png" alt-text="Screenshot of the Copy activity lineage in Azure Purview." lightbox="./media/data-factory-purview/copy-lineage.png":::

- :::image type="content" source="./media/data-factory-purview/dataflow-lineage.png" alt-text="Screenshot of the Data Flow lineage in Azure Purview." lightbox="./media/data-factory-purview/dataflow-lineage.png":::

- :::image type="content" source="./media/data-factory-purview/ssis-lineage.png" alt-text="Screenshot of the Execute SSIS lineage in Azure Purview." lightbox="./media/data-factory-purview/ssis-lineage.png":::

-[Connect Data Factory to Azure Purview](connect-data-factory-to-azure-purview.md)

-1. Verify the SMB credentials for the share. In the local web UI of your device, go to **Connect and copy**, and select **SMB** for the share. You'll see the following dialog box.

-description: Use Azure Purview's data sensitivity classifications in Microsoft Defender for Cloud

-[Azure Purview](../purview/overview.md), Microsoft's data governance service, provides rich insights into the *sensitivity of your data*. With automated data discovery, sensitive data classification, and end-to-end data lineage, Azure Purview helps organizations manage and govern data in hybrid and multi-cloud environments.

-Microsoft Defender for Cloud customers using Azure Purview can benefit from an additional vital layer of metadata in alerts and recommendations: information about any potentially sensitive data involved. This knowledge helps solve the triage challenge and ensures security professionals can focus their attention on threats to sensitive data.

-This page explains the integration of Azure Purview's data sensitivity classification labels within Defender for Cloud.

-|Pricing:|You'll need an Azure Purview account to create the data sensitivity classifications and run the scans. Viewing the scan results and using the output is free for Defender for Cloud users|

-Azure Purview's data sensitivity classifications and data sensitivity labels provide that knowledge.

-To provide the vital information about discovered sensitive data, and help ensure you have that information when you need it, Defender for Cloud displays information from Azure Purview in multiple locations.

-> If a resource is scanned by multiple Azure Purview accounts, the information shown in Defender for Cloud relates to the most recent scan.

-The [asset inventory page](asset-inventory.md) has a collection of powerful filters to group your resources with outstanding alerts and recommendations according to the criteria relevant for any scenario. These filters include **Data sensitivity classifications** and **Data sensitivity labels**. Use these filters to evaluate the security posture of resources on which Azure Purview has discovered sensitive data.

-When reviewing the health of a specific resource, you'll see the Azure Purview information on this page and can use it determine what data has been discovered on this resource alongside the Azure Purview account used to scan the resource.

-The dedicated **Information protection** tile in Defender for CloudΓÇÖs [overview dashboard](overview-page.md) shows Azure PurviewΓÇÖs coverage. It also shows the resource types with the most sensitive data discovered.

-A graph shows the number of recommendations and alerts by classified resource types. The tile also includes a link to Azure Purview to scan additional resources. Select the tile to see classified resources in Defender for CloudΓÇÖs asset inventory page.

-### Prioritize security actions by data sensitivity (powered by Azure Purview) (in preview)

-To address this challenge, Microsoft Defender for Cloud now integrates sensitivity information from [Azure Purview](../purview/overview.md). Azure Purview is a unified data governance service that provides rich insights into the sensitivity of your data within multi-cloud, and on-premises workloads.

-The integration with Azure Purview extends your security visibility in Defender for Cloud from the infrastructure level down to the data, enabling an entirely new way to prioritize resources and security activities for your security teams.

-Use the following procedure to create a scale set to use with your sensor connection. For more information, see [What are virtual machine scale sets?](/azure/virtual-machine-scale-sets/overview)

-For more information, see the [Azure Load Balancer documentation](/azure/load-balancer/load-balancer-overview).

-For more information, see [Sensor connection methods](architecture-connections.md).

-**For IoT device builders**, Microsoft Defender for IoT also offers a lightweight, micro-agent that supports standard IoT operating systems, such as Linux and RTOS. The Microsoft Defender device builder agent helps you ensure that security is built into your IoT/OT projects, from the cloud. For more information, see [Microsoft Defender for IoT for device builders documentation](/azure/defender-for-iot/device-builders/overview).

-[Azure Monitor workbooks](/azure/azure-monitor/visualize/workbooks-overview) provide graphs and dashboards that visually reflect your data, and are now available directly in Microsoft Defender for IoT with data from [Azure Resource Graph](/azure/governance/resource-graph/).

-[Getting started with Defender for IoT](getting-started.md)

-For management, such as adding users or adding lab owners, all lab users must belong to the [Azure Active Directory (Azure AD)](https://azure.microsoft.com/services/active-directory) tenant for the Azure subscription the pilot uses. Many enterprises set up [hybrid identity](/azure/active-directory/hybrid/whatis-hybrid-identity) to enable users to use their on-premises identities in the cloud. You don't need a hybrid identity for a DevTest Labs proof of concept.

-The enterprise-level requirements for resources in Azure can extend beyond the [available quotas within a single subscription](/azure/azure-resource-manager/management/azure-subscription-service-limits). You might need several Azure subscriptions, or you might need to make service requests to increase initial subscription limits. For more information, see [Scalability considerations](devtest-lab-reference-architecture.md#scalability-considerations).

-The [default network infrastructure](/azure/app-service/networking-features) that DevTest Labs automatically creates might not meet requirements and constraints for enterprise users. For example, enterprises often use:

-You can add a DevTest Labs User to a lab by using the following Azure PowerShell script. The script requires the user to be in the Azure Active Directory (Azure AD). For information about adding an external user to Azure AD as a guest, see [Add a new guest user](/azure/active-directory/fundamentals/add-users-azure-active-directory#add-a-new-guest-user). If the user isn't in Azure AD, use the portal procedure instead.

-This article explains how to attach and detach a lab virtual machine (VM) data disk in Azure DevTest Labs. You can create, attach, detach, and reattach [data disks](/azure/virtual-machines/managed-disks-overview) for lab VMs that you own. This functionality is useful for managing storage or software separately from individual VMs.

-To attach or detach a data disk, you need to own the lab VM, and the VM must be running. The VM size determines how many data disks you can attach. For more information, see [Sizes for virtual machines](/azure/virtual-machines/sizes).

- - For **Disk type**, select a [disk type](/azure/virtual-machines/disks-types) from the drop-down list.

-For information about transferring data disks for claimable lab VMs, see [Transfer the data disk](devtest-lab-add-claimable-vm.md#transfer-the-data-disk).

-In this architecture, [Azure Active Directory (Azure AD)](/azure/active-directory/fundamentals/active-directory-whatis) provides identity and access management across all networks. Lab VMs usually have a local administrative account for access. If there's an Azure AD, on-premises, or [Azure AD Domain Services](../active-directory-domain-services/overview.md) domain available, you can join lab VMs to the domain. Users can then use their domain-based identities to connect to the VMs.

-See the next article in this series: [Deliver a proof of concept](deliver-proof-concept.md).

-For in-depth troubleshooting, see [Configure Azure Storage firewalls and virtual networks](/azure/storage/common/storage-network-security).

-For instructions on finding the log files on a **Linux** VM, see [Use the Azure Custom Script Extension Version 2 with Linux virtual machines](/azure/virtual-machines/extensions/custom-script-linux#troubleshooting).

-Ensure that the [Azure Virtual Machine Agent (VM Agent)](/azure/virtual-machines/extensions/agent-windows) is installed and ready.

-When the VM first starts, or when the CSE first installs to serve the request to apply artifacts, the VM might need to either upgrade the VM Agent or wait for the VM Agent to initialize. The VM Agent might depend on services that take a long time to initialize. For further troubleshooting, see [Azure Virtual Machine Agent overview](/azure/virtual-machines/extensions/agent-windows).

-For general information about Azure extensions, see [Azure virtual machine extensions and features](/azure/virtual-machines/extensions/overview).

-[Az.DevTestLabs PowerShell reference](/powershell/module/az.devtestlabs/)

-For more information about managing Azure Storage, see [Optimize costs by automatically managing the data lifecycle](../storage/blobs/lifecycle-management-overview.md).

-By default, Azure DevTest Labs creates a new [Azure virtual network](/azure/virtual-network/virtual-networks-overview) for each lab. The virtual network acts as a security boundary to isolate lab resources from the public internet. To ensure lab resources follow organizational networking policies, you can use several other networking options:

-You can automate these steps with PowerShell or Azure CLI to configure network isolation for multiple labs. For more information, see [Configure Azure Storage firewalls and virtual networks](/azure/storage/common/storage-network-security).

-For more information, see [Connect to a storage account using an Azure Private Endpoint](/azure/private-link/tutorial-private-endpoint-storage-portal).

-For more information, see [Assign a Key Vault access policy](/azure/key-vault/general/assign-access-policy).

-To add users to a lab, you must be a [User Access Administrator](/azure/role-based-access-control/built-in-roles#user-access-administrator) or [Owner](/azure/role-based-access-control/built-in-roles#owner) of the subscription the lab is in. For more information, see [Add lab owners, contributors, and users in Azure DevTest Labs](devtest-lab-add-devtest-user.md).

-> [Tutorial: Access the lab](tutorial-use-custom-lab.md)

-You need at least [DevTest Labs User](/azure/role-based-access-control/built-in-roles#devtest-labs-user) access to the lab created in [Tutorial: Set up a lab in Azure DevTest Labs](tutorial-create-custom-lab.md), or to another lab that has a claimable VM.

-Once you connect to the VM, you can use it to do your work. You have [Owner](/azure/role-based-access-control/built-in-roles#owner) role on all lab VMs you claim or create, unless you unclaim them.

-In this tutorial, you learned how to claim and connect to claimable VMs in Azure DevTest Labs. To create your own lab VMs, see [Create lab virtual machines in Azure DevTest Labs](devtest-lab-add-vm.md).

-DevTest Labs supports [nested ARM templates](/azure/azure-resource-manager/templates/linked-templates). To use `_artifactsLocation` and `_artifactsLocationSasToken` tokens to create a URI to a nested ARM template, see [Deploy DevTest Labs environments by using nested templates](deploy-nested-template-environments.md). For more information, see the **Deployment artifacts** section of the [Azure Resource Manager Best Practices Guide](https://github.com/Azure/azure-quickstart-templates/blob/master/1-CONTRIBUTION-GUIDE/best-practices.md#deployment-artifacts-nested-templates-scripts).

-It also contains a sample twin graph and telemetry scenario that you can use to see the historized twin updates in Azure Data Explorer.

-If you already have twins in your Azure Digital Twins instance that are receiving telemetry updates, you can skip this section and visualize the results using your own resources.

-Otherwise, continue through this section to set up a sample graph containing twins that can receive telemetry updates.

-You can set up a sample graph for this scenario using the **Azure Digital Twins Data Simulator**. The Azure Digital Twins Data Simulator continuously pushes telemetry to several twins in an Azure Digital Twins instance.

-You can use the **Azure Digital Twins Data Simulator** to provision a sample twin graph and push telemetry data to it. The twin graph created here models pasteurization processes for a dairy company.

-description: This article explains transition from Event Grid on Azure IoT Edge to Azure IoT Edge MQTT Broker or IoT Hub message routing.

-## Why are we retiring?

-There are multiple reasons for deciding to retire Event Grid on IoT Edge, which is currently in Preview, in March 2023.

-Here's the list of the features that will be removed with the retirement of Event Grid on Azure IoT Edge and a list of the new IoT Edge native capabilities.

-| Event Grid on Azure IoT Edge | MQTT broker on Azure IoT Edge |

-| - Publishing and subscribing to events locally/cloud<br/>- Forwarding events to Event Grid<br/>- Forwarding events to IoT Hub<br/>- React to Blob Storage events locally | - Connectivity to IoT Edge hub<br/>- Publish and subscribe on user-defined topics<br/>- Publish and subscribe on IoT Hub topics<br/>- Publish and subscribe between MQTT brokers |

-| Publish, subscribe and forward events locally or cloud | You can use Azure IoT Edge MQTT broker to publish and subscribe messages. To learn how to connect to this broker, publish and subscribe to messages over user-defined topics, and use IoT Hub messaging primitives, see [publish and subscribe with Azure IoT Edge](../../iot-edge/how-to-publish-subscribe.md). The IoT Edge MQTT broker is built in the IoT Edge hub. For more information, see [the brokering capabilities of the IoT Edge hub](../../iot-edge/iot-edge-runtime.md). </br> </br> If you're subscribing to IoT Hub, itΓÇÖs possible to create an event to publish to Event Grid if you need. For details, see [Azure IoT Hub and Event Grid](../../iot-hub/iot-hub-event-grid.md). |

-| Forward events to IoT Hub | You can use IoT Hub message routing to send device-cloud messages to different endpoints. For details, see [Understand Azure IoT Hub message routing](../../iot-hub/iot-hub-devguide-messages-d2c.md). |

-| React to Blob Storage events on IoT Edge (Preview) | You can use Azure Function Apps to react to blob storage events on cloud when a blob is created or updated. For more information, see [Azure Blob storage trigger for Azure Functions](../../azure-functions/functions-bindings-storage-blob-trigger.md) and [Tutorial: Deploy Azure Functions as modules - Azure IoT Edge](../../iot-edge/tutorial-deploy-function.md). Blob triggers in IoT Edge blob storage module aren't supported. |

-|Start/Stop doesnΓÇÖt work with a firewall configured in forced-tunnel mode|Start/stop doesnΓÇÖt work with Azure firewall configured in forced-tunnel mode. Attempting to start Azure Firewall with forced tunneling configured results in the following error:<br><br>*Set-AzFirewall: AzureFirewall FW-xx management IP configuration cannot be added to an existing firewall. Redeploy with a management IP configuration if you want to use forced tunneling support.<br>StatusCode: 400<br>ReasonPhrase: Bad Request*|Under investigation.<br><br>As a workaround, you can delete the existing firewall and create a new one with the same parameters.|

-## Azure Purview

-1. Create an [Azure Policy](/azure/governance/policy/tutorials/create-and-manage) in the Azure portal.

- There are several [predefined sample policies](/azure/governance/policy/samples/)

-> [Azure Policy definition structure](../concepts/definition-structure.md)

-The **FHIR to Synapse Sync Agent** is a Microsoft OSS project released under MIT License. It's an Azure function that extracts data from a FHIR server using FHIR Resource APIs, converts it to hierarchical Parquet files, and writes it to Azure Data Lake in near real time. This also contains a script to create external tables and views in [Synapse Serverless SQL pool](https://docs.microsoft.com/azure/synapse-analytics/sql/on-demand-workspace-overview) pointing to the Parquet files.

-To specify the Azure Storage account, you need to use [Rest API](https://docs.microsoft.com/rest/api/healthcareapis/services/create-or-update) to update the FHIR service.

->[Copy data from FHIR service to Azure Synapse Analytics](copy-to-synapse.md)

-The **FHIR to Synapse Sync Agent** is a Microsoft OSS project released under MIT License. It's an Azure function that extracts data from a FHIR server using FHIR Resource APIs, converts it to hierarchical Parquet files, and writes it to Azure Data Lake in near real time. This also contains a script to create external tables and views in [Synapse Serverless SQL pool](https://docs.microsoft.com/azure/synapse-analytics/sql/on-demand-workspace-overview) pointing to the Parquet files.

-The **FHIR to CDM pipeline generator** is a Microsoft OSS project released under MIT License. It's a tool to generate an ADF pipeline for copying a snapshot of data from a FHIR server using $export API, transforming it to csv format, and writing to a [CDM folder](https://docs.microsoft.com/common-data-model/data-lake) in Azure Data Lake Storage Gen 2. The tool requires a user-created configuration file containing instructions to project and flatten FHIR Resources and fields into tables. You can also follow the instructions for creating a downstream pipeline in Synapse workspace to move data from CDM folder to Synapse dedicated SQL pool.

->[Exporting de-identified data](./de-identified-export.md)

- - [Device Provisioning Service](https://docs.microsoft.com/azure/iot-dps/) used for deploying and provisioning the simulation gateways

-> [Customize the configuration of the components](tutorial-configure-industrial-iot-components.md)

-Private endpoints use private IP addresses from a virtual network address space to connect your devices privately to your IoT Central application. Network traffic between devices on the virtual network and the IoT platform traverses the virtual network and a private link on the [Microsoft backbone network](/azure/networking/microsoft-global-network), eliminating exposure on the public internet.

-To learn more, see the Azure [Azure Virtual Network FAQ](/azure/virtual-network/virtual-networks-faq).

-> [Create a private endpoint for Azure IoT Central application](howto-create-private-endpoint.md).

-Private endpoints use private IP addresses from a virtual network address space to connect your devices privately to your IoT Central application. Network traffic between devices on the virtual network and the IoT platform traverses the virtual network and a private link on the [Microsoft backbone network](/azure/networking/microsoft-global-network), eliminating exposure on the public internet. This article shows you how to create a private endpoint for your IoT Central application.

-If after trying all these checks you're still experiencing an issue, try the [private endpoint troubleshooting guide](/azure/private-link/troubleshoot-private-endpoint-connectivity).

-> [Administer your application](howto-administer.md)

-Microsoft Defender for IoT provides agentless monitoring of devices in an IoT solution. Monitoring can be enhanced by including the [Microsoft Defender for IOT micro-agent for Azure RTOS](/azure/defender-for-iot/device-builders/iot-security-azure-rtos) on your device. For more information, see the [Runtime security monitoring and threat detection](#runtime-security-monitoring-and-threat-detection) recommendation.

- | **IoT Hub Device ID** | This ID will represent your device. It must follow the rules for a device ID. For more information, see [Device identity properties](../iot-hub/iot-hub-devguide-identity-registry. If the device ID is left unspecified, then the registration ID will be used.|

-9. Select the two IoT Hubs you created in the **Select the IoT hubs this group c an be assigned to** drop down.

-> [How to deprovision devices that were previously auto-provisioned](how-to-unprovision-devices.md)

-2. In the config file, find the `[agent]` section, which contains all the configuration information for the edgeAgent module to use on startup. The IoT Edge agent definition includes an `[agent.env]` subsection where you can add environment variables.

-|Azure Purview|[Using credentials for source authentication in Azure Purview](../../purview/manage-credentials.md)

- `http://<logic-app-name>.azurewebsites.net:443/api/<workflow-name>/triggers/manual/invoke?api-version=2020-05-01-preview&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=<shared-access-signature>`

- `https://fabrikam-workflows.azurewebsites.net:443/api/Fabrikam-Stateful-Workflow/triggers/manual/invoke?api-version=2020-05-01-preview&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=xxxxxXXXXxxxxxXXXXxxxXXXXxxxxXXXX`

- | **Cancelled** | The run was triggered and started but received a cancel request. |

- | **Cancelled** | The action was running but received a cancel request. |

- [cancelled-icon]: ./media/create-single-tenant-workflows-azure-portal/cancelled.png

- `http://localhost:7071/api/<workflow-name>/triggers/manual/invoke?api-version=2020-05-01-preview&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=<shared-access-signature>`

- | **Cancelled** | The run was triggered and started but received a cancellation request. |

- | **Cancelled** | The action was running but received a request to cancel. |

-azureml:/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>/compute/<compute-name>

-description: Learn how to call Grafana APIs in your automation with Azure Active Directory (Azure AD) and an Azure service principal

-# How to call Grafana APIs in your automation within Azure Managed Grafana Preview

-## Assign roles to the service principal of your application and of your Azure Managed Grafana workspace

-Replace `<access-token>` with the access token retrieved in the previous step and replace `<grafana-url>` with the URL of your Grafana instance. For example `https://grafanaworkspace-abcd.cuse.grafana.azure.com`. This URL is displayed in the Azure platform, in the **Overview** page of your Managed Grafana workspace.

- - Use only lowercase letters and numbers. The ID can include hyphens and underscores, but no spaces, and is limited to 50 characters. For example, if your Publisher ID is `testpublisherid` and you enter **test-offer-1**, the offer web address will be `https://appsource.microsoft.com/product/dynamics-365/testpublisherid.test-offer-1`.

- - Use only lowercase letters and numbers. The ID can include hyphens and underscores, but no spaces, and is limited to 50 characters. For example, if your Publisher ID is `testpublisherid` and you enter **test-offer-1**, the offer web address will be `https://appsource.microsoft.com/product/dynamics-365/testpublisherid.test-offer-1`.

- - Use only lowercase letters and numbers. The ID can include hyphens and underscores, but no spaces, and is limited to 50 characters. For example, if you enter **test-offer-1**, the offer web address will be `https://azuremarketplace.microsoft.com/marketplace/../test-offer-1`.

-|

-|||

-If you want to make changes after previewing the offer, you can edit and resubmit your publication request. If your offer is already live and available to the public in the marketplace, any updates you make won't go live until you select **Go live*. For more information, see [Update an existing offer in the commercial marketplace](update-existing-offer.md)

- * **Postgresql Version**: the version of PostgreSQL server to deploy, such as *9.5*, *9.6*, *10*, or *11* (the default).

-|Azure Purview | Southeast Asia, Australia East, Brazil South, North Europe, West Europe, Canada Central, East US, East US 2, EAST US 2 EUAP, South Central US, West Central US, West US 2, Central India, UK South | [Select for known limitations](../purview/catalog-private-link-troubleshoot.md#known-limitations) | GA <br/> [Learn how to create a private endpoint for Azure Purview.](../purview/catalog-private-link.md) |

-| Azure Purview (Microsoft.Purview) / account | privatelink.purview.azure.com | purview.azure.com |

-| Azure Purview (Microsoft.Purview) / portal| privatelink.purviewstudio.azure.com | purview.azure.com |

-| Azure Purview | Microsoft.Purview/accounts | account |

-| Azure Purview | Microsoft.Purview/accounts | portal |

-| Azure Static Web Apps | Microsoft.Web/staticSites | staticSite |

-When you scan [SAP ECC](register-scan-sapecc-source.md), [SAP S/4HANA](register-scan-saps4hana-source.md), and [SAP BW](register-scan-sap-bw.md) sources in Azure Purview, you need to create the dependent ABAP function module in your SAP server. Azure Purview invokes this function module to extract the metadata from your SAP system during scan.

-Download the SAP ABAP function module source code from Azure Purview Studio. After you register a source for [SAP ECC](register-scan-sapecc-source.md), [SAP S/4HANA](register-scan-saps4hana-source.md), or [SAP BW](register-scan-sap-bw.md), you can find a download link on top as shown in the following image. You can also see the link when you create a new scan or edit a scan.

- 1. On the main menu, upload the text file you downloaded from Azure Purview Studio as described in [Prerequisites](#prerequisites). To do so, select **Utilities** > **More Utilities** > **Upload/Download** > **Upload**.

-# Apply classifications on assets in Azure Purview

-Classifications can be system or custom types. System classifications are present in Azure Purview by default. Custom classifications can be created based on a regular expression pattern. Classifications can be applied to assets either automatically or manually.

-In Azure Purview, you can apply system or custom classifications on a file, table, or column asset. This article describes the steps to manually apply classifications on your assets.

-Azure Purview can scan and automatically classify documentation files. For example, if you have a file named *multiple.docx* and it has a National ID number in its content, Azure Purview adds the classification **EU National Identification Number** to the file asset's detail page.

-When Azure Purview scans your data sources, it doesn't automatically assign classifications to table assets. If you want your table asset to have a classification, you must add it manually.

-1. On the **Overview** page, verify that Azure Purview added your new classifications.

-Azure Purview automatically scans and adds classifications to all column assets. However, if you want to change the classification, you can do so at the column level.

-description: This how-to guide describes how to view and use Azure Purview Insights asset reporting on your data.

-# Asset insights on your data in Azure Purview

-This how-to guide describes how to access, view, and filter Azure Purview Asset insight reports for your data.

-> Azure Purview Insights are currently in PREVIEW. The [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-> * View insights from your Azure Purview account.

-Before getting started with Azure Purview insights, make sure that you've completed the following steps:

-For more information, see [Manage data sources in Azure Purview](manage-data-sources.md).

-## Use Azure Purview Asset Insights

-In Azure Purview, you can register and scan source types. Once the scan is complete, you can view the asset distribution in Asset Insights, which tells you the state of your data estate by classification and resource sets. It also tells you if there is any change in data size.

-1. Navigate to your Azure Purview account in the Azure portal.

-1. On the **Overview** page, in the **Get Started** section, select the **Open Azure Purview Studio** tile.

- :::image type="content" source="./media/asset-insights/portal-access.png" alt-text="Launch Azure Purview from the Azure portal":::

-1. On the Azure Purview **Home** page, select **Insights** on the left menu.

-1. In the **Insights** area, select **Assets** to display the Azure Purview **Asset insights** report.

-Learn more about Azure Purview insight reports with

-description: This article provides details about supported data sources, file types, and functionalities in Azure Purview.

-This article discusses currently supported data sources, file types, and scanning concepts in Azure Purview.

-## Azure Purview data sources

-> Currently, Azure Purview can't scan an asset that has `/`, `\`, or `#` in its name. To scope your scan and avoid scanning assets that have those characters in the asset name, use the example in [Register and scan an Azure SQL Database](register-scan-azure-sql-database.md#creating-the-scan).

-The following is a list of all the Azure data source (data center) regions where the Azure Purview scanner runs. If your Azure data source is in a region outside of this list, the scanner will run in the region of your Azure Purview instance.

-### Azure Purview scanner regions

- > * Azure Purview scanner only supports schema extraction for the structured file types listed above.

- > * For AVRO, ORC, and PARQUET file types, Azure Purview scanner does not support schema extraction for files that contain complex data types (for example, MAP, LIST, STRUCT).

- > * Azure Purview scanner supports scanning snappy compressed PARQUET types for schema extraction and classification.

-In Azure Purview terminology,

-For all structured file formats, Azure Purview scanner samples files in the following way:

- - If a document file is larger than 20 MB, then it is not subject to a deep scan (subject to classification). In that case, Azure Purview captures only basic meta data like file name and fully qualified name.

-A folder or group of partition files is detected as a *resource set* in Azure Purview, if it matches with a system resource set policy or a customer defined resource set policy. If a resource set is detected, then Azure Purview will sample each folder that it contains. Learn more about resource sets [here](concept-resource-sets.md).

-All 206 system classification rules apply to structured file formats. Only the MCE classification rules apply to document file types (Not the data scan native regex patterns, bloom filter-based detection). For more information on supported classifications, see [Supported classifications in Azure Purview](supported-classifications.md).

-# View, edit and delete assets in Azure Purview catalog

-You can discover your assets in the Azure Purview data catalog by either:

-Even on edited assets, after a scan Azure Purview will reflect the truth of the source system. For example: if you edit a column and it's deleted from the source, it will be deleted from your asset in Azure Purview.

-> If you update the **name or data type of a column** in an Azure Purview asset, later scans **will not** update the asset schema. New columns and classifications **will not** be detected.

-Any asset you delete using the delete button is permanently deleted in Azure Purview. However, if you run a **full scan** on the source from which the asset was ingested into the catalog, then the asset is reingested and you can discover it using the Azure Purview catalog.

-If you have a scheduled scan (weekly or monthly) on the source, the **deleted asset will not get re-ingested** into the catalog unless the asset is modified by an end user since the previous run of the scan. For example, if a SQL table was deleted from Azure Purview, but after the table was deleted a user added a new column to the table in SQL, at the next scan the asset will be rescanned and ingested into the catalog.

-If you delete an asset, only that asset is deleted. Azure Purview does not currently support cascaded deletes. For example, if you delete a storage account asset in your catalog - the containers, folders and files within them are not deleted.

-description: This article describes steps how to configure Azure AD Conditional Access for Azure Purview

-# Customer intent: As an identity and security admin, I want to set up Azure Active Directory Conditional Access for Azure Purview, for secure access.

-# Conditional Access with Azure Purview

-[Azure Purview](./overview.md) supports Microsoft Conditional Access.

-The following steps show how to configure Azure Purview to enforce a Conditional Access policy.

-1. Select **Cloud apps**, select **Select apps**. You see all apps available for Conditional Access. Select **Azure Purview**, at the bottom select **Select**, and then select **Done**.

-description: This article provides an overview of the catalog lineage feature of Azure Purview.

-# Azure Purview Data Catalog lineage user guide

-This article provides an overview of the data lineage features in Azure Purview Data Catalog.

-One of the platform features of Azure Purview is the ability to show the lineage between datasets created by data processes. Systems like Data Factory, Data Share, and Power BI capture the lineage of data as it moves. Custom lineage reporting is also supported via Atlas hooks and REST API.

- Metadata collected in Azure Purview from enterprise data systems are stitched across to show an end to end data lineage. Data systems that collect lineage into Azure Purview are broadly categorized into following three types:

-Data integration and ETL tools can push lineage into Azure Purview at execution time. Tools such as Data Factory, Data Share, Synapse, Azure Databricks, and so on, belong to this category of data processing systems. The data processing systems reference datasets as source from different databases and storage solutions to create target datasets. The list of data processing systems currently integrated with Azure Purview for lineage are listed in below table.

-Databases & storage solutions such as Oracle, Teradata, and SAP have query engines to transform data using scripting language. Data lineage from views/stored procedures/etc are collected into Azure Purview and stitched with lineage from other systems. Lineage is supported for the following data sources via Azure Purview data scan. Learn more about the supported lineage scenarios from the respective article.

-Data analytics and reporting systems like Azure ML and Power BI report lineage into Azure Purview. These systems will use the datasets from storage systems and process through their meta model to create BI Dashboards, ML experiments and so on.

-Lineage in Azure Purview includes datasets and processes. Datasets are also referred to as nodes while processes can be also called edges:

-* **Dataset (Node)**: A dataset (structured or unstructured) provided as an input to a process. For example, a SQL Table, Azure blob, and files (such as .csv and .xml), are all considered datasets. In the lineage section of Azure Purview, datasets are represented by rectangular boxes.

-* **Process (Edge)**: An activity or transformation performed on a dataset is called a process. For example, ADF Copy activity, Data Share snapshot and so on. In the lineage section of Azure Purview, processes are represented by round-edged boxes.

-To access lineage information for an asset in Azure Purview, follow the steps:

-1. In the Azure portal, go to the [Azure Purview accounts page](https://aka.ms/purviewportal).

-1. Select your Azure Purview account from the list, and then select **Open Azure Purview Studio** from the **Overview** page.

-1. On the Azure Purview Studio **Home** page, search for a dataset name or the process name such as ADF Copy or Data Flow activity. And then press Enter.

-Azure Purview supports asset level lineage for the datasets and processes. To see the asset level lineage go to the **Lineage** tab of the current asset in the catalog. Select the current dataset asset node. By default the list of columns belonging to the data appears in the left pane.

-Data process can take one or more input datasets to produce one or more outputs. In Azure Purview, column level lineage is available for process nodes.

-description: This article describes Managed Virtual Network and managed private endpoints in Azure Purview.

-# Customer intent: As a Azure Purview admin, I want to set up Managed Virtual Network and managed private endpoints for my Azure Purview account.

-# Use a Managed VNet with your Azure Purview account

-> Azure Purview Managed Vnet, VNet Integration Runtime, and managed private endpoint connections are currently in PREVIEW. The [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-> Currently, Managed Virtual Network and managed private endpoints are available for Azure Purview accounts that are deployed in the following regions:

-This article describes how to configure Managed Virtual Network and managed private endpoints for Azure Purview.

-Currently, Managed Virtual Network and managed private endpoints are available for Azure Purview accounts that are deployed in the following regions:

-Currently, the following data sources are supported to have a managed private endpoint and can be scanned using Managed VNet Runtime in Azure Purview:

-> If you are planning to scan Azure Synapse workspaces using Managed Virtual Network, you are also required to [configure Azure Synapse workspace firewall access](register-scan-synapse-workspace.md#set-up-azure-synapse-workspace-firewall-access) to enable **Allow Azure services and resources to access this workspace**. Currently, we do not support setting up scans for an Azure Synapse workspace from Azure Purview Studio, if you cannot enable **Allow Azure services and resources to access this workspace** on your Azure Synapse workspaces. If you cannot enable the firewall:

-> - You can use [Azure Purview Rest API - Scans - Create Or Update](/rest/api/purview/scanningdataplane/scans/create-or-update/) to create a new scan for your Synapse workspaces including dedicated and serverless pools.

-A Managed Virtual Network in Azure Purview is a virtual network which is deployed and managed by Azure inside the same region as Azure Purview account to allow scanning Azure data sources inside a managed network, without having to deploy and manage any self-hosted integration runtime virtual machines by the customer in Azure.

-You can deploy an Azure Managed Integration Runtime within an Azure Purview Managed Virtual Network. From there, the Managed VNet Runtime will leverage private endpoints to securely connect to and scan supported data sources.

-> Currently, the Managed Virtual Network is only supported in the same region as Azure Purview account region.

-A Managed VNet is created for your Azure Purview account when you create a Managed VNet Runtime for the first time in your Azure Purview account. You can't view or manage the Managed VNets.

-Managed private endpoints are private endpoints created in the Azure Purview Managed Virtual Network establishing a private link to Azure Purview and Azure resources. Azure Purview manages these private endpoints on your behalf.

-Azure Purview supports private links. Private link enables you to access Azure (PaaS) services (such as Azure Storage, Azure Cosmos DB, Azure Synapse Analytics).

-> If an Azure PaaS data store (Blob, Azure Data Lake Storage Gen2, Azure Synapse Analytics) has a private endpoint already created against it, and even if it allows access from all networks, Azure Purview would only be able to access it using a managed private endpoint. If a private endpoint does not already exist, you must create one in such scenarios.

-A private endpoint connection is created in a "Pending" state when you create a managed private endpoint in Azure Purview. An approval workflow is initiated. The private link resource owner is responsible to approve or reject the connection.

-Before deploying a Managed VNet and Managed VNet Runtime for an Azure Purview account, ensure you meet the following prerequisites:

-1. An Azure Purview account deployed in one of the [supported regions](#supported-regions).

-2. From Azure Purview roles, you must be a data curator at root collection level in your Azure Purview account.

-3. From Azure RBAC roles, you must be contributor on the Azure Purview account and data source to approve private links.

-1. Go to the [Azure portal](https://portal.azure.com), and navigate to the **Azure Purview accounts** page and select your _Purview account_.

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-azure-portal.png" alt-text="Screenshot that shows the Azure Purview account":::

-2. **Open Azure Purview Studio** and navigate to the **Data Map --> Integration runtimes**.

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-vnet.png" alt-text="Screenshot that shows Azure Purview Data Map menus":::

-5. Deploying the Managed VNet Runtime for the first time triggers multiple workflows in Azure Purview Studio for creating managed private endpoints for Azure Purview and its Managed Storage Account. Click on each workflow to approve the private endpoint for the corresponding Azure resource.

-6. In Azure portal, from your Azure Purview account resource blade, approve the managed private endpoint. From Managed storage account blade approve the managed private endpoints for blob and queue

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-pe-purview.png" alt-text="Screenshot that shows how to approve a managed private endpoint for Azure Purview":::

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-pe-purview-approved.png" alt-text="Screenshot that shows how to approve a managed private endpoint for Azure Purview - approved":::

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-pe-list.png" alt-text="Screenshot that shows managed private endpoints in Azure Purview":::

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-pe-list-approved.png" alt-text="Screenshot that shows managed private endpoints in Azure Purview - approved ":::

-7. Inside Azure Purview Studio, the managed private endpoint must be shown as approved as well.

-It is important to register the data source in Azure Purview prior to setting up a scan for the data source. Follow these steps to register data source if you haven't yet registered it.

-1. Go to your Azure Purview account.

-For more information, see [Manage data sources in Azure Purview](manage-data-sources.md).

-You can use any of the following options to scan data sources using Azure Purview Managed VNet Runtime:

- - Account Key or SQL Authentication- Secrets can be created inside an Azure Key Vault to store credentials in order to enable access for Azure Purview to scan data sources securely using the secrets. A secret can be a storage account key, SQL login password, or a password.

-To scan a data source using a Managed VNet Runtime and Azure Purview managed identity perform these steps:

-1. Select the **Data Map** tab on the left pane in the Azure Purview Studio.

-You can also use other supported options to scan data sources using Azure Purview Managed Runtime. This requires setting up a private connection to Azure Key Vault where the secret is stored.

-1. [Grant Azure Purview access to your Azure Key Vault](manage-credentials.md#grant-azure-purview-access-to-your-azure-key-vault).

-2. [Create a new credential in Azure Purview](manage-credentials.md#create-a-new-credential).

- :::image type="content" source="media/catalog-managed-vnet/purview-managed-pe-key-vault-create.png" alt-text="Screenshot that shows how to create a managed private endpoint for Azure Key Vault in Azure Purview Studio":::

-9. Inside Azure Purview Studio, the managed private endpoint must be shown as approved as well.

-10. Select the **Data Map** tab on the left pane in the Azure Purview Studio.

-description: This article gives an overview permission, access control, and collections in Azure Purview. Role-based access control (RBAC) is managed within Azure Purview itself, so this guide will cover the basics to secure your information.

-# Access control in Azure Purview

-Azure Purview uses **Collections** to organize and manage access across its sources, assets, and other artifacts. This article describes collections and access management in your Azure Purview account.

-A collection is a tool Azure Purview uses to group assets, sources, and other artifacts into a hierarchy for discoverability and to manage access control. All accesses to Azure Purview's resources are managed from collections in the Azure Purview account itself.

-Azure Purview uses a set of predefined roles to control who can access what within the account. These roles are currently:

-> At this time, Azure Purview Policy author role is not sufficient to create policies. The Azure Purview Data source admin role is also required.

-|My application's Service Principal needs to push data to Azure Purview|Data curator|

-|I need to set up scans via the Azure Purview Studio|Data curator on the collection **or** data curator **and** data source administrator where the source is registered.|

-|I need to enable a Service Principal or group to set up and monitor scans in Azure Purview without allowing them to access the catalog's information |Data source administrator|

-|I need to put users into roles in Azure Purview | Collection administrator |

-|I need to create workflows for my Azure Purview account | Workflow administrator |

-## Understand how to use Azure Purview's roles and collections

-All access control is managed in Azure Purview's collections. Azure Purview's collections can be found in the [Azure Purview Studio](https://web.purview.azure.com/resource/). Open your Azure Purview account in the [Azure portal](https://portal.azure.com) and select the Azure Purview Studio tile on the Overview page. From there, navigate to the data map on the left menu, and then select the 'Collections' tab.

-When an Azure Purview account is created, it starts with a root collection that has the same name as the Azure Purview account itself. The creator of the Azure Purview account is automatically added as a Collection Admin, Data Source Admin, Data Curator, and Data Reader on this root collection, and can edit and manage this collection.

-Sources, assets, and objects can be added directly to this root collection, but so can other collections. Adding collections will give you more control over who has access to data across your Azure Purview account.

-All other users can only access information within the Azure Purview account if they, or a group they're in, are given one of the above roles. This means, when you create an Azure Purview account, no one but the creator can access or use its APIs until they're [added to one or more of the above roles in a collection](how-to-create-and-manage-collections.md#add-role-assignments).

-You can assign Azure Purview roles to users, security groups and service principals from your Azure Active Directory that is associated with your purview account's subscription.

-After creating an Azure Purview account, the first thing to do is create collections and assign users to roles within those collections.

-> If you created your Azure Purview account using a service principal, to be able to access the Azure Purview Studio and assign permissions to users, you will need to grant a user collection admin permissions on the root collection.

-> az purview account add-root-collection-admin --account-name [Azure Purview Account Name] --resource-group [Resource Group Name] --object-id [User Object Id]

-Collections can be customized for structure of the sources in your Azure Purview account, and can act like organized storage bins for these resources. When you're thinking about the collections you might need, consider how your users will access or discover information. Are your sources broken up by departments? Are there specialized groups within those departments that will only need to discover some assets? Are there some sources that should be discoverable by all your users?

-The [data reader role](#roles) can access information within the catalog, but not manage or edit it. So for our example above, adding the Data Reader permission to a group on the root collection and allowing inheritance will give all users in that group reader permissions on Azure Purview sources and assets. This makes these resources discoverable, but not editable, by everyone in that group. [Restricting inheritance](how-to-create-and-manage-collections.md#restrict-inheritance) on the Revenue group will control access to those assets. Users who need access to revenue information can be added separately to the Revenue collection.

-Role assignment is managed through the collections. Only a user with the [collection admin role](#roles) can grant permissions to other users on that collection. When new permissions need to be added, a collection admin will access the [Azure Purview Studio](https://web.purview.azure.com/resource/), navigate to data map, then the collections tab, and select the collection where a user needs to be added. From the Role Assignments tab they'll be able to add and manage users who need permissions.

-Now that you have a base understanding of collections, and access control, follow the guides below to create and manage those collections, or get started with registering sources into your Azure Purview Resource.

-description: This article describes how you can set up a private endpoint to connect to your Azure Purview account from restricted network.

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account for secure access.

-# Connect privately and securely to your Azure Purview account

-In this guide, you will learn how to deploy private endpoints for your Azure Purview account to allow you to connect to your Azure Purview account only from VNets and private networks. To achieve this goal, you need to deploy _account_ and _portal_ private endpoints for your Azure Purview account.

-The Azure Purview _account_ private endpoint is used to add another layer of security by enabling scenarios where only client calls that originate from within the virtual network are allowed to access the Azure Purview account. This private endpoint is also a prerequisite for the portal private endpoint.

-The Azure Purview _portal_ private endpoint is required to enable connectivity to [Azure Purview Studio](https://web.purview.azure.com/resource/) using a private network.

- :::image type="content" source="media/catalog-private-link/purview-private-link-account-portal.png" alt-text="Diagram that shows Azure Purview and Private Link architecture.":::

-Using one of the deployment options from this guide, you can deploy a new Azure Purview account with _account_ and _portal_ private endpoints or you can choose to deploy these private endpoints for an existing Azure Purview account:

-1. Choose an appropriate Azure virtual network and a subnet to deploy Azure Purview private endpoints. Select one of the following options:

-2. Define an appropriate [DNS name resolution method](./catalog-private-link-name-resolution.md#deployment-options), so Azure Purview account and web portal can be accessible through private IP addresses. You can use any of the following options:

-3. Deploy a [new Azure Purview account](#option-1deploy-a-new-azure-purview-account-with-account-and-portal-private-endpoints) with account and portal private endpoints, or deploy account and portal private endpoints for an [existing Azure Purview account](#option-2enable-account-and-portal-private-endpoint-on-existing-azure-purview-accounts).

-6. Validate your network and name resolution from management machine to Azure Purview.

-## Option 1 - Deploy a new Azure Purview account with _account_ and _portal_ private endpoints

-1. Go to the [Azure portal](https://portal.azure.com), and then go to the **Azure Purview accounts** page. Select **+ Create** to create a new Azure Purview account.

-3. Under **Account and portal** select **+ Add** to add a private endpoint for your Azure Purview account.

-4. On the **Create a private endpoint** page, for **Azure Purview sub-resource**, choose your location, provide a name for _account_ private endpoint and select **account**. Under **networking**, select your virtual network and subnet, and optionally, select **Integrate with private DNS zone** to create a new Azure Private DNS zone.

-6. In **Create Azure Purview account** wizard, select **+Add** again to add _portal_ private endpoint.

-7. On the **Create a private endpoint** page, for **Azure Purview sub-resource**,choose your location, provide a name for _portal_ private endpoint and select **portal**. Under **networking**, select your virtual network and subnet, and optionally, select **Integrate with private DNS zone** to create a new Azure Private DNS zone.

-## Option 2 - Enable _account_ and _portal_ private endpoint on existing Azure Purview accounts

-There are two ways you can add Azure Purview _account_ and _portal_ private endpoints for an existing Azure Purview account:

-### Use the Azure portal (Azure Purview account)

-1. Go to the [Azure portal](https://portal.azure.com), and then select your Azure Purview account, and under **Settings** select **Networking**, and then select **Private endpoint connections**.

-5. For **Resource**, select the Azure Purview account, and for **Target sub-resource**, select **account**.

-1. For **Resource**, select the already created Azure Purview account. For **Target sub-resource**, select **account**.

-> If your VM, VPN gateway, or VNet Peering gateway has public internet access, it can access the Azure Purview portal and the Azure Purview account enabled with private endpoints. For this reason, you don't have to follow the rest of the instructions. If your private network has network security group rules set to deny all public internet traffic, you'll need to add some rules to enable Azure Active Directory (Azure AD) access. Follow the instructions to do so.

-These instructions are provided for accessing Azure Purview securely from an Azure VM. Similar steps must be followed if you're using VPN or other VNet Peering gateways.

-1. After the new rule is created, go back to the VM and try to sign in by using your Azure AD credentials again. If sign-in succeeds, then the Azure Purview portal is ready to use. But in some cases, Azure AD redirects to other domains to sign in based on a customer's account type. For example, for a live.com account, Azure AD redirects to live.com to sign in, and then those requests are blocked again. For Microsoft employee accounts, Azure AD accesses msft.sts.microsoft.com for sign-in information.

-1. Now your Azure AD sign-in should be successful. The Azure Purview portal will load successfully, but listing all the Azure Purview accounts won't work because it can only access a specific Azure Purview account. Enter `web.purview.azure.com/resource/{PurviewAccountName}` to directly visit the Azure Purview account that you successfully set up a private endpoint for.

-description: This article describes how you can set up a private endpoint to connect to your Azure Purview account and scan data sources from restricted network for an end to end isolation

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account to access purview account and scan data sources from restricted network.

-# Connect to your Azure Purview and scan data sources privately and securely

-In this guide, you will learn how to deploy _account_, _portal_ and _ingestion_ private endpoints for your Azure Purview account to access purview account and scan data sources using a self-hosted integration runtime securely and privately, thereby enabling end-to-end network isolation.

-The Azure Purview _account_ private endpoint is used to add another layer of security by enabling scenarios where only client calls that originate from within the virtual network are allowed to access the Azure Purview account. This private endpoint is also a prerequisite for the portal private endpoint.

-The Azure Purview _portal_ private endpoint is required to enable connectivity to [Azure Purview Studio](https://web.purview.azure.com/resource/) using a private network.

-Azure Purview can scan data sources in Azure or an on-premises environment by using _ingestion_ private endpoints. Three private endpoint resources are required to be deployed and linked to Azure Purview managed resources when ingestion private endpoint is deployed:

- :::image type="content" source="media/catalog-private-link/purview-private-link-architecture.png" alt-text="Diagram that shows Azure Purview and Private Link architecture.":::

-Using one of the deployment options explained further in this guide, you can deploy a new Azure Purview account with _account_, _portal_ and _ingestion_ private endpoints or you can choose to deploy these private endpoints for an existing Azure Purview account:

-1. Choose an appropriate Azure virtual network and a subnet to deploy Azure Purview private endpoints. Select one of the following options:

-2. Define an appropriate [DNS name resolution method](./catalog-private-link-name-resolution.md#deployment-options), so you can access Azure Purview account and scan data sources using private network. You can use any of the following options:

-3. Deploy a [new Azure Purview account](#option-1deploy-a-new-azure-purview-account-with-account-portal-and-ingestion-private-endpoints) with account, portal and ingestion private endpoints, or deploy private endpoints for an [existing Azure Purview account](#option-2enable-account-portal-and-ingestion-private-endpoint-on-existing-azure-purview-accounts).

-5. Deploy and register [Self-hosted integration runtime](#deploy-self-hosted-integration-runtime-ir-and-scan-your-data-sources) inside the same VNet or a peered VNet where Azure Purview account and ingestion private endpoints are deployed.

-7. Validate your network and name resolution between management machine, self-hosted IR VM and data sources to Azure Purview.

-## Option 1 - Deploy a new Azure Purview account with _account_, _portal_ and _ingestion_ private endpoints

-1. Go to the [Azure portal](https://portal.azure.com), and then go to the **Azure Purview accounts** page. Select **+ Create** to create a new Azure Purview account.

-3. Under **Account and portal** select **+ Add** to add a private endpoint for your Azure Purview account.

-4. On the **Create a private endpoint** page, for **Azure Purview sub-resource**, choose your location, provide a name for _account_ private endpoint and select **account**. Under **networking**, select your virtual network and subnet, and optionally, select **Integrate with private DNS zone** to create a new Azure Private DNS zone.

-7. On the **Create a private endpoint** page, for **Azure Purview sub-resource**,choose your location, provide a name for _portal_ private endpoint and select **portal**. Under **networking**, select your virtual network and subnet, and optionally, select **Integrate with private DNS zone** to create a new Azure Private DNS zone.

- > It is important to select correct Azure Private DNS Zones to allow correct name resolution between Azure Purview and data sources. You can also use your existing Azure Private DNS Zones or create DNS records in your DNS Servers manually later. For more information, see [Configure DNS Name Resolution for private endpoints](./catalog-private-link-name-resolution.md).

-## Option 2 - Enable _account_, _portal_ and _ingestion_ private endpoint on existing Azure Purview accounts

-1. Go to the [Azure portal](https://portal.azure.com), and then select your Azure Purview account, and under **Settings** select **Networking**, and then select **Private endpoint connections**.

-5. For **Resource**, select the Azure Purview account, and for **Target sub-resource**, select **account**.

-9. From your Azure Purview account, under **Settings** select **Networking**, and then select **Ingestion private endpoint connections**.

-> If your VM, VPN gateway, or VNet Peering gateway has public internet access, it can access the Azure Purview portal and the Azure Purview account enabled with private endpoints. For this reason, you don't have to follow the rest of the instructions. If your private network has network security group rules set to deny all public internet traffic, you'll need to add some rules to enable Azure Active Directory (Azure AD) access. Follow the instructions to do so.

-These instructions are provided for accessing Azure Purview securely from an Azure VM. Similar steps must be followed if you're using VPN or other VNet Peering gateways.

-1. After the new rule is created, go back to the VM and try to sign in by using your Azure AD credentials again. If sign-in succeeds, then the Azure Purview portal is ready to use. But in some cases, Azure AD redirects to other domains to sign in based on a customer's account type. For example, for a live.com account, Azure AD redirects to live.com to sign in, and then those requests are blocked again. For Microsoft employee accounts, Azure AD accesses msft.sts.microsoft.com for sign-in information.

-1. Now your Azure AD sign-in should be successful. The Azure Purview portal will load successfully, but listing all the Azure Purview accounts won't work because it can only access a specific Azure Purview account. Enter `web.purview.azure.com/resource/{PurviewAccountName}` to directly visit the Azure Purview account that you successfully set up a private endpoint for.

-Once you deploy ingestion private endpoints for your Azure Purview, you need to setup and register at least one self-hosted integration runtime (IR):

-To cut off access to the Azure Purview account completely from the public internet, follow these steps. This setting applies to both private endpoint and ingestion private endpoint connections.

-1. Go to the Azure Purview account from the Azure portal, and under **Settings** > **Networking**, select **Private endpoint connections**.

-description: This article answers frequently asked questions about Azure Purview private endpoints.

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account for secure access.

-# FAQ about Azure Purview private endpoints

-This article answers common questions that customers and field teams often ask about Azure Purview network configurations by using [Azure Private Link](../private-link/private-link-overview.md). It's intended to clarify questions about Azure Purview firewall settings, private endpoints, DNS configuration, and related configurations.

-To set up Azure Purview by using Private Link, see [Use private endpoints for your Azure Purview account](./catalog-private-link.md).

-### What's the purpose of deploying the Azure Purview account private endpoint?

-The Azure Purview account private endpoint is used to add another layer of security by enabling scenarios where only client calls that originate from within the virtual network are allowed to access the account. This private endpoint is also a prerequisite for the portal private endpoint.

-### What's the purpose of deploying the Azure Purview portal private endpoint?

-The Azure Purview portal private endpoint provides private connectivity to Azure Purview Studio.

-### What's the purpose of deploying the Azure Purview ingestion private endpoints?

-Azure Purview can scan data sources in Azure or an on-premises environment by using ingestion private endpoints. Three other private endpoint resources are deployed and linked to Azure Purview managed resources when ingestion private endpoints are created:

-### Can I scan data through a public endpoint if a private endpoint is enabled on my Azure Purview account?

-Yes. Data sources that aren't connected through a private endpoint can be scanned by using a public endpoint while Azure Purview is configured to use a private endpoint.

-Yes. Data sources that aren't connected through a private endpoint can be scanned by using a service endpoint while Azure Purview is configured to use a private endpoint.

-### Can I access Azure Purview Studio from a public network if Public network access is set to Deny in Azure Purview account networking?

-No. Connecting to Azure Purview from a public endpoint where **Public network access** is set to **Deny** results in the following error message:

-"Not authorized to access this Azure Purview account. This Azure Purview account is behind a private endpoint. Please access the account from a client in the same virtual network (VNet) that has been configured for the Azure Purview account's private endpoint."

-In this case, to open Azure Purview Studio, either use a machine that's deployed in the same virtual network as the Azure Purview portal private endpoint or use a VM that's connected to your CorpNet in which hybrid connectivity is allowed.

-### Is it possible to restrict access to the Azure Purview managed storage account and event hub namespace (for private endpoint ingestion only) but keep portal access enabled for users across the web?

-No. When you set **Public network access** to **Deny**, access to the Azure Purview managed storage account and event hub namespace is automatically set for private endpoint ingestion only. When you set **Public network access** to **Allow**, access to the Azure Purview managed storage account and event hub namespace is automatically set for **All Networks**. You can't modify the private endpoint ingestion manually for the managed storage account or event hub namespace manually.

-No. As protected resources, access to the Azure Purview managed storage account and event hub namespace is restricted to Azure Purview only. These resources are deployed with a deny assignment to all principals, which prevents any applications, users, or groups from gaining access to them.

-### What private DNS zones are required for Azure Purview for a private endpoint?

-For Azure Purview _account_ and _portal_ private endpoints:

-For Azure Purview _ingestion_ private endpoints:

-### Do I have to use a dedicated virtual network and dedicated subnet when I deploy Azure Purview private endpoints?

-No. However, `PrivateEndpointNetworkPolicies` must be disabled in the destination subnet before you deploy the private endpoints. Consider deploying Azure Purview into a virtual network that has network connectivity to data source virtual networks through VNet Peering and access to an on-premises network if you plan to scan data sources cross-premises.

-### Can I deploy Azure Purview private endpoints and use existing private DNS zones in my subscription to register the A records?

-Yes. Your private endpoint DNS zones can be centralized in a hub or data management subscription for all internal DNS zones required for Azure Purview and all data source records. We recommend this method to allow Azure Purview to resolve data sources by using their private endpoint internal IP addresses.

-### What are the outbound ports and firewall requirements for virtual machines with self-hosted integration runtime for Azure Purview when you use a private endpoint?

-The VMs in which self-hosted integration runtime is deployed must have outbound access to Azure endpoints and an Azure Purview private IP address through port 443.

-No. However, it's expected that the virtual machine running self-hosted integration runtime can connect to your instance of Azure Purview through an internal IP address by using port 443. Use common troubleshooting tools for name resolution and connectivity testing, such as nslookup.exe and Test-NetConnection.

-### Why do I receive the following error message when I try to launch Azure Purview Studio from my machine?

-"This Azure Purview account is behind a private endpoint. Please access the account from a client in the same virtual network (VNet) that has been configured for the Azure Purview account's private endpoint."

-It's likely your Azure Purview account is deployed by using Private Link and public access is disabled on your Azure Purview account. As a result, you have to browse Azure Purview Studio from a virtual machine that has internal network connectivity to Azure Purview.

-1. Validate if you can resolve the following addresses through your Azure Purview account's private IP addresses.

-1. Verify network connectivity to your Azure Purview account by using the following PowerShell command:

-To set up Azure Purview by using Private Link, see [Use private endpoints for your Azure Purview account](./catalog-private-link.md).

-description: This article describes an overview of how you can use a private end point for your Azure Purview account

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account, for secure access.

-# Configure and verify DNS Name Resolution for Azure Purview private endpoints

-Accurate name resolution is a critical requirement when setting up private endpoints for your Azure Purview accounts.

-You may require enabling internal name resolution in your DNS settings to resolve the private endpoint IP addresses to the fully qualified domain name (FQDN) from data sources and your management machine to Azure Purview account and self-hosted integration runtime, depending on scenarios that you are deploying.

-The following example shows Azure Purview DNS name resolution from outside the virtual network or when an Azure private endpoint is not configured.

- :::image type="content" source="media/catalog-private-link/purview-name-resolution-external.png" alt-text="Screenshot that shows Azure Purview name resolution from outside CorpNet.":::

-The following example shows Azure Purview DNS name resolution from inside the virtual network.

- :::image type="content" source="media/catalog-private-link/purview-name-resolution-private-link.png" alt-text="Screenshot that shows Azure Purview name resolution from inside CorpNet.":::

-Use any of the following options to set up internal name resolution when using private endpoints for your Azure Purview account:

-To enable internal name resolution, you can deploy the required Azure DNS Zones inside your Azure subscription where Azure Purview account is deployed.

-When you create ingestion, portal and account private endpoints, the DNS CNAME resource records for Azure Purview is automatically updated to an alias in few subdomains with the prefix `privatelink`:

-The following table shows an example of Azure Private DNS zones and DNS A Records that are deployed as part of configuration of private endpoint for an Azure Purview account if you enable _Private DNS integration_ during the deployment:

-|Account |Azure Purview |`privatelink.purview.azure.com` |Contoso-Purview |

-|Portal |Azure Purview |`privatelink.purviewstudio.azure.com` |Web |

-|Ingestion |Azure Purview managed Storage Account - Blob |`privatelink.blob.core.windows.net` |scaneastusabcd1234 |

-|Ingestion |Azure Purview managed Storage Account - Queue |`privatelink.queue.core.windows.net` |scaneastusabcd1234 |

-|Ingestion |Azure Purview managed Storage Account - Event Hub |`privatelink.servicebus.windows.net` |atlas-12345678-1234-1234-abcd-123456789abc |

-When you resolve the Azure Purview endpoint URL from outside the virtual network with the private endpoint, it resolves to the public endpoint of Azure Purview. When resolved from the virtual network hosting the private endpoint, the Azure Purview endpoint URL resolves to the private endpoint's IP address.

-As an example, if an Azure Purview account name is 'Contoso-Purview', when it is resolved from outside the virtual network that hosts the private endpoint, it will be:

-| `Contoso-Purview.privatelink.purview.azure.com` | CNAME | \<Azure Purview public endpoint\> |

-| \<Azure Purview public endpoint\> | A | \<Azure Purview public IP address\> |

-| `Web.purview.azure.com` | CNAME | \<Azure Purview Studio public endpoint\> |

-| `Contoso-Purview.privatelink.purview.azure.com` | A | \<Azure Purview account private endpoint IP address\> |

-| `Web.purview.azure.com` | CNAME | \<Azure Purview portal private endpoint IP address\> |

-The following list shows the required Azure DNS zones and A records for Azure Purview private endpoints:

-> Update all names with `Contoso-Purview`,`scaneastusabcd1234` and `atlas-12345678-1234-1234-abcd-123456789abc` with corresponding Azure resources name in your environment. For example, instead of `scaneastusabcd1234` use the name of your Azure Purview managed storage account.

-|Account |Azure Purview |`privatelink.purview.azure.com` |Contoso-Purview |

-|Portal |Azure Purview |`privatelink.purviewstudio.azure.com` |Web |

-|Ingestion |Azure Purview managed Storage Account - Blob |`privatelink.blob.core.windows.net` |scaneastusabcd1234 |

-|Ingestion |Azure Purview managed Storage Account - Queue |`privatelink.queue.core.windows.net` |scaneastusabcd1234 |

-|Ingestion |Azure Purview managed Storage Account - Event Hub |`privatelink.servicebus.windows.net` |atlas-12345678-1234-1234-abcd-123456789abc |

- :::image type="content" source="media/catalog-private-link/purview-name-resolution-diagram.png" alt-text="Diagram that shows Azure Purview name resolution"lightbox="media/catalog-private-link/purview-name-resolution-diagram.png":::

-When you resolve the Azure Purview endpoint URL from outside the virtual network with the private endpoint, it resolves to the public endpoint of Azure Purview. When resolved from the virtual network hosting the private endpoint, the Azure Purview endpoint URL resolves to the private endpoint's IP address.

-As an example, if an Azure Purview account name is 'Contoso-Purview', when it is resolved from outside the virtual network that hosts the private endpoint, it will be:

-| `Contoso-Purview.privatelink.purview.azure.com` | CNAME | \<Azure Purview public endpoint\> |

-| \<Azure Purview public endpoint\> | A | \<Azure Purview public IP address\> |

-| `Web.purview.azure.com` | CNAME | \<Azure Purview Studio public endpoint\> |

-| `Contoso-Purview.privatelink.purview.azure.com` | A | \<Azure Purview account private endpoint IP address\> |

-| `Web.purview.azure.com` | CNAME | \<Azure Purview portal private endpoint IP address\> |

-> Update all names with `Contoso-Purview`,`scaneastusabcd1234` and `atlas-12345678-1234-1234-abcd-123456789abc` with corresponding Azure resources name in your environment. For example, instead of `scaneastusabcd1234` use the name of your Azure Purview managed storage account.

-| `web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview> |

-| `scaneastusabcd1234.blob.core.windows.net` | A | \<blob-ingestion private endpoint IP address of Azure Purview> |

-| `scaneastusabcd1234.queue.core.windows.net` | A | \<queue-ingestion private endpoint IP address of Azure Purview> |

-| `atlas-12345678-1234-1234-abcd-123456789abc.servicebus.windows.net`| A | \<namespace-ingestion private endpoint IP address of Azure Purview> |

-| `Contoso-Purview.Purview.azure.com` | A | \<account private endpoint IP address of Azure Purview> |

-| `Contoso-Purview.scan.Purview.azure.com` | A | \<account private endpoint IP address of Azure Purview> |

-| `Contoso-Purview.catalog.Purview.azure.com` | A | \<account private endpoint IP address of Azure Purview\> |

-| `Contoso-Purview.proxy.purview.azure.com` | A | \<account private endpoint IP address of Azure Purview\> |

-| `Contoso-Purview.guardian.purview.azure.com` | A | \<account private endpoint IP address of Azure Purview\> |

-| `gateway.purview.azure.com` | A | \<account private endpoint IP address of Azure Purview\> |

-| `manifest.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `cdn.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `hub.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `catalog.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `cseo.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `datascan.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `datashare.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `datasource.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `policy.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

-| `sensitivity.prod.ext.web.purview.azure.com` | A | \<portal private endpoint IP address of Azure Purview\> |

- |Account |Azure Purview |`privatelink.purview.azure.com` |Contoso-Purview |

- |Portal |Azure Purview |`privatelink.purviewstudio.azure.com` |Web |

- |Ingestion |Azure Purview managed Storage Account - Blob |`privatelink.blob.core.windows.net` |scaneastusabcd1234 |

- |Ingestion |Azure Purview managed Storage Account - Queue |`privatelink.queue.core.windows.net` |scaneastusabcd1234 |

- |Ingestion |Azure Purview managed Storage Account - Event Hub |`privatelink.servicebus.windows.net` |atlas-12345678-1234-1234-abcd-123456789abc |

-3. From your management PC and self-hosted integration runtime VM, test name resolution and network connectivity to your Azure Purview account using tools such as Nslookup.exe and PowerShell

-description: This article describes how to troubleshoot problems with your Azure Purview account related to private endpoints configurations

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account, for secure access.

-# Troubleshooting private endpoint configuration for Azure Purview accounts

-This guide summarizes known limitations related to using private endpoints for Azure Purview and provides a list of steps and solutions for troubleshooting some of the most common relevant issues.

-1. Once you deploy private endpoints for your Azure Purview account, review your Azure environment to make sure private endpoint resources are deployed successfully. Depending on your scenario, one or more of the following Azure private endpoints must be deployed in your Azure subscription:

- |Account |Azure Purview Account |mypurview-private-account |

- |Portal |Azure Purview Account |mypurview-private-portal |

-3. If portal private endpoint is deployed, and public network access is set to deny in your Azure Purview account, make sure you launch [Azure Purview Studio](https://web.purview.azure.com/resource/) from internal network.

-4. Test network connectivity and name resolution from management machine to Azure Purview endpoint and purview web url. If account and portal private endpoints are deployed, the endpoints must be resolved through private IP addresses.

-5. If you have created your Azure Purview account after 18 August 2021, make sure you download and install the latest version of self-hosted integration runtime from [Microsoft download center](https://www.microsoft.com/download/details.aspx?id=39717).

-6. From self-hosted integration runtime VM, test network connectivity and name resolution to Azure Purview endpoint.

-7. From self-hosted integration runtime, test network connectivity and name resolution to Azure Purview managed resources such as blob queue and Event Hub through port 443 and private IP addresses. (Replace the managed storage account and Event Hubs namespace with corresponding managed resource name assigned to your Azure Purview account).

-8. From the network where data source is located, test network connectivity and name resolution to Azure Purview endpoint and managed resources endpoints.

-9. If data sources are located in on-premises network, review your DNS forwarder configuration. Test name resolution from within the same network where data sources are located to self-hosted integration runtime, Azure Purview endpoints and managed resources. It is expected to obtain a valid private IP address from DNS query for each endpoint.

-11. If ingestion private endpoint is used, make sure self-hosted integration runtime is registered successfully inside Azure Purview account and shows as running both inside the self-hosted integration runtime VM and in the [Azure Purview Studio](https://web.purview.azure.com/resource/) .

-This can be an indication of issues related to connectivity or name resolution between the VM running self-hosted integration runtime and Azure Purview's managed resources storage account or Event Hub.

-Azure Purview account with private endpoint deployment failed with Azure Policy validation error during the deployment.

-> - Azure Purview (Microsoft.Purview/Accounts)

-Not authorized to access this Azure Purview account. This Azure Purview account is behind a private endpoint. Please access the account from a client in the same virtual network (VNet) that has been configured for the Azure Purview account's private endpoint.

-User is trying to connect to Azure Purview from a public endpoint or using Azure Purview public endpoints where **Public network access** is set to **Deny**.

-In this case, to open Azure Purview Studio, either use a machine that is deployed in the same virtual network as the Azure Purview portal private endpoint or use a VM that is connected to your CorpNet in which hybrid connectivity is allowed.

-description: This article describes a high level overview of how you can use a private end point for your Azure Purview account

-# Customer intent: As an Azure Purview admin, I want to set up private endpoints for my Azure Purview account, for secure access.

-# Use private endpoints for your Azure Purview account

-This article describes how to configure private endpoints for Azure Purview.

-You can use [Azure private endpoints](../private-link/private-endpoint-overview.md) for your Azure Purview accounts to allow users on a virtual network (VNet) to securely access the catalog over a Private Link. A private endpoint uses an IP address from the VNet address space for your Azure Purview account. Network traffic between the clients on the VNet and the Azure Purview account traverses over the VNet and a private link on the Microsoft backbone network.

-You can deploy Azure Purview _account_ private endpoint, to allow only client calls to Azure Purview that originate from within the private network.

-To connect to Azure Purview Studio using a private network connectivity, you can deploy _portal_ private endpoint.

-You can deploy _ingestion_ private endpoints if you need to scan Azure IaaS and PaaS data sources inside Azure virtual networks and on-premises data sources through a private connection. This method ensures network isolation for your metadata flowing from the data sources to Azure Purview Data Map.

-Before deploying private endpoints for Azure Purview account, ensure you meet the following prerequisites:

-## Azure Purview private endpoint deployment scenarios

-Use the following recommended checklist to perform deployment of Azure Purview account with private endpoints:

-|**Scenario 1** - [Connect to your Azure Purview and scan data sources privately and securely](./catalog-private-link-end-to-end.md) |You need to restrict access to your Azure Purview account only via a private endpoint, including access to Azure Purview Studio, Atlas APIs and scan data sources in on-premises and Azure behind a virtual network using self-hosted integration runtime ensuring end to end network isolation. (Deploy _account_, _portal_ and _ingestion_ private endpoints.) |

-|**Scenario 2** - [Connect privately and securely to your Azure Purview account](./catalog-private-link-account-portal.md) | You need to enable access to your Azure Purview account, including access to _Azure Purview Studio_ and Atlas API through private endpoints. (Deploy _account_ and _portal_ private endpoints). |

-|**Scenario 3** - [Scan data source securely using Managed Virtual Network](./catalog-managed-vnet.md) | You need to scan Azure data sources securely, without having to manage a virtual network or a self-hosted integration runtime VM. (Deploy managed private endpoint for Azure Purview, managed storage account and Azure data sources). |

-For scenarios where _ingestion_ private endpoint is used in your Azure Purview account, and public access on your data sources is disabled, Azure Purview can scan the following data sources that are behind a private endpoint:

-For FAQs related to private endpoint deployments in Azure Purview, see [FAQ about Azure Purview private endpoints](./catalog-private-link-faqs.md).

-For troubleshooting private endpoint configuration for Azure Purview accounts, see [Troubleshooting private endpoint configuration for Azure Purview accounts](./catalog-private-link-troubleshoot.md).

-To view list of current limitations related to Azure Purview private endpoints, see [Azure Purview private endpoints known limitations](./catalog-private-link-troubleshoot.md#known-limitations).

-description: This how-to guide describes how to view and use Azure Purview classification reporting on your data.

-# Customer intent: As a security officer, I need to understand how to use Azure Purview Insights to learn about sensitive data identified and classified and labeled during scanning.

-# Classification insights about your data from Azure Purview

-This how-to guide describes how to access, view, and filter Azure Purview Classification insight reports for your data.

-> Azure Purview Insights are currently in PREVIEW. The [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-> - Launch your Azure Purview account from Azure

-Before getting started with Azure Purview insights, make sure that you've completed the following steps:

-For more information, see [Manage data sources in Azure Purview](manage-data-sources.md).

-## Use Azure Purview classification insights

-In Azure Purview, classifications are similar to subject tags, and are used to mark and identify data of a specific type that's found within your data estate during scanning.

-Azure Purview uses the same sensitive information types as Microsoft 365, allowing you to stretch your existing security policies and protection across your entire data estate.

-1. Go to the **Azure Purview** [instance screen in the Azure portal](https://aka.ms/purviewportal) and select your Azure Purview account.

-1. On the **Overview** page, in the **Get Started** section, select the **Azure Purview Studio** tile.

-1. In Azure Purview, select the **Insights** :::image type="icon" source="media/insights/ico-insights.png" border="false"::: menu item on the left to access your **Insights** area.

-1. In the **Insights** :::image type="icon" source="media/insights/ico-insights.png" border="false"::: area, select **Classification** to display the Azure Purview **Classification insights** report.

-|**Browse assets** | To browse through the assets found with a specific classification or source, select a classification or source, depending on the report you're viewing, and then select **Browse assets** :::image type="icon" source="medi). |

-Learn more about Azure Purview insight reports

-description: Learn how Azure Purview prevents duplicate assets in your data map through asset normalization

-When ingesting assets into the Azure Purview data map, different sources updating the same data asset may send similar, but slightly different qualified names. While these qualified names represent the same asset, slight differences such as an extra character or different capitalization may cause these assets on the surface to appear different. To avoid storing duplicate entries and causing confusion when consuming the data catalog, Azure Purview applies normalization during ingestion to ensure all fully qualified names of the same entity type are in the same format.

-Below are the normalization rules applied by Azure Purview.

-[Scan in an Azure Blob Storage](register-scan-azure-blob-storage-source.md) account into the Azure Purview data map.

-description: This article provides examples of Azure Purview accounts architectures and describes best practices.

-# Azure Purview accounts architectures and best practices

-Azure Purview is a unified data governance solution. You deploy an Azure Purview account to centrally manage data governance across your data estate, spanning both cloud and on-prem environments. To use Azure Purview as your centralized data governance solution, you need to deploy one or more Azure Purview accounts inside your Azure subscription. We recommend keeping the number of Azure Purview instances as minimum, however, in some cases more Azure Purview instances are needed to fulfill business security and compliance requirements.

-## Single Azure Purview account

-Consider deploying minimum number of Azure Purview accounts for the entire organization. This approach takes maximum advantage of the "network effects" where the value of the platform increases exponentially as a function of the data that resides inside the platform.

-Use [Azure Purview collections hierarchy](./concept-best-practices-collections.md) to lay out your organization's data management structure inside a single Azure Purview account. In this scenario, one Azure Purview account is deployed in an Azure subscription. Data sources from one or more Azure subscriptions can be registered and scanned inside the Azure Purview. You can also register and scan data sources from your on-premises or multi-cloud environments.

-## Multiple Azure Purview accounts

-Some organizations may require setting up multiple Azure Purview accounts. Review the following scenarios as few examples when defining your Azure Purview accounts architecture:ΓÇ»

-It is recommended to create a new instance of Azure Purview account when testing scan configurations or classifications in isolated environments. For some scenarios, there is a "versioning" feature in some areas of the platform such as glossary, however, it would be easier to have a "disposable" instance of Azure Purview to freely test expected functionality and then plan to roll out the feature into the production instance.

-Additionally, consider using a test Azure Purview account when you cannot perform a rollback. For example, currently you cannot remove a glossary term attribute from an Azure Purview instance once it is added to your Azure Purview account. In this case, it is recommended using a test Azure Purview account first.

-Consider deploying separate instances of Azure Purview accounts for development, testing and production environments, specially when you have separate instances of data for each environment.

-In this scenario, production and non-production data sources can be registered and scanned inside their corresponding Azure Purview instances.

-Optionally, you can register a data source in more than one Azure Purview instance, if needed.

-When you scan data sources in Azure Purview, information related to your metadata is ingested and stored inside your Azure Purview Data Map in the Azure region where your Azure Purview account is deployed. Consider deploying separate instances of Azure Purview if you have specific regulatory and compliance requirements that include even having metadata in a specific geographical location.

-If your organization has data in multiple geographies and you must keep metadata in the same region as the actual data, you have to deploy multiple Azure Purview instances, one for each geography. In this case, data sources from each regions should be registered and scanned in the Azure Purview account that corresponds to the data source region or geography.

-Currently, Azure Purview doesn't support multi-tenancy. If you have Azure data sources distributed across multiple Azure subscriptions under different Azure Active Directory tenants, it is recommended deploying separate Azure Purview accounts under each tenant.

-An exception applies to VM-based data sources and Power BI tenants.For more information about how to scan and register a cross tenant Power BI in a single Azure Purview account, see, [Register and scan a cross-tenant Power BI](./register-scan-power-bi-tenant.md).

-Review [Azure Purview Pricing model](https://azure.microsoft.com/pricing/details/azure-purview) when defining budgeting model and designing Azure Purview architecture for your organization. One billing is generated for a single Azure Purview account in the subscription where Azure Purview account is deployed. This model also applies to other Azure Purview costs such as scanning and classifying metadata inside Azure Purview Data Map.

-Some organizations often have many business units (BUs) that operate separately, and, in some cases, they don't even share billing with each other. In those cases, the organization will end up creating an Azure Purview instance for each BU. This model is not ideal, however, may be necessary, especially because Business Units are often not willing to share Azure billing.

-description: This article provides process and best practice guidance to effectively manage the lifecycle of assets in the Azure Purview catalog

-As data and content has a lifecycle that requires active management (for example, acquisition - processing - disposal) assets in the Azure Purview data catalog need active management in a similar way. "Assets" in the catalog include the technical metadata that describes collection, lineage and scan information. Metadata describing the business structure of data such as glossary, classifications and ownership also needs to be managed.

-## Why do you need business processes for managing assets in Azure Purview?

-An organization employing Azure Purview should define processes and people structure to manage the lifecycle of assets and ensure data is valuable to users of the catalog. Metadata in the catalog must be maintained to be able to manage data at scale for discovery, quality, security and privacy.

-### Best practice processes that should be considered when starting the data governance journey with Azure Purview:

-The [Data Curator](catalog-permissions.md) role in Azure Purview controls read/write permission to assets within a collection group. To support the data governance processes, the Data Curator role has been granted to separate data governance personas in the organization:

-> The 4 **personas** listed are suggested read/write users, and would all be assigned Data Curator role in Azure Purview.

-This process describes the high-level steps and suggested roles to capture and maintain assets in the Azure Purview data catalog.

-| 1 | [Azure Purview collections architecture and best practices](concept-best-practices-collections.md) |

-| 3 & 4 | [Understand Azure Purview access and permissions](catalog-permissions.md)

-| 5 | [Azure Purview supported sources](purview-connector-overview.md) <br> [Azure Purview private endpoint networking](catalog-private-link.md) |

-| 7 | [Best practices for scanning data sources in Azure Purview](concept-best-practices-scanning.md)

-This process describes the high-level steps and roles to manage and define the business glossary and classifications metadata to enrich the Azure Purview data catalog.

-| 1 & 2 | [Understand Azure Purview access and permissions](catalog-permissions.md) |

-> It is not currently possible to edit glossary term attributes (for example, Status) in bulk using the Azure Purview UI, but it is possible to export the glossary in bulk, edit in Excel and re-import with amendments.

-This process describes the high-level steps and roles to move assets between collections using the Azure Purview portal.

-| 1 & 2 | [Azure Purview collections architecture and best practice](concept-best-practices-collections.md) |

-| 7 | [Browse the Azure Purview Catalog](how-to-browse-catalog.md)

-> It is not currently possible to bulk move assets from one collection to another using the Azure Purview portal.

-This process describes the high-level steps and roles to delete asset metadata from the data catalog using the Azure Purview portal.

-> - Deleting a collection, registered source or scan from Azure Purview does not delete all associated asset metadata.

-> - It is not possible to bulk delete asset metadata using the Azure Purview Portal

-description: This article provides an overview of Azure Purview automation tools and guidance on what to use when.

-# Azure Purview automation best practices

-While Azure Purview provides an out of the box user experience with Azure Purview Studio, not all tasks are suited to the point-and-click nature of the graphical user experience.

-Azure Purview provides several tools in which we can use to interact with the underlying platform, in an automated, and programmatic fashion. Because of the open nature of the Azure Purview service, we can automate different aspects, from the control plane, made accessible via Azure Resource Manager, to Azure Purview's multiple data planes (catalog, scanning, administration, and more).

-* Scenarios that require repeated Azure Purview deployments, templates ensure Azure Purview along with any other dependent resources are deployed in a consistent manner.

-* When coupled with [deployment scripts](../azure-resource-manager/templates/deployment-script-template.md), templated solutions can traverse the control and data planes, enabling the deployment of end-to-end solutions. For example, create an Azure Purview account, register sources, trigger scans.

-Azure CLI and Azure PowerShell are command-line tools that enable you to manage Azure resources such as Azure Purview. While the list of commands will grow over time, only a subset of Azure Purview control plane operations is currently available. For an up-to-date list of commands currently available, check out the documentation ([Azure CLI](/cli/azure/purview) | [Azure PowerShell](/powershell/module/az.purview)).

-* **Azure CLI** - A cross-platform tool that allows the execution of commands through a terminal using interactive command-line prompts or a script. Azure CLI has a **purview extension** that allows for the management of Azure Purview accounts. For example, `az purview account`.

-* **Azure PowerShell** - A cross-platform task automation program, consisting of a set of cmdlets for managing Azure resources. Azure PowerShell has a module called **Az.Purview** that allows for the management of Azure Purview accounts. For example, `Get-AzPurviewAccount`.

-REST APIs are HTTP endpoints that surface different methods (`POST`, `GET`, `PUT`, `DELETE`), triggering actions such as create, read, update, or delete (CRUD). Azure Purview exposes a large portion of the Azure Purview platform via multiple [service endpoints](/rest/api/purview/).

-Each Azure Purview account comes with an optional fully managed event hub, accessible via the Atlas Kafka endpoint found via the Azure portal > Azure Purview Account > Properties. Azure Purview events can be monitored by consuming messages from the event hub. External systems can also use the event hub to publish events to Azure Purview as they occur.

-* **Consume Events** - Azure Purview will send notifications about metadata changes to Kafka topic **ATLAS_ENTITIES**. Applications interested in metadata changes can monitor for these notifications. Supported operations include: `ENTITY_CREATE`, `ENTITY_UPDATE`, `ENTITY_DELETE`, `CLASSIFICATION_ADD`, `CLASSIFICATION_UPDATE`, `CLASSIFICATION_DELETE`.

-* **Publish Events** - Azure Purview can be notified of metadata changes via notifications to Kafka topic **ATLAS_HOOK**. Supported operations include: `ENTITY_CREATE_V2`, `ENTITY_PARTIAL_UPDATE_V2`, `ENTITY_FULL_UPDATE_V2`, `ENTITY_DELETE_V2`.

-Azure Purview can send platform logs and metrics via "Diagnostic settings" to one or more destinations (Log Analytics Workspace, Storage Account, or Azure Event Hubs). [Available metrics](./how-to-monitor-with-azure-monitor.md#available-metrics) include `Data Map Capacity Units`, `Data Map Storage Size`, `Scan Canceled`, `Scan Completed`, `Scan Failed`, and `Scan Time Taken`.

-Once configured, Azure Purview automatically sends these events to the destination as a JSON payload. From there, application subscribers that need to consume and act on these events can do so with the option of orchestrating downstream logic.

-Microsoft provides Azure SDKs to programmatically manage and interact with Azure services. Azure Purview client libraries are available in several languages (.NET, Java, JavaScript, and Python), designed to be consistent, approachable, and idiomatic.

-* [Azure Purview REST API](/rest/api/purview)

-description: This article provides best practices for classification in Azure Purview.

-# Azure Purview classification best practices

-Data classification, in the context of Azure Purview, is a way of categorizing data assets by assigning unique logical labels or classes to the data assets. Classification is based on the business context of the data. For example, you might classify assets by *Passport Number*, *Driver's License Number*, *Credit Card Number*, *SWIFT Code*, *PersonΓÇÖs Name*, and so on.

-* Azure Purview supports the following two methods for creating custom classification rules:

- * Azure Purview also provides a feature to generate a suggested regex pattern. After you upload a sample data file, select one of the suggested patterns, and then select **Add to patterns** to use the suggested data and column patterns. You can modify the suggested patterns, or you can type your own patterns without having to upload a file.

-* With Azure Purview, you can assign classifications at the asset or column level automatically by including relevant classifications in the scan rule, or you can assign them manually after you ingest the metadata into Azure Purview.

-* For automatic assignment, see [Supported data stores in Azure Purview](./azure-purview-connector-overview.md).

-* Before you scan your data sources in Azure Purview, it is important to understand your data and configure the appropriate scan rule set for it (for example, by selecting relevant system classification, custom classifications, or a combination of both), because it could affect your scan performance. For more information, see [Supported classifications in Azure Purview](./supported-classifications.md).

-* The Azure Purview scanner applies data sampling rules for deep scans (subject to classification) for both system and custom classifications. The sampling rule is based on the type of data sources. For more information, see the "Sampling within a file" section in [Supported data sources and file types in Azure Purview](./sources-and-scans.md#sampling-within-a-file).

-* The sampling rules apply to resource sets as well. For more information, see the "Resource set file sampling" section in [Supported data sources and file types in Azure Purview](./sources-and-scans.md#resource-set-file-sampling).

-* If you apply classifications manually from Azure Purview Studio, such classifications are retained in subsequent scans.

-* For *encrypted source* data assets, Azure Purview picks only file names, fully qualified names, schema details for structured file types, and database tables. For classification to work, decrypt the encrypted data before you run scans.

-description: This article provides examples of Azure Purview collections architectures and describes best practices.

-# Azure Purview collections architectures and best practices

-At the core of Azure Purview, the data map is a platform as a service (PaaS) component that keeps an up-to-date map of assets and their metadata across your data estate. To hydrate the data map, you need to register and scan your data sources. In an organization, there might be thousands of sources of data that are managed and governed by either centralized or decentralized teams.

-[Collections](./how-to-create-and-manage-collections.md) in Azure Purview support organizational mapping of metadata. By using collections, you can manage and maintain data sources, scans, and assets in a hierarchy instead of a flat structure. Collections allow you to build a custom hierarchical model of your data landscape based on how your organization plans to use Azure Purview to govern your landscape.

-A collection also provides a security boundary for your metadata in the data map. Access to collections, data sources, and metadata is set up and maintained based on the collections hierarchy in Azure Purview, following a least-privilege model:

-## Why do you need to define collections and an authorization model for your Azure Purview account?

-Consider deploying collections in Azure Purview to fulfill the following requirements:

-Azure Purview data-plane roles are managed in Azure Purview. After you deploy an Azure Purview account, the creator of the Azure Purview account is automatically assigned the following roles at the root collection. You can use [Azure Purview Studio](https://web.purview.azure.com/resource/) or a programmatic method to directly assign and manage roles in Azure Purview.

- - **Collection Admins** can edit Azure Purview collections and their details and add subcollections. They can also add users to other Azure Purview roles on collections where they're admins.

- > In certain scenarios, you might need to use an emergency account to sign in to Azure Purview. You might need this type of account to fix organization-level access problems when nobody else can sign in to Azure Purview or when other admins can't accomplish certain operations because of corporate authentication problems. We strongly recommended that you follow Microsoft best practices around implementing [emergency access accounts](/azure/active-directory/users-groups-roles/directory-emergency-access) by using cloud-only users.

- > Follow the instructions in [this article](./concept-account-upgrade.md#what-happens-when-your-upgraded-account-doesnt-have-a-collection-admin) to recover access to your Azure Purview root collection if your previous Collection Admin is unavailable.

-You can deploy your Azure Purview collection based on centralized, decentralized, or hybrid data management and governance models. Base this decision on your business and security requirements.

-In this scenario, each region has a subcollection of its own under the top-level collection in the Azure Purview account. Data sources are registered and scanned in the corresponding subcollections in their own geographic locations. So assets also appear in the subcollection hierarchy for the region.

-Each region has a subcollection of its own under the top-level collection in the Azure Purview account. Data sources are registered and scanned in the corresponding subcollections in their own geographic locations. So assets are added to the subcollection hierarchy for the region.

-You can apply a combination of these two scenarios in your Azure Purview data map if total data democratization is required with a few exceptions for some collections. You can assign Azure Purview roles at the top-level collection and restrict inheritance to the specific child collections.

-description: This article provides examples of Azure Purview glossary best practices.

-# Azure Purview glossary best practices

-This article is intended to provide guidance around how to establish and govern your organizations A to Z business glossary of commonly used terms and it is aimed to provide more guidance to ensure you are able to focus on promoting a shared understanding for business data governance ownership. Adopting these considerations and recommendations will help your organization achieve success with Azure Purview.

-Creating terms is necessary to build the business vocabulary and apply it to assets within Azure Purview. When a new Azure Purview account is created, by default, there are no built-in terms in the account.

-Glossary terms in Azure Purview are case sensitive and allow white space. The following shows a poorly executed example of implementing glossary terms and demonstrates the confusion caused:

-When building new term templates in Azure Purview, review the following considerations:

-Exporting terms may be useful in Azure Purview account to account, Backup, or Disaster Recovery scenarios. Exporting terms in Azure Purview Studio must be done one term template at a time. Choosing terms from multiple templates will disable the "Export terms" button. As a best practice, using the "Term template" filter before bulk selecting will make the export process quick.

- - Manually, using Azure Purview Studio.

- - Using Bulk Edit mode to update up to 25 assets, using Azure Purview Studio.

-description: This article provides best practices for data Lineage various data sources in Azure Purview.

-# Azure Purview Data Lineage best practices

-Data Lineage is broadly understood as the lifecycle that spans the dataΓÇÖs origin, and where it moves over time across the data estate. Azure Purview can capture lineage for data in different parts of your organization's data estate, and at different levels of preparation including:

-* Each Data Factory instance can connect to only one Azure Purview account. You can establish new connection in another Azure Purview account, but this will turn existing connection to disconnected.

-* Data factory's managed identity is used to authenticate lineage in Azure Purview account, the data factory's managed identity Data Curator role on Azure Purview root collection is required.

-* Azure Purview captures runtime lineage from the following Azure Data Factory activities:

-* Azure Purview drops lineage if the source or destination uses an unsupported data storage system.

-* Azure Purview cannot capture lineage if Azure Data Factory copy activity use copy activity features listed in **Limitations on copy activity lineage** of [Connect to Azure Data Factory](how-to-link-azure-data-factory.md)

-* For the lineage of Dataflow activity, Azure Purview only support source and sink. The lineage for Dataflow transformation is not supported yet.

-* Data flow lineage doesn't integrate with Azure Purview resource set.

- :::image type="content" source="./media/concept-best-practices-lineage/ssis-lineage.png" alt-text="Screenshot of the Execute SSIS lineage in Azure Purview." lightbox="./media/concept-best-practices-lineage/ssis-lineage.png":::

-* Please refer the following step-by-step guide to [push Azure Data Factory lineage in Azure Purview](../data-factory/tutorial-push-lineage-to-purview.md).

-# Azure Purview backup and recovery for migration best practices

-This article provides guidance on backup and recovery strategy when your organization has Azure Purview in production deployment. You can also use this general guideline to implement account migration. The scope of this article is to cover [manual BCDR methods](disaster-recovery.md) where you could automate using APIs. There is some key information to consider upfront:

-Most of enterprise organizations have critical requirement for Azure Purview for capabilities such as Backup, Business Continuity, and Disaster Recovery (BCDR). To get into more details of this requirement, you need to differentiate between Backup, High Availability (HA), and Disaster recovery (DR).

-You may need to create regular backups from an Azure Purview account and use a backup in case a piece of data or configuration is accidentally or deliberately deleted from the Azure Purview account by the users.

-The backup should allow saving a point in time copy of the following configurations from the Azure Purview account:

-In computing, the term availability is used to describe the period of time when a service is available, and the time required by a system to respond to a request made by a user. For Azure Purview, high availability means ensuring that Azure Purview instances are available in the case of a problem that is local to a data center or single region in the cloud region.

-> Information about Azure Purview availability is available at [SLA for Azure Purview](https://azure.microsoft.com/support/legal/sla/purview/v1_0/).

-> Azure Purview ensures no data loss but a recovery from outages may require you to restart your workflows such as scans.

-Organizations need a failover mechanism for their Azure Purview instances, so when the primary region experiences a catastrophic event like an earthquake or flood, the business must be prepared to have its systems come online elsewhere.

-> Not all Azure mirrored regions support deploying Azure Purview accounts. For example, For a DR scenario, you cannot choose to deploy a new Azure Purview account in Canada East if the primary region is Canada Central. Even with Customers managed DR, not all customer may able to trigger a DR.

-This section provides high level guidance on required tasks to copy assets, glossaries, classifications & relationships across regions or subscriptions either using the Azure Purview Studio or the REST APIs. The approach is to perform the tasks as programmatically as possible at scale.

-You will need to create a new Azure Purview account by following below instruction:

-* [Quickstart: Create an Azure Purview account in the Azure portal](create-catalog-portal.md)

-Below steps are referring to [Azure Purview API documentation](/rest/api/purview/) so that you can programmatically stand up the backup account quickly:

-|**ADF connections**|Currently an ADF can be connected to one Azure Purview at a time. You must disconnect ADF from failed Azure Purview account and reconnect it to the new account later.|

-* Azure Purview comes out with new features regularly so you can benefit from other features when running new scans.

-Running the scans is the most effective way to get all assets of data sources that Azure Purview is already supporting.

-1. Prepare the relationship payload so that there is no hard reference to old `guids` in the old Azure Purview accounts. You need to update those `guids` to the new account's `guids`.

-#### Using Azure Purview Portal

-The quickest way to migrate glossary terms is to [export terms to a .csv file](how-to-create-import-export-glossary.md). You can do this using the Azure Purview Studio.

-#### Using Azure Purview API

-description: This article provides examples of Azure Purview network architecture options and describes best practices.

-# Azure Purview network architecture and best practices

-Azure Purview is a platform as a service (PaaS) solution for data governance. Azure Purview accounts have public endpoints that are accessible through the internet to connect to the service. However, all endpoints are secured through Azure Active Directory (Azure AD) logins and role-based access control (RBAC).

-For an added layer of security, you can create private endpoints for your Azure Purview account. You then get a private IP address from your virtual network in Azure to the Azure Purview account and its managed resources. This address will restrict all traffic between your virtual network and the Azure Purview account to a private link for user interaction with the APIs and Azure Purview Studio, or for scanning and ingestion.

-Currently, the Azure Purview firewall provides access control for the public endpoint of your purview account. You can use the firewall to allow all access or to block all access through the public endpoint when using private endpoints.

-Based on your network, connectivity, and security requirements, you can set up and maintain Azure Purview accounts to access underlying services or ingestion. Use this best practices guide to define and prepare your network environment so you can access Azure Purview and scan data sources from various locations in your network or cloud.

-This guide describes a few of the most common network architecture scenarios for Azure Purview. Though you're not limited to those scenarios, keep in mind the [limitations](#current-limitations) of the service when you're planning networking for your Azure Purview accounts.

-By default, you can use Azure Purview accounts through public endpoints accessible through the internet. Allow public networks in your Azure Purview account if you have the following requirements:

-To scan data sources while the Azure Purview account firewall is set to allow public access, you can use both the Azure integration runtime and a [self-hosted integration runtime](./manage-integration-runtimes.md). How you use them depends on the [supportability of your data sources](manage-data-sources.md).

- :::image type="content" source="media/concept-best-practices/network-azure-runtime.png" alt-text="Screenshot that shows the connection flow between Azure Purview, the Azure runtime, and data sources."lightbox="media/concept-best-practices/network-azure-runtime.png":::

- 1. A manual or automatic scan is initiated from the Azure Purview data map through the Azure integration runtime.

- 3. Metadata is queued in Azure Purview managed storage and stored in Azure Blob Storage.

- 4. Metadata is sent to the Azure Purview data map.

- :::image type="content" source="media/concept-best-practices/network-self-hosted-runtime.png" alt-text="Screenshot that shows the connection flow between Azure Purview, a self-hosted runtime, and data sources."lightbox="media/concept-best-practices/network-self-hosted-runtime.png":::

- 2. The scan is initiated from the Azure Purview data map through a self-hosted integration runtime.

- 4. Metadata is processed in VM memory for the self-hosted integration runtime. Metadata is queued in Azure Purview managed storage and then stored in Azure Blob Storage.

- 5. Metadata is sent to the Azure Purview data map.

-When you're scanning a data source in Azure Purview, you need to provide a credential. Azure Purview can then read the metadata of the assets by using the Azure integration runtime in the destination data source. When you're using a public network, authentication options and requirements vary based on the following factors:

- If the data source is Azure Blob Storage, you can use an Azure Purview managed identity or a service principal in Azure Active Directory added as a Blob Storage Data Reader role on the Azure storage account. Or simply use the storage account's key.

- 1. Register the key vault inside Azure Purview.

- 1. Inside Azure Purview, create a new credential by using the secret saved in the key vault.

-You can use [Azure private endpoints](../private-link/private-endpoint-overview.md) for your Azure Purview accounts. This option is useful if you need to do either of the following:

-Similar to other PaaS solutions, Azure Purview does not support deploying directly into a virtual network. So you can't use certain networking features with the offering's resources, such as network security groups, route tables, or other network-dependent appliances such as Azure Firewall. Instead, you can use private endpoints that can be enabled on your virtual network. You can then disable public internet access to securely connect to Azure Purview.

-You must use private endpoints for your Azure Purview account if you have any of the following requirements:

-In this scenario, all Azure data sources, self-hosted integration runtime VMs, and Azure Purview private endpoints are deployed in the same virtual network in an Azure subscription.

-If on-premises data sources exist, connectivity is provided through a site-to-site VPN or Azure ExpressRoute connectivity to an Azure virtual network where Azure Purview private endpoints are deployed.

-In a hub-and-spoke architecture, you can deploy Azure Purview and one or more self-hosted integration runtime VMs in the hub subscription and virtual network. You can register and scan data sources from other virtual networks from multiple subscriptions in the same region.

-#### Name resolution for multiple Azure Purview accounts

-It is recommended to follow these recommendations, if your organization needs to deploy and maintain multiple Azure Purview accounts using private endpoints:

-1. Deploy at least one _account_ private endpoint for each Azure Purview account.

-2. Deploy at least one set of _ingestion_ private endpoints for each Azure Purview account.

-3. Deploy one _portal_ private endpoint for one of the Azure Purview accounts in your Azure environments. Create one DNS A record for _portal_ private endpoint to resolve `web.purview.azure.com`. The _portal_ private endpoint can be used by all purview accounts in the same Azure virtual network or virtual networks connected through VNet peering.

-This scenario also applies if multiple Azure Purview accounts are deployed across multiple subscriptions and multiple VNets that are connected through VNet peering. _Portal_ private endpoint mainly renders static assets related to Azure Purview Studio, thus, it is independent of Azure Purview account, therefore, only one _portal_ private endpoint is needed to visit all Azure Purview accounts in the Azure environment if VNets are connected.

-> You may need to deploy separate _portal_ private endpoints for each Azure Purview account in the scenarios where Azure Purview accounts are deployed in isolated network segmentations.

-> Azure Purview _portal_ is static contents for all customers without any customer information. Optionally, you can use public network, (without portal private endpoint) to launch `web.purview.azure.com` if your end users are allowed to launch the Internet.

-1. Use private endpoints for your Azure Purview account.

-1. Set **Public network access** to **allow** on your Azure Purview account.

- When you're using a private endpoint with Azure Purview, you need to allow network connectivity from data sources to a self-hosted integration VM on the Azure virtual network where Azure Purview private endpoints are deployed.

- - You can't use an Azure Purview managed identity. Instead, use a service principal, an account key, or SQL authentication, based on the data source type.

- - Make sure that your credentials are stored in an Azure key vault and registered inside Azure Purview.

- - You must create a credential in Azure Purview based on each secret that you create in Azure Key Vault. At minimum, assign _get_ and _list_ access for secrets for Azure Purview on the Key Vault resource in Azure. Otherwise, the credentials won't work in the Azure Purview account.

-description: This article provides best practices for registering and scanning various data sources in Azure Purview.

-# Azure Purview scanning best practices