+### Provider name

+You can optionally include the `ProviderName` attribute in the SAML authorization request. Set the metadata item as shown below to include the provider name for all requests to the external SAML IDP. The following example shows the `ProviderName` property set to `Contoso app`:

+```xml

+<Metadata>

+ ...

+ <Item Key="ProviderName">Contoso app</Item>

+ ...

+</Metadata>

+```

+The following example shows the `ProviderName` property in an authorization request:

+```xml

+<samlp:AuthnRequest AssertionConsumerServiceURL="https://..." ...

+ ProviderName="Contoso app">

+ ...

+</samlp:AuthnRequest>

+```

+1. Under **Password configuration**, select **Self-service password reset**.

+* You need global administrator privileges in your Azure AD tenant to enable secure LDAP.

+The Azure AD provisioning service includes a feature to help avoid accidental deletions. This feature ensures that users aren't disabled or deleted in an application unexpectedly.

+1. Sign in to the [Microsoft Endpoint Manager admin center](https://endpoint.microsoft.com).

+1. Sign in to the [Microsoft Endpoint Manager admin center](https://endpoint.microsoft.com).

+1. Browse to **Device** > **Windows** > **Configuration Profiles** > **Create profile**.

+ - Profile type: Template > Custom

+[Learn more about Azure AD Multi-Factor Authentication](../authentication/howto-mfa-getstarted.md)

+1. Sign in to the [Azure portal](https://portal.azure.com) as a Global administrator.

+1. Select **Per-user MFA**. You may need to scroll to the right to see this menu option. Select the example screenshot below to see the full Azure portal window and menu location:

+To understand why a user was prompted or not prompted to perform MFA, see [Azure AD Multi-Factor Authentication reports](howto-mfa-reporting.md).

+2. Go to **Azure Active Directory** > **User settings** > **Manage user feature settings**.

+* Months and weekdays with your company's local languages

+> [Enable risk-based Azure AD Multi-Factor Authentication](./tutorial-enable-azure-mfa.md)

+- An on-premises AD DS environment configured with Azure AD Connect cloud sync version 1.1.587 or later. Learn how to [identify the agent's current version](../cloud-sync/how-to-automatic-upgrade.md). If needed, configure Azure AD Connect cloud sync using [this tutorial](tutorial-enable-sspr.md).

+For public preview, you need to enable password writeback in Azure AD Connect cloud sync by using the Set-AADCloudSyncPasswordWritebackConfiguration cmdlet on the servers with the provisioning agents. You will need global administrator credentials:

+If you don't assign these permissions, writeback may appear to be configured correctly, but users encounter errors when they manage their on-premises passwords from the cloud. When setting "Unexpire Password" permissions in Active Directory, it must be applied to **This object and all descendant objects**, **This object only**, or **All descendant objects**, or the "Unexpire Password" permission can't be displayed.

+* To change the user consent settings in your organization, see [Configure how users consent to applications](../manage-apps/configure-user-consent.md).

+> Microsoft itself uses the default configuration allowing users to register applications and only allows user consent for a very limited set of permissions.

+[apps_service_principals_directory]:../media/active-directory-how-applications-are-added/HowAppsAreAddedToAAD.jpg

+ Title: Build a web app that authenticates users and calls web APIs | Azure

+description: Learn how to build a web app that authenticates users and calls web APIs (overview)

+#Customer intent: As an application developer, I want to know how to write a web app that authenticates users and calls web APIs by using the Microsoft identity platform.

+# Scenario: A web app that authenticates users and calls web APIs

+[App registration](scenario-web-app-call-api-app-registration.md).

+The Microsoft Graph beta endpoint (`https://graph.microsoft.com/beta`) exposes REST APIs to create, update, delete [federatedIdentityCredentials](/graph/api/resources/federatedidentitycredential?view=graph-rest-beta&preserve-view=true) on applications. Launch [Azure Cloud Shell](https://portal.azure.com/#cloudshell/) and sign in to your tenant.

+## Configure a federated identity credential

+Run the Microsoft Graph [create a new federated identity credential](/graph/api/application-post-federatedidentitycredentials?view=graph-rest-beta&preserve-view=true) operation on your app (specified by the object ID of the app).

+*issuer* and *subject* are the key pieces of information needed to set up the trust relationship. *issuer* is the URL of the external identity provider and must match the `issuer` claim of the external token being exchanged. *subject* is the identifier of the external software workload and must match the `sub` (`subject`) claim of the external token being exchanged. *subject* has no fixed format, as each IdP uses their own - sometimes a GUID, sometimes a colon delimited identifier, sometimes arbitrary strings. The combination of `issuer` and `subject` must be unique on the app. When the external software workload requests Microsoft identity platform to exchange the external token for an access token, the *issuer* and *subject* values of the federated identity credential are checked against the `issuer` and `subject` claims provided in the external token. If that validation check passes, Microsoft identity platform issues an access token to the external software workload.

+> [!IMPORTANT]

+> If you accidentally add the incorrect external workload information in the *subject* setting the federated identity credential is created successfully without error. The error does not become apparent until the token exchange fails.

+### GitHub Actions example

+### Kubernetes example

+Run the following command to configure a federated identity credential on an app and create a trust relationship with a Kubernetes service account. The *issuer* is your service account issuer URL. *subject* is the subject name in the tokens issued to the service account. Kubernetes uses the following format for subject names: `system:serviceaccount:<SERVICE_ACCOUNT_NAMESPACE>:<SERVICE_ACCOUNT_NAME>`.

+```azurecli

+az rest --method POST --uri 'https://graph.microsoft.com/beta/applications/f6475511-fd81-4965-a00e-41e7792b7b9c/federatedIdentityCredentials' --body '{"name":"Kubernetes-federated-credential","issuer":"https://aksoicwesteurope.blob.core.windows.net/9d80a3e1-2a87-46ea-ab16-e629589c541c/","subject":"system:serviceaccount:erp8asle:pod-identity-sa","description":"Kubernetes service account federated credential","audiences":["api://AzureADTokenExchange"]}'

+```

+And you get the response:

+```azurecli

+{

+ "@odata.context": "https://graph.microsoft.com/beta/$metadata#applications('f6475511-fd81-4965-a00e-41e7792b7b9c')/federatedIdentityCredentials/$entity",

+ "audiences": [

+ "api://AzureADTokenExchange"

+ ],

+ "description": "Kubernetes service account federated credential",

+ "id": "51ecf9c3-35fc-4519-a28a-8c27c6178bca",

+ "issuer": "https://aksoicwesteurope.blob.core.windows.net/9d80a3e1-2a87-46ea-ab16-e629589c541c/",

+ "name": "Kubernetes-federated-credential",

+ "subject": "system:serviceaccount:erp8asle:pod-identity-sa"

+}

+```

+## List federated identity credentials on an app

+## Delete a federated identity credential

+- To learn how to use workload identity federation for Kubernetes, see [Azure AD Workload Identity for Kubernetes](https://azure.github.io/azure-workload-identity/docs/quick-start.html) open source project.

+- To learn how to use workload identity federation for GitHub Actions, see [Configure a GitHub Actions workflow to get an access token](/azure/developer/github/connect-from-azure).

+- For more information, read about how Azure AD uses the [OAuth 2.0 client credentials grant](v2-oauth2-client-creds-grant-flow.md#third-case-access-token-request-with-a-federated-credential) and a client assertion issued by another IdP to get a token.

+You can use workload identity federation in scenarios such as GitHub Actions and workloads running on Kubernetes.

+- Workloads running on Kubernetes. Install the Azure AD workload identity webhook and establish a trust relationship between your app in Azure AD and a Kubernetes workload (described in the [Kubernetes quickstart](https://azure.github.io/azure-workload-identity/docs/quick-start.html)).

+Sometimes the invited external guest user's email may conflict with an existing [Contact object](/graph/api/resources/contact), resulting in the guest user being created without a proxyAddress. This is a known limitation that prevents guest users from redeeming an invitation through a direct link using [SAML/WS-Fed IdP](./direct-federation.md), [Microsoft Accounts](./microsoft-account.md), [Google Federation](./google-federation.md), or [Email One-Time Passcode](./one-time-passcode.md) accounts.

+However, the following scenarios should continue to work:

+- Redeeming an invitation through an invitation email redemption link using [SAML/WS-Fed IdP](./direct-federation.md), [Email One-Time Passcode](./one-time-passcode.md), and [Google Federation](./google-federation.md) accounts.

+- Signing back into an application after redemption using [SAML/WS-Fed IdP](./direct-federation.md) and [Google Federation](./google-federation.md) accounts.

+1. Delete the conflicting Contact object.

+2. Delete the guest user in the Azure portal (the user's "Invitation accepted" property should be in a pending state).

+3. Re-invite the guest user.

+4. Wait for the user to redeem invitation.

+5. Add the user's Contact email back into Exchange and any DLs they should be a part of.

+- [Leave an organization as a guest user](leave-the-organization.md)

+ :::image type="content" source="media/configure-admin-consent-workflow/enable-admin-consent-workflow.png" alt-text="Configure admin consent workflow settings":::

+1. Connect to [Azure AD PowerShell](/powershell/module/azuread/).

+For more information on setting up your web front end, see the tutorial [Configure your Azure AD to issue verifiable credentials](../verifiable-credentials/enable-your-tenant-verifiable-credentials.md).

+| `kubelet` | The Kubernetes agent that processes the orchestration requests from the control plane along with scheduling and running the requested containers. |

+ az aks show -g <resource-group> -n <cluster-name> --query addonProfiles.azureKeyvaultSecretsProvider.identity.clientId -o tsv

+description: Learn how to quickly create a Kubernetes cluster and deploy an application in Azure Kubernetes Service (AKS) using PowerShell.

+#Customer intent: As a developer or cluster operator, I want to quickly create an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+1. Create an AKS cluster using the [New-AzAksCluster][new-azakscluster] cmdlet.

+ kubectl get nodes

+ kubectl apply -f azure-vote.yaml

+kubectl get service azure-vote-front --watch

+- Verify *Microsoft.OperationsManagement* and *Microsoft.OperationalInsights* are registered on your subscription. To check the registration status:

+ ```azurecli

+ az provider show -n Microsoft.OperationsManagement -o table

+ az provider show -n Microsoft.OperationalInsights -o table

+ ```

+

+ If they are not registered, register *Microsoft.OperationsManagement* and *Microsoft.OperationalInsights* using:

+

+ ```azurecli

+ az provider register --namespace Microsoft.OperationsManagement

+ az provider register --namespace Microsoft.OperationalInsights

+ ```

+```

+Create an AKS cluster using the [az aks create][az-aks-create] command with the *--enable-addons monitoring* parameter to enable [Azure Monitor container insights][azure-monitor-containers]. The following example creates a cluster named *myAKSCluster* with one node:

+View the cluster nodes' and pods' health metrics captured by [Azure Monitor container insights][azure-monitor-containers] in the Azure portal.

+> If the AKS cluster was created with system-assigned managed identity (default identity option used in this quickstart), the identity is managed by the platform and does not require removal.

+> If the AKS cluster was created with service principal as the identity option instead, then when you delete the cluster, the service principal used by the AKS cluster is not removed. For steps on how to remove the service principal, see [AKS service principal considerations and deletion][sp-delete].

+Using the preceding Dockerfile, run the [az acr build][az-acr-build] command to build and push an image to the registry. The `.` at the end of the command provides the location of the source code directory path (in this case, the current directory). The `--file` parameter takes in the path of the Dockerfile relative to this source code directory path.

+> If the AKS cluster was created with system-assigned managed identity (default identity option used in this quickstart), the identity is managed by the platform and does not require removal.

+> If the AKS cluster was created with service principal as the identity option instead, then when you delete the cluster, the service principal used by the AKS cluster is not removed. For steps on how to remove the service principal, see [AKS service principal considerations and deletion][sp-delete].

+* When deploying into an [internal VNet](./api-management-using-with-internal-vnet.md), the instance requires an extra IP address for the internal load balancer.

+| \*.prod.warm.ingest.monitor.core.windows.net:443 |

+| \*.prod.warm.ingest.monitor.core.usgovcloudapi.net:443 |

+ Title: 'App Service outbound traffic control with Azure Firewall'

+description: Outbound traffic from App Service to internet, private IP addresses, and Azure services are routed through Azure Firewall. Learn how to control App Service outbound traffic by using Virtual Network integration and Azure Firewall.

+# Control outbound traffic with Azure Firewall

+This article shows you how to lock down the outbound traffic from your App Service app to back-end Azure resources or other network resources with [Azure Firewall](../firewall/overview.md). This configuration helps prevent data exfiltration or the risk of malicious program implantation.

+By default, an App Service app can make outbound requests to the public internet (for example, when installing required Node.js packages from NPM.org.). If your app is [integrated with an Azure virtual network](overview-vnet-integration.md), you can control outbound traffic with [network security groups](../virtual-network/network-security-groups-overview.md) to a limited extent, such as the target IP address, port, and protocol. Azure Firewall lets you control outbound traffic at a much more granular level and filter traffic based on real-time threat intelligence from Microsoft Cyber Security. You can centrally create, enforce, and log application and network connectivity policies across subscriptions and virtual networks (see [Azure Firewall features](../firewall/features.md)).

+For detailed network concepts and security enhancements in App Service, see [Networking features](networking-features.md) and [Zero to Hero with App Service, Part 6: Securing your web app](https://azure.github.io/AppService/2020/08/14/zero_to_hero_pt6.html).

+## Prerequisites

+* [Enable regional virtual network integration](configure-vnet-integration-enable.md) for your app.

+* [Verify that **Route All** is enabled](configure-vnet-integration-routing.md). This setting is enabled by default, which tells App Service to route all outbound traffic through the integrated virtual network. If you disable it, only traffic to private IP addresses will be routed through the virtual network.

+* If you want to route access to back-end Azure services through Azure Firewall as well, [disable all service endpoints](../virtual-network/virtual-network-service-endpoints-overview.md) on the App Service subnet in the integrated virtual network. After Azure Firewall is configured, outbound traffic to Azure Services will be routed through the firewall instead of the service endpoints.

+## 1. Create the required firewall subnet

+To deploy a firewall into the integrated virtual network, you need a subnet called **AzureFirewallSubnet**.

+1. In the [Azure portal](https://portal.azure.com), navigate to the virtual network that's integrated with your app.

+1. From the left navigation, select **Subnets** > **+ Subnet**.

+1. In **Name**, type **AzureFirewallSubnet**.

+1. **Subnet address range**, accept the default or specify a range that's [at least /26 in size](../firewall/firewall-faq.yml#why-does-azure-firewall-need-a--26-subnet-size).

+1. Select **Save**.

+## 2. Deploy the firewall and get its IP

+1. On the [Azure portal](https://portal.azure.com) menu or from the **Home** page, select **Create a resource**.

+1. Type *firewall* in the search box and press **Enter**.

+1. Select **Firewall** and then select **Create**.

+1. On the **Create a Firewall** page, configure the firewall as shown in the following table:

+ | Setting | Value |

+ | - | - |

+ | **Resource group** | Same resource group as the integrated virtual network. |

+ | **Name** | Name of your choice |

+ | **Region** | Same region as the integrated virtual network. |

+ | **Firewall policy** | Create one by selecting **Add new**. |

+ | **Virtual network** | Select the integrated virtual network. |

+ | **Public IP address** | Select an existing address or create one by selecting **Add new**. |

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/create-azfw.png" alt-text="Screenshot of creating an Azure Firewall in the Azure portal.":::

+1. Click **Review + create**.

+1. Select **Create** again.

+ This will take a few minutes to deploy.

+1. After deployment completes, go to your resource group, and select the firewall.

+1. In the firewall's **Overview** page, copy private IP address. The private IP address will be used as next hop address in the routing rule for the virtual network.

+

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/firewall-private-ip.png" alt-text="Screenshot of get Azure Firewall private IP address.":::

+## 3. Route all traffic to the firewall

+When you create a virtual network, Azure automatically creates a [default route table](../virtual-network/virtual-networks-udr-overview.md#default) for each of its subnets and adds system default routes to the table. In this step, you create a user-defined route table that routes all traffic to the firewall, and then associate it with the App Service subnet in the integrated virtual network.

+1. On the [Azure portal](https://portal.azure.com) menu, select **All services** or search for and select **All services** from any page.

+1. Under **Networking**, select **Route tables**.

+1. Select **Add**.

+1. Configure the route table like the following example:

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/create-route-table.png" alt-text="Screenshot of creating a routing route table in Azure portal.":::

+ Make sure you select the same region as the firewall you created.

+1. Select **Review + create**.

+1. Select **Create**.

+1. After deployment completes, select **Go to resource**.

+1. From the left navigation, select **Routes** > **Add**.

+1. Configure the new route as shown in the following table:

+ | Setting | Value |

+ | - | - |

+ | **Address prefix** | *0.0.0.0/0* |

+ | **Next hop type** | **Virtual appliance** |

+ | **Next hop address** | The private IP address for the firewall that you copied in [2. Deploy the firewall and get its IP](#2-deploy-the-firewall-and-get-its-ip). |

+1. From the left navigation, select **Subnets** > **Associate**.

+1. In **Virtual network**, select your integrated virtual network.

+1. In **Subnet**, select the App Service subnet.

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/associate-route-table.png" alt-text="Screenshot of associate the route table to the App Service subnet.":::

+1. Select **OK**.

+## 4. Configure firewall policies

+Outbound traffic from your app is now routed through the integrated virtual network to the firewall. To control App Service outbound traffic, add an application rule to firewall policy.

+1. Navigate to the firewall's overview page and select its firewall policy.

+1. In the firewall policy page, from the left navigation, select **Application Rules** > **Add a rule collection**.

+1. In **Rules**, add a network rule with the App Service subnet as the source address, and specify an FQDN destination. In the screenshot below, the destination FQDN is set to `api.my-ip.io`.

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/config-azfw-policy-app-rule.png" alt-text="Screenshot of configure Azure Firewall policy rule.":::

+ > [!NOTE]

+ > Instead of specifying the App Service subnet as the source address, you can also use the private IP address of the app in the subnet directly. You can find your app's private IP address in the subnet by using the [`WEBSITE_PRIVATE_IP` environment variable](reference-app-settings.md#networking).

+1. Select **Add**.

+## 5. Verify the outbound traffic

+An easy way to verify your configuration is to use the `curl` command from your app's SCM debug console to verify the outbound connection.

+1. In a browser, navigate to `https://<app-name>.scm.azurewebsites.net/DebugConsole`.

+1. In the console, run `curl -s <protocol>://<fqdn-address>` with a URL that matches the application rule you configured, To continue example in the previous screenshot, you can use **curl -s https://api.my-ip.io/api**. The following screenshot shows a successful response from the API, showing the public IP address of your App Service app.

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/verify-outbound-traffic-fw-allow-rule.png" alt-text="Screenshot of verifying the success outbound traffic by using curl command in SCM debug console.":::

+1. Run `curl -s <protocol>://<fqdn-address>` again with a URL that doesn't match the application rule you configured. In the following screenshot, you get no response, which indicates that your firewall has blocked the outbound request from the app.

+ :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/verify-outbound-traffic-fw-no-rule.png" alt-text="Screenshot of sending outbound traffic by using curl command in SCM debug console.":::

+> [!TIP]

+> Because these outbound requests are going through the firewall, you can capture them in the firewall logs by [enabling diagnostic logging for the firewall](../firewall/firewall-diagnostics.md#enable-diagnostic-logging-through-the-azure-portal) (enable the **AzureFirewallApplicationRule**).

+>

+> If you run the `curl` commands with diagnostic logs enabled, you can find them in the firewall logs.

+>

+> 1. In the Azure portal, navigate to your firewall.

+> 1. From the left navigation, select **Logs**.

+> 1. Close the welcome message by selecting **X**.

+> 1. From All Queries, select **Firewall Logs** > **Application rule log data**.

+> 1. Click **Run**. You can see these two access logs in query result.

+>

+> :::image type="content" source="./media/network-secure-outbound-traffic-azure-firewall/azfw-application-log-min.png" alt-text="Screenshot of SCM debug console to verify the failed outbound traffic by using curl command." lightbox="./media/network-secure-outbound-traffic-azure-firewall/azfw-application-log.png":::

+## More resources

+[Monitor Azure Firewall logs and metrics](../firewall/firewall-diagnostics.md).

+> [!NOTE]

+> If your Azure Storage account is secured by firewall rules, see [Networking considerations](#networking-considerations).

+> [!NOTE]

+> If your Azure Storage account is secured by firewall rules, see [Networking considerations](#networking-considerations).

+## Networking considerations

+If you secure your Azure Storage account by [only allowing selected networks](../storage/common/storage-network-security.md#change-the-default-network-access-rule), it can receive logs from App Service only if both of the following are true:

+- The Azure Storage account is in a different Azure region from the App Service app.

+- All outbound addresses of the App Service app are [added to the Storage account's firewall rules](../storage/common/storage-network-security.md#managing-ip-network-rules). To find the outbound addresses for your app, see [Find outbound IPs](overview-inbound-outbound-ips.md#find-outbound-ips).

+* Agent-based hybrid worker uses MMA proxy setting. You have to pass the proxy setting while installing the log analytics extension(MMA) and this setting will be stored under MMA configuration(registry) on VM.

+> [!NOTE]

+> You can set up the proxy settings either by PowerShell cmdlets or API.

+**Proxy server settings**

+# [Windows](#tab/windows)

+

+```azurepowershell

+$settings = @{

+ "AutomationAccountURL" = "<registrationurl>/<subscription-id>";

+ "ProxySettings" = @{

+ "ProxyServer" = "<ipaddress>:<port>";

+ "UserName"="test";

+ }

+};

+$protectedsettings = @{

+"ProxyPassword" = "password";

+};

+```

+# [Linux](#tab/linux)

+```

+$protectedsettings = @{

+ "Proxy_URL"="http://username:password@<IP Address>"

+};

+$settings = @{

+ "AutomationAccountURL" = "<registration-url>/<subscription-id>";

+};

+```

+This article covers how to unregister a node managed by Automation State Configuration, and safely remove a PowerShell Desired State Configuration (DSC) configuration from managed nodes. For both Windows and Linux nodes, you need to [unregister the node](#unregister-a-node) and [Delete a configuration from the node](#delete-a-configuration-from-the-node). For Linux nodes only, you can optionally delete the DSC packages from the nodes as well. See [Remove the DSC package from a Linux node](#remove-the-dsc-package-from-a-linux-node).

+>[!NOTE]

+> Unregistering a node from the service only sets the Local Configuration Manager settings so the node is no longer connecting to the service. This does not effect the configuration that's currently applied to the node, and leaves the related files in place on the node. After you unregister/delete the node, to re-register it, clear the existing configuration files. See [Delete a configuration from the node](#delete-a-configuration-from-the-node).

+If you no longer want a node to be managed by State Configuration (DSC), you can unregister it from the Azure portal or with Azure PowerShell using the following steps.

+ # [Azure portal](#tab/azureportal)

+# [Azure PowerShell](#tab/powershell)

+> If your organization still uses the deprecated AzureRM modules, you can use [Unregister-AzureRmAutomationDscNode](/powershell/module/azurerm.automation/unregister-azurermautomationdscnode).

+## Delete a configuration from the node

+When you're ready to remove an imported DSC configuration document (which is a Managed Object Format (MOF) or .mof file) that's assigned to one or more nodes, follow either of these steps.

+# [Azure portal](#tab/delete-azureportal)

+# [Manual Deletion](#tab/manual-delete-azureportal)

+To manually delete the .mof configuration files, follow the steps:

+**Delete a Windows configuration using PowerShell**

+**Delete a Linux configuration**

+## Re-register a node

+You can re-register a node just as you registered the node initially, using any of the methods described in [Enable Azure Automation State Configuration](/azure/automation/automation-dsc-onboarding)

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+> [!IMPORTANT]

+> In the interest of ensuring new features are documented no later than their release, this page may include documentation for features that may not yet be publicly available.

+If your client library or tool doesn't support TLS, then enabling unencrypted connections is possible through the [Azure portal](cache-configure.md#access-ports) or [management APIs](/rest/api/redis/2021-06-01/redis/update). In cases where encrypted connections aren't possible, we recommend placing your cache and client application into a virtual network. For more information about which ports are used in the virtual network cache scenario, see this [table](cache-how-to-premium-vnet.md#outbound-port-requirements).

+ az functionapp create --consumption-plan-location westeurope --runtime python --runtime-version 3.8 --functions-version 3 --name <APP_NAME> --os-type linux --storage-account <STORAGE_NAME>

+* `POCO` - The message is formatted as a C# object. For complete code, see C# [example](#example).

+The following example shows a [C# function](functions-dotnet-class-library.md) that is executed each time the minutes have a value divisible by five (eg if the function starts at 18:55:00, the next performance will be at 19:00:00). The [`TimerInfo`](https://github.com/Azure/azure-webjobs-sdk-extensions/blob/master/src/WebJobs.Extensions/Extensions/Timers/TimerInfo.cs) object is passed into the function.

+|[Computer Professionals International](https://cb20.com/)|

+|The Azure Percept DK Wi-Fi access point passphrase/password doesn't work| We have heard reports that some welcome cards may have incorrect passphrase/password printed.|In order to retrieve the Wi-Fi SoftAP password of your Percept Devkit, you must connect and use an Ethernet cable. Once the cable is attached and the device powered on, youΓÇÖll need to find the IP address that was assigned to your devkit. In ΓÇ£HomeΓÇ¥ situations you may be able to log in to your home router to get this info. Look for an ASUS device named ΓÇ£apdk-xxxxxxxΓÇ¥. The article [Connect to Azure Percept DK over Ethernet](./how-to-connect-over-ethernet.md) can guide you if youΓÇÖre not able to get the IP from the router. Once you have the EthernetΓÇÖs IP, start a web browser and manually copy and paste this address: IE: http://192.168.0.222 to go to the Onboarding experience. <ul><li>DonΓÇÖt go through the full setup just yet.</li><li>Setup Wi-Fi and create your SSH User and pause there (you can leave that window open and complete setup after we get the SoftAP password).</li><li>Open Putty or an SSH client and connect to the devkit using the user/pw you just created.</li><li>**Run: sudo tpm2_handle2psk 0x81000009.** The output from this command will be your password for the SoftAP. ΓÇô Please write it down on the card ΓÇô</li></ul>

+There are two ways to authenticate and authorize access to Azure Relay resources: Azure Active Directory (Azure AD) and Shared Access Signatures (SAS). This article gives you details on using these two types of security mechanisms.

+[0]: ./media/relay-authentication-and-authorization/hcanon.png

+The size of a template spec is limited to approximated 2 MB. If a template spec size exceeds the limit, you will get the **TemplateSpecTooLarge** error code. The error message says:

+```error

+The size of the template spec content exceeds the maximum limit. For large template specs with many artifacts, the recommended course of action is to split it into multiple template specs and reference them modularly via TemplateLinks.

+```

+The size of a template spec is limited to approximated 2 MB. If a template spec size exceeds the limit, you will get the **TemplateSpecTooLarge** error code. The error message says:

+```error

+The size of the template spec content exceeds the maximum limit. For large template specs with many artifacts, the recommended course of action is to split it into multiple template specs and reference them modularly via TemplateLinks.

+```

+| RegionDoesNotAllowProvisioning | Select a different region or submit a quota support request for **Region access**. | |

+ Title: List of pipeline topologies

+description: This article lists validated sample pipeline topologies for Azure Video Analyzer.

+# List of pipeline topologies

+The following tables list validated sample Azure Video Analyzer [live pipeline topologies](terminology.md#pipeline-topology). These topologies can be further customized according to solution needs. The tables also provide

+* A short description,

+* Topology's corresponding sample tutorial(s), and

+* The corresponding pipeline topology name of the Visual Studio Code (VSCode) [Video Analyzer extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.azure-video-analyzer).

+Clicking on a topology name redirects to the corresponding JSON file located in [this GitHub folder](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/), clicking on a sample redirects to the corresponding sample document, and clicking on a VSCode name redirects to a screenshot of the sample topology.

+## Live pipeline topologies

+### Continuous video recording

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[cvr-video-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/cvr-video-sink/topology.json) | Perform continuous video recording (CVR). Capture video and continuously record it to an Azure Video Analyzer video. | [Continuous video recording and playback](edge/use-continuous-video-recording.md) | [Record to Video Analyzer video](./visual-studio-code-extension.md#record-to-video-analyzer-video)

+[cvr-with-grpcExtension](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/cvr-with-grpcExtension/topology.json) | Perform CVR. A subset of the video frames is sent to an external AI inference engine using the sharedMemory mode for data transfer via the gRPC extension. The results are then published to the IoT Edge Hub. | | [Record using gRPC Extension](./visual-studio-code-extension.md#record-using-grpc-extension)

+[cvr-with-httpExtension](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/cvr-with-httpExtension/topology.json) | Perform CVR. A subset of the video frames is sent to an external AI inference engine via the HTTP extension. The results are then published to the IoT Edge Hub. | | [Record using HTTP Extension](./visual-studio-code-extension.md#record-using-http-extension)

+[cvr-with-httpExtension-and-objectTracking](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/cvr-with-httpExtension-and-objectTracking/topology.json) | Perform CVR and track objects in a live feed. Inference metadata from an external AI inference engine is published to the IoT Edge Hub, and can be played back with the video. | [Record and stream inference metadata with video](edge/record-stream-inference-data-with-video.md)

+[cvr-with-motion](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/cvr-with-motion/topology.json) | Perform CVR. When motion is detected from a live video feed, relevant inferencing events are published to the IoT Edge Hub. | | [Record on motion detection](./visual-studio-code-extension.md#record-on-motion-detection)

+[audio-video](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/audio-video/topology.json) | Perform CVR and record audio using the outputSelectors property. | | [Record audio with video](./visual-studio-code-extension.md#record-audio-with-video)

+### Event-based video recording

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[evr-grpcExtension-video-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-grpcExtension-video-sink/topology.json) | When an event of interest is detected by the external AI inference engine via the gRPC extension, those events are published to the IoT Edge Hub. The events are used to trigger the signal gate processor node that results in the appending of new clips to the Azure Video Analyzer video, corresponding to when the event of interest was detected. | [Develop and deploy gRPC inference server](edge/develop-deploy-grpc-inference-srv.md) | [Record using gRPC Extension](./visual-studio-code-extension.md#record-using-grpc-extension-1)

+[evr-httpExtension-video-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-httpExtension-video-sink/topology.json) | When an event of interest is detected by the external AI inference engine via the HTTP extension, those events are published to the IoT Edge Hub. The events are used to trigger the signal gate processor node that results in the appending of new clips to the Azure Video Analyzer video, corresponding to when the event of interest was detected. | | [Record using HTTP Extension](./visual-studio-code-extension.md#record-using-http-extension-1)

+[evr-hubMessage-video-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-hubMessage-video-sink/topology.json) | Use an object detection AI model to look for objects in the video, and record video clips only when a certain type of object is detected. The trigger to generate these clips is based on the AI inference events published onto the IoT Hub. | [Event-based video recording and playback](edge/record-event-based-live-video.md)| [Record to Video Analyzer video based on inference events](./visual-studio-code-extension.md#record-to-video-analyzer-video-based-on-inference-events)

+[evr-hubMessage-file-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-hubMessage-file-sink/topology.json) | Record video clips to the local file system of the edge device whenever an external sensor sends a message to the pipeline topology. For example, the sensor can be a door sensor. | | [Record to local files based on inference events](./visual-studio-code-extension.md#record-to-local-files-based-on-inference-events)

+[evr-motion-video-sink-file-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-motion-video-sink-file-sink/topology.json) | Perform event-based recording of video clips to the cloud and to the edge. When motion is detected from a live video feed, events are sent to a signal gate processor node that opens, allowing video to pass through to a file sink node and a video sink node. As a result, new files are created on the local file system of the Edge device, and new video clips are appended to your Video Analyzer video. The recordings contain the frames where motion was detected. | | [Record motion events to Video Analyzer video and local files](./visual-studio-code-extension.md#record-motion-events-to-video-analyzer-video-and-local-files)

+[evr-motion-video-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-motion-video-sink/topology.json) | When motion is detected, those events are published to the IoT Edge Hub. In addition, the motion events are used to trigger the signal gate processor node that will send frames to the video sink node when motion is detected. As a result, new video clips are appended to the Azure Video Analyzer video, corresponding to when motion was detected. | [Detect motion, record video to Video Analyzer](edge/detect-motion-record-video-clips-cloud.md) | [Record motion events to Video Analyzer video](./visual-studio-code-extension.md#record-motion-events-to-video-analyzer-video)

+[evr-motion-file-sink](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/evr-motion-file-sink/topology.json) | When motion is detected from a live video feed, events are sent to a signal gate processor node that opens, sending frames to a file sink node. As a result, new files are created on the local file system of the edge device, containing the frames where motion was detected. | [Detect motion and record video on edge devices](edge/detect-motion-record-video-edge-devices.md) | [Record motion events to local files](./visual-studio-code-extension.md#record-motion-events-to-local-files)

+### Motion detection

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[motion-detection](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/motion-detection/topology.json) | Detect motion in a live video feed. When motion is detected, those events are published to the IoT Hub. | [Get started with Azure Video Analyzer](edge/get-started-detect-motion-emit-events.md), [Get started with Video Analyzer in the portal](edge/get-started-detect-motion-emit-events-portal.md), [Detect motion and emit events](edge/detect-motion-emit-events-quickstart.md) | [Publish motion events to IoT Hub](./visual-studio-code-extension.md#publish-motion-events-to-iot-hub)

+[motion-with-grpcExtension](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/motion-with-grpcExtension/topology.json) | Perform event-based recording in the presence of motion. When motion is detected from a live video feed, those events are published to the IoT Edge Hub. In addition, the motion events are used to trigger a signal gate processor node that will send frames to a video sink node only when motion is present. As a result, new video clips are appended to the Azure Video Analyzer video, corresponding to when motion was detected. Additionally, run video analytics only when motion is detected. Upon detecting motion, a subset of the video frames is sent to an external AI inference engine via the gRPC extension. The results are then published to the IoT Edge Hub. | [Analyze live video with your own model - gRPC](edge/analyze-live-video-use-your-model-grpc.md) | [Analyzer motion events using gRPC Extension](./visual-studio-code-extension.md#analyze-motion-events-using-grpc-extension)

+[motion-with-httpExtension](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/motion-with-httpExtension/topology.json) | Perform event-based recording in the presence of motion. When motion is detected in a live video feed, those events are published to the IoT Edge Hub. In addition, the motion events are used to trigger a signal gate processor node that will send frames to a video sink node only when motion is present. As a result, new video clips are appended to the Azure Video Analyzer video, corresponding to when motion was detected. Additionally, run video analytics only when motion is detected. Upon detecting motion, a subset of the video frames is sent to an external AI inference engine via the HTTP extension. The results are then published to the IoT Edge Hub. | [Analyze live video with your own model - HTTP](edge/analyze-live-video-use-your-model-http.md#generate-and-deploy-the-iot-edge-deployment-manifest) | [Analyze motion events using HTTP Extension](./visual-studio-code-extension.md#analyze-motion-events-using-http-extension)

+### Extensions

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[grpcExtensionOpenVINO](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/grpcExtensionOpenVINO/topology.json) | Run video analytics on a live video feed. The gRPC extension allows you to create images at video frame rate from the camera that are converted to images, and sent to the [OpenVINO™ DL Streamer - Edge AI Extension module](https://aka.ms/ava-intel-ovms) provided by Intel. The results are then published to the IoT Edge Hub. | [Analyze live video with Intel OpenVINO™ DL Streamer – Edge AI Extension](edge/use-intel-grpc-video-analytics-serving-tutorial.md)

+[httpExtension](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/httpExtension/topology.json) | Run video analytics on a live video feed. A subset of the video frames from the camera are converted to images, and sent to an external AI inference engine. The results are then published to the IoT Edge Hub. | [Analyze live video with your own model - HTTP](edge/analyze-live-video-use-your-model-http.md), [Analyze live video with Azure Video Analyzer on IoT Edge and Azure Custom Vision](edge/analyze-live-video-custom-vision.md) | [Analyze video using HTTP Extension](./visual-studio-code-extension.md#analyze-video-using-http-extension)

+[httpExtensionOpenVINO](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/httpExtensionOpenVINO/topology.json) | Run video analytics on a live video feed. A subset of the video frames from the camera are converted to images, and sent to the [OpenVINO™ Model Server – AI Extension module](https://aka.ms/ava-intel-ovms) provided by Intel. The results are then published to the IoT Edge Hub. | [Analyze live video using OpenVINO™ Model Server – AI Extension from Intel](https://aka.ms/ava-intel-ovms-tutorial) | [Analyze video with Intel OpenVINO Model Server](./visual-studio-code-extension.md#analyze-video-with-intel-openvino-model-server)

+### Computer vision

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[spatial-analysis/person-count-operation-topology](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/spatial-analysis/person-count-operation-topology.json) | Live video is sent to an external [spatialAnalysis](../../cognitive-services/computer-vision/spatial-analysis-operations.md) module that counts people in a designated zone. When the criteria defined by the AI operation is met, events are sent to a signal gate processor that opens, sending the frames to a video sink node. As a result, a new clip is appended to the Azure Video Analyzer video resource. | | [Person count operation with Computer Vision for Spatial Analysis](./visual-studio-code-extension.md#person-count-operation-with-computer-vision-for-spatial-analysis)

+[spatial-analysis/person-line-crossing-operation-topology](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/spatial-analysis/person-line-crossing-operation-topology.json) | Live video is sent to an external [spatialAnalysis](../../cognitive-services/computer-vision/spatial-analysis-operations.md) module that tracks when a person crosses a designated line. When the criteria defined by the AI operation is met, events are sent to a signal gate processor that opens, sending the frames to a video sink node. As a result, a new clip is appended to the Azure Video Analyzer video resource. | | [Person crossing line operation with Computer Vision for Spatial Analysis](./visual-studio-code-extension.md#person-crossing-line-operation-with-computer-vision-for-spatial-analysis)

+[spatial-analysis/person-zone-crossing-operation-topology](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/spatial-analysis/person-zone-crossing-operation-topology.json) | Live video is sent to an external [spatialAnalysis](../../cognitive-services/computer-vision/spatial-analysis-operations.md) module that emits an event when a person enters or exists a zone. When the criteria defined by the AI operation is met, events are sent to a signal gate processor that opens, sending the frames to a video sink node. As a result, a new clip is appended to the Azure Video Analyzer video resource. | [Live Video with Computer Vision for Spatial Analysis](https://aka.ms/ava-spatial-analysis) | [Person crossing zone operation with Computer Vision for Spatial Analysis](./visual-studio-code-extension.md#person-crossing-zone-operation-with-computer-vision-for-spatial-analysis)

+[spatial-analysis/person-distance-operation-topology](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/spatial-analysis/person-distance-operation-topology.json) | Live video is sent to an external [spatialAnalysis](../../cognitive-services/computer-vision/spatial-analysis-operations.md) module that tracks when people violate a distance rule. When the criteria defined by the AI operation is met, events are sent to a signal gate processor that opens, sending the frames to a video sink node. As a result, a new clip is appended to the Azure Video Analyzer video resource. | | [Person distance operation with Computer Vision for Spatial Analysis](./visual-studio-code-extension.md#person-distance-operation-with-computer-vision-for-spatial-analysis)

+[spatial-analysis/custom-operation-topology](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/spatial-analysis/custom-operation-topology.json) | Live video is sent to an external [spatialAnalysis](../../cognitive-services/computer-vision/spatial-analysis-operations.md) module that carries out a supported AI operation. When the criteria defined by the AI operation is met, events are sent to a signal gate processor that opens, sending the frames to a video sink node. As a result, a new clip is appended to the Azure Video Analyzer video resource. | | [Custom operation with Computer Vision for Spatial Analysis](./visual-studio-code-extension.md#custom-operation-with-computer-vision-for-spatial-analysis)

+### AI composition

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[ai-composition](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/ai-composition/topology.json) | Run 2 AI inferencing models of your choice. In this example, classified video frames are sent from an AI inference engine using the [Tiny YOLOv3 model](https://github.com/Azure/video-analyzer/tree/main/edge-modules/extensions/yolo/tinyyolov3/grpc-cpu) to another engine using the [YOLOv3 model](https://github.com/Azure/video-analyzer/tree/main/edge-modules/extensions/yolo/yolov3/grpc-cpu). Having such a topology enables you to trigger a heavy AI module, only when a light AI module indicates a need to do so. | [Analyze live video streams with multiple AI models using AI composition](edge/analyze-ai-composition.md) | [Record to the Video Analyzer service using multiple AI models](./visual-studio-code-extension.md#record-to-the-video-analyzer-service-using-multiple-ai-models)

+### Miscellaneous

+Name | Description | Samples | VSCode Name

+:-- | :- | :- | :

+[object-tracking](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/object-tracking/topology.json) | Track objects in a live video feed. The object tracker comes in handy when you need to detect objects in every frame, but the edge device does not have the necessary compute power to be able to apply the vision model on every frame. | [Track objects in a live video](edge/track-objects-live-video.md) | [Record video based on the object tracking AI model](./visual-studio-code-extension.md#record-video-based-on-the-object-tracking-ai-model)

+[line-crossing](https://github.com/Azure/video-analyzer/blob/main/pipelines/live/topologies/line-crossing/topology.json) | Use a computer vision model to detect objects in a subset of frames when they cross a virtual line in a live video feed. The object tracker node is used to track those objects in the frames and pass them through a line-crossing node. The line-crossing node comes in handy when you want to detect objects that cross the imaginary line and emit events. | [Detect when objects cross a virtual line in a live video](edge/use-line-crossing.md) | [Record video based on the line crossing AI model](./visual-studio-code-extension.md#record-video-based-on-the-line-crossing-ai-model)

+## Next steps

+[Understand Video Analyzer pipelines](pipeline.md).

+## Quotas and limitations - cloud pipelines

+To create a topology, along the left panel under the Video Analyzer Edge module, right-click on `Pipelines topologies` and select `Create pipeline topology`. This will open up a new blank topology. Either load one of the pre-made topologies by selecting from the `Try sample topologies` dropdown at the top, or build one by dragging and dropping the available modules and connecting them.

+After all required areas are complete, save the topology with the `Save` in the top right. For sample topologies, the required fields should be pre-filled. This will make it available for use with creating live pipelines.

+To edit an existing topology, on the left panel under `Pipeline topologies` right-click on the name of the topology, and select `Edit pipeline topology`.

+To delete an existing topology, on the left under `Pipeline topologies` right-click on the name of the topology, and select `Delete pipeline topology`. Live pipelines will need to be removed first.

+To view the underlying JSON behind an existing topology, on the left panel under `Pipeline topologies` right-click on the name of the topology, and select `Show pipeline topology JSON`.

+To create a live pipeline, along the left panel under `Pipeline topologies` right-click on the name of the topology and select `Create live pipeline`. Fill in a live pipeline name and any required parameters before continuing. In the top right, either click `Save` which will save it in an inactive state, or `Save and activate` which will start the live pipeline immediately.

+To activate an existing live pipeline, along the left panel under `Pipeline topologies` right-click on the name of the live pipeline and select `Activate live pipeline`.

+To deactivate a running instance, along the left panel under `Pipeline topologies` right-click on the live pipeline and select `Deactivate live pipeline`. This will not delete the live pipeline.

+To delete an existing live pipeline, along the left panel under `Pipeline topologies` right-click on the live pipeline and select `Delete live pipeline`. Active live pipelines cannot be deleted.

+To view the underlying JSON behind an existing live pipeline, on the left panel under `Pipeline topologies` right-click on the live pipeline and select `Show live pipeline JSON`.

+## Remote device adapters

+To create a [remote device adapter](./cloud/connect-cameras-to-cloud.md#connect-via-a-remote-device-adapter), along the left panel under the Video Analyzer Edge module, right-click on `Remote device adapters` and select `Create remote device adapter`. Three additional dialog boxes appear, prompting for additional information:

+1. Enter a unique name for the remote device adapter. (There should be no other remote device adapters with this name.)

+2. Select a IoT Device. (For instance, select the IoT device that represents the network camera that will be connected.)

+3. Enter a hostname or IP address of the remote device adapter. (For instance, enter the IP address of the network camera that will be connected.)

+After entering all the necessary information, the remote device adapter will be saved and listed under the `Remote device adapters` section.

+To delete an existing remote device adapter, along the left panel under `Remote device adapters` right-click on a remote device adapter and select `Delete remote device adapter`.

+To view the underlying JSON behind an existing remote device adapter, along the left panel under `Remote device adapters` right-click on a remote device adapter and select `Show remote device adapter JSON`.

+If you wish to manage your existing parameters, this can be done with the `Manage parameters` option along the top. The pane that comes allows you to add new parameters, and either edit or delete existing ones.

+When creating a series of live pipelines, there are likely cases where you want to use variables to help name files or outputs. For example, you may wish to name a video clip with the live pipeline name and date / time so you know where it came from and at what time. Video Analyzer provides three system variables you can use in your modules to help here.

+When you create a topology, you will need to connect the various modules together. This is done with connections. From the circle on the edge of a module, drag to the circle on the next module you want data to flow to. This will produce a connection.

+## Sample pipeline topologies

+The following sample pipeline topologies are available on the extension:

+### Continuous Video Recording

+#### Record to Video Analyzer video

+[  ](./media/visual-studio-code-extension/cvr-to-video-sink.png)

+#### Record using gRPC Extension

+[  ](./media/visual-studio-code-extension/cvr-with-grpc-extension.png)

+#### Record using HTTP Extension

+[  ](./media/visual-studio-code-extension/cvr-with-http-extension.png)

+#### Record on motion detection

+[  ](./media/visual-studio-code-extension/cvr-with-motion-detection.png)

+#### Record audio with video

+[  ](./media/visual-studio-code-extension/audio-video.png)

+### Event-based Video Recording

+#### Record using gRPC Extension

+[  ](./media/visual-studio-code-extension/evr-to-video-sink-by-grpc-extension.png)

+#### Record using HTTP Extension

+[  ](./media/visual-studio-code-extension/evr-to-video-sink-by-http-extension.png)

+#### Record to Video Analyzer video based on inference events

+[  ](./media/visual-studio-code-extension/evr-to-video-sink-on-obj-detect.png)

+#### Record to local files based on inference events

+[  ](./media/visual-studio-code-extension/evr-to-files-based-on-hub-messages.png)

+#### Record motion events to Video Analyzer video and local files

+[  ](./media/visual-studio-code-extension/evr-to-files-and-video-sink-on-motion.png)

+#### Record motion events to Video Analyzer video

+[  ](./media/visual-studio-code-extension/evr-to-video-sink-on-motion-detection.png)

+#### Record motion events to local files

+[  ](./media/visual-studio-code-extension/evr-to-files-on-motion-detection.png)

+### Motion Detection

+#### Publish motion events to IoT Hub

+[  ](./media/visual-studio-code-extension/motion-detection.png)

+#### Analyze motion events using gRPC Extension

+[  ](./media/visual-studio-code-extension/evr-on-motion-plus-grpc-extension.png)

+#### Analyze motion events using HTTP Extension

+[  ](./media/visual-studio-code-extension/evr-on-motion-plus-http-extension.png)

+### Extensions

+#### Analyze video using HTTP Extension

+[  ](./media/visual-studio-code-extension/inferencing-with-http-extension.png)

+#### Analyze video with Intel OpenVINO Model Server

+[  ](./media/visual-studio-code-extension/inferencing-with-openvino.png)

+### Computer Vision

+#### Person count operation with Computer Vision for Spatial Analysis

+[  ](./media/visual-studio-code-extension/person-count-topology.png)

+#### Person crossing line operation with Computer Vision for Spatial Analysis

+[  ](./media/visual-studio-code-extension/person-crossing-line-topology.png)

+#### Person crossing zone operation with Computer Vision for Spatial Analysis

+[  ](./media/visual-studio-code-extension/person-zone-crossing-topology.png)

+#### Person distance operation with Computer Vision for Spatial Analysis

+[  ](./media/visual-studio-code-extension/person-distance-topology.png)

+#### Custom operation with Computer Vision for Spatial Analysis

+[  ](./media/visual-studio-code-extension/person-attributes-topology.png)

+### AI Composition

+#### Record to the Video Analyzer service using multiple AI models

+[  ](./media/visual-studio-code-extension/ai-composition.png)

+### Miscellaneous

+#### Record video based on the object tracking AI model

+[  ](./media/visual-studio-code-extension/object-tracking-with-http-extension.png)

+#### Record video based on the line crossing AI model

+[  ](./media/visual-studio-code-extension/line-crossing-with-http-extension.png)

+To read more on how to create a **new ARM-Based** Video Analyzer for Media account, read this [article](create-video-analyzer-for-media-account.md)

+| **HANA versions** | SDC on HANA 1.x, MDC on HANA 2.x SPS04, SPS05 Rev <= 56, SPS 06 (validated for encryption enabled scenarios as well) | |

+* Back up of databases with extensions in their names arenΓÇÖt supported. This is because the IIS server performs the [file extension request filtering](/iis/configuration/system.webserver/security/requestfiltering/fileextensions). However, note that we have allowlisted _.ad_, _.cs_, and _.master_ that can be used in the database names.

+|Status|Required|

+|Validation|Device to be validated through toolset to ensure that code integrity is enabled by validating dm-verity and IMA|

+| Rel 21-12 | [4578950] | .NET Framework 3.5 Security and Quality Rollup | [3.104] | Feb 16, 2021 |

+| Rel 21-12 | [4578954] | . NET Framework 4.5.2 Security and Quality Rollup | [3.104] | Feb 16, 2021 |

+| Rel 21-12 | [5004335] | . NET Framework 3.5 and 4.7.2 Cumulative Update | [6.38] | Aug 10, 2021 |

+| Rel 21-12 | [5001401] | Servicing Stack update | [3.104] | Apr 13, 2021 |

+| Rel 21-12 | [5001403] | Servicing Stack update | [4.97] | Apr 13, 2021 |

+| Rel 21-12 | [5008287] | Servicing Stack update | [6.38] | Aug 10, 2021 |

+| Rel 21-12 | [4494175] | Microcode | [5.62] | Sep 1, 2020 |

+[4578950]: https://support.microsoft.com/kb/4578950  

+[4578954]: https://support.microsoft.com/kb/4578954 

+[5004335]: https://support.microsoft.com/kb/5004335 

+[5001401]: https://support.microsoft.com/kb/5001401 

+[5001403]: https://support.microsoft.com/kb/5001403 

+[5008287]: https://support.microsoft.com/kb/5008287 

+* A __Docker container__ for Windows, Linux, or ARM architecture. The container includes a TensorFlow model and service code to use the Custom Vision API.

+In this quickstart, you'll learn how to use the Custom Vision website to create an object detector model. Once you build a model, you can test it with new images and integrate it into your own image recognition app.

+1. Next, select one of the available domains. Each domain optimizes the detector for specific types of images, as described in the following table. You can change the domain later if you want to.

+ |__General__| Optimized for a broad range of object detection tasks. If none of the other domains are appropriate, or if you're unsure about which domain to choose, select the __General__ domain. |

+In this section, you'll upload and manually tag images to help train the detector.

+1. You'll see your uploaded images in the **Untagged** section of the UI. The next step is to manually tag the objects that you want the detector to learn to recognize. Select the first image to open the tagging dialog window.

+The **Overlap Threshold** slider deals with how correct an object prediction must be to be considered "correct" in training. It sets the minimum allowed overlap between the predicted object's bounding box and the actual user-entered bounding box. If the bounding boxes don't overlap to this degree, the prediction won't be considered correct.

+description: Learn how to test an image and then use it to retrain your model in the Custom Vision service.

+After you train your Custom Vision model, you can quickly test it using a locally stored image or a URL pointing to a remote image. The test uses the most recently trained iteration of your model. Then you can decide whether further training is needed.

+1. From the [Custom Vision web portal](https://customvision.ai), select your project. Select **Quick Test** on the right of the top menu bar. This action opens a window labeled **Quick Test**.

+2. In the **Quick Test** window, select in the **Submit Image** field and enter the URL of the image you want to use for your test. If you want to use a locally stored image instead, select the **Browse local files** button and select a local image file.

+The image you select appears in the middle of the page. Then the prediction results appear below the image in the form of a table with two columns, labeled **Tags** and **Confidence**. After you view the results, you may close the **Quick Test** window.

+You can now take the image submitted previously for testing and use it to retrain your model.

+ To add an image to your training data, select the image, manually select the tag(s), and then select __Save and close__. The image is removed from __Predictions__ and added to the training images. You can view it by selecting the __Training Images__ tab.

+| Arabic (Jordan) | `ar-JO` | Female | `ar-JO-SanaNeural` ^New | General |

+| Arabic (Jordan) | `ar-JO` | Male | `ar-JO-TaimNeural` ^New | General |

+ Title: Azure Communication Services Closed Caption overview

+description: Learn about the Azure Communication Services Closed Captions.

+# Closed Captions overview

+Azure Communication Services allows one to enable Closed Captions for the VoIP calls in private preview.

+Closed Captions is the conversion of a voice or video call audio track into written words that appear in real time. Closed Captions are never saved and are only visible to the user that has enabled it.

+Here are main scenarios where Closed Captions are useful:

+- **Accessibility**. In the workplace or consumer apps, Closed Captioning for meetings, conference calls, and training videos can make a huge difference.

+- **Accessibility**. Scenarios when audio can't be heard, either because of a noisy environment, such as an airport, or because of an environment that must be kept quiet, such as a hospital.

+- **Inclusivity**. Closed Captioning was developed to aid hearing-impaired people, but it could be useful for a language proficiency as well.

+## When to use Closed Captions

+- Closed Captions help maintain concentration and engagement, which can provide a better experience for viewers with learning disabilities, a language barrier, attention deficit disorder, or hearing impairment.

+- Closed Captions allow participants to be on the call in loud or sound-sensitive environments.

+## Feature highlights

+- Support for multiple platforms with cross-platform support.

+- Async processing with client subscription to events and callbacks.

+- Multiple languages to choose from for recognition.

+- Support for existing SkypeToken Authentication

+## Availability

+The private preview will be available on all platforms.

+- Android

+- iOS

+- Web

+## Next steps

+- Get started with a Closed Caption Quickstart (TBD)

+$env:PSModulePath += ";$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules"

+## Accounts portal is retired

+Accounts portal was retired December 31, 2021. The features supported in the Accounts portal were migrated to the Azure portal. This article explains how to perform some of the most common operations in the Azure portal.

+- That permission is always required in Git mode since every time after the customer publishes, the factory object with the last commit ID needs to be updated.

+| ArraySize |The number of bytes the connector can fetch in a single network round trip. For example, `ArraySize=‭10485760‬`.<br/><br/>Larger values increase throughput by reducing the number of times to fetch data across the network. Smaller values increase response time, as there is less of a delay waiting for the server to transmit data. | An integer from 1 to 4294967296 (4 GB). Default value is `60000`. The value 1 does not define the number of bytes, but indicates allocating space for exactly one row of data. |

+| Load a large amount of data by using a custom query, without physical partitions, while with an integer column for data partitioning. | **Partition options**: Dynamic range partition.<br>**Query**: `SELECT * FROM <TABLENAME> WHERE ?AdfRangePartitionColumnName <= ?AdfRangePartitionUpbound AND ?AdfRangePartitionColumnName >= ?AdfRangePartitionLowbound AND <your_additional_where_clause>`.<br>**Partition column**: Specify the column used to partition data. You can partition against the column with integer data type.<br>**Partition upper bound** and **partition lower bound**: Specify if you want to filter against partition column to retrieve data only between the lower and upper range.<br><br>During execution, the service replaces `?AdfRangePartitionColumnName`, `?AdfRangePartitionUpbound`, and `?AdfRangePartitionLowbound` with the actual column name and value ranges for each partition, and sends to Amazon RDS for Oracle. <br>For example, if your partition column "ID" is set with the lower bound as 1 and the upper bound as 80, with parallel copy set as 4, the service retrieves data by 4 partitions. Their IDs are between [1, 20], [21, 40], [41, 60], and [61, 80], respectively. |

+ - Windows Server 2022

+After you're confident that your applications and workloads are stable, you can begin using Azure Synapse Analytics to satisfy your business scenarios. Turn off any remaining pipelines that are running on Azure Data Lake Analytics and retire your Azure Data Lake Analytics accounts.

+| SsemUserErrorCrossTenantIdentityAccessForbidden | Managed identity access operation failed. <br> Note: This error can occur when a subscription is moved to different tenant. The customer has to manually move the identity to the new tenant. | Try adding a different user-assigned identity to your key vault to enable access to the customer-managed key. Or move the identity to the new tenant under which the subscription is present. For more information, see how to [Enable the key](#enable-key). |

+| Generic error | Could not fetch the passkey. | This error is a generic error. Contact Microsoft Support to troubleshoot the error and determine the next steps.|

+If the device isn't available, you receive a notification. If the device is available, Microsoft identifies the device for shipment and prepares the shipment. During device preparation, following actions occur:

+description: Learn how to add a private artifact repository to your lab to store your custom artifacts.

+# Add an artifact repository to a lab

+This article tells you how to add an *artifact* repository to your lab in Azure DevTest Labs. Artifacts are tools or applications to install on virtual machines (VMs). You define artifacts in a JSON file that you load from a GitHub or Azure Repos Git repository.

+The public [DevTest Labs GitHub artifact repository](https://github.com/Azure/azure-devtestlab/tree/master/Artifacts) provides many common artifacts for Windows and Linux. The artifacts in this public repository are available by default in DevTest Labs. For information about adding artifacts to VMs, see [Add artifacts to DevTest Labs VMs](add-artifact-vm.md).

+You can also create custom artifacts that aren't available in the public artifact repository. To learn about creating custom artifacts, see [Create custom artifacts](devtest-lab-artifact-author.md). You can add your custom artifacts to your own artifact repository, and add the repository to your lab so all lab users can use the artifacts.

+This article shows you how to add an artifact repository to your lab by using the Azure portal, an Azure Resource Management (ARM) template, or Azure PowerShell. You can also use an Azure PowerShell or Azure CLI script to automate adding an artifact repository to a lab.

+To add an artifact repository to a lab, you need to know the Git HTTPS clone URL and the personal access token for the GitHub or Azure Repos repository that has the artifact files.

+### Get the clone URL and personal access token for GitHub

+1. On the home page of the GitHub repository that has your artifacts, select **Code**, and under **Clone**, copy the HTTPS URL.

+1. Select your profile image in the upper-right corner of GitHub, and then select **Settings**.

+1. On your profile page, in the left menu, select **Developer Settings**, and then select **Personal access tokens**.

+1. Select **Generate new token**.

+1. On the **New personal access token** page, under **Note**, enter an optional description for the token. Accept all the defaults, and then select **Generate token**.

+1. Save the generated token.

+### Get the clone URL and personal access token for Azure Repos

+1. On the main page of the repository that has your artifacts, select **Clone**. On the **Clone Repository** page, copy the clone URL.

+1. In the upper-right corner of the Azure DevOps page, select **User settings** > **Personal access tokens**.

+1. On the **Personal Access Tokens** page, select **New Token**.

+1. Fill out the information for the token, selecting **Read** for the scopes, and then select **Create**.

+1. On the **Success** page, be sure to copy the token, because Azure Repos doesn't store the token or show it again.

+## Add an artifact repository to a lab in the Azure portal

+1. On the lab's **Overview** page, select **Configuration and policies** from the left navigation.

+1. On the **Configuration and policies** page, select **Repositories** under **External resources** in the left navigation.

+ On the **Repositories** page, the **Public Artifact Repo** is automatically present and connects to the [DevTest Labs public GitHub repository](https://github.com/Azure/azure-devtestlab). If this repo isn't enabled for your lab, you can enable it by selecting the checkbox next to **Public Artifact Repo**, and then selecting **Enable** on the top menu bar.

+1. To add your artifact repository to the lab, select **Add** in the top menu bar.

+ 

+1. In the **Repository** pane, enter the following information:

+ - **Name**: A repository name to use in the lab.

+ - **Git clone URL**: The Git HTTPS clone URL from GitHub or Azure Repos.

+ - **Branch** (optional): The branch that has your artifact definitions.

+ - **Personal access token**: The personal access token from GitHub or Azure Repos.

+ - **Folder paths**: The folder for your ARM template definitions, relative to the Git clone URL. Be sure to include the initial forward slash in the folder path.

+1. Select **Save**.

+ 

+The repository now appears in the **Repositories** list for the lab.

+## Add an artifact repository by using an ARM template

+ARM templates are JSON files that describe Azure resources to create. For more information about ARM templates, see [Understand the structure and syntax of ARM templates](../azure-resource-manager/templates/syntax.md).

+The following ARM template adds an artifact repository to a lab. The template creates the lab if it doesn't already exist.

+### Review the ARM template

+The sample template gathers the following information in parameters. Some of the parameters have defaults, but the deployment command must specify the lab name, artifact repository URI, repository type, and repository personal access token.

+- Display name for the artifact repository in DevTest Labs. The default value is `Team Repository`.

+- URI of the artifact repository, which you copied earlier.

+- Repository branch that contains the artifacts. The default value is `main`.

+- Name of the folder that contains the artifacts. The default value is: `/Artifacts`.

+- Repository type. The allowed values are `VsoGit`, for Azure Repos, or `GitHub`.

+- Personal access token for the repository, which you copied earlier.

+```json

+{

+ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",

+ "contentVersion": "1.0.0.0",

+ "parameters": {

+ "labName": {

+ "type": "string"

+ },

+ "artifactRepositoryDisplayName": {

+ "type": "string",

+ "defaultValue": "Team Repository"

+ },

+ "artifactRepoUri": {

+ "type": "string"

+ "artifactRepoBranch": {

+ "type": "string",

+ "defaultValue": "main"

+ "artifactRepoFolder": {

+ "type": "string",

+ "defaultValue": "/Artifacts"

+ },

+ "artifactRepoType": {

+ "type": "string",

+ "allowedValues": ["VsoGit", "GitHub"]

+ },

+ "artifactRepoSecurityToken": {

+ "type": "securestring"

+ }

+ },

+ "variables": {

+ "artifactRepositoryName": "[concat('Repo-', uniqueString(subscription().subscriptionId))]"

+ },

+ "resources": [{

+ "apiVersion": "2016-05-15",

+ "type": "Microsoft.DevTestLab/labs",

+ "name": "[parameters('labName')]",

+ "location": "[resourceGroup().location]",

+ "resources": [

+ {

+ "apiVersion": "2016-05-15",

+ "name": "[variables('artifactRepositoryName')]",

+ "type": "artifactSources",

+ "dependsOn": [

+ "[resourceId('Microsoft.DevTestLab/labs', parameters('labName'))]"

+ ],

+ "properties": {

+ "uri": "[parameters('artifactRepoUri')]",

+ "folderPath": "[parameters('artifactRepoFolder')]",

+ "branchRef": "[parameters('artifactRepoBranch')]",

+ "displayName": "[parameters('artifactRepositoryDisplayName')]",

+ "securityToken": "[parameters('artifactRepoSecurityToken')]",

+ "sourceType": "[parameters('artifactRepoType')]",

+ "status": "Enabled"

+ }

+ ]

+ }

+ ]

+}

+```

+There are several ways to deploy ARM templates to create or update Azure resources. For information and instructions, see the following articles:

+- [Deploy resources with ARM templates and Azure PowerShell](../azure-resource-manager/templates/deploy-powershell.md)

+- [Deploy resources with ARM templates and Azure CLI](../azure-resource-manager/templates/deploy-cli.md)

+- [Deploy resources with ARM templates in the Azure portal](../azure-resource-manager/templates/deploy-portal.md)

+- [Deploy resources with ARM templates and Resource Manager REST API](../azure-resource-manager/templates/deploy-rest.md)

+For this example, deploy the template by using Azure PowerShell.

+> [!NOTE]

+> The cmdlets that deploy the template are context-specific, so they use the current tenant and subscription. If you need to change the context, use [Set-AzContext](/powershell/module/az.accounts/set-azcontext) before you deploy the template

+1. Create a resource group by using [New-AzResourceGroup](/powershell/module/az.resources/new-azresourcegroup). If the resource group you want to use already exists, skip this step.

+ ```powershell

+ New-AzResourceGroup -Name MyLabResourceGroup1 -Location westus

+ ```

+1. Create a deployment to the resource group by using [New-AzResourceGroupDeployment](/powershell/module/az.resources/new-azresourcegroupdeployment). You can make several resource deployments to the same resource group. If you're deploying several times to the same resource group, make sure each deployment name is unique.

+ ```powershell

+ New-AzResourceGroupDeployment `

+ -Name MyLabResourceGroup-Deployment1 `

+ -ResourceGroupName MyLabResourceGroup1 `

+ -TemplateFile azuredeploy.json `

+ -TemplateParameterFile azuredeploy.parameters.json

+ ```

+

+After `New-AzResourceGroupDeployment` runs successfully, the output shows important information like the provisioning state, which should be `succeeded`, and any outputs for the template.

+## Add an artifact repository by using Azure PowerShell

+The following sample PowerShell script, *New-DevTestLabArtifactRepository.ps1*, adds an artifact repository to a lab. The full script includes some verbose messages and comments.

+Name for the new artifact repository. The script creates a random name for the repository if not specified.

+This name appears in the list of artifact repositories for a lab.

+Uri to the artifact repository.

+Branch that contains the artifact files. Defaults to 'main'.

+Folder that contains the artifact files. Defaults to '/Artifacts'

+Personal access token for the GitHub or Azure Repos repository.

+Whether the artifact repository is a VSOGit (Azure Repos) or GitHub repository.

+The script uses the current Azure context. To set the context, use Set-AzContext.

+ $RepositoryBranch = 'main',

+# Set artifact repository internal name if not specified.

+# Sign in to Azure.

+#Get Lab Resource.

+#Prepare properties object for the call to New-AzResource.

+#Add resource to the current subscription.

+# Using resourceId lets you specify the $SubscriptionId rather than using the

+# Check the result.

+#Return the newly created resource to use in later scripts.

+The PowerShell script takes the following parameters:

+| `LabName` | The name of the lab. |

+| `ArtifactRepositoryName` | Name for the new artifact repository. The script creates a random name for the repository if it isn't specified. |

+| `ArtifactRepositoryDisplayName` | Display name that appears in the lab's artifact repository list. |

+| `RepositoryUri` | URI of the artifact repository, which you copied earlier.

+| `RepositoryBranch` | Repository branch that contains the artifacts. The default value is `main`.|

+| `FolderPath` | Folder that contains the artifacts. The default value is: `/Artifacts`.|

+| `PersonalAccessToken` | Security token for accessing the repository, which you copied earlier.|

+| `SourceType` | Whether the artifact repository is a VSOGit (Azure Repos) or GitHub repository.|

+The repository needs an internal name for identification, which is different than the display name in the Azure portal. You don't see the internal name when using the Azure portal, but you see it when using Azure REST APIs or Azure PowerShell. The script creates a random name if the deployment command doesn't specify one.

+### PowerShell commands

+The script uses the following PowerShell commands:

+| Command | Notes |

+| | -- |

+| [Get-AzResource](/powershell/module/az.resources/get-azresource) | Gets details about the lab, such as its location. You create the artifact repository source in the same location and under the same resource group as the lab.|

+| [New-AzResource](/powershell/module/az.resources/new-azresource) | Adds the Azure resource. There's no specific command for adding artifact repositories. This cmdlet needs either the `ResourceId` or the `ResourceName` and `ResourceType` pair to know the type of resource to create. The current script uses the `ResourceName` and `ResourceType` pair. |

+A good way to discover resource name and resource type information is to use the [Azure REST API Browser](https://azure.github.io/projects/apis/) website. DevTest Labs [Artifact Sources](/rest/api/dtl/artifact-sources) shows REST APIs for creating and managing DevTest Labs artifact sources. The current script uses the following resource ID:

+https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DevTestLab/labs/{labName}/artifactsources/{name}

+The resource type is everything listed after `providers` in the URI, except for items in curly brackets. The resource name is everything in the curly brackets. If you use more than one item for the resource name, separate each item with a slash:

+### Run the PowerShell script

+Run the PowerShell script, substituting your own values for the example values in `LabName`, `LabResourceGroupName`, `ArtifactRepositoryName`, `RepositoryUri`, `PersonalAccessToken`, and `SourceType`:

+```powershell

+Set-AzContext -SubscriptionId <Your Azure subscription ID>

+.\New-DevTestLabArtifactRepository.ps1 -LabName "mydevtestlab" -LabResourceGroupName "mydtlrg" -ArtifactRepositoryName "myteamrepository" -RepositoryUri "https://github.com/myteam/myteamrepository.git" - "1111...." -SourceType "GitHub"

+```

+- [Specify mandatory artifacts for DevTest Labs VMs](devtest-lab-mandatory-artifacts.md)

+description: Learn how to add an artifact to a virtual machine in a lab in Azure DevTest Labs.

+# Add artifacts to DevTest Labs VMs

+This article describes how to add *artifacts* to Azure DevTest Labs virtual machines (VMs). Artifacts specify actions to take to provision a VM, such as running Windows PowerShell scripts, running Bash commands, or installing software. You can use parameters to customize the artifacts for your own needs.

+DevTest Labs artifacts can come from the [public DevTest Labs Git repository](https://github.com/Azure/azure-devtestlab/tree/master/Artifacts) or from private Git repositories. To create your own custom artifacts and store them in a repository, see [Create custom artifacts](devtest-lab-artifact-author.md). To add your artifact repository to a lab so lab users can access the custom artifacts, see [Add an artifact repository to your lab](add-artifact-repository.md).

+DevTest Labs lab owners can specify mandatory artifacts to be installed on all lab VMs at creation. For more information, see [Specify mandatory artifacts for DevTest Labs VMs](devtest-lab-mandatory-artifacts.md).

+You can't change or remove mandatory artifacts at VM creation time, but you can add any available individual artifacts. This article describes how to add available artifacts to VMs by using the Azure portal or Azure PowerShell.

+## Add artifacts to VMs from the Azure portal

+You can add artifacts during VM creation, or add artifacts to existing lab VMs.

+To add artifacts during VM creation:

+1. On the lab's home page, select **Add**.

+1. On the **Choose a base** page, select the type of VM you want.

+1. On the **Create lab resource** screen, select **Add or Remove Artifacts**.

+1. On the **Add artifacts** page, select the arrow next to each artifact you want to add to the VM.

+1. On each **Add artifact** pane, enter any required and optional parameter values, and then select **OK**. The artifact appears under **Selected artifacts**, and the number of configured artifacts updates.

+ 

+1. You can change the artifacts after adding them.

+ - By default, artifacts install in the order you add them. To rearrange the order, select the ellipsis **...** next to the artifact in the **Selected artifacts** list, and select **Move up**, **Move down**, **Move to top**, or **Move to bottom**.

+ - To edit the artifact's parameters, select **Edit** to reopen the **Add artifact** pane.

+ - To delete the artifact from the **Selected artifacts** list, select **Delete**.

+1. When you're done adding and arranging artifacts, select **OK**.

+1. The **Create lab resource** screen shows the number of artifacts added. To add, edit, rearrange, or delete the artifacts before you create the VM, select **Add or Remove Artifacts** again.

+After you create the VM, the installed artifacts appear on the VM's **Artifacts** page. To see details about each artifact's installation, select the artifact name.

+To install artifacts on an existing VM:

+1. From the lab's home page, select the VM from the **My virtual machines** list.

+1. On the VM page, select **Artifacts** in the top menu bar or left navigation.

+1. On the **Artifacts** page, select **Apply artifacts**.

+ 

+1. On the **Add artifacts** page, select and configure artifacts the same as for a new VM.

+1. When you're done adding artifacts, select **Install**. The artifacts install on the VM immediately.

+## Add artifacts to VMs by using Azure PowerShell

+The following PowerShell script applies an artifact to a VM by using the [Invoke-AzResourceAction](/powershell/module/az.resources/invoke-azresourceaction) cmdlet.

+[Parameter(Mandatory=$true, HelpMessage="The name of the lab that has the VM")]

+[Parameter(Mandatory=$true, HelpMessage="The name of the VM")]

+[Parameter(Mandatory=$true, HelpMessage="The artifact to apply to the VM")]

+# Get the internal repository name

+# Find the VM in Azure

+# Handle the input parameters to pass through

+# Fill the artifact parameter with the additional -param_ data and strip off the -param_

+# Create a structure to pass the artifact data to the action

+# Apply the artifact

+- [Specify mandatory artifacts](devtest-lab-mandatory-artifacts.md)

+ Title: Create custom artifacts for virtual machines

+description: Learn how to create and use artifacts to deploy and set up applications on DevTest Labs virtual machines.

+# Create custom artifacts for DevTest Labs

+This article describes how to create custom artifact files for Azure DevTest Labs virtual machines (VMs). DevTest Labs artifacts specify actions to take to provision a VM. An artifact consists of an artifact definition file and other script files that you store in a folder in a Git repository.

+- For information about adding your artifact repositories to labs, see [Add an artifact repository to your lab](add-artifact-repository.md).

+- For information about adding the artifacts you create to VMs, see [Add artifacts to DevTest Labs VMs](add-artifact-vm.md).

+- For information about specifying mandatory artifacts to be added to all lab VMs, see [Specify mandatory artifacts for DevTest Labs VMs](devtest-lab-mandatory-artifacts.md).

+## Artifact definition files

+Artifact definition files are JSON expressions that specify what you want to install on a VM. The files define the name of an artifact, a command to run, and available parameters for the command. You can refer to other script files by name in the artifact definition file.

+The following example shows the sections that make up the basic structure of an *artifactfile.json* artifact definition file:

+| Element name | Description |

+| | |

+| `$schema` |Location of the JSON schema file. The JSON schema file can help you test the validity of the definition file.|

+| `title` |Name of the artifact to display in the lab. **Required.**|

+| `description` |Description of the artifact to display in the lab. **Required.**|

+| `iconUri` |URI of the artifact icon to display in the lab.|

+| `targetOsType` |Operating system of the VM to install the artifact on. Supported values: `Windows`, `Linux`. **Required.**|

+| `parameters` |Values to customize the artifact when installing on the VM.|

+| `runCommand` |The artifact install command to execute on the VM. **Required.**|

+In the parameters section of the definition file, specify the values a user can input when installing an artifact. You can refer to these values in the artifact install command.

+| Element name | Description |

+| | |

+| `type` |Type of parameter value. **Required.**|

+| `displayName` |Name of the parameter to display to the lab user. **Required.**|

+| `description` |Description of the parameter to display to the lab user. **Required.**|

+The allowed parameter value types are:

+| Type | Description |

+| | |

+|`string`|Any valid JSON string|

+|`int`|Any valid JSON integer|

+|`bool`|Any valid JSON boolean|

+|`array`|Any valid JSON array|

+### Secrets as secure strings

+To declare secrets as secure string parameters with masked characters in the UI, use the following syntax in the `parameters` section of the *artifactfile.json* file:

+The artifact install command to run the PowerShell script takes the secure string created by using the `ConvertTo-SecureString` command.

+Don't log secrets to the console, because the script captures output for user debugging.

+### Artifact expressions and functions

+You can use expressions and functions to construct the artifact install command. Expressions evaluate when the artifact installs. Expressions can appear anywhere in a JSON string value, and always return another JSON value. Enclose expressions with brackets, \[ \]. If you need to use a literal string that starts with a bracket, use two brackets \[\[.

+You usually use expressions with functions to construct a value. Function calls are formatted as `functionName(arg1, arg2, arg3)`.

+Common functions include:

+| Function | Description |

+| | |

+|`parameters(parameterName)`|Returns a parameter value to provide when the artifact command runs.|

+|`concat(arg1, arg2, arg3, ...)`|Combines multiple string values. This function can take various arguments.|

+The following example uses expressions and functions to construct a value:

+To create a custom artifact:

+- Install a JSON editor to work with artifact definition files. [Visual Studio Code](https://code.visualstudio.com/) is available for Windows, Linux, and macOS.

+- Start with a sample *artifactfile.json* definition file.

+ The public [DevTest Labs artifact repository](https://github.com/Azure/azure-devtestlab/tree/master/Artifacts) has a rich library of artifacts you can use. You can download an artifact definition file and customize it to create your own artifacts.

+ This article uses the *artifactfile.json* definition file and *artifact.ps1* PowerShell script at [https://github.com/Azure/azure-devtestlab/tree/master/Artifacts/windows-test-paramtypes](https://github.com/Azure/azure-devtestlab/tree/master/Artifacts/windows-test-paramtypes).

+- Use IntelliSense to see valid elements and value options that you can use to construct an artifact definition file. For example, when you edit the `targetOsType` element, IntelliSense shows you `Windows` or `Linux` options.

+- Store your artifacts in public or private Git artifact repositories.

+ - Store each *artifactfile.json* artifact definition file in a separate directory named the same as the artifact name.

+ - Store the scripts that the install command references in the same directory as the artifact definition file.

+ The following screenshot shows an example artifact folder:

+ 

+- To store your custom artifacts in the public [DevTest Labs artifact repository](https://github.com/Azure/azure-devtestlab/tree/master/Artifacts), open a pull request against the repo.

+- To add your private artifact repository to a lab, see [Add an artifact repository to your lab in DevTest Labs](add-artifact-repository.md).

+- [Add artifacts to DevTest Labs VMs](add-artifact-vm.md)

+- [Diagnose artifact failures in the lab](devtest-lab-troubleshoot-artifact-failure.md)

+- [Troubleshoot issues when applying artifacts](devtest-lab-troubleshoot-apply-artifacts.md)

+ Title: Create an Azure DevTest Labs virtual machine custom image from a VHD file

+description: Learn how to use a VHD file to create an Azure DevTest Labs virtual machine custom image in the Azure portal.

+You can create a virtual machine (VM) custom image for Azure DevTest Labs by using a virtual hard drive (VHD) file.

+This article describes how to create a custom image in the Azure portal. You can also [use PowerShell](devtest-lab-create-custom-image-from-vhd-using-powershell.md) to create a custom image.

+## Azure portal instructions

+To create a custom image from a VHD file in DevTest Labs in the Azure portal, follow these steps:

+1. In the [Azure portal](https://go.microsoft.com/fwlink/p/?LinkID=525040), go to the **Overview** page for the lab that has the uploaded VHD file.

+1. Select **Configuration and policies** in the left navigation.

+1. On the **Configuration and policies** pane, select **Custom images** under **Virtual machine bases** in the left navigation.

+1. On the **Custom images** page, select **Add**.

+ 

+1. On the **Add custom image** page:

+ - Enter a name for the custom image to display in the list of base images for creating a VM.

+ - Enter an optional description to display in the base image list.

+ - Under **OS type**, select whether the OS for the VHD and custom image is **Windows** or **Linux**.

+ - If you choose **Windows**, select the checkbox if you ran *sysprep* on the machine before creating the VHD file.

+ - If you choose **Linux**, select the checkbox if you ran *deprovision* on the machine before creating the VHD file.

+1. Under **VHD**, select the uploaded VHD file for the custom image from the drop-down menu.

+1. Optionally, enter a plan name, plan offer, and plan publisher if the VHD image isn't a licensed image published by Microsoft. If the image is a licensed image, these fields are pre-populated with the plan information.

+ - **Plan name:** Name of the non-Microsoft Marketplace image or SKU used to create the VHD image.

+ - **Plan offer:** Product or offer name for the Marketplace image.

+ - **Plan publisher:** Publisher of the Marketplace image.

+1. Select **OK**.

+ 

+After creation, the custom image is stored in the lab's storage account. The custom image appears in the list of VM base images for the lab. Lab users can create new VMs based on the custom image.

+

+- [Add a VM to your lab](./devtest-lab-add-vm.md)

+- [Compare custom images and formulas in DevTest Labs](devtest-lab-comparing-vm-base-image-types.md)

+- [Copying Custom Images between Azure DevTest Labs](https://www.visualstudiogeeks.com/blog/DevOps/How-To-Move-CustomImages-VHD-Between-AzureDevTestLabs#copying-custom-images-between-azure-devtest-labs)

+ Title: Specify mandatory artifacts for lab virtual machines

+description: Learn how to specify mandatory artifacts to install at creation of every lab virtual machine (VM) in Azure DevTest Labs.

+# Specify mandatory artifacts for DevTest Labs VMs

+This article describes how to specify mandatory *artifacts* in Azure DevTest Labs to install on every lab virtual machine (VM). Artifacts are tools and applications to add to VMs. Installing mandatory artifacts ensures all lab VMs have standardized, up-to-date artifacts. Lab users don't have to spend time and effort to add needed artifacts individually.

+Mandatory artifacts can include any software that every VM in your lab must have. If you create a custom image from a VM that has mandatory artifacts applied to it, and create new VMs from that image, those VMs also have the mandatory artifacts. Even if the custom image is old, VM creation applies the most updated versions of the mandatory artifacts.

+Only artifacts that have no parameters can be mandatory artifacts. Lab users don't have to enter extra parameter values, making the VM creation process simple.

+During VM creation, mandatory artifacts install before any artifacts the user chooses to install on the machine.

+You can select mandatory artifacts for Windows and Linux lab machines separately.

+1. On your lab's home page, under **Settings** in the left navigation, select **Configuration and policies**.

+1. On the **Configuration and policies** screen, under **External resources** in the left navigation, select **Mandatory artifacts**.

+1. For Windows VMs, select **Windows**, and then select **Edit Windows artifacts**. For Linux VMs, select **Linux**, and then select **Edit Linux artifacts**.

+ 

+1. On the **Mandatory artifacts** page, select the arrow next to each artifact you want to add to the VM.

+1. On each **Add artifact** pane, select **OK**. The artifact appears under **Selected artifacts**, and the number of configured artifacts updates.

+ 

+1. By default, artifacts install in the order you add them. To rearrange the order, select the ellipsis **...** next to the artifact in the **Selected artifacts** list, and select **Move up**, **Move down**, **Move to top**, or **Move to bottom**. To delete the artifact from the list, select **Delete**.

+1. When you're done adding and arranging artifacts, select **Save**.

+## Delete or rearrange mandatory artifacts

+After you add mandatory artifacts, the lists of selected artifacts appear on the **Configuration and policies | Mandatory artifacts** screen under **Windows** and **Linux**. You can rearrange or delete the specified mandatory artifacts.

+To delete a mandatory artifact from the list, select the checkbox next to the artifact, and then select **Delete**.

+

+To rearrange the order of the mandatory artifacts:

+1. Select **Edit Windows artifacts** or **Edit Linux artifacts**.

+1. On the **Mandatory artifacts** page, select the ellipsis **...** next to the artifact in the **Selected artifacts** list.

+1. Select **Move up**, **Move down**, **Move to top**, or **Move to bottom**.

+1. Select **Save**.

+## See mandatory artifacts for a VM

+Once you specify mandatory artifacts for a lab, all lab VMs for that operating system (Windows or Linux) have those artifacts installed at creation. Lab users can see the mandatory artifacts to be installed on their VMs.

+For example, to see the mandatory artifacts specified for lab Windows VMs in the earlier procedure:

+1. On your lab's home page, select **Add**.

+1. On the **Choose a base** page, select a Windows image, such as **Windows 11 Pro**.

+1. On the **Create lab resource** page, under **Artifacts**, note the number of mandatory artifacts. To see what the mandatory artifacts are, select **Add or Remove Artifacts**.

+ 

+1. On the **Add artifacts** screen, an informational message lists the mandatory artifacts to be installed, in order.

+ 

+You can't remove, rearrange, or change mandatory artifacts when you create an individual VM. However, you can add other available artifacts to the VM. For more information and instructions, see [Add artifacts to DevTest Labs VMs](add-artifact-vm.md).

+You can also create your own artifacts for VMs. For more information, see [Create custom artifacts for DevTest Labs VMs](devtest-lab-artifact-author.md).

+- Learn how to [add a Git artifact repository to a lab](add-artifact-repository.md).

+An [update policy](/azure/data-explorer/kusto/management/updatepolicy) in Azure Data Explorer allows you to automatically transform and append data to a target table whenever new data is inserted into a source table.

+.create-merge table rawData (Timestamp:datetime, someId:string, Value:string, ValueType:string)

+.create-merge table mappingTable (someId:string, twinId:string, otherMetadata:string)

+.create-merge table timeseriesSilver (twinId:string, Timestamp:datetime, someId:string, otherMetadata:string, ValueNumeric:real, ValueString:string)

+.create-or-alter function with (folder = "Update", skipvalidation = "true") Update_rawData() {

+| join kind=leftouter mappingTable on someId

+| project

+ Timestamp, ValueNumeric = toreal(Value), ValueString = Value, ...

+| timestamp | twinId | modelId | name | value | relationshipTarget | relationshipID |

+| 2021-02-01 17:24 | ConfRoomTempSensor | dtmi:com:example:TemperatureSensor;1 | temperature | 301.0 | | |

+Microsoft has partnered with [RealEstateCore](https://www.realestatecore.io/) to deliver this open-source DTDL ontology for the real estate industry. [RealEstateCore](https://www.realestatecore.io/) is a Swedish consortium of real estate owners, software vendors, and research institutions.

+This smart buildings ontology provides common ground for modeling smart buildings, using industry standards (like [BRICK Schema](https://brickschema.org/ontology/) or [W3C Building Topology Ontology](https://w3c-lbd-cg.github.io/bot/https://docsupdatetracker.net/index.html)) to avoid reinvention. The ontology also comes with best practices for how to consume and properly extend it.

+There's a sample application available that converts an RDF-based model file to [DTDL (version 2)](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md) for use by the Azure Digital Twins service. It has been validated for the [Brick](https://brickschema.org/ontology/) schema, and can be extended for other schemas in the building industry (such as [Building Topology Ontology (BOT)](https://w3c-lbd-cg.github.io/bot/), [Semantic Sensor Network](https://www.w3.org/TR/vocab-ssn/), or [buildingSmart Industry Foundation Classes (IFC)](https://technical.buildingsmart.org/standards/ifc/ifc-schema-specifications/)).

+Setting up an [Azure Active Directory (Azure AD)](../active-directory/fundamentals/active-directory-whatis.md) **managed identity** for an Azure Digital Twins instance can allow the instance to easily access other Azure AD-protected resources, such as [Azure Key Vault](../key-vault/general/overview.md). The identity is managed by the Azure platform, and doesn't require you to provision or rotate any secrets. For more about managed identities in Azure AD, see [Managed identities for Azure resources](../active-directory/managed-identities-azure-resources/overview.md).

+You can use a system-assigned managed identity for your Azure Digital Instance to authenticate to a [custom-defined endpoint](concepts-route-events.md#create-an-endpoint). Azure Digital Twins supports system-assigned identity-based authentication to endpoints for [Event Hubs](../event-hubs/event-hubs-about.md) and [Service Bus](../service-bus-messaging/service-bus-messaging-overview.md) destinations, and to an [Azure Storage Container](../storage/blobs/storage-blobs-introduction.md) endpoint for [dead-letter events](concepts-route-events.md#dead-letter-events). [Event Grid](../event-grid/overview.md) endpoints are currently not supported for managed identities.

+* **Pricing**: For pricing details, see [Azure Private Link pricing](https://azure.microsoft.com/pricing/details/private-link).

+For information on the limits of Private Link, see [Azure Private Link documentation: Limitations](../private-link/private-link-service-overview.md#limitations).

+A **service tag** represents a group of IP address prefixes from a given Azure service. Microsoft manages the address prefixes encompassed by the service tag and automatically updates the service tag as addresses change, minimizing the complexity of frequent updates to network security rules. For more information about service tags, see [Virtual network tags](../virtual-network/service-tags-overview.md).

+You can use service tags to define network access controls on [network security groups](../virtual-network/network-security-groups-overview.md#security-rules) or [Azure Firewall](../firewall/service-tags.md), by using service tags in place of specific IP addresses when you create security rules. By specifying the service tag name (in this case, **AzureDigitalTwins**) in the appropriate *source* or *destination* field of a rule, you can allow or deny the traffic for the corresponding service.

+This article describes how to enable a [system-assigned identity for an Azure Digital Twins instance](concepts-security.md#managed-identity-for-accessing-other-resources), and use the identity when forwarding events to supported routing destinations. Setting up a managed identity isn't required for routing, but it can help the instance to easily access other Azure AD-protected resources, such as [Event Hubs](../event-hubs/event-hubs-about.md), [Service Bus](../service-bus-messaging/service-bus-messaging-overview.md) destinations, and [Azure Storage Container](../storage/blobs/storage-blobs-introduction.md).

+Learn more about managed identities in Azure AD:

+`MATCH` supports any query that finds a path between twins with an unpredictable number of hops, based on certain relationship conditions.

+A query with a `MATCH` clause must also use the [WHERE clause](reference-query-clause-where.md) to specify the `$dtId` for at least one of the twins it references.

+ Title: Finding IDs for Azure Education Hub APIs

+description: Learn how to find all the IDs needed to call the education hub APIs

+# Tutorial: Find all IDs needed to call Azure Education Hub APIs

+This article helps you gather the necessary IDs needed to call the education hub APIs. If you go through the Education Hub UI, these IDs are gathered for you, but to call them publicly, you must have a billing account ID, billing profile ID, and invoice section ID.

+In this tutorial, you learn how to:

+> [!div class="checklist"]

+> * Find your billing account ID

+> * Find your billing profile ID

+> * Find your invoice section ID

+## Prerequisites

+You must have an Azure account linked with education hub.

+## Sign in to Azure

+* Sign in to the Azure portal at https://portal.azure.com.

+## Navigate to Cost Management + Billing

+While in the Azure portal, search for Cost Management + Billing and click on the service from the dropdown menu.

+## Get Billing account ID

+This section will show you how to get your Billing Account ID.

+1. Click on the "Properties" tab under Settings

+2. The string listed in the ID box is your Billing Account ID

+3. Copy this and save it for later

+## Get Billing profile ID

+This section will show you how to get your Billing Profile ID.

+1. Click on "Billing Profiles" tab under the Billing section

+2. Click on the desired billing profile

+3. Click on the "Properties" tab under the Settings section

+4. This page will display your billing profile ID at the top of the page

+5. Copy this and save it for later. You can also see your Billing Account ID at the bottom of the page.

+## Get Invoice section ID

+This section will show you how to get your Invoice Section ID.

+1. Click on "Invoice sections" tab under the Billing tab. Note you must be in Billing Profile to see Invoice Sections

+2. Click on the desired Invoice Section

+3. Click on the "Properties" tab under the Settings section

+4. This page will display your invoice section ID at the top of the page

+5. Copy this and save it for later. You can also see your Billing Account ID at the bottom of the page.

+## Next steps

+- [Manage your Academic Grant using the Overview page](hub-overview-page.md)

+- [Support options](educator-service-desk.md)

+Event Grid provides durable delivery. It tries to deliver each message **at least once** for each matching subscription immediately. If a subscriber's endpoint doesn't acknowledge receipt of an event or if there's a failure, Event Grid retries delivery based on a fixed [retry schedule](#retry-schedule) and [retry policy](#retry-policy). By default, the Event Grid module delivers one event at a time to the subscriber. The payload is however an array with a single event.

+When Event Grid receives an error for an event delivery attempt, Event Grid decides whether it should retry the delivery, dead-letter the event, or drop the event based on the type of the error.

+If the error returned by the subscribed endpoint is a configuration-related error that can't be fixed with retries (for example, if the endpoint is deleted), Event Grid will either perform dead-lettering on the event or drop the event if dead-letter isn't configured.

+If the error returned by the subscribed endpoint isn't among the above list, Event Grid performs the retry using policies described below:

+There's a five-minute delay between the last attempt to deliver an event and when it's delivered to the dead-letter location. This delay is intended to reduce the number of Blob storage operations. If the dead-letter location is unavailable for four hours, the event is dropped.

+Here are the possible values of `lastDeliveryOutcome` and their descriptions.

+| LastDeliveryOutcome | Description |

+| - | -- |

+| NotFound | Destination resource wasn't found. |

+| Disabled | Destination has disabled receiving events. Applicable for Azure Service Bus and Azure Event Hubs. |

+| Full | Exceeded maximum number of allowed operations on the destination. Applicable for Azure Service Bus and Azure Event Hubs. |

+| Unauthorized | Destination returned unauthorized response code. |

+| BadRequest | Destination returned bad request response code. |

+| TimedOut | Delivery operation timed out. |

+| Busy | Destination server is busy. |

+| PayloadTooLarge | Size of the message exceeded the maximum allowed size by the destination. Applicable for Azure Service Bus and Azure Event Hubs. |

+| Probation | Destination is put in probation by Event Grid. Delivery isn't attempted during probation. |

+| Canceled | Delivery operation canceled. |

+| Aborted | Delivery was aborted by Event Grid after a time interval. |

+| SocketError | Network communication error occurred during delivery. |

+| ResolutionError | DNS resolution of destination endpoint failed. |

+| Delivering | Delivering events to the destination. |

+| SessionQueueNotSupported | Event delivery without session ID is attempted on an entity, which has session support enabled. Applicable for Azure Service Bus entity destination. |

+| Forbidden | Delivery is forbidden by destination endpoint (could be because of IP firewalls or other restrictions) |

+| InvalidAzureFunctionDestination | Destination Azure function isn't valid. Probably because it doesn't have the EventGridTrigger type. |

+**LastDeliveryOutcome: Probation**

+An event subscription is put into probation for a duration by Event Grid if event deliveries to that destination start failing. Probation time is different for different errors returned by the destination endpoint. If an event subscription is in probation, events may get dead-lettered or dropped without even trying delivery depending on the error code due to which it's in probation.

+| Error | Probation Duration |

+| -- | |

+| Busy | 10 seconds |

+| NotFound | 5 minutes |

+| SocketError | 30 seconds |

+| ResolutionError | 5 minutes |

+| Disabled | 5 minutes |

+| Full | 5 minutes |

+| TimedOut | 10 seconds |

+| Unauthorized | 5 minutes |

+| Forbidden | 5 minutes |

+| InvalidAzureFunctionDestination | 10 minutes |

+> [!NOTE]

+> Event Grid uses probation duration for better delivery management and the duration might change in the future.

+## .NET 6.2.0-preview (2021-06)

+This release corresponds to api-version 2021-06-01-preview which includes the following new features:

+- [Azure Active Directory authentication for topics and domains, and partner namespaces](authenticate-with-active-directory.md)

+- Private link support for partner namespaces. Azure portal doesn't support it yet.

+- IP Filtering for partner namespaces. Azure portal doesn't support it yet.

+- System Identity for partner topics. Azure portal doesn't support it yet.

+- [User Identity for system topics, custom topics and domains](enable-identity-custom-topics-domains.md)

+Azure Firewall provides automatic SNAT for all outbound traffic to public IP addresses. Azure Firewall doesnΓÇÖt SNAT when the destination IP address is a private IP address range per IANA RFC 1918. This logic works perfectly when you egress directly to the Internet. However, with forced tunneling enabled, Internet-bound traffic is SNATed to one of the firewall private IP addresses in the AzureFirewallSubnet. This hides the source address from your on-premises firewall. You can configure Azure Firewall to not SNAT regardless of the destination IP address by adding *0.0.0.0/0* as your private IP address range. With this configuration, Azure Firewall can never egress directly to the Internet. For more information, see [Azure Firewall SNAT private IP address ranges](snat-private-range.md).

+You can configure Forced Tunneling during Firewall creation by enabling Forced Tunnel mode as shown below. To support forced tunneling, Service Management traffic is separated from customer traffic. An additional dedicated subnet named **AzureFirewallManagementSubnet** (minimum subnet size /26) is required with its own associated public IP address. This public IP address is for management traffic. It is used exclusively by the Azure platform and can't be used for any other purpose.

+In both HTTP and TLS inspected HTTPS cases, the firewall ignores the packet's destination IP address and uses the DNS resolved IP address from the Host header. The firewall expects to get port number in the Host header, otherwise it assumes the standard port 80. If there's a port mismatch between the actual TCP port and the port in the host header, the traffic is dropped. DNS resolution is done by Azure DNS or by a custom DNS if configured on the firewall. 

+From a security standpoint, Microsoft doesn't recommend disabling certificate subject name check. In certain use cases such as for testing, as a work-around to resolve failing HTTPS connection, you can disable certificate subject name check for your Azure Front Door. Note that the origin still needs to present a certificate with a valid trusted chain, but doesn't have to match the origin host name. The option to disable is present under the Azure Front Door settings in the Azure portal and on the BackendPoolsSettings in the Azure Front Door API.

+ Title: Quickstart - Connect a device to an Azure IoT Central application | Microsoft Docs

+description: Quickstart - Connect your first device to a new IoT Central application. This quickstart uses a smartphone app from either the Google Play or Apple app store as an IoT device.

+# Quickstart - Use your smartphone as a device to send telemetry to an IoT Central application

+- Germany West Central

+- West Us 2

+* Learn how to [migrate your existing outbound connectivity method to NAT gateway](../virtual-network/nat-gateway/tutorial-migrate-outbound-nat.md)

+- Learn how to [parameterize a load test](./how-to-parameterize-load-tests.md) with secrets.

+- Learn how to [configure automated performance testing](./tutorial-cicd-azure-pipelines.md).

+1. Create an [Azure Virtual Network](../virtual-network/virtual-networks-overview.md) that will contain the resources used by the workspace.

+1. Create an [Azure Virtual Networks](../virtual-network/virtual-networks-overview.md) that will contain the workspace and other resources. Then create a [Private Link-enabled workspace](how-to-secure-workspace-vnet.md#secure-the-workspace-with-private-endpoint) to enable communication between your VNet and workspace.

+1. If your compute cluster or compute instance does not use a public IP address, you must [Allow inbound communication](how-to-secure-training-vnet.md#required-public-internet-access) so that management services can submit jobs to your compute resources.

+ > [!TIP]

+ > Compute cluster and compute instance can be created with or without a public IP address. If created with a public IP address, they communicate with the Azure Batch Services over the public IP. If created without a public IP, they communicate with Azure Batch Services over the private IP. When using a private IP, you need to allow inbound communications from Azure Batch.

+1. The compute resource receives the job and begins training. The compute resource accesses secure storage accounts to download training files and upload output.

+1. Right-click the **Automobile price data (Raw)** and select **Preview Data**.

+1. Right-click the **Score Model** component, and select **Preview data** > **Scored dataset** to view its output.

+1. Right-click the **Evaluate Model** component and select **Preview data** > **Evaluation results** to view its output.

+The **Long running queries** tab shows the top 5 Query IDs by average duration per execution, aggregated in 15-minute intervals. You can view more Query IDs by selecting from the **Number of Queries** drop down. The chart colors may change for a specific Query ID when you do this.

+> [!Note]

+> Displaying the Query Text is no longer supported and will show as empty. The query text is removed to avoid unauthorized access to the query text or underlying schema which can pose a security risk.

+The recommended steps to view the query text is shared below:

+ 1. Identify the query_id of the top queries from the Query Performance Insight blade in the Azure portal.

+1. Log in to your Azure Database for MySQL server from MySQL Workbench or mysql.exe client or your preferred query tool and execute the following queries.

+

+```sql

+ SELECT * FROM mysql.query_store where query_id = '<insert query id from Query performance insight blade in Azure portal'; // for queries in Query Store

+ SELECT * FROM mysql.query_store_wait_stats where query_id = '<insert query id from Query performance insight blade in Azure portal'; // for wait statistics

+```

+> [!Note]

+> Displaying the Query Text is no longer supported and will show as empty. The query text is removed to avoid unauthorized access to the query text or underlying schema which can pose a security risk.

+The recommended steps to view the query text is shared below:

+ 1. Identify the query_id of the top queries from the Query Performance Insight blade in the Azure portal.

+1. Log in to your Azure Database for MySQL server from MySQL Workbench or mysql.exe client or your preferred query tool and execute the following queries.

+

+```sql

+ SELECT * FROM mysql.query_store where query_id = '<insert query id from Query performance insight blade in Azure portal'; // for queries in Query Store

+ SELECT * FROM mysql.query_store_wait_stats where query_id = '<insert query id from Query performance insight blade in Azure portal'; // for wait statistics

+```

+5. Deploy and register [Self-hosted integration runtime](#deploy-self-hosted-integration-runtime-ir-and-scan-your-data-sources) inside the same VNet or a peered VNet where Azure Purview account and ingestion private endpoints are deployed.

+- For all Azure source types like Azure Blob Storage and Azure SQL Database, you must explicitly choose to run the scan by using a self-hosted integration runtime that is deployed in the same VNet or a peered VNet where Azure Purview account and ingestion private endpoints are deployed.

+- Self-hosted integration runtime machine must be deployed in the same VNet or a peered VNet where Azure Purview account and ingestion private endpoints are deployed.

+ Title: Azure Purview accounts architecture and best practices

+Azure Purview is a unified data governance solution. You deploy an Azure Purview account to centrally manage data governance across your data estate, spanning both cloud and on-prem environments. To use Azure Purview as your centralized data governance solution, you need to deploy one or more Azure Purview accounts inside your Azure subscription. We recommend keeping the number of Azure Purview instances as minimum, however, in some cases more Azure Purview instances are needed to fulfill business security and compliance requirements.

+## Single Azure Purview account

+Consider deploying minimum number of Azure Purview accounts for the entire organization. This approach takes maximum advantage of the "network effects" where the value of the platform increases exponentially as a function of the data that resides inside the platform.

+Use [Azure Purview collections hierarchy](./concept-best-practices-collections.md) to lay out your organization's data management structure inside a single Azure Purview account. In this scenario, one Purview account is deployed in an Azure subscription. Data sources from one or more Azure subscriptions can be registered and scanned inside the Azure Purview. You can also register and scan data sources from your on-premises or multi-cloud environments.

+## Multiple Azure Purview accounts

+It is recommended to create a new instance of Azure Purview account when testing scan configurations or classifications in isolated environments. For some scenarios, there is a "versioning" feature in some areas of the platform such as glossary, however, it would be easier to have a "disposable" instance of Azure Purview to freely test expected functionality and then plan to roll out the feature into the production instance.

+Additionally, consider using a test Azure Purview account when you cannot perform a rollback. For example, currently you cannot remove a glossary term attribute from an Azure Purview instance once it is added to your Azure Purview account. In this case, it is recommended using a test Azure Purview account first.

+Consider deploying separate instances of Azure Purview accounts for development, testing and production environments, specially when you have separate instances of data for each environment.

+In this scenario, production and non-production data sources can be registered and scanned inside their corresponding Azure Purview instances.

+Optionally, you can register a data source in more than one Azure Purview instance, if needed.

+When you scan data sources in Azure Purview, information related to your metadata is ingested and stored inside your Azure Purview Data Map in the Azure region where your Azure Purview account is deployed. Consider deploying separate instances of Azure Purview if you have specific regulatory and compliance requirements that include even having metadata in a specific geographical location.

+If your organization has data in multiple geographies and you must keep metadata in the same region as the actual data, you have to deploy multiple Azure Purview instances, one for each geography. In this case, data sources from each regions should be registered and scanned in the Azure Purview account that corresponds to the data source region or geography.

+Currently, Azure Purview doesn't support multi-tenancy. If you have Azure data sources distributed across multiple Azure subscriptions under different Azure Active Directory tenants, it is recommended deploying separate Azure Purview accounts under each tenant.

+An exception applies to VM-based data sources and Power BI tenants.For more information about how to scan and register a cross tenant Power BI in a single Azure Purview account, see, [Register and scan a cross-tenant Power BI](./register-scan-power-bi-tenant.md).

+Review [Azure Purview Pricing model](https://azure.microsoft.com/pricing/details/azure-purview) when defining budgeting model and designing Azure Purview architecture for your organization. One billing is generated for a single Azure Purview account in the subscription where Azure Purview account is deployed. This model also applies to other Azure Purview costs such as scanning and classifying metadata inside Azure Purview Data Map.

+Some organizations often have many business units (BUs) that operate separately, and, in some cases, they don't even share billing with each other. In those cases, the organization will end up creating a Azure Purview instance for each BU. This model is not ideal, however, may be necessary, especially because Business Units are often not willing to share Azure billing.

+- Keep the number of Azure Purview accounts low for simplified administrative overhead. If you plan building multiple Azure Purview accounts, you may require creating and managing additional scans, access control model, credentials, and runtimes across your Azure Purview accounts. Additionally, you may need to manage classifications and glossary terms for each Azure Purview account.

+- Review your budgeting and financial requirements. If possible, use chargeback or showback model when using Azure services and divide the cost of Azure Purview across the organization to keep the number of Azure Purview accounts minimum.

+- Review [Azure Purview limits](./how-to-manage-quotas.md#azure-purview-limits) before deploying any new Azure Purview accounts. Currently, the default limit of Azure Purview accounts per region, per tenant (all subscriptions combined) is 3. You may need to contact Microsoft support to increase this limit in your subscription or tenant before deploying extra instances of Azure Purview.ΓÇ»

+- Review [Azure Purview prerequisites](./create-catalog-portal.md#prerequisites) before deploying any new Azure Purview accounts in your environment.

+While Azure Purview provides an out of the box user experience with Azure Purview Studio, not all tasks are suited to the point-and-click nature of the graphical user experience.

+* [Azure Purview REST API](/rest/api/purview)

+* If you apply classifications manually from Azure Purview Studio, such classifications are retained in subsequent scans.

+ Title: Azure Purview collections architecture and best practices

+- Review the [Azure Purview account best practices](./deployment-best-practices.md#determine-the-number-of-azure-purview-instances) and define the adequate number of Azure Purview accounts required in your organization before you plan the collection structure.

+- Each Azure Purview account is created with a default _root collection_. The root collection name is the same as your Azure Purview account name. The root collection can't be removed. To change the root collection's friendly name, you can change the friendly name of your Azure Purview account from Azure Purview Management center.

+- By design, you can't register data sources multiple times in a single Azure Purview account. This architecture helps to avoid the risk of assigning different levels of access control to a single data source. If multiple teams consume the metadata of a single data source, you can register and manage the data source in a parent collection. You can then create corresponding scans under each subcollection so that relevant assets appear under each child collection.

+Azure Purview data-plane roles are managed in Azure Purview. After you deploy an Azure Purview account, the creator of the Azure Purview account is automatically assigned the following roles at the root collection. You can use [Azure Purview Studio](https://web.purview.azure.com/resource/) or a programmatic method to directly assign and manage roles in Azure Purview.

+ - **Collection Admins** can edit Azure Purview collections and their details and add subcollections. They can also add users to other Azure Purview roles on collections where they're admins.

+- Consider implementing [emergency access](/azure/active-directory/users-groups-roles/directory-emergency-access) or a break-glass strategy for the Collection Admin role at your Azure Purview root collection level to avoid Azure Purview account-level lockouts. Document the process for using emergency accounts.

+ > In certain scenarios, you might need to use an emergency account to sign in to Azure Purview. You might need this type of account to fix organization-level access problems when nobody else can sign in to Azure Purview or when other admins can't accomplish certain operations because of corporate authentication problems. We strongly recommended that you follow Microsoft best practices around implementing [emergency access accounts](/azure/active-directory/users-groups-roles/directory-emergency-access) by using cloud-only users.

+ > Follow the instructions in [this article](./concept-account-upgrade.md#what-happens-when-your-upgraded-account-doesnt-have-a-collection-admin) to recover access to your Azure Purview root collection if your previous Collection Admin is unavailable.

+- In Azure Purview, you can assign roles to users, security groups, and service principals (including managed identities) from Azure Active Directory (Azure AD) on the same Azure AD tenant where the Azure Purview account is deployed.

+- You must first add guest accounts to your Azure AD tenant as B2B users before you can assign Azure Purview roles to external users.

+- If you need to connect to Azure Data Factory for lineage, grant the Data Curator role to the data factory's managed identity at your Azure Purview root collection level. When you connect Data Factory to Azure Purview in the authoring UI, Data Factory tries to add these role assignments automatically. If you have the Collection Admin role on the Azure Purview root collection, this operation will work.

+In this scenario, each region has a subcollection of its own under the top-level collection in the Azure Purview account. Data sources are registered and scanned in the corresponding subcollections in their own geographic locations. So assets also appear in the subcollection hierarchy for the region.

+Each region has a subcollection of its own under the top-level collection in the Azure Purview account. Data sources are registered and scanned in the corresponding subcollections in their own geographic locations. So assets are added to the subcollection hierarchy for the region.

+You can apply a combination of these two scenarios in your Azure Purview data map if total data democratization is required with a few exceptions for some collections. You can assign Azure Purview roles at the top-level collection and restrict inheritance to the specific child collections.

+- [Create a collection and assign permissions in Azure Purview](./quickstart-create-collection.md)

+Creating terms is necessary to build the business vocabulary and apply it to assets within Azure Purview. When a new Azure Purview account is created, by default, there are no built-in terms in the account.

+Glossary terms in Azure Purview are case sensitive and allow white space. The following shows a poorly executed example of implementing glossary terms and demonstrates the confusion caused:

+When building new term templates in Azure Purview, review the following considerations:

+- By default, Azure Purview offers several [out-of-the-box term attributes](./concept-business-glossary.md#custom-attributes) such as Name, Nick Name, Status, Definition, Acronym, Resources, Related terms, Synonyms, Stewards, Experts, and Parent term, which are found in the "System Default" template.

+- When importing terms from a .CSV file, be sure that terms already existing in Azure Purview are intended to be updated. When using the import feature, Azure Purview will overwrite existing terms.

+Exporting terms may be useful in Azure Purview account to account, Backup, or Disaster Recovery scenarios. Exporting terms in Azure Purview Studio must be done one term template at a time. Choosing terms from multiple templates will disable the "Export terms" button. As a best practice, using the "Term template" filter before bulk selecting will make the export process quick.

+- In Azure Purview, terms can be added to assets in different ways:

+Data Lineage is broadly understood as the lifecycle that spans the dataΓÇÖs origin, and where it moves over time across the data estate. Azure Purview can capture lineage for data in different parts of your organization's data estate, and at different levels of preparation including:

+* Each Data Factory instance can connect to only one Azure Purview account. You can establish new connection in another Azure Purview account, but this will turn existing connection to disconnected.

+* Data factory's managed identity is used to authenticate lineage in Azure Purview account, the data factory's managed identity Data Curator role on Azure Purview root collection is required.

+* Azure Purview cannot capture lineage if Azure Data Factory copy activity use copy activity features listed in **Limitations on copy activity lineage** of [Connect to Azure Data Factory](how-to-link-azure-data-factory.md)

+* For the lineage of Dataflow activity, Azure Purview only support source and sink. The lineage for Dataflow transformation is not supported yet.

+* Data flow lineage doesn't integrate with Azure Purview resource set.

+ :::image type="content" source="./media/concept-best-practices-lineage/ssis-lineage.png" alt-text="Screenshot of the Execute SSIS lineage in Azure Purview." lightbox="./media/concept-best-practices-lineage/ssis-lineage.png":::

+* Please refer the following step-by-step guide to [push Azure Data Factory lineage in Azure Purview](../data-factory/tutorial-push-lineage-to-purview.md).

+ Title: Azure Purview migration best practices

+|**ADF connections**|Currently an ADF can be connected to one Azure Purview at a time. You must disconnect ADF from failed Azure Purview account and reconnect it to the new account later.|

+The quickest way to migrate glossary terms is to [export terms to a .csv file](how-to-create-import-export-glossary.md). You can do this using the Azure Purview Studio.

+- [Create an Azure Purview account](./create-catalog-portal.md)

+- If your data sources are in Azure, you need to set up and use a self-hosted integration runtime on a Windows virtual machine that's deployed inside the same or a peered virtual network where Azure Purview ingestion private endpoints are deployed. The Azure integration runtime won't work with ingestion private endpoints.

+The self-hosted integration runtime VMs can be deployed inside the same Azure virtual network or a peered virtual network where the account and ingestion private endpoints are deployed.

+You can optionally deploy an additional self-hosted integration runtime in the spoke virtual networks.

+For performance and cost optimization, we highly recommended deploying one or more self-hosted integration runtime VMs in each region where data sources are located.

+- To scan an Azure data source that's configured with a private endpoint, you need to set up and use a self-hosted integration runtime on a Windows virtual machine that's deployed inside the same or a peered virtual network where Azure Purview account and ingestion private endpoints are deployed.

+ Title: Best practices for scanning of data sources in Azure Purview

+Azure Purview supports automated scanning of on-prem, multi-cloud, and SaaS data sources. Running a "scan" invokes the process to ingest metadata from the registered data sources. The metadata curated at the end of scan and curation process includes technical metadata like data asset names (table names/ file names), file size, columns, data lineage and so on. For structured data sources (for example Relational Database Management System) the schema details are also captured. The curation process applies automated classification labels on the schema attributes based on the scan rule set configured, and sensitivity labels if your Azure Purview account is connected to a Microsoft 365 Security & Compliance Center.

+- By design, you cannot register data sources multiple times in the same Azure Purview account. This architecture helps to avoid the risk of assigning different access control to the same data source.

+ - **Scan name**: By default, Azure Purview uses a naming convention **SCAN-[A-Z][a-z][a-z]** which is not helpful when trying to identify a scan that you have run. As a best practice, use a meaningful naming convention. An instance could be naming the scan as _environment-source-frequency-time_, for example DEVODS-Daily-0200, which would represent a daily scan at 0200 hrs.

+ - Azure Purview MSI - Managed Identity (for example, for Azure Data Lake Gen2 sources)

+- If a field / column, table, or a file is removed from the source system after the scan was executed, it will only be reflected (removed) in Azure Purview after the next scheduled full / incremental scan.

+- To understand the behavior of subsequent scans after *manually* editing a data asset or an underlying schema through Azure Purview Studio, refer to [Catalog asset details](./catalog-asset-details.md#scans-on-edited-assets).

+ Title: Azure Purview security best practices

+If you need to use Azure Purview from inside your private network, it is recommended to use Azure Private Link Service with your Azure Purview accounts for partial or [end-to-end isolation](catalog-private-link-end-to-end.md) to connect to Azure Purview Studio, access Azure Purview endpoints and to scan data sources.

+- To access Azure Purview Studio and Azure Purview endpoints, you need to use a management machine that is connected to private network to access Azure Purview through private network.

+The following NSG rules are required on **data sources** for Azure Purview scanning:

+The following NSG rules are required on from the **management machines** to access Azure Purview Studio:

+|Outbound | Management machines' private IP addresses or subnets | * | Azure Purview account and portal private endpoint IP addresses or subnets | 443 | Any | Allow |

+The following NSG rules are required on **self-hosted integration runtime VMs** for Azure Purview scanning and metadata ingestion:

+|Outbound | Self-hosted integration runtime VMs' private IP addresses or subnets | * | Azure Purview account and ingestion private endpoint IP addresses or Subnets | 443 | Any | Allow |

+The following NSG rules are required on for **Azure Purview account, portal and ingestion private endpoints**:

+|Inbound | Self-hosted integration runtime VMs' private IP addresses or subnets | * | Azure Purview account and ingestion private endpoint IP addresses or subnets | 443 | Any | Allow |

+|Inbound | Management machines' private IP addresses or subnets | * | Azure Purview account and ingestion private endpoint IP addresses or subnets | 443 | Any | Allow |

+- Enforce multi-factor authentication for Azure Purview users, especially, for users with privileged roles such as collection admins, data source admins or data curators.

+|Deploy an Azure Purview account | Control plane | Azure subscription owner or contributor | Azure RBAC roles |

+|Delete an Azure Purview account | Control plane | Contributor  | Azure RBAC roles |

+|View Azure Purview metrics to get current capacity units | Control plane | Reader | Azure RBAC roles |

+|Search inside Azure Purview Data Catalog | Data plane | Data source admin and data reader or data curator | Azure Purview roles |

+Azure Purview plane roles are defined and managed inside Azure Purview instance in Azure Purview collections. For more information, see [Access control in Azure Purview](catalog-permissions.md#roles).

+To gain access to Azure Purview, users must be authenticated and authorized. Authentication is the process of proving the user is who they claim to be. Authorization refers to controlling access inside Azure Purview assigned on collections.

+We use Azure Active Directory to provide authentication and authorization mechanisms for Azure Purview inside Collections. You can assign Azure Purview roles to the following security principals from your Azure Active Directory tenant which is associated with Azure subscription where your Azure Purview instance is hosted:

+Azure Purview fine-grained roles can be assigned to a flexible Collections hierarchy inside the Azure Purview instance.

+In Azure Purview, data sources, assets and scans can be organized using [Azure Purview Collections](quickstart-create-collection.md). Collections are hierarchical grouping of metadata in Azure Purview, but at the same time they provide a mechanism to manage access across Azure Purview. Roles in Azure Purview can be assigned to a collection based on your collection's hierarchy.

+For more information how to assign least privilege access model in Azure Purview, based on Azure Purview collection hierarchy, see [Access control in Azure Purview](catalog-permissions.md#assign-permissions-to-your-users).

+Reduce the number of users with write access inside your Azure Purview instance. Keep the number of collection admins and data curator roles minimum at root collection.

+By using Azure Active Directory Conditional Access policies, apply Azure AD Multi-Factor Authentication at sign-in for all individual users who are assigned to Azure Purview roles with modify access inside your Azure Purview instances: Collection Admin, Data Source Admin, Data Curator.

+Adding a `CanNotDelete` or `ReadOnly` lock to Azure Purview account does not prevent deletion or modification operations inside Azure Purview data plane, however, it prevents any operations in control plane, such as deleting the Azure Purview account, deploying a private endpoint or configuration of diagnostic settings.

+Resource locks can be assigned to Azure Purview resource groups or resources, however, you cannot assign an Azure resource lock to Azure Purview Managed resources or managed Resource Group.

+For more information about Azure Purview break glass strategy, see [Azure Purview collections best practices and design recommendations](concept-best-practices-collections.md#design-recommendations).

+Azure Purview is a data governance solution in cloud. You can register and scan different data sources from various data systems from your on-premises, Azure, or multi-cloud environments into Azure Purview. While data source is registered and scanned in Azure Purview, the actual data and data sources stay in their original locations, only metadata is extracted from data sources and stored in Azure Purview Data Map which means, you do not need to move data out of the region or their original location to extract the metadata into Azure Purview.

+When an Azure Purview account is deployed, in addition, a managed resource group is also deployed in your Azure subscription. A managed Azure Storage Account and a Managed Event Hub are deployed inside this resource group. The managed storage account is used to ingest metadata from data sources during the scan. Since these resources are consumed by the Azure Purview they cannot be accessed by any other users or principals, except the Azure Purview account. This is because an Azure role-based access control (RBAC) deny assignment is added automatically for all principals to this resource group at the time of Azure Purview account deployment, preventing any CRUD operations on these resources if they are not initiated from Azure Purview.

+Azure Purview extracts only the metadata from different data source systems into [Azure Purview Data Map](concept-elastic-data-map.md) during the scanning process.

+You can deploy a Azure Purview account inside your Azure subscription in any [supported Azure regions](https://azure.microsoft.com/global-infrastructure/services/?products=purview&regions=all).

+1. Azure Purview Managed Identity

+|Power BI tenant | Azure Runtime | Azure Purview Managed Identity |

+### Define required number of Azure Purview accounts for your organization

+As part of security planning for implementation of Azure Purview in your organization, review your business and security requirements to define [how many Azure Purview accounts are needed](concept-best-practices-accounts.md) in your organization. various factors may impact the decision, such as [multi-tenancy](/azure/cloud-adoption-framework/ready/enterprise-scale/enterprise-enrollment-and-azure-ad-tenants#define-azure-ad-tenants) billing or compliance requirements.

+- Limit the number of software in the self-hosted integration runtime VMs. Although it is not a mandatory requirement to have a dedicated VM for a self-hosted runtime for Azure Purview, we highly suggest using dedicated VMs especially for production environments.

+ Title: Best practices for applying sensitivity labels in Azure Purview

+Azure Purview supports labeling of both structured and unstructured data stored across various data sources. Labeling of data within Azure Purview allows users to easily find data that matches pre-defined auto-labeling rules that have been configured in the Microsoft 365 Security and Compliance Center (SCC). Azure Purview extends the use of Microsoft 365 sensitivity labels to assets stored in infrastructure cloud locations and structured data sources.

+- When configuring sensitivity labels for Azure Purview, you may define autolabeling rules for files, database columns, or both within the label properties. Azure Purview will label files within the Azure Purview data map when the autolabeling rule is configured to automatically apply the label or recommend that the label is applied.

+- If you are defining new autolabeling rules for files when configuring labels for Azure Purview, make sure that you have the condition for applying the label set appropriately.

+- Run test scans from Azure Purview on different Data Sources (for Example Hybrid-Cloud, On-Premise) to identify Sensitivity Labels.

+- Gather and consider insights (for example by using Azure Purview insights) and use alerting mechanism to mitigate potential breaches of Regulations.

+# Default Azure Purview Account

+In general, our guidance is to have a single Azure Purview account for entire customer's data estate. However, there are cases in which customers would like to have multiple Azure Purview accounts in their organization. The top reasons for different Azure Purview accounts are listed below:

+* Conglomerates - Conglomerates often have many business units (BUs) that operate separately to the extent that they won't even share billing with each other. Hence, this might require the conglomerates to create different Azure Purview accounts for different BUs.

+* Compliance - There are some strict compliance regulations, which treat even metadata as sensitive and require it to be in a particular geography. For the same reason customers might end up with multiple Azure Purview accounts per region.

+Having multiple Azure Purview accounts in a tenant now poses the challenge of which Azure Purview account should all other services like PBI, Synapse connect to. A PBI admin or Synapse Admin who is given the responsibility of pairing their PBI tenant or Synapse account with right Azure Purview account. This is where default Azure Purview account will help our customers. Azure global administrator (or tenant admin) can designate an Azure Purview account as default Azure Purview account at tenant level. At any point in time a tenant can have only 0 or 1 default accounts. Once this is set PBI Admin or Synapse Admin or any user in your organization has clear understanding that this account is the "right" one, discover the same and all other services should connect to this one.

+* Changing the default account is a two-step process. First you need to change the flag as 'No' to the current default Azure Purview account and then set the flag as 'Yes' to the new Azure Purview account.

+* Setting up default account is a control plane operation and hence Azure Purview studio will not have any changes if an account is defined as default. However, in the studio you can see the account name is appended with "(default)" for the default Azure Purview account.

+- [Azure Purview Pricing](https://azure.microsoft.com/pricing/details/azure-purview/)

+ Title: Azure Purview pricing guidelines

+description: This article provides a guideline towards understanding the various components in Azure Purview pricing.

+- While the pricing for Azure Purview is on a subscription-based **Pay-As-You-Go** model, there are various dimensions that you can consider while budgeting for Azure Purview

+- This guideline is intended to help you plan the budgeting for Azure Purview by providing a view on the various control factors that impact the budget

+There are **direct** and **indirect** costs that need to be considered while planning the Azure Purview budgeting and cost management.

+- The **Data map** is the foundation of the Azure Purview architecture and so needs to be up to date with asset information in the data estate at any given point

+- Use the basic resource set feature, before switching on the Advanced Resource Sets in Azure Purview to verify if requirements are met

+ - your data lakes schema is constantly changing, and you are looking for additional value beyond the basic Resource Set feature to enable Azure Purview to compute parameters such as #partitions, size of the data estate, etc., as a service

+ - When an Azure Purview account is provisioned, a storage account and event hub queue are created within the subscription in order to cater to secured scanning, which may be charged separately

+ - Azure private end points are used for Azure Purview accounts where it is required for users on a virtual network (VNet) to securely access the catalog over a private link

+This article identifies common tasks that can help you deploy Azure Purview into production. These tasks can be completed in phases, over the course of a month or more. Even organizations who have already deployed Azure Purview can use this guide to ensure they're getting the most out of their investment.

+- Ability to create Azure resources including Azure Purview

+2. For data sources that are not supported yet by Azure Purview, what are my options?

+3. How many Azure Purview instances do we need?

+6. Who can modify content inside of Azure Purview?

+7. What process can I use to improve the data quality in Azure Purview?

+To ensure the success of implementing Azure Purview for the entire enterprise, itΓÇÖs important to involve the right stakeholders. Only a few people are involved in the initial phase. However, as the scope expands, you will require additional personas to contribute to the project and provide feedback.

+|**Chief Data Officer**|The CDO oversees a range of functions that may include data management, data quality, master data management, data science, business intelligence, and creating data strategy. They can be the sponsor of the Azure Purview implementation project.|

+|**Data Security Specialist**|Assess overall network and data security, which involves data coming in and out of Azure Purview|

+Azure Purview can be used to centrally manage data governance across an organizationΓÇÖs data estate spanning cloud and on-premises environments. To have a successful implementation, you must identify key scenarios that are critical to the business. These scenarios can cross business unit boundaries or impact multiple user personas either upstream or downstream.

+4. Detail scenarios ΓÇô How the users use Azure Purview to solve problems?

+|Govern data assets with friendly user experience|I need to have a Business glossary for business-specific metadata. The business users can use Azure Purview for self-service scenarios to annotate their data and enable the data to be discovered easily via search.|Domain/Business Owner, Business Analyst, Data Scientist, Data Engineer|

+If you have only one small group using Azure Purview with basic consumption use cases, the approach could be as simple as having one Azure Purview instance to service the entire group. However, you may also wonder whether your organization needs more than one Azure Purview instance. And if using multiple Azure Purview instances, how can employees promote the assets from one stage to another.

+### Determine the number of Azure Purview instances

+In most cases, there should only be one Azure Purview account for the entire organization. This approach takes maximum advantage of the ΓÇ£network effectsΓÇ¥ where the value of the platform increases exponentially as a function of the data that resides inside the platform.

+3. **Conglomerates and federated model** ΓÇô Conglomerates often have many business units (BUs) that operate separately, and, in some cases, they won't even share billing with each other. In those cases, the organization will end up creating an Azure Purview instance for each BU. This model is not ideal, but may be necessary, especially because BUs are often not willing to share billing.

+4. **Compliance** ΓÇô There are some strict compliance regimes, which treat even metadata as sensitive and require it to be in a specific geography. If a company has multiple geographies, the only solution is to have multiple Azure Purview instances, one for each geography.

+Some organizations may decide to keep things simple by working with a single production version of Azure Purview. They probably donΓÇÖt need to go beyond discovery, search, and browse scenarios. If some assets have incorrect glossary terms, itΓÇÖs quite forgiving to let people self-correct. However, most organizations that want to deploy Azure Purview across various business units will want to have some form of process and control.

+Another important aspect to include in your production process is how classifications and labels can be migrated. Azure Purview has over 90 system classifiers. You can apply system or custom classifications on file, table, or column assets. Classifications are like subject tags and are used to mark and identify content of a specific type found within your data estate during scanning. Sensitivity labels are used to identify the categories of classification types within your organizational data, and then group the policies you wish to apply to each category. It makes use of the same sensitive information types as Microsoft 365, allowing you to stretch your existing security policies and protection across your entire content and data estate. It can scan and automatically classify documents. For example, if you have a file named multiple.docx and it has a National ID number in its content, Azure Purview will add classification such as EU National Identification Number in the Asset Detail page.

+In Azure Purview, there are several areas where the Catalog Administrators need to ensure consistency and maintenance best practices over its life cycle:

+* **Data assets** ΓÇô Data sources will need to be rescanned across environments. ItΓÇÖs not recommended to scan only in development and then regenerate them using APIs in Production. The main reason is that the Azure Purview scanners do a lot more ΓÇ£wiringΓÇ¥ behind the scenes on the data assets, which could be complex to move them to a different Azure Purview instance. ItΓÇÖs much easier to just add the same data source in production and scan the sources again. The general best practice is to have documentation of all scans, connections, and authentication mechanisms being used.

+* **Resource set pattern policies** ΓÇô This functionality is very advanced for any typical organizations to apply. In some cases, your Azure Data Lake Storage has folder naming conventions and specific structure that may cause problems for Azure Purview to generate the resource set. Your business unit may also want to change the resource set construction with additional customizations to fit the business needs. For this scenario, itΓÇÖs best to keep track of all changes via REST API, and document the changes through external versioning platform.

+* **Role assignment** ΓÇô This is where you control who has access to Azure Purview and which permissions they have. Azure Purview also has REST API to support export and import of users and roles but this is not Atlas API-compatible. The recommendation is to assign an Azure Security Group and manage the group membership instead.

+### Plan and implement different integration points with Azure Purview

+ItΓÇÖs likely that a mature organization already has an existing data catalog. The key question is whether to continue to use the existing technology and sync with Azure Purview or not. To handle syncing with existing products in an organization, Azure Purview provides Atlas REST APIs. Atlas APIs provide a powerful and flexible mechanism handling both push and pull scenarios. Information can be published to Azure Purview using Atlas APIs for bootstrapping or to push latest updates from another system into Azure Purview. The information available in Azure Purview can also be read using Atlas APIs and then synced back to existing products.

+For other integration scenarios such as ticketing, custom user interface, and orchestration you can use Atlas APIs and Kafka endpoints. In general, there are four integration points with Azure Purview:

+* **Data Asset** ΓÇô This enables Azure Purview to scan a storeΓÇÖs assets in order to enumerate what those assets are and collect any readily available metadata about them. So for SQL this could be a list of DBs, tables, stored procedures, views and config data about them kept in places like `sys.tables`. For something like Azure Data Factory (ADF) this could be enumerating all the pipelines and getting data on when they were created, last run, current state.

+* **Lineage** ΓÇô This enables Azure Purview to collect information from an analysis/data mutation system on how data is moving around. For something like Spark this could be gathering information from the execution of a notebook to see what data the notebook ingested, how it transformed it and where it outputted it. For something like SQL, it could be analyzing query logs to reverse engineer what mutation operations were executed and what they did. We support both push and pull based lineage depending on the needs.

+* **Classification** ΓÇô This enables Azure Purview to take physical samples from data sources and run them through our classification system. The classification system figures out the semantics of a piece of data. For example, we may know that a file is a Parquet file and has three columns and the third one is a string. But the classifiers we run on the samples will tell us that the string is a name, address, or phone number. Lighting up this integration point means that we have defined how Azure Purview can open up objects like notebooks, pipelines, parquet files, tables, and containers.

+* **Embedded Experience** ΓÇô Products that have a ΓÇ£studioΓÇ¥ like experience (such as ADF, Synapse, SQL Studio, PBI, and Dynamics) usually want to enable users to discover data they want to interact with and also find places to output data. Azure PurviewΓÇÖs catalog can help to accelerate these experiences by providing an embedding experience. This experience can occur at the API or the UX level at the partnerΓÇÖs option. By embedding a call to Azure Purview, the organization can take advantage of Azure PurviewΓÇÖs map of the data estate to find data assets, see lineage, check schemas, look at ratings, contacts etc.

+In this phase, Azure Purview must be created and configured for a very small set of users. Usually, it is just a group of 2-3 people working together to run through end-to-end scenarios. They are considered the advocates of Azure Purview in their organization. The main goal of this phase is to ensure key functionalities can be met and the right stakeholders are aware of the project.

+|Navigating Azure Purview|Understand how to use Azure Purview from the home page.|1 Day|

+|Search and browse|Allow end users to access Azure Purview and perform end-to-end search and browse scenarios.|1 Day|

+* Azure Purview account is created successfully in organization subscription under the organization tenant.

+* A small group of users with multiple roles can access Azure Purview.

+* Azure Purview is configured to scan at least one data source.

+* Users should be able to extract key values of Azure Purview such as:

+Once you have the agreed requirements and participated business units to onboard Azure Purview, the next step is to work on a Minimum Viable Product (MVP) release. In this phase, you will expand the usage of Azure Purview to more users who will have additional needs horizontally and vertically. There will be key scenarios that must be met horizontally for all users such as glossary terms, search, and browse. There will also be in-depth requirements vertically for each business unit or group to cover specific end-to-end scenarios such as lineage from Azure Data Lake Storage to Azure Synapse DW to Power BI.

+|[Create custom classifications and rules](create-a-custom-classification-and-classification-rule.md)|Once your assets are scanned, your users may realize that there are additional use cases for more classification beside the default classifications from Azure Purview.|2-4 Weeks|

+|[Import glossary terms](how-to-create-import-export-glossary.md)|In most cases, your organization may already develop a collection of glossary terms and term assignment to assets. This will require an import process into Azure Purview via .csv file.|1 Week|

+|Get classification and sensitive insights|For reporting and insight in Azure Purview, you can access this functionality to get various reports and provide presentation to management.|1 Day|

+|Onboard additional users using Azure Purview managed users|This step will require the Azure Purview Admin to work with the Azure Active Directory Admin to establish new Security Groups to grant access to Azure Purview.|1 Week|

+* Successfully onboard a larger group of users to Azure Purview (50+)

+Once the MVP phase has passed, itΓÇÖs time to plan for pre-production milestone. Your organization may decide to have a separate instance of Azure Purview for pre-production and production, or keep the same instance but restrict access. Also in this phase, you may want to include scanning on on-premises data sources such as SQL Server. If there is any gap in data sources not supported by Azure Purview, it is time to explore the Atlas API to understand additional options.

+|Understand firewall concept when scanning|This step requires some exploration of how the organization configures its firewall and how Azure Purview can authenticate itself to access the data sources for scanning.|1 Day|

+|Use Azure Purview REST API for integration scenarios|If you have requirements to integrate Azure Purview with other 3rd party technologies such as orchestration or ticketing system, you may want to explore REST API area.|1-4 Weeks|

+|Understand Azure Purview pricing|This step will provide the organization important financial information to make decision.|1-5 Days|

+1. Navigate to Azure Purview Policy management app using the left side panel.

+1. Navigate to the Azure Purview Policy management app using the left side panel.

+If the [Azure built-in roles](built-in-roles.md) don't meet the specific needs of your organization, you can create your own Azure custom roles. Just like built-in roles, you can assign custom roles to users, groups, and service principals at management group (in preview only), subscription and resource group scopes. Custom roles are stored in an Azure Active Directory (Azure AD) directory and can be shared across subscriptions. Each directory can have up to 5000 custom roles. Custom roles can be created using the Azure portal, Azure PowerShell, Azure CLI, or the REST API. This article describes how to create custom roles using the Azure portal.

+In Azure ML notebooks, the authentication defaults to using the credentials you used to authenticate to the Azure ML workspace.

+**Authenticate by using managed identity**

+Run the following code to authenticate to your Sentinel workspace.

+ # Get the default Microsoft Sentinel workspace details from msticpyconfig.yaml

+ # Connect to Microsoft Sentinel with our QueryProvider and config details

+Output similar to the following is displayed in your notebook:

+ :::image type="content" source="media/notebook-get-started/authorization-connected-workspace.png" alt-text="Screenshot that shows authentication to Azure that ends with a connected message.":::

+ - **SAP version 750 or later**: Install the SAP change request *NPLK900180*

+ - **SAP version 740**: Install the SAP change request *NPLK900179*

+- **SAP Basis versions 7.50 and higher**, install NPLK900180

+- **For lower versions**, install NPLK900179

+When a client sends a message, it usually wants to know whether the message has been properly transferred to and accepted by the broker or whether some sort of error occurred. This positive or negative acknowledgment settles the understanding of both the client and broker about the transfer state of the message. Therefore, it's referred to as a *settlement*.

+- To learn more about Service Bus messaging in general, see [Service Bus queues, topics, and subscriptions](service-bus-queues-topics-subscriptions.md)

+- West Europe

+You can enable Blob storage versioning to automatically maintain previous versions of an object. When blob versioning is enabled, you can access earlier versions of a blob to recover your data if it is modified or deleted.

+After versioning is disabled, modifying the current version creates a blob that is not a version. All subsequent updates to the blob will overwrite its data without saving the previous state. All existing versions persist as previous versions.

+Blob versions, like blob snapshots, are billed at the same rate as active data. How versions are billed depends on whether you have explicitly set the tier for the current or previous versions of a blob (or snapshots). For more information about blob tiers, see [Hot, Cool, and Archive access tiers for blob data](access-tiers-overview.md).

+If you have not explicitly set the blob tier for any versions of a blob, then you are charged for unique blocks or pages across all versions, and any snapshots it may have. Data that is shared across blob versions is charged only once. When a blob is updated, then data in the new current version diverges from the data stored in previous versions, and the unique data is charged per block or page.

+In scenario 3, the blob has been updated, but the version has not. Block 3 was replaced with block 4 in the current blob, but the previous version still reflects block 3. As a result, the account is charged for four blocks.

+In scenario 4, the current version has been completely updated and contains none of its original blocks. As a result, the account is charged for all eight unique blocks &mdash; four in the current version, and four combined in the two previous versions. This scenario can occur if you are writing to a blob with the [Put Blob](/rest/api/storageservices/put-blob) operation, because it replaces the entire contents of the blob.

+| When blob tier is set… | Then you are billed for... |

+| Explicitly on a version, whether current or previous | The full content length of that version. Versions that don't have an explicitly set tier are billed only for unique blocks.¹ |

+| To archive | The full content length of all versions and snapshots.¹. |

+When blob soft delete is enabled, all soft-deleted entities are billed at full content length. If you delete or overwrite a current version that has had its tier explicitly set, then any previous versions of the soft-deleted blob are billed at full content length. For more information about how blob versioning and soft delete work together, see [Blob versioning and soft delete](#blob-versioning-and-soft-delete).

+**Input validation** is a technique to use to protect the main query logic from malformed or unexpected events. The query is upgraded to explicitly process and check records so they can't break the main logic.

+To implement input validation, we add two initial steps to a query. We first make sure the schema submitted to the core business logic matches its expectations. We then triage exceptions, and optionally route invalid records into a secondary output.

+A query with input validation will be structured as follows:

+```SQL

+WITH preProcessingStage AS (

+ SELECT

+ -- Rename incoming fields, used for audit and debugging

+ field1 AS in_field1,

+ field2 AS in_field2,

+ ...

+ -- Try casting fields in their expected type

+ TRY_CAST(field1 AS bigint) as field1,

+ TRY_CAST(field2 AS array) as field2,

+ ...

+ FROM myInput TIMESTAMP BY myTimestamp

+),

+triagedOK AS (

+ SELECT -- Only fields in their new expected type

+ field1,

+ field2,

+ ...

+ FROM preProcessingStage

+ WHERE ( ... ) -- Clauses make sure that the core business logic expectations are satisfied

+),

+triagedOut AS (

+ SELECT -- All fields to ease diagnostic

+ *

+ FROM preProcessingStage

+ WHERE NOT (...) -- Same clauses as triagedOK, opposed with NOT

+)

+-- Core business logic

+SELECT

+ ...

+INTO myOutput

+FROM triagedOK

+...

+-- Audit output. For human review, correction, and manual re-insertion downstream

+SELECT

+ *

+INTO BlobOutput -- To a storage adapter that doesn't require strong typing, here blob/adls

+FROM triagedOut

+```

+To see a comprehensive example of a query set up with input validation, see the section: [Example of query with input validation](#example-of-query-with-input-validation).

+This article illustrates how to implement this technique.

+## Scenario: input validation for unreliable event producers

+A composite key is one that is composed of two or more columns that are together required to uniquely identify a record in a table. For example, in an Order table, both OrderNumber and ProductId may be required to uniquely identify a record.

+Currently, you can choose from 11 database templates in Azure Synapse Studio to start creating your lake database:

+ - **Agriculture**ΓÇè-ΓÇèfor companies engaged in growing crops, raising livestock, and dairy production.

+ - **Banking** - for companies that analyze banking data.

+ - **Energy & Commodity Trading**ΓÇè-ΓÇèfor traders of energy, commodities, or carbon credits.

+ - **Freight & Logistics**ΓÇè-ΓÇèfor companies that provide freight and logistics services.

+ - **Fund Management** - for companies that manage investment funds for investors.

+ - **Life Insurance & Annuities** - for companies that provide life insurance, sell annuities, or both.

+ - **Oil & Gas**ΓÇè-ΓÇèfor companies that are involved in various phases of the Oil & Gas value chain.

+ - **Property & Casualty Insurance** - for companies that provide insurance against risks to property and various forms of liability coverage.

+ - **Utilities**ΓÇè-ΓÇèfor gas, electric, and water utilities; power generators; and water desalinators.

+> Make sure you have [placed the sample data in the primary storage account](get-started-create-workspace.md#place-sample-data-into-the-primary-storage-account).

+1. In Synapse Studio, go to the **Develop** hub.

+1. Create a new notebook.

+1. Create a new code cell and paste the following code in that cell:

+1. Modify the load URI, so it references the sample file in your storage account according to the [abfss URI scheme](../storage/blobs/data-lake-storage-introduction-abfs-uri.md).

+ Title: "Azure Synapse Analytics security white paper: Access control"

+description: Use different approaches or a combination of techniques to control access to data with Azure Synapse Analytics.

+# Azure Synapse Analytics security white paper: Access control

+Depending on how the data has been modeled and stored, data governance and access control might require that developers and security administrators use different approaches, or combination of techniques, to implement a robust security foundation.

+Azure Synapse supports a wide range of capabilities to control who can access what data. These capabilities are built upon a set of advanced access control features, including:

+- [Object-level security](#object-level-security)

+- [Row-level security](#row-level-security)

+- [Column-level security](#column-level-security)

+- [Dynamic data masking](#dynamic-data-masking)

+- [Synapse role-based access control](#synapse-role-based-access-control)

+## Object-level security

+Every object in a dedicated SQL pool has associated permissions that can be granted to a principal. In the context of users and service accounts, that's how individual tables, views, stored procedures, and functions are secured. Object permissions, like SELECT, can be granted to user accounts (SQL logins, Azure Active Directory users or groups) and [database roles](/sql/relational-databases/security/authentication-access/database-level-roles?view=sql-server-ver15&preserve-view=true), which provides flexibility for database administrators. Further, permissions granted on tables and views can be combined with other access control mechanisms (described below), such as column-level security, row-level security, and dynamic data masking.

+In Azure Synapse, all permissions are granted to database-level users and roles. Additionally, any user granted the built-in [Synapse Administrator RBAC role](../security/synapse-workspace-synapse-rbac-roles.md) at the workspace level is automatically granted full access to all dedicated SQL pools.

+In addition to securing SQL tables in Azure Synapse, dedicated SQL pool (formerly SQL DW), serverless SQL pool, and Spark tables can be secured too. By default, users assigned to the **Storage Blob Data Contributor** role of data lakes connected to the workspace have READ, WRITE, and EXECUTE permissions on all Spark-created tables *when users interactively execute code in notebook*. It's called *Azure Active Directory (Azure AD) pass-through*, and it applies to all data lakes connected to the workspace. However, if the same user executes the same notebook *through a pipeline*, the workspace Managed Service Identity (MSI) is used for authentication. So, for the pipeline to successfully execute workspace MSI, it must also belong to the **Storage Blob Data Contributor** role of the data lake that's accessed.

+## Row-level security

+[Row-level security](/sql/relational-databases/security/row-level-security?view=azure-sqldw-latest&preserve-view=true) allows security administrators to establish and control fine grained access to specific table rows based on the profile of a user (or a process) running a query. Profile or user characteristics may refer to group membership or execution context. Row-level security helps prevent unauthorized access when users query data from the same tables but must see different subsets of data.

+> [!NOTE]

+> Row-level security is supported in Azure Synapse and dedicated SQL pool (formerly SQL DW), but it's not supported for Apache Spark pool and serverless SQL pool.

+## Column-level security

+[Column-level security](../sql-data-warehouse/column-level-security.md) allows security administrators to set permissions that limit who can access sensitive columns in tables. It's set at the database level and can be implemented without the need to change the design of the data model or application tier.

+> [!NOTE]

+> Column-level security is supported in Azure Synapse and dedicated SQL pool (formerly SQL DW), but it's not supported for Apache Spark pool and serverless SQL pool.

+## Dynamic data masking

+[Dynamic data masking](../../azure-sql/database/dynamic-data-masking-overview.md) allows security administrators to restrict sensitive data exposure by masking it on read to non-privileged users. It helps prevent unauthorized access to sensitive data by enabling administrators to determine how the data is displayed at query time. Based on the identity of the authenticated user and their group assignment in the SQL pool, a query returns either masked or unmasked data. Masking is always applied regardless of whether data is accessed directly from a table or by using a view or stored procedure.

+> [!NOTE]

+> Dynamic data masking is supported in Azure Synapse and dedicated SQL pool (formerly SQL DW), but it's not supported for Apache Spark pool and serverless SQL pool.

+## Synapse role-based access control

+Azure Synapse also includes [Synapse role-based access control (RBAC) roles](../security/synapse-workspace-understand-what-role-you-need.md) to manage different aspects of Synapse Studio. Leverage these built-in roles to assign permissions to users, groups, or other security principals to manage who can:

+- Publish code artifacts and list or access published code artifacts.

+- Execute code on Apache Spark pools and integration runtimes.

+- Access linked (data) services that are protected by credentials.

+- Monitor or cancel job executions, review job output and execution logs.

+## Next steps

+In the [next article](security-white-paper-authentication.md) in this white paper series, learn about authentication.

+ Title: "Azure Synapse Analytics security white paper: Authentication"

+description: Implement authentication mechanisms with Azure Synapse Analytics.

+# Azure Synapse Analytics security white paper: Authentication

+Authentication is the process of proving the user is who they claim to be. Authentication activities can be logged with [Azure SQL Auditing](../../azure-sql/database/auditing-overview.md), and an IT administrator can configure reports and alerts whenever a login from a suspicious location is attempted.

+## Benefits

+Some of the benefits of these robust authentication mechanisms include:

+- Strong password policies to deter brute force attacks.

+- User password encryption.

+- [Firewall rules](../../azure-sql/database/firewall-configure.md).

+- SQL endpoints with [Multi-factor authentication](../sql/mfa-authentication.md).

+- Elimination of the need to manage credentials with [managed identity](../../data-factory/data-factory-service-identity.md).

+Azure Synapse, dedicated SQL pool (formerly SQL DW), and serverless SQL pool currently support [Azure Active Directory](../../active-directory/fundamentals/active-directory-whatis.md) (Azure AD) authentication and [SQL authentication](../sql/sql-authentication.md), while Apache Spark pool supports only Azure AD authentication. Multi-factor authentication and managed identity are fully supported for Azure Synapse, dedicated SQL pool (formerly SQL DW), serverless SQL pool, and Apache Spark pool.

+## Next steps

+In the [next article](security-white-paper-network-security.md) in this white paper series, learn about network security.

+ Title: "Azure Synapse Analytics security white paper: Data protection"

+description: Protect data to comply with federal, local, and company guidelines with Azure Synapse Analytics.

+# Azure Synapse Analytics security white paper: Data protection

+## Data discovery and classification

+Organizations need to protect their data to comply with federal, local, and company guidelines to mitigate risks of data breach. One challenge organizations face is: *How do you protect the data if you don't know where it is?* Another is: *What level of protection is needed?*ΓÇöbecause some datasets require more protection than others.

+Imagine an organization with hundreds or thousands of files stored in their data lake, and hundreds or thousands of tables in their databases. It would benefit from a process that automatically scans every row and column of the file system or table and classifies columns as *potentially* sensitive data. This process is known as *data discovery*.

+Once the data discovery process is complete, it provides classification recommendations based on a predefined set of patterns, keywords, and rules. Someone can then review the recommendations and apply sensitivity-classification labels to appropriate columns. This process is known as *classification*.

+Azure Synapse provides two options for data discovery and classification:

+- [Data Discovery & Classification](../../azure-sql/database/data-discovery-and-classification-overview.md), which is built into Azure Synapse and dedicated SQL pool (formerly SQL DW).

+- [Azure Purview](https://azure.microsoft.com/services/purview/), which is a unified data governance solution that helps manage and govern on-premises, multicloud, and software-as-a-service (SaaS) data. It can automate data discovery, lineage identification, and data classification. By producing a unified map of data assets and their relationships, it makes data easily discoverable.

+> [!NOTE]

+> Azure Purview data discovery and classification is in public preview for Azure Synapse, dedicated SQL pool (formerly SQL DW), and serverless SQL pool. However, data lineage is currently not supported for Azure Synapse, dedicated SQL pool (formerly SQL DW), and serverless SQL pool. Apache Spark pool only supports [lineage tracking](../../purview/how-to-lineage-spark-atlas-connector.md).

+## Data encryption

+Data is encrypted at rest and in transit.

+### Data at rest

+By default, Azure Storage [automatically encrypts all data](../../storage/common/storage-service-encryption.md) using 256-bit Advanced Encryption Standard encryption (AES 256). It's one of the strongest block ciphers available and is FIPS 140-2 compliant. The platform manages the encryption key, and it forms the *first layer* of data encryption. This encryption applies to both user and system databases, including the **master** database.

+Enabling [Transparent Data Encryption](../../azure-sql/database/transparent-data-encryption-tde-overview.md) (TDE) can add a *second layer* of data encryption. It performs real-time I/O encryption and decryption of database files, transaction logs files, and backups at rest without requiring any changes to the application. By default, it uses AES 256.

+By default, TDE protects the database encryption key (DEK) with a built-in server certificate (service managed). There's an option to bring your own key (BYOK) that can be securely stored in [Azure Key Vault](../../key-vault/general/basic-concepts.md).

+Azure Synapse SQL serverless pool and Apache Spark pool are analytic engines that work directly on [Azure Data Lake Gen2](../../storage/blobs/data-lake-storage-introduction.md) (ALDS Gen2) or [Azure Blob Storage](../../storage/blobs/storage-blobs-introduction.md). These analytic runtimes don't have any permanent storage and rely on Azure Storage encryption technologies for data protection. By default, Azure Storage encrypts all data using [server-side encryption](../../storage/common/storage-service-encryption.md) (SSE). It's enabled for all storage types (including ADLS Gen2) and cannot be disabled. SSE encrypts and decrypts data transparently using AES 256.

+There are two SSE encryption options:

+- **Microsoft-managed keys:** Microsoft manages every aspect of the encryption key, including key storage, ownership, and rotations. It's entirely transparent to customers.

+- **Customer-managed keys:** In this case, the symmetric key used to encrypt data in Azure Storage is encrypted using a customer-provided key. It supports RSA and RSA-HSM (Hardware Security Modules) keys of sizes 2048, 3072, and 4096. Keys can be securely stored in [Azure Key Vault](../../key-vault/general/overview.md) or [Azure Key Vault Managed HSM](../../key-vault/managed-hsm/overview.md). It provides fine grain access control of the key and its management, including storage, backup, and rotations. For more information, see [Customer-managed keys for Azure Storage encryption](../../storage/common/customer-managed-keys-overview.md).

+While SSE forms the first layer of encryption, cautious customers can double encrypt by enabling a second layer of [256-bit AES encryption at the Azure Storage infrastructure layer](../../storage/common/storage-service-encryption.md#doubly-encrypt-data-with-infrastructure-encryption). Known as *infrastructure encryption*, it uses a platform-managed key together with a separate key from SSE. So, data in the storage account is encrypted twice; once at the service level and once at the infrastructure level with two different encryption algorithms and different keys.

+### Data in transit

+Azure Synapse, dedicated SQL pool (formerly SQL DW), and serverless SQL pool use the [Tabular Data Stream](/openspecs/windows_protocols/ms-tds/893fcc7e-8a39-4b3c-815a-773b7b982c50) (TDS) protocol to communicate between the SQL pool endpoint and a client machine. TDS depends on Transport Layer Security (TLS) for channel encryption, ensuring all data packets are secured and encrypted between endpoint and client machine. It uses a signed server certificate from the Certificate Authority (CA) used for TLS encryption, managed by Microsoft. Azure Synapse supports data encryption in transit with TLS v1.2, using AES 256 encryption.

+Azure Synapse leverages TLS to ensure data is encrypted in motion. SQL dedicated pools support TLS 1.0, TLS 1.1, and TLS 1.2 versions for encryption wherein Microsoft-provided drivers use TLS 1.2 by default. Serverless SQL pool and Apache Spark pool use TLS 1.2 for all outbound connections.

+## Next steps

+In the [next article](security-white-paper-access-control.md) in this white paper series, learn about access control.

+ Title: Azure Synapse Analytics security white paper

+description: Overview of the Azure Synapse Analytics security white paper series of articles.

+# Azure Synapse Analytics security white paper: Introduction

+**Summary:** [Azure Synapse Analytics](https://azure.microsoft.com/services/synapse-analytics/) is a Microsoft limitless analytics platform that integrates enterprise data warehousing and big data processing into a single managed environment with no system integration required. Azure Synapse provides the end-to-end tools for your analytic life cycle with:

+- [Pipelines](../../data-factory/concepts-pipelines-activities.md?context=/azure/synapse-analytics/context/context&amp;tabs=synapse-analytics) for data integration.

+- [Apache Spark pool](../spark/apache-spark-overview.md) for big data processing.

+- [Data Explorer](../data-explorer/data-explorer-overview.md) for log and time series analytics.

+- [Serverless SQL pool](../sql/on-demand-workspace-overview.md) for data exploration over [Azure Data Lake](https://azure.microsoft.com/solutions/data-lake/).

+- [Dedicated SQL pool](../sql-data-warehouse/sql-data-warehouse-overview-what-is.md?context=/azure/synapse-analytics/context/context) (formerly SQL DW) for enterprise data warehousing.

+- Deep integration with [Power BI](https://powerbi.microsoft.com/), [Azure Cosmos DB](../../cosmos-db/synapse-link.md?context=/azure/synapse-analytics/context/context), and [Azure Machine Learning](../machine-learning/what-is-machine-learning.md).

+Azure Synapse data security and privacy are non-negotiable. The purpose of this white paper, then, is to provide a comprehensive overview of Azure Synapse security features, which are enterprise-grade and industry-leading. The white paper comprises a series of articles that cover the following five layers of security:

+- Data protection

+- Access control

+- Authentication

+- Network security

+- Threat protection

+This white paper targets all enterprise security stakeholders. They include security administrators, network administrations, Azure administrators, workspace administrators, and database administrators.

+**Writers:** Vengatesh Parasuraman, Fretz Nuson, Ron Dunn, Khendr'a Reid, John Hoang, Nithesh Krishnappa, Mykola Kovalenko, Brad Schacht, Pedro Matinez, and Mark Pryce-Maher.

+**Technical Reviewers:** Nandita Valsan, Rony Thomas, Daniel Crawford, and Tammy Richter Jones.

+**Applies to:** Azure Synapse Analytics, dedicated SQL pool (formerly SQL DW), serverless SQL pool, and Apache Spark pool.

+> [!IMPORTANT]

+> This white paper does not apply to Azure SQL Database, Azure SQL Managed Instance, Azure Machine Learning, or Azure Databricks.

+## Introduction

+Frequent headlines of data breaches, malware infections, and malicious code injection are among an extensive list of security concerns for companies looking to cloud modernization. The enterprise customer requires a cloud provider or service solution that can address their concerns as they can't afford to get it wrong.

+Some common security questions include:

+- How can I control who can see what data?

+- What are the options for verifying a user's identity?

+- How is my data protected?

+- What network security technology can I use to protect the integrity, confidentiality, and access of my networks and data?

+- What are the tools that detect and notify me of threats?

+The purpose of this white paper is to provide answers to these common security questions, and many others.

+## Security layers

+Azure Synapse implements a multi-layered security architecture for end-to-end protection of your data. There are five layers:

+- [**Data protection**](security-white-paper-data-protection.md) to identify and classify sensitive data, and encrypt data at rest and in motion.

+- [**Access control**](security-white-paper-access-control.md) to determine a user's right to interact with data.

+- [**Authentication**](security-white-paper-authentication.md) to prove the identity of users and applications.

+- [**Network security**](security-white-paper-network-security.md) to isolate network traffic with private endpoints and virtual private networks.

+- [**Threat protection**](security-white-paper-threat-protection.md) to identify potential security threats, such as unusual access locations, SQL injection attacks, authentication attacks, and more.

+## Next steps

+In the [next article](security-white-paper-data-protection.md) in this white paper series, learn about data protection.

+ Title: "Azure Synapse Analytics security white paper: Network security"

+description: Manage secure network access with Azure Synapse Analytics.

+# Azure Synapse Analytics security white paper: Network security

+To secure Azure Synapse, there are a range of network security options to consider.

+## Network security terminology

+This opening section provides an overview and definitions of some of key Azure Synapse terms related to network security. Keep these definitions in mind while reading this article.

+### Synapse workspace

+A [*Synapse workspace*](../get-started-create-workspace.md) is a securable logical collection of all services offered by Azure Synapse. It includes dedicated SQL pools (formerly SQL DW), serverless SQL pools, Apache Spark pools, pipelines, and other services. Certain network configuration settings, such as IP firewall rules, managed virtual network, and approved tenants for exfiltration protection, are configured and secured at the workspace level.

+### Synapse workspace endpoints

+An endpoint is a point of an incoming connection to access a service. Each Synapse workspace has three distinct endpoints:

+- **Dedicated SQL endpoint** for accessing dedicated SQL pools.

+- **Serverless SQL endpoint** for accessing serverless SQL pools.

+- **Development endpoint** for accessing Apache Spark pools and pipeline resources in the workspace.

+These endpoints are automatically created when the Synapse workspace is created.

+### Synapse Studio

+[*Synapse Studio*](/learn/modules/explore-azure-synapse-studio/) is a secure web front-end development environment for Azure Synapse. It supports various roles, including the data engineer, data scientist, data developer, data analyst, and Synapse administrator.

+Use Synapse Studio to performing various data and management operations in Azure Synapse, such as:

+- Connecting to dedicated SQL pools, serverless SQL pools, and running SQL scripts.

+- Developing and running notebooks on Apache Spark pools.

+- Developing and running pipelines.

+- Monitoring dedicated SQL pools, serverless SQL pools, Apache Spark pools, and pipeline jobs.

+- Managing [Synapse RBAC permissions](../security/synapse-workspace-understand-what-role-you-need.md) of workspace items.

+- Creating [managed private endpoint connections](#managed-private-endpoint-connection) to data sources and sinks.

+Connections to workspace endpoints can be made using Synapse Studio. Also, it's possible to create [private endpoints](#private-endpoints) to ensure that communication to the workspace endpoints is private.

+## Public network access and firewall rules

+By default, the workspace endpoints are *public endpoints* when they're provisioned. Access to these workspace endpoints from any public network is enabled, including networks that are outside the customer's organization, without requiring a VPN connection or an ExpressRoute connection to Azure.

+All Azure services, including PaaS services like Azure Synapse, are protected by [DDoS basic protection](../../ddos-protection/ddos-protection-overview.md) to mitigate malicious attacks (active traffic monitoring, always on detection, and automatic attack mitigations).

+All traffic to workspace endpointsΓÇöeven via public networksΓÇöis encrypted and secured in transit by Transport Level Security (TLS) protocol.

+To protect any sensitive data, it's recommended to disable public access to the workspace endpoints entirely. By doing so, it ensures all workspace endpoints can only be accessed using [private endpoints](#private-endpoints).

+Disabling public access for all the Synapse workspaces in a subscription or a resource group is enforced by assigning an [Azure Policy](../../governance/policy/overview.md). It's also possible to disable public network access on per-workspace basis based on the sensitivity of data processed by the workspace.

+However, if public access needs to be enabled, it's highly recommended to configure the IP firewall rules to allow inbound connections only from the specified list of public IP addresses.

+Consider enabling public access when the on-premises environment doesn't have VPN access or ExpressRoute to Azure, and it requires access to the workspace endpoints. In this case, specify a list of public IP addresses of the on-premises data centers and gateways in the IP firewall rules.

+## Private endpoints

+An [Azure private endpoint](../../private-link/private-endpoint-overview.md) is a virtual network interface with a private IP address that's created in the customer's own [Azure Virtual Network](../../virtual-network/virtual-networks-overview.md) (VNet) subnet. A private endpoint can be created for any Azure service that supports private endpoints, such as Azure Synapse, dedicated SQL pools (formerly SQL DW), Azure SQL Databases, Azure Storage, or any service in Azure powered by [Azure Private Link service](../../private-link/private-link-service-overview.md).

+It's possible to create private endpoints in the VNet for all three Synapse workspace endpoints, individually. This way, there could be three private endpoints created for three endpoints of a Synapse workspace: one for dedicated SQL pool, one for serverless SQL pool, and one for the development endpoint.

+Private endpoints have many security benefits compared to the public endpoints. Private endpoints in an Azure VNet can be accessed only from within:

+- The same VNet that contains this private endpoint.

+- Regionally or globally [peered](../../virtual-network/virtual-network-peering-overview.md) Azure VNets.

+- On-premises networks connected to Azure via [VPN Gateway](../../vpn-gateway/vpn-gateway-about-vpngateways.md) or ExpressRoute.

+The main benefit of private endpoints is that it's no longer necessary to expose workspace endpoints to the public internet. *The less exposure, the better.*

+The following diagram depicts private endpoints.

+The above diagram depicts the following key points:

+| **Item** | **Description** |

+| | |

+|  | Workstations from within the customer VNet access the Azure Synapse private endpoints. |

+|  | Peering between customer VNet and another VNet. |

+|  | Workstation from peered VNet access the Azure Synapse private endpoints. |

+|  | On-premises network access the Azure Synapse private endpoints through VPN or ExpressRoute. |

+|  | Workspace endpoints are mapped into customer's VNet through private endpoints using Azure Private Link service. |

+|  | Public access is disabled on the Synapse workspace. |

+In the following diagram, a private endpoint is mapped to an instance of a PaaS resource instead of the entire service. In the event of a security incident within the network, only the mapped resource instance is exposed, minimizing the exposure and threat of data leakage and exfiltration.

+The above diagram depicts the following key points:

+| **Item** | **Description** |

+| | |

+|  | The private endpoint in the customer VNet is mapped to a single dedicated SQL pool (formerly SQL DW) endpoint in Workspace A. |

+|  | Other SQL pool endpoints in the other workspaces (B and C) aren't accessible through this private endpoint, minimizing exposure. |

+Private endpoint works across Azure Active Directory (Azure AD) tenants and regions, so it's possible to create private endpoint connections to Synapse workspaces across tenants and regions. In this case, it goes through the [private endpoint connection approval workflow](../../private-link/private-endpoint-overview.md#access-to-a-private-link-resource-using-approval-workflow). The resource owner controls which private endpoint connections are approved or denied. The resource owner is in full control of who can connect to their workspaces.

+The following diagram depicts a private endpoint connection approval workflow.

+The above diagram depicts the following key points:

+| **Item** | **Description** |

+| | |

+|  | Dedicated SQL pool (formerly SQL DW) in Workspace A in Tenant A is accessed by a private endpoint in the customer VNet in Tenant A. |

+|  | The same dedicated SQL pool (formerly SQL DW) in Workspace A in Tenant A is accessed by a private endpoint in the customer VNet in Tenant B through a connection approval workflow. |

+## Managed VNet

+The [Synapse Managed VNet](../security/synapse-workspace-managed-vnet.md) feature provides a fully managed network isolation for the Apache Spark pool and pipeline compute resources between Synapse workspaces. It can be configured at workspace creation time. In addition, it also provides network isolation for Spark clusters within the same workspace. Each workspace has its own virtual network, which is fully managed by Synapse. The Managed VNet isn't visible to the users to make any modifications. Any pipeline or Apache Spark pool compute resources that are spun up by Azure Synapse in a Managed VNet gets provisioned inside its own VNet. This way, there's full network isolation from other workspaces.

+This configuration eliminates the need to create and manage VNets and network security groups for the Apache Spark pool and pipeline resources, as is typically done by [VNet Injection](/azure/databricks/administration-guide/cloud-configurations/azure/vnet-inject).

+As such, multi-tenant services in a Synapse workspace, such as dedicated SQL pools and serverless SQL pools, are **not** provisioned inside the Managed VNet.

+The following diagram depicts network isolation between two Managed VNets of Workspaces A and B with their Apache Spark pools and pipeline resources inside the Managed VNets.

+## Managed private endpoint connection

+A [managed private endpoint connection](../security/synapse-workspace-managed-private-endpoints.md) enables connections to any Azure PaaS service (that supports Private Link), securely and seamlessly, without the need to create a private endpoint for that service from the customer's VNet. Synapse automatically creates and manages the private endpoint. These connections are used by the compute resources that are provisioned inside the Synapse Managed VNet, such as Apache Spark pools and pipeline resources, to connect to the Azure PaaS services *privately*.

+For example, if you want to connect to your Azure storage account *privately* from your pipeline, the usual approach is to create a private endpoint for the storage account and use a self-hosted integration runtime to connect to your storage private endpoint. With Synapse Managed VNets, you can privately connect to your storage account using Azure integration runtime simply by creating a managed private endpoint connection directly to that storage account. This approach eliminates the need to have a self-hosted integration runtime to connect to your Azure PaaS services privately.

+As such, multi-tenant services in a Synapse workspace, such as dedicated SQL pools and serverless SQL pools, are **not** provisioned inside the Managed VNet. So, they don't use the managed private endpoint connections created in the workspace for their outbound connectivity.

+The following diagram depicts a managed private endpoint connecting to an Azure storage account from a Managed VNet in Workspace A.

+## Advanced Spark security

+A Managed VNet also provides some added advantages for Apache Spark pool users. There's no need to worry about configuring a *fixed* subnet address space as would be done in [VNet Injection](/azure/databricks/administration-guide/cloud-configurations/azure/vnet-inject). Azure Synapse automatically takes care of allocating these address spaces dynamically for workloads.

+In addition, Spark pools operate as a job cluster. It means each user gets their own Spark cluster when interacting with the workspace. Creating an Spark pool within the workspace is metadata information for what will be assigned to the user when executing Spark workloads. It means each user will get their own Spark cluster *in a dedicated subnet inside the Managed VNet* to execute workloads. Spark pool sessions from the same user execute on the same compute resources. By providing this functionality, there are three main benefits:

+- Greater security due to workload isolation based on the user.

+- Reduction of noisy neighbors.

+- Greater performance.

+## Data exfiltration protection

+Synapse workspaces with Managed VNet have an additional security feature called *[data exfiltration protection](../security/workspace-data-exfiltration-protection.md)*. It protects all egress traffic going out from Azure Synapse from all services, including dedicated SQL pools, serverless SQL pools, Apache spark pools, and pipelines. It's configured by enabling data exfiltration protection at the workspace level (at workspace creation time) to restrict the outbound connections to an allowed list of Azure Active Directory (Azure AD) tenants. By default, only the home tenant of the workspace is added to the list, but it's possible to add or modify the list of Azure AD tenants anytime after the workspace is created. Adding additional tenants is a highly privileged operation that requires the elevated role of [Synapse Administrator](../security/synapse-workspace-synapse-rbac-roles.md). It effectively controls exfiltration of data from Azure Synapse to other organizations and tenants, without the need to have complicated network security policies in place.

+For workspaces with data exfiltration protection enabled, Synapse pipelines and Apache Spark pools must use managed private endpoint connections for all their outbound connections.

+Dedicated SQL pool and serverless SQL pool don't use managed private endpoints for their outbound connectivity; however, any outbound connectivity from SQL pools can only be made to the *approved targets*, which are the targets of managed private endpoint connections.

+## Private link hubs for Synapse Studio

+[Azure Private Link Hubs](../security/synapse-private-link-hubs.md) allows securely connecting to Synapse Studio from the customer's VNet using Azure Private Link. This feature is useful for customers who want to access the Synapse workspace using the Synapse Studio from a controlled and restricted environment, where the outbound internet traffic is restricted to a limited set of Azure services.

+It's achieved by creating a private link hub resource and a private endpoint to this hub from the VNet. This private endpoint is then used to access the studio using its fully qualified domain name (FQDN), *web.azuresynapse.net*, with a private IP address from the VNet. The private link hub resource downloads the static contents of Synapse Studio over Azure Private Link to the user's workstation. In addition, separate private endpoints must be created for the individual workspace endpoints to ensure that communication to the workspace endpoints is private.

+The following diagram depicts private link hubs for Synapse Studio.

+The above diagram depicts the following key points:

+| **Item** | **Description** |

+| | |

+|  | The workstation in a restricted customer VNet accesses the Synapse Studio using a web browser. |

+|  | A private endpoint created for private link hubs resource is used to download the static studio contents using Azure Private Link. |

+|  | Private endpoints created for Synapse workspace endpoints access the workspace resources securely using Azure Private Links. |

+|  | Network security group rules in the restricted customer VNet allow outbound traffic over port 443 to a limited set of Azure services, such as Azure Resource Manager, Azure Front Door, and Azure Active Directory. |

+|  | Network security group rules in the restricted customer VNet deny all other outbound traffic from the VNet. |

+|  | Public access is disabled on the Synapse workspace. |

+## Dedicated SQL pool (formerly SQL DW)

+Prior to the Azure Synapse offering, an Azure SQL data warehouse product named SQL DW was offered. It's now renamed as [dedicated SQL pool (formerly SQL DW)](../sql-data-warehouse/sql-data-warehouse-overview-what-is.md).

+Dedicated SQL pool (formerly SQL DW) is created inside a logical Azure SQL server. It's a securable logical construct that acts as a central administrative point for a collection of databases including SQL DW and other Azure SQL databases.

+Most of the core network security features discussed in the previous sections of this article for Azure Synapse are also applicable to dedicated SQL pool (formerly SQL DW). They include:

+> [!div class="checklist"]

+> - IP firewall rules

+> - Disabling public network access

+> - Private endpoints

+> - Data exfiltration protection through outbound firewall rules

+Since dedicated SQL pool (formerly SQL DW) is a multi-tenant service, it's not provisioned inside a Managed VNet. It means some of the features, such as Managed VNet and managed private endpoint connections, aren't applicable to it.

+## Network security feature matrix

+The following comparison table provides a high-level overview of network security features supported across the Azure Synapse offerings:

+| **Feature** | **Azure Synapse: Apache Spark pool** | **Azure Synapse: Dedicated SQL pool** | **Azure Synapse: Serverless SQL pool** | **Dedicated SQL pool (formerly SQL DW)** |

+| | :-: | :-: | :-: | :-: |

+| IP firewall rules | Yes | Yes | Yes | Yes |

+| Disabling public access | Yes | Yes | Yes | Yes |

+| Private endpoints | Yes | Yes | Yes | Yes |

+| Data exfiltration protection | Yes | Yes | Yes | Yes |

+| Secure access using Synapse Studio | Yes | Yes | Yes | No |

+| Access from restricted network using Synapse private link hub | Yes | Yes | Yes | No |

+| Managed VNet and workspace-level network isolation | Yes | N/A | N/A | N/A |

+| Managed private endpoint connections for outbound connectivity | Yes | N/A | N/A | N/A |

+| User-level network isolation | Yes | N/A | N/A | N/A |

+## Next steps

+In the [next article](security-white-paper-threat-protection.md) in this white paper series, learn about threat protection.

+ Title: "Azure Synapse Analytics security white paper: Threat detection"

+description: Audit, protect, and monitor Azure Synapse Analytics.

+# Azure Synapse Analytics security white paper: Threat detection

+Azure Synapse provides SQL Auditing, SQL Threat Detection, and Vulnerability Assessment to audit, protect, and monitor databases.

+## Auditing

+[Auditing for Azure SQL Database](../../azure-sql/database/auditing-overview.md#overview) and Azure Synapse tracks database events and writes them to an audit log in an Azure storage account, Log Analytics workspace, or Event Hubs. For any database, auditing is important. It produces an audit trail over time to help understand database activity and gain insight into discrepancies and anomalies that could indicate business concerns or suspected security violations.

+Used with [Data discovery and classification](../../azure-sql/database/data-discovery-and-classification-overview.md), when any sensitive columns or tables are queried by users, entries will appear in a field named **data_sensitivity_information** of the **sql_audit_information** table.

+> [!NOTE]

+> Azure SQL Auditing applies to Azure Synapse, dedicated SQL pool (formerly SQL DW), and serverless SQL pool, but it doesn't apply to Apache Spark pool.

+## Threat detection

+[Microsoft Defender for Cloud](../../defender-for-cloud/defender-for-cloud-introduction.md) is a tool for security posture management and threat detection. It protects workloads running in Azure, including (but not exclusively) servers, app service, key vaults, Kubernetes services, storage accounts, and Azure SQL Databases.

+As one of the options available with Microsoft Defender for Cloud, [Microsoft Defender for SQL](../../azure-sql/database/azure-defender-for-sql.md) extends Defender for Cloud's data security package to secure databases. It can discover and mitigate potential database vulnerabilities by detecting anomalous activities that could be a potential threat to the database. Specifically, it continually monitors your database for:

+> [!div class="checklist"]

+> - Potential SQL injection attacks

+> - Anomalous database access and queries

+> - Suspicious database activity

+Alert notifications include details of the incident, and recommendations on how to investigate and remediate threats.

+> [!NOTE]

+> Microsoft Defender for SQL applies to Azure Synapse and dedicated SQL pool (formerly SQL DW). It doesn't apply to serverless SQL pool or Apache Spark pool.

+## Vulnerability assessment

+[SQL vulnerability assessment](../../azure-sql/database/sql-vulnerability-assessment.md) is part of the Microsoft Defender for SQL offering. It continually monitors the data warehouse, ensuring that databases are always maintained at a high level of security and that organizational policies are met. It provides a comprehensive security report along with actionable remediation steps for each issue found, making it easy to proactively manage database security stature even if you're not a security expert.

+> [!NOTE]

+> SQL vulnerability assessment applies to Azure Synapse and dedicated SQL pool (formerly SQL DW). It doesn't apply to serverless SQL pool or Apache Spark pool.

+## Compliance

+For an overview of Azure compliance offerings, download the latest version of the [Microsoft Azure Compliance Offerings](https://azure.microsoft.com/resources/microsoft-azure-compliance-offerings/) document.

+## Next steps

+For more information related to this white paper, check out the following resources:

+- [Azure Synapse Analytics Blog](https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/bg-p/AzureSynapseAnalyticsBlog)

+- [Azure security baseline for Azure Synapse dedicated SQL pool (formerly SQL DW)](/security/benchmark/azure/baselines/synapse-analytics-security-baseline)

+- [Overview of the Azure Security Benchmark (v3)](/security/benchmark/azure/overview)

+- [Security baselines for Azure](/security/benchmark/azure/security-baselines-overview)

+|  |**Incorta**<br>Incorta enables organizations to go from raw data to quickly discovering actionable insights in Azure by automating the various data preparation steps typically required to analyze complex data. which. Using a proprietary technology called Direct Data Mapping and Incorta's Blueprints (pre-built content library and best practices captured from real customer implementations), customers experience unprecedented speed and simplicity in accessing, organizing, and presenting data and insights for critical business decision-making.|[Product page](https://www.incorta.com/solutions/microsoft-azure-synapse)<br>[Azure Marketplace](https://azuremarketplace.microsoft.com/marketplace/apps/incorta.incorta_direct_data_platform)<br>|

+ | Members |workspace1_SynapseAdministrators, workspace1_SynapseContributors, and workspace1_SynapseComputeOperators|

+| Scala | 2.12.10 |

+| Hadoop | 3.1.1 |

+VegasConnector-1.0.25.1_2.12.jar

+adal4j-1.6.3.jar

+azure-eventhubs-3.3.0.jar

+azure-eventhubs-spark_2.12-2.3.21.jar

+azure-synapse-ml-pandas_2.12-1.0.0.jar

+azure-synapse-ml-predict_2.12-1.0.jar

+commons-pool2-2.6.2.jar

+cosmos-analytics-spark-connector_3-1_2-12-assembly-3.0.4.jar

+cudf-21.10.0-cuda11.jar

+delta-core_2.12-1.0.0.2b.jar

+hadoop-annotations-3.1.1.5.0-50849917.jar

+hadoop-auth-3.1.1.5.0-50849917.jar

+hadoop-aws-3.1.1.5.0-50849917.jar

+hadoop-azure-3.1.1.5.0-50849917.jar

+hadoop-client-3.1.1.5.0-50849917.jar

+hadoop-common-3.1.1.5.0-50849917.jar

+hadoop-hdfs-client-3.1.1.5.0-50849917.jar

+hadoop-mapreduce-client-common-3.1.1.5.0-50849917.jar

+hadoop-mapreduce-client-core-3.1.1.5.0-50849917.jar

+hadoop-mapreduce-client-jobclient-3.1.1.5.0-50849917.jar

+hadoop-openstack-3.1.1.5.0-50849917.jar

+hadoop-yarn-api-3.1.1.5.0-50849917.jar

+hadoop-yarn-client-3.1.1.5.0-50849917.jar

+hadoop-yarn-common-3.1.1.5.0-50849917.jar

+hadoop-yarn-registry-3.1.1.5.0-50849917.jar

+hadoop-yarn-server-common-3.1.1.5.0-50849917.jar

+hadoop-yarn-server-web-proxy-3.1.1.5.0-50849917.jar

+javatuples-1.2.jar

+kafka-clients-2.4.1.5.0-50849917.jar

+libvegasjni.so

+mmlspark-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-cognitive-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-core-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-deep-learning-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-lightgbm-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-opencv-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+mmlspark-vw-1.0.0-rc3-179-327be83c-SNAPSHOT.jar

+notebook-utils-3.0.0-20211110.6.jar

+onnxruntime_gpu-1.8.1.jar

+proton-j-0.33.8.jar

+qpid-proton-j-extensions-1.2.4.jar

+rapids-4-spark_2.12-21.10.0.jar

+spark-avro_2.12-3.1.2.5.0-50849917.jar

+spark-catalyst_2.12-3.1.2.5.0-50849917.jar

+spark-cdm-connector-assembly-1.19.2.jar

+spark-core_2.12-3.1.2.5.0-50849917.jar

+spark-enhancement_2.12-3.1.2.5.0-50849917.jar

+spark-graphx_2.12-3.1.2.5.0-50849917.jar

+spark-hadoop-cloud_2.12-3.1.2.5.0-50849917.jar

+spark-hive-thriftserver_2.12-3.1.2.5.0-50849917.jar

+spark-hive_2.12-3.1.2.5.0-50849917.jar

+spark-kvstore_2.12-3.1.2.5.0-50849917.jar

+spark-launcher_2.12-3.1.2.5.0-50849917.jar

+spark-microsoft-telemetry_2.12-3.1.2.5.0-50849917.jar

+spark-microsoft-tools_2.12-3.1.2.5.0-50849917.jar

+spark-mllib-local_2.12-3.1.2.5.0-50849917.jar

+spark-mllib_2.12-3.1.2.5.0-50849917.jar

+spark-mssql-connector-1.2.0.jar

+spark-network-common_2.12-3.1.2.5.0-50849917.jar

+spark-network-shuffle_2.12-3.1.2.5.0-50849917.jar

+spark-repl_2.12-3.1.2.5.0-50849917.jar

+spark-sketch_2.12-3.1.2.5.0-50849917.jar

+spark-sql-kafka-0-10_2.12-3.1.2.5.0-50849917.jar

+spark-sql_2.12-3.1.2.5.0-50849917.jar

+spark-streaming-kafka-0-10-assembly_2.12-3.1.2.5.0-50849917.jar

+spark-streaming-kafka-0-10_2.12-3.1.2.5.0-50849917.jar

+spark-streaming_2.12-3.1.2.5.0-50849917.jar

+spark-tags_2.12-3.1.2.5.0-50849917.jar

+spark-token-provider-kafka-0-10_2.12-3.1.2.5.0-50849917.jar

+spark-unsafe_2.12-3.1.2.5.0-50849917.jar

+spark-yarn_2.12-3.1.2.5.0-50849917.jar

+spark_diagnostic_cli-1.0.10_spark-3.1.2.jar

+sqlanalyticsconnector_3.1.2-1.0.1.jar

+synapse-spark-telemetry_2.12-0.0.6.jar

+synfs-3.0.0-20211110.6.jar

+zookeeper-3.4.6.5.0-50849917.jar

+- [PyTorch](https://pytorch.org/) & [TensorFlow](https://www.tensorflow.org/) are powerful Python deep learning libraries. Within an Apache Spark pool in Azure Synapse Analytics, you can use these libraries to build single-machine models by setting the number of executors on your pool to zero. Even though Apache Spark is not functional under this configuration, it is a simple and cost-effective way to create single-machine models.

+string connectionString = TokenLibrary.GetSecret("<AZURE KEY VAULT NAME>", "<SECRET KEY>", "<LINKED SERVICE NAME>");

+To mount container **mycontainer**, mssparkutils need to check whether you have the permission to access the container at first, currently we support three authentication methods to trigger mount operation, **LinkedService**, **accountKey**, and **sastoken**.

+

+> [!NOTE]

+> For the newest release updates on Azure Synapse Analytics, including dedicated SQL pools, please refer to the [Azure Synapse Analytics blog](https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/bg-p/AzureSynapseAnalyticsBlog/label-name/Monthly%20Update), [What's new in Azure Synapse Analytics?](../whats-new.md), or the Synapse Studio homepage in the Azure portal.

+- [Create a dedicated SQL pool(formerly SQL DW)](create-data-warehouse-portal.md)

+- Increases the maximum number of concurrent queries and concurrency slots

+| **Tables** | [Yes](/sql/t-sql/statements/create-table-azure-sql-data-warehouse?view=azure-sqldw-latest&preserve-view=true) | No, the in-database tables are not supported. Serverless SQL pool can query only [external tables](develop-tables-external-tables.md?tabs=native) that reference data placed on [Azure Storage](#data-access) |

+| **Temporary tables** | [Yes](../sql-data-warehouse/sql-data-warehouse-tables-temporary.md?context=/azure/synapse-analytics/context/context) | Temporary tables might be used just to store some information from the system views, literals, or other temp tables. UPDATE/DELETE on temp table is also supported. You can join temp tables with the system views. You cannot select data from an external table to insert it into temp table or join temp table with external table - these operations will fail because external data and temp-tables cannot be mixed in the same query. |

+| **User defined procedures** | [Yes](/sql/t-sql/statements/create-procedure-transact-sql?view=azure-sqldw-latest&preserve-view=true) | Yes, stored procedures can be placed in any user databases (not `master` database). Procedures can just read external data and use [query language elements](#query-language) that are available in serverless pool. |

+| **External tables** | [Yes](/sql/t-sql/statements/create-external-table-transact-sql?view=azure-sqldw-latest&preserve-view=true). See supported [data formats](#data-formats). | Yes, [external tables](/sql/t-sql/statements/create-external-table-transact-sql?view=azure-sqldw-latest&preserve-view=true) are available. See the supported [data formats](#data-formats). |

+| **Caching queries** | Yes, multiple forms (SSD-based caching, in-memory, resultset caching). In addition, Materialized View are supported | No, only the file statistics are cached. |

+| **Table partitioning** | [Yes](../sql-data-warehouse/sql-data-warehouse-tables-partition.md?context=/azure/synapse-analytics/context/context). | External tables do not support partitioning. You can partition files using Hive-partition folder structure and create partitioned tables in Spark. The Spark partitioning will be [synchronized with the serverless pool](../metadat#partitioned-views) on folder partition structure, but external tables cannot be created on partitioned folders. |

+| **SELECT statement** | Yes. `SELECT` statement is supported, but some Transact-SQL query clauses, such as [FOR XML/FOR JSON](/sql/t-sql/queries/select-for-clause-transact-sql?view=azure-sqldw-latest&preserve-view=true), [MATCH](/sql/t-sql/queries/match-sql-graph?view=azure-sqldw-latest&preserve-view=true), OFFSET/FETCH are not supported. | Yes, `SELECT` statement is supported, but some Transact-SQL query clauses like [FOR XML](/sql/t-sql/queries/select-for-clause-transact-sql?view=azure-sqldw-latest&preserve-view=true), [MATCH](/sql/t-sql/queries/match-sql-graph?view=azure-sqldw-latest&preserve-view=true), [PREDICT](/sql/t-sql/queries/predict-transact-sql?view=azure-sqldw-latest&preserve-view=true), GROUPNG SETS, and query hints are not supported. |

+| **INSERT statement** | Yes | No, upload new data to Data lake using Spark or other tools. Use Cosmos DB with the analytical storage for highly transactional workloads. |

+| **UPDATE statement** | Yes | No, update Parquet/CSV data using Spark and the changes will be automatically available in serverless pool. Use Cosmos DB with the analytical storage for highly transactional workloads. |

+| **DELETE statement** | Yes | No, delete Parquet/CSV data using Spark and the changes will be automatically available in serverless pool. Use Cosmos DB with the analytical storage for highly transactional workloads.|

+| **MERGE statement** | Yes ([preview](/sql/t-sql/statements/merge-transact-sql?view=azure-sqldw-latest&preserve-view=true)) | No, merge Parquet/CSV data using Spark and the changes will be automatically available in serverless pool. |

+| **Types** | Yes, all Transact-SQL types except [cursor](/sql/t-sql/data-types/cursor-transact-sql?view=azure-sqldw-latest&preserve-view=true), [hierarchyid](/sql/t-sql/data-types/hierarchyid-data-type-method-reference?view=azure-sqldw-latest&preserve-view=true), [ntext, text, and image](/sql/t-sql/data-types/ntext-text-and-image-transact-sql?view=azure-sqldw-latest&preserve-view=true), [rowversion](/sql/t-sql/data-types/rowversion-transact-sql?view=azure-sqldw-latest&preserve-view=true), [Spatial Types](/sql/t-sql/spatial-geometry/spatial-types-geometry-transact-sql?view=azure-sqldw-latest&preserve-view=true), [sql\_variant](/sql/t-sql/data-types/sql-variant-transact-sql?view=azure-sqldw-latest&preserve-view=true), and [xml](/sql/t-sql/xml/xml-transact-sql?view=azure-sqldw-latest&preserve-view=true) | Yes, all Transact-SQL types are supported, except [cursor](/sql/t-sql/data-types/cursor-transact-sql?view=azure-sqldw-latest&preserve-view=true), [hierarchyid](/sql/t-sql/data-types/hierarchyid-data-type-method-reference?view=azure-sqldw-latest&preserve-view=true), [ntext, text, and image](/sql/t-sql/data-types/ntext-text-and-image-transact-sql?view=azure-sqldw-latest&preserve-view=true), [rowversion](/sql/t-sql/data-types/rowversion-transact-sql?view=azure-sqldw-latest&preserve-view=true), [Spatial Types](/sql/t-sql/spatial-geometry/spatial-types-geometry-transact-sql?view=azure-sqldw-latest&preserve-view=true), [sql\_variant](/sql/t-sql/data-types/sql-variant-transact-sql?view=azure-sqldw-latest&preserve-view=true), [xml](/sql/t-sql/xml/xml-transact-sql?view=azure-sqldw-latest&preserve-view=true), and Table type. See how to [map Parquet column types to SQL types here](develop-openrowset.md#type-mapping-for-parquet). |

+| **Cross-database queries** | No | Yes, 3-part-name references are supported including [USE](/sql/t-sql/language-elements/use-transact-sql?view=azure-sqldw-latest&preserve-view=true) statement. The queries can reference the serverless SQL databases or the Lake databases in the same workspace. |

+| **Built-in/system table-value functions** | Yes, [Transact-SQL Rowset functions](/sql/t-sql/functions/functions?view=azure-sqldw-latest&preserve-view=true#rowset-functions), except [OPENXML](/sql/t-sql/functions/openxml-transact-sql?view=azure-sqldw-latest&preserve-view=true), [OPENDATASOURCE](/sql/t-sql/functions/opendatasource-transact-sql?view=azure-sqldw-latest&preserve-view=true), [OPENQUERY](/sql/t-sql/functions/openquery-transact-sql?view=azure-sqldw-latest&preserve-view=true), and [OPENROWSET](/sql/t-sql/functions/openrowset-transact-sql?view=azure-sqldw-latest&preserve-view=true) | Yes, all [Transact-SQL Rowset functions](/sql/t-sql/functions/functions?view=azure-sqldw-latest&preserve-view=true#rowset-functions) are supported, except [OPENXML](/sql/t-sql/functions/openxml-transact-sql?view=azure-sqldw-latest&preserve-view=true), [OPENDATASOURCE](/sql/t-sql/functions/opendatasource-transact-sql?view=azure-sqldw-latest&preserve-view=true), and [OPENQUERY](/sql/t-sql/functions/openquery-transact-sql?view=azure-sqldw-latest&preserve-view=true) |

+| **Built-in/system aggregates** | Transact-SQL built-in aggregates except, except [CHECKSUM_AGG](/sql/t-sql/functions/checksum-agg-transact-sql?view=azure-sqldw-latest&preserve-view=true) and [GROUPING_ID](/sql/t-sql/functions/grouping-id-transact-sql?view=azure-sqldw-latest&preserve-view=true) | Yes, all Transact-SQL built-in [aggregates](/sql/t-sql/functions/aggregate-functions-transact-sql?view=sql-server-ver15) are supported. |

+| **DDL statements (CREATE, ALTER, DROP)** | Yes. All Transact-SQL DDL statement applicable to the supported object types | Yes, all Transact-SQL DDL statement applicable to the supported object types are supported. |

+| **Logins** | N/A (only contained users are supported in databases) | Yes, server-level Azure AD and SQL logins are supported. |

+| **Users** | N/A (only contained users are supported in databases) | Yes, database users are supported. |

+| **[Contained users](/sql/relational-databases/security/contained-database-users-making-your-database-portable?view=azure-sqldw-latest&preserve-view=true)** | Yes. **Note:** only one Azure AD user can be unrestricted admin | No, the contained users are not supported. |

+| **Storage Azure Active Directory (Azure AD) passthrough authentication** | Yes | Yes, [Azure AD passthrough authentication](develop-storage-files-storage-access-control.md?tabs=user-identity#supported-storage-authorization-types) is applicable to Azure AD logins. The identity of the Azure AD user is passed to the storage if a credential is not specified. Azure AD passthrough authentication is not available for the SQL users. |

+| **Storage [Managed Identity](../../data-factory/data-factory-service-identity.md?context=/azure/synapse-analytics/context/context&tabs=synapse-analytics) authentication** | Yes, using [Managed Service Identity Credential](../../azure-sql/database/vnet-service-endpoint-rule-overview.md?bc=%2fazure%2fsynapse-analytics%2fbreadcrumb%2ftoc.json&preserve-view=true&toc=%2fazure%2fsynapse-analytics%2ftoc.json&view=azure-sqldw-latest&preserve-view=true) | Yes, The query can access the storage using the workspace [Managed Identity](develop-storage-files-storage-access-control.md?tabs=managed-identity#database-scoped-credential) credential. |

+| **Storage Application identity/Service principal (SPN) authentication** | [Yes](/sql/t-sql/statements/create-external-data-source-transact-sql?view=azure-sqldw-latest&preserve-view=true) | Yes, you can create a [credential](develop-storage-files-storage-access-control.md?tabs=service-principal#database-scoped-credential) with a [service principal application ID](develop-storage-files-storage-access-control.md?tabs=service-principal#supported-storage-authorization-types) that will be used to authenticate on the storage. |

+| **Permissions - Schema-level** | Yes, including ability to GRANT, DENY, and REVOKE permissions to users/logins on the schema | Yes, you can specify schema-level permissions including ability to GRANT, DENY, and REVOKE permissions to users/logins on the schema |

+| **Permissions - Object-level** | Yes, including ability to GRANT, DENY, and REVOKE permissions to users | Yes, you can GRANT, DENY, and REVOKE permissions to users/logins on the system objects that are supported |

+| **Permissions - [Column-level security](../sql-data-warehouse/column-level-security.md?bc=%2fazure%2fsynapse-analytics%2fbreadcrumb%2ftoc.json&toc=%2fazure%2fsynapse-analytics%2ftoc.json)** | Yes | Yes, column-level security is supported in serverless SQL pools. |

+| **Built-in/system security &amp; identity functions** | Some Transact-SQL security functions and operators: `CURRENT_USER`, `HAS_DBACCESS`, `IS_MEMBER`, `IS_ROLEMEMBER`, `SESSION_USER`, `SUSER_NAME`, `SUSER_SNAME`, `SYSTEM_USER`, `USER`, `USER_NAME`, `EXECUTE AS`, `OPEN/CLOSE MASTER KEY` | Some Transact-SQL security functions and operators are supported: `CURRENT_USER`, `HAS_DBACCESS`, `HAS_PERMS_BY_NAME`, `IS_MEMBER', 'IS_ROLEMEMBER`, `IS_SRVROLEMEMBER`, `SESSION_USER`, `SESSION_CONTEXT`, `SUSER_NAME`, `SUSER_SNAME`, `SYSTEM_USER`, `USER`, `USER_NAME`, `EXECUTE AS`, and `REVERT`. Security functions cannot be used to query external data (store the result in variable that can be used in the query). |

+| **Advanced Threat Protection** | [Yes](../../azure-sql/database/threat-detection-overview.md) | No |

+| **[Firewall rules](../security/synapse-workspace-ip-firewall.md)**| Yes | Yes, the firewall rules can be set on serverless SQL endpoint. |

+| **[Private endpoint](../security/synapse-workspace-managed-private-endpoints.md)**| Yes | Yes, the private endpoint can be set on serverless SQL pool. |

+| **Synapse Studio** | Yes, SQL scripts | Yes, SQL scripts. Use SSMS or ADS instead of Synapse Studio if you are returning a large amount of data as a result. |

+| **Azure Data Studio** | Yes | [Yes](get-started-azure-data-studio.md), version 1.18.0 or higher. SQL scripts and SQL Notebooks are supported. |

+| **SQL Server Management Studio** | Yes | [Yes](get-started-ssms.md), version 18.5 or higher |

+## Data access

+| **[Delta Lake](https://delta.io/)** | No | [Yes](query-delta-lake-format.md), including files with [nested types](query-parquet-nested-types.md) |

+|catalogartifact.azureedge.net|443|Azure Marketplace|AzureFrontDoor.Frontend|

+## Share an existing disk

+To share an existing disk, or update how many VMs it can mount to, set the `maxShares` parameter with either the Azure PowerShell module or Azure CLI. You can also set `maxShares` to 1, if you want to disable sharing.

+> [!IMPORTANT]

+> The value of `maxShares` can only be set or changed when a disk is unmounted from all VMs. See the [Disk sizes](#disk-sizes) for the allowed values for `maxShares`.

+> Before detaching a disk, record the LUN ID for when you re-attach it.

+### PowerShell

+```azurepowershell

+$datadiskconfig = Get-AzDisk -DiskName "mySharedDisk"

+$datadiskconfig.maxShares = 3

+Update-AzDisk -ResourceGroupName 'myResourceGroup' -DiskName 'mySharedDisk' -Disk $datadiskconfig

+```

+### CLI

+```azurecli

+#Modifying a disk to enable or modify sharing configuration

+az disk update --name mySharedDisk --max-shares 5

+```

+- [RHEL 8.3 and above](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/deploying_red_hat_enterprise_linux_8_on_public_cloud_platforms/index?lb_target=production#azure-configuring-shared-block-storage_configuring-rhel-high-availability-on-azure)

+ - It may be possible to use RHEL 7 or an older version of RHEL 8 with shared disks, contact SharedDiskFeedback @microsoft.com

+This extension installs InfiniBand OFED drivers on InfiniBand and SR-IOV-enabled ('r' sizes) [H-series](../sizes-hpc.md) and [N-series](../sizes-gpu.md) VMs running Linux. Depending on the VM family, the extension installs the appropriate drivers for the Connect-X NIC. It does not install the InfiniBand ND drivers on the non-SR-IOV enabled [H-series](../sizes-hpc.md) and [N-series](../sizes-gpu.md) VMs.

+| Distribution | Version | InfiniBand NIC drivers |

+||||

+| Ubuntu | 16.04 LTS, 18.04 LTS, 20.04 LTS | CX3-Pro, CX5, CX6 |

+| CentOS | 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8.1, 8,2 | CX3-Pro, CX5, CX6 |

+| Red Hat Enterprise Linux | 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8.1, 8,2 | CX3-Pro, CX5, CX6 |

+For latest list of supported OS and driver versions, refer to [resources.json](https://github.com/Azure/azhpc-extensions/blob/master/InfiniBand/resources.json)

+For latest list of supported OS and driver versions, refer to [resources.json](https://github.com/Azure/azhpc-extensions/blob/master/InfiniBand/resources.json)

+For more details about subscription licenses that qualify to run Windows 10 on Azure, download the [Windows 10 licensing breif for Virtual Desktops](https://download.microsoft.com/download/3/D/4/3D42BDC2-6725-4B29-B75A-A5B04179958B/Licensing_brief_PLT_Windows_10_licensing_for_Virtual_Desktops.pdf)

+### <a name="connectivity"></a> Connectivity

+Because you can create multiple connection configurations using VPN Gateway, you need to determine which configuration best fits your needs. Point-to-Site, Site-to-Site, and coexisting ExpressRoute/Site-to-Site connections all have different instructions and configuration requirements. For connection diagrams and corresponding links to configuration steps, see [VPN Gateway design](design.md).

+* [Site-to-Site VPN connections](design.md#s2smulti)

+* [Point-to-Site VPN connections](design.md#P2S)

+* [VNet-to-VNet VPN connections](design.md#V2V)

+The following table can help you decide the best connectivity option for your solution. Note that ExpressRoute is not a part of VPN Gateway, but is included in the table.

+VPN gateways can be deployed in Azure Availability Zones. This brings resiliency, scalability, and higher availability to virtual network gateways. Deploying gateways in Azure Availability Zones physically and logically separates gateways within a region, while protecting your on-premises network connectivity to Azure from zone-level failures. See [About zone-redundant virtual network gateways in Azure Availability Zones](about-zone-redundant-vnet-gateways.md).

+> [!NOTE]

+> Modifying immutability policies is not supported from Storage Explorer.

+[19]: ./media/vs-azure-tools-storage-explorer-blobs/blob-container-open-editor-context-menu.png