+| Custom policy deployment | Azure AD B2C relies on caching to deliver performance to your end users. When you deploy a custom policy using whatever method, expect a delay of up to **30 minutes** for your users to see the changes. As a result of this behavior, consider the following practices when you deploy your custom policies: <br> - If you're deploying to a development environment, set the `DeploymentMode` attribute to `Development` in your custom policy file's `<TrustFrameworkPolicy>` element. <br> - Deploy your updated policy files to a production environment when traffic in your app is low. <br> - When you deploy to a production environment to update existing policy files, upload the updated files with new name(s), and then update your app reference to the new name(s). You can then remove the old policy files afterwards.<br> - You can set the `DeploymentMode` to `Development` in a production environment to bypass the caching behavior. However, we don't recommend this practice. If you [Collect Azure AD B2C logs with Application Insights](troubleshoot-with-application-insights.md), all claims sent to and from identity providers are collected, which is a security and performance risk. |

+- **refresh_token_lifetime_secs** Refresh token lifetimes (seconds). The default is 1,209,600 (14 days). The minimum is 86,400 (24 hours). The maximum is 7,776,000 (90 days).

+- Learn how to build [Resilience through developer best practices](../active-directory/fundamentals/resilience-b2c-developer-best-practices.md?bc=%2fazure%2factive-directory-b2c%2fbread%2ftoc.json&toc=%2fazure%2factive-directory-b2c%2fTOC.json).

+## Known issues

+Some customers may see the control to enable Voice call grayed out due to a licensing requirement, despite having a premium license. This is a known issue that we are actively working to fix.

+Azure AD supports the use of OATH-TOTP SHA-1 tokens that refresh codes every 30 or 60 seconds. Customers can purchase these tokens from the vendor of their choice. Hardware OATH tokens are available for users with an Azure AD Premium P1 or P2 license.

+| AADSTS50199 | CmsiInterrupt - For security reasons, user confirmation is required for this request. Because this is an "interaction_required" error, the client should do interactive auth. This occurs because a system webview has been used to request a token for a native application - the user must be prompted to ask if this was actually the app they meant to sign into. To avoid this prompt, the redirect URI should be part of the following safe list: <br />http://<br />https://<br />chrome-extension:// (desktop Chrome browser only) |

+| | [Windows Server 2019 and newer Virtual Machines running in Azure](howto-vm-sign-in-azure-ad-windows.md) (Server core isn't supported) |

+> [!NOTE]

+> Access Reviews for Service Principals requires an Entra Workload Identities Premium plan in addition to Azure AD Premium P2 license. You can view and acquire licenses on the [Workload Identities blade](https://portal.azure.com/#view/Microsoft_Azure_ManagedServiceIdentity/WorkloadIdentitiesBlade) in the Azure portal.

+The **Azure AD application activity (preview)** report shows the list of applications with one or more sign-in attempts. Any application activity during the selected date range appears in the report. The report allows you to sort by the number of successful sign-ins, failed sign-ins, and the success rate.

+It's possible that activity for a deleted application may appear in the report if the activity took place during the selected date range and before the application was deleted. Other scenarios could include a user attempting to sign in to an application that doesn't have a service principal associated with the app. For these types of scenarios, you may need to review the audit logs or sign-in logs to investigate further.

+Users with this role have permissions to manage compliance-related features in the Microsoft Purview compliance portal, Microsoft 365 admin center, Azure, and Office 365 Security & Compliance Center. Assignees can also manage all features within the Exchange admin center and create support tickets for Azure and Microsoft 365. More information is available at [About Microsoft 365 admin roles](https://support.office.com/article/About-Office-365-admin-roles-da585eea-f576-4f55-a1e0-87090b6aaa9d).

+ > [!NOTE]

+ > The Identifier value should match the entry in the Agiloft SAML Configuration Entity ID field. That field in Agiloft may need to be updated as follows:

+ > 1. Add https:// to the beginning.

+ > 1. If there are any spaces in the URL, replace each one with an underscore (_).

+| Name |Value |

+|`Content-Type`| `application/json`|

+Authorization: Bearer <token>

+ "includeQRCode": true,

+ "callback":ΓÇ»{

+ "url":ΓÇ»"https://wwww.contoso.com/vc/callback",

+ "state": "Aaaabbbb11112222",

+ "headers":ΓÇ»{

+ "api-key":ΓÇ»"an-api-key-can-go-here"

+ }

+ },

+ ...

+If there is an error with the request, an [error response](error-codes.md) will be returned and should be handled appropriately by the app.

+Create a cluster with Azure CNI Overlay. Use the argument `--network-plugin-mode` to specify that this is an overlay cluster. If the pod CIDR is not specified then AKS assigns a default space, viz. 10.244.0.0/16. Replace the values for the variables `clusterName`, `resourceGroup`, and `location`.

+```azurecli-interactive

+clusterName="myOverlayCluster"

+resourceGroup="myResourceGroup"

+location="westcentralus"

+az aks create -n $clusterName -g $resourceGroup --location $location --network-plugin azure --network-plugin-mode overlay --pod-cidr 192.168.0.0/16

+```

+[az-feature-show]: /cli/azure/feature#az-feature-show

+Run this commands to create a cluster with an overlay network and Cilium. Replace the values for `<clusterName>`, `<resourceGroupName>`, and `<location>`:

+The following are the results from the [CIS Kubernetes V1.24 Benchmark v1.0.0][cis-benchmark-kubernetes] recommendations on AKS. These are applicable to AKS 1.21.x through AKS 1.24.x.

+|1.1.1|Ensure that the API server pod specification file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.3|Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.5|Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.7|Ensure that the etcd pod specification file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.9|Ensure that the Container Network Interface file permissions are set to 600 or more restrictive|Not Scored|L1|N/A|

+|1.1.13|Ensure that the admin.conf file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.15|Ensure that the scheduler.conf file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.17|Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.1.20|Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|1.2.2|Ensure that the `--token-auth-file` parameter is not set|Scored|L1|Fail|

+|1.2.3|Ensure that `--DenyServiceExternalIPs` is not set|Scored|L1|Pass|

+|1.2.4|Ensure that the `--kubelet-client-certificate` and `--kubelet-client-key` arguments are set as appropriate|Scored|L1|Pass|

+|1.2.5|Ensure that the `--kubelet-certificate-authority` argument is set as appropriate|Scored|L1|Fail|

+|1.2.6|Ensure that the `--authorization-mode` argument is not set to AlwaysAllow|Scored|L1|Pass|

+|1.2.7|Ensure that the `--authorization-mode` argument includes Node|Scored|L1|Pass|

+|1.2.8|Ensure that the `--authorization-mode` argument includes RBAC|Scored|L1|Pass|

+|1.2.9|Ensure that the admission control plugin EventRateLimit is set|Not Scored|L1|Fail|

+|1.2.10|Ensure that the admission control plugin AlwaysAdmit is not set|Scored|L1|Pass|

+|1.2.11|Ensure that the admission control plugin AlwaysPullImages is set|Not Scored|L1|Fail|

+|1.2.12|Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used|Not Scored|L1|Fail|

+|1.2.13|Ensure that the admission control plugin ServiceAccount is set|Scored|L1|Pass|

+|1.2.14|Ensure that the admission control plugin NamespaceLifecycle is set|Scored|L1|Pass|

+|1.2.15|Ensure that the admission control plugin NodeRestriction is set|Scored|L1|Pass|

+|1.2.16|Ensure that the `--secure-port` argument is not set to 0|Scored|L1|Pass|

+|1.2.17|Ensure that the `--profiling` argument is set to false|Scored|L1|Pass|

+|1.2.18|Ensure that the `--audit-log-path` argument is set|Scored|L1|Pass|

+|1.2.19|Ensure that the `--audit-log-maxage` argument is set to 30 or as appropriate|Scored|L1|Equivalent Control|

+|1.2.20|Ensure that the `--audit-log-maxbackup` argument is set to 10 or as appropriate|Scored|L1|Equivalent Control|

+|1.2.21|Ensure that the `--audit-log-maxsize` argument is set to 100 or as appropriate|Scored|L1|Pass|

+|1.2.22|Ensure that the `--request-timeout` argument is set as appropriate|Scored|L1|Pass|

+|1.2.23|Ensure that the `--service-account-lookup` argument is set to true|Scored|L1|Pass|

+|1.2.24|Ensure that the `--service-account-key-file` argument is set as appropriate|Scored|L1|Pass|

+|1.2.25|Ensure that the `--etcd-certfile` and `--etcd-keyfile` arguments are set as appropriate|Scored|L1|Pass|

+|1.2.26|Ensure that the `--tls-cert-file` and `--tls-private-key-file` arguments are set as appropriate|Scored|L1|Pass|

+|1.2.27|Ensure that the `--client-ca-file` argument is set as appropriate|Scored|L1|Pass|

+|1.2.28|Ensure that the `--etcd-cafile` argument is set as appropriate|Scored|L1|Pass|

+|1.2.29|Ensure that the `--encryption-provider-config` argument is set as appropriate|Scored|L1|Depends on Environment|

+|1.2.30|Ensure that encryption providers are appropriately configured|Scored|L1|Depends on Environment|

+|1.2.31|Ensure that the API Server only makes use of Strong Cryptographic Ciphers|Not Scored|L1|Pass|

+|1.3.6|Ensure that the RotateKubeletServerCertificate argument is set to true|Scored|L2|Fail|

+|1.3.7|Ensure that the `--bind-address` argument is set to 127.0.0.1|Scored|L1|Equivalent Control|

+|1.4.2|Ensure that the `--bind-address` argument is set to 127.0.0.1|Scored|L1|Equivalent Control|

+|4.1.1|Ensure that the kubelet service file permissions are set to 600 or more restrictive|Scored|L1|Pass|

+|4.1.3|If a proxy kubeconfig file exists, ensure permissions are set to 600 or more restrictive|Scored|L1|N/A|

+|4.1.4|If a proxy kubeconfig file exists, ensure ownership is set to root:root|Scored|L1|N/A|

+|4.1.5|Ensure that the `--kubeconfig` kubelet.conf file permissions are set to 600 or more restrictive|Scored|L1|Pass|

+|4.1.6|Ensure that the `--kubeconfig` kubelet.conf file ownership is set to root:root|Scored|L1|Pass|

+|4.1.7|Ensure that the certificate authorities file permissions are set to 600 or more restrictive|Scored|L1|Pass|

+|4.1.9|If the kubelet config.yaml configuration file is being used, ensure permissions set to 600 or more restrictive|Scored|L1|Pass|

+|4.1.10|If the kubelet config.yaml configuration file is being used, ensure file ownership is set to root:root|Scored|L1|Pass|

+|4.2.9|Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture|Not Scored|L2|Pass|

+|4.2.10|Ensure that the `--tls-cert-file`and `--tls-private-key-file` arguments are set as appropriate|Scored|L1|Pass|

+|4.2.12|Ensure that the RotateKubeletServerCertificate argument is set to true|Scored|L1|Pass|

+ Title: Configure the Dapr extension for your Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes project

+description: Learn how to configure the Dapr extension specifically for your Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes project

+# Configure the Dapr extension for your Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes project

+Once you've [created the Dapr extension](./dapr.md), you can configure the [Dapr](https://dapr.io/) extension to work best for you and your project using various configuration options, like:

+- Limiting which of your nodes use the Dapr extension

+- Setting automatic CRD updates

+- Configuring the Dapr release namespace

+The extension enables you to set Dapr configuration options by using the `--configuration-settings` parameter. For example, to provision Dapr with high availability (HA) enabled, set the `global.ha.enabled` parameter to `true`:

+```azurecli

+az k8s-extension create --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=2"

+```

+> [!NOTE]

+> If configuration settings are sensitive and need to be protected, for example cert related information, pass the `--configuration-protected-settings` parameter and the value will be protected from being read.

+If no configuration-settings are passed, the Dapr configuration defaults to:

+```yaml

+ ha:

+ enabled: true

+ replicaCount: 3

+ disruption:

+ minimumAvailable: ""

+ maximumUnavailable: "25%"

+ prometheus:

+ enabled: true

+ port: 9090

+ mtls:

+ enabled: true

+ workloadCertTTL: 24h

+ allowedClockSkew: 15m

+```

+For a list of available options, see [Dapr configuration][dapr-configuration-options].

+## Limiting the extension to certain nodes

+In some configurations, you may only want to run Dapr on certain nodes. You can limit the extension by passing a `nodeSelector` in the extension configuration. If the desired `nodeSelector` contains `.`, you must escape them from the shell and the extension. For example, the following configuration will install Dapr to only nodes with `topology.kubernetes.io/zone: "us-east-1c"`:

+```azurecli

+az k8s-extension create --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=2" \

+--configuration-settings "global.nodeSelector.kubernetes\.io/zone: us-east-1c"

+```

+For managing OS and architecture, use the [supported versions](https://github.com/dapr/dapr/blob/b8ae13bf3f0a84c25051fcdacbfd8ac8e32695df/docker/docker.mk#L50) of the `global.daprControlPlaneOs` and `global.daprControlPlaneArch` configuration:

+```azurecli

+az k8s-extension create --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=2" \

+--configuration-settings "global.daprControlPlaneOs=linuxΓÇ¥ \

+--configuration-settings "global.daprControlPlaneArch=amd64ΓÇ¥

+```

+## Configure the Dapr release namespace

+You can configure the release namespace. The Dapr extension gets installed in the `dapr-system` namespace by default. To override it, use `--release-namespace`. Include the cluster `--scope` to redefine the namespace.

+```azurecli

+az k8s-extension create \

+--cluster-type managedClusters \

+--cluster-name dapr-aks \

+--resource-group dapr-rg \

+--name my-dapr-ext \

+--extension-type microsoft.dapr \

+--release-train stable \

+--auto-upgrade false \

+--version 1.9.2 \

+--scope cluster \

+--release-namespace dapr-custom

+```

+[Learn how to configure the Dapr release namespace if you already have Dapr installed](./dapr-migration.md).

+## Show current configuration settings

+Use the `az k8s-extension show` command to show the current Dapr configuration settings:

+```azurecli

+az k8s-extension show --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr

+```

+## Update configuration settings

+> [!IMPORTANT]

+> Some configuration options cannot be modified post-creation. Adjustments to these options require deletion and recreation of the extension, applicable to the following settings:

+> * `global.ha.*`

+> * `dapr_placement.*`

+>

+> HA is enabled enabled by default. Disabling it requires deletion and recreation of the extension.

+To update your Dapr configuration settings, recreate the extension with the desired state. For example, assume we've previously created and installed the extension using the following configuration:

+```azurecli-interactive

+az k8s-extension create --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=2"

+```

+To update the `dapr_operator.replicaCount` from two to three, use the following command:

+```azurecli-interactive

+az k8s-extension create --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=3"

+```

+## Set the outbound proxy for Dapr extension for Azure Arc on-premises

+If you want to use an outbound proxy with the Dapr extension for AKS, you can do so by:

+1. Setting the proxy environment variables using the [`dapr.io/env` annotations](https://docs.dapr.io/reference/arguments-annotations-overview/):

+ - `HTTP_PROXY`

+ - `HTTPS_PROXY`

+ - `NO_PROXY`

+1. [Installing the proxy certificate in the sidecar](https://docs.dapr.io/operations/configuration/install-certificates/).

+## Disable automatic CRD updates

+With Dapr version 1.9.2, CRDs are automatically upgraded when the extension upgrades. To disable this setting, you can set `hooks.applyCrds` to `false`.

+```azurecli

+az k8s-extension upgrade --cluster-type managedClusters \

+--cluster-name myAKSCluster \

+--resource-group myResourceGroup \

+--name dapr \

+--extension-type Microsoft.Dapr \

+--auto-upgrade-minor-version true \

+--configuration-settings "global.ha.enabled=true" \

+--configuration-settings "dapr_operator.replicaCount=2" \

+--configuration-settings "global.daprControlPlaneOs=linuxΓÇ¥ \

+--configuration-settings "global.daprControlPlaneArch=amd64ΓÇ¥ \

+--configuration-settings "hooks.applyCrds=false"

+```

+> [!NOTE]

+> CRDs are only applied in case of upgrades and are skipped during downgrades.

+## Meet network requirements

+The Dapr extension for AKS and Arc for Kubernetes requires outbound URLs on `https://:443` to function. In addition to the `https://mcr.microsoft.com/daprio` URL for pulling Dapr artifacts, verify you've included the [outbound URLs required for AKS or Arc for Kubernetes](../azure-arc/kubernetes/quickstart-connect-cluster.md#meet-network-requirements).

+## Next Steps

+Once you have successfully provisioned Dapr in your AKS cluster, try deploying a [sample application][sample-application].

+<!-- LINKS INTERNAL -->

+[deploy-cluster]: ./tutorial-kubernetes-deploy-cluster.md

+[az-feature-register]: /cli/azure/feature#az-feature-register

+[az-feature-list]: /cli/azure/feature#az-feature-list

+[az-provider-register]: /cli/azure/provider#az-provider-register

+[sample-application]: ./quickstart-dapr.md

+[k8s-version-support-policy]: ./supported-kubernetes-versions.md?tabs=azure-cli#kubernetes-version-support-policy

+[arc-k8s-cluster]: ../azure-arc/kubernetes/quickstart-connect-cluster.md

+[update-extension]: ./cluster-extensions.md#update-extension-instance

+[install-cli]: /cli/azure/install-azure-cli

+[dapr-migration]: ./dapr-migration.md

+[dapr-settings]: ./dapr-settings.md

+<!-- LINKS EXTERNAL -->

+[kubernetes-production]: https://docs.dapr.io/operations/hosting/kubernetes/kubernetes-production

+[building-blocks-concepts]: https://docs.dapr.io/developing-applications/building-blocks/

+[dapr-configuration-options]: https://github.com/dapr/dapr/blob/master/charts/dapr/README.md#configuration

+[sample-application]: https://github.com/dapr/quickstarts/tree/master/hello-kubernetes#step-2create-and-configure-a-state-store

+[dapr-security]: https://docs.dapr.io/concepts/security-concept/

+[dapr-deployment-annotations]: https://docs.dapr.io/operations/hosting/kubernetes/kubernetes-overview/#adding-dapr-to-a-kubernetes-deployment

+[dapr-oss-support]: https://docs.dapr.io/operations/support/support-release-policy/

+[dapr-supported-version]: https://docs.dapr.io/operations/support/support-release-policy/#supported-versions

+[dapr-troubleshooting]: https://docs.dapr.io/operations/troubleshooting/common_issues/

+[supported-cloud-regions]: https://azure.microsoft.com/global-infrastructure/services/?products=azure-arc

+- Learn more about [additional settings and preferences you can set on the Dapr extension][dapr-settings].

+[dapr-settings]: ./dapr-settings.md

+* Azure Key Vault with Firewall enabled to allow public access isn't supported because it blocks traffic from KMS plugin to the Key Vault.

+>

+> Once you rotate the key, the old key (key1) is still cached and shouldn't be deleted. If you want to delete the old key (key1) immediately, you need to rotate the key twice. Then key2 and key3 are cached, and key1 can be deleted without impacting existing cluster.

+The following query returns durations for each query end/refresh end event for a model database and server. If scaled out, the results are broken out by replica because the replica number is included in ServerName_s. Grouping by RootActivityId_g reduces the row count retrieved from the Azure Diagnostics REST API and helps stay within the limits as described in Log Analytics Rate limits.

+>[!NOTE]

+>When you use Update management, ensure that the execution policy for scripts is *RemoteSigned*.

+ Title: Runbook authoring using VS code in Azure Automation

+description: This article provides an overview authoring runbooks in Azure Automation using the visual studio code.

+# Runbook authoring through VS Code in Azure Automation

+This article explains about the Visual Studio extension that you can use to create and manage runbooks.

+Azure Automation provides a new extension from VS Code to create and manage runbooks. Using this extension, you can perform all runbook management operations such as, creating and editing runbooks, triggering a job, tracking recent jobs output, linking a schedule, asset management, and local debugging.

+## Prerequisites

+- An Azure account with an active subscription.ΓÇ»[Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- [Visual Studio Code](https://code.visualstudio.com/).

+- PowerShell modules and Python packages used by runbook must be locally installed on the machine to run the runbook locally.

+## Supported operating systems

+The test matrix includes the following operating systems:

+1. **Windows Server 2022** with Windows PowerShell 5.1 and PowerShell Core 7.2.7

+1. **Windows Server 2019** with Windows PowerShell 5.1 and PowerShell Core 7.2.7

+1. **macOS 11** with PowerShell Core 7.2.7

+1. **Ubuntu** 20.04 with PowerShell Core 7.2.7

+>[!NOTE]

+>- The extension should work anywhere in VS Code and it supports [PowerShell 7.2 or higher](https://learn.microsoft.com/powershell/scripting/install/PowerShell-Support-Lifecycle?view=powershell-7.3). For Windows PowerShell, only version 5.1 is supported.

+>- PowerShell Core 6 is end-of-life and not supported.

+## Key Features

+- **Simplified onboarding** ΓÇô You can sign in using an Azure account in a simple and secure way.

+- **Multiple languages** - Supports all Automation runtime stack such as PowerShell 5, PowerShell 7, Python 2, and Python 3 Runbooks.

+- **Supportability**- Supports test execution of job, publishing Automation job and triggering job in Azure and Hybrid workers. You can execute runbooks locally.

+- Supports Python positional parameters and PowerShell parameters to trigger job.

+- **Webhooks simplified** ΓÇô You can create a webhook, start a job through a webhook in simpler way. Also, support to link a schedule to a Runbook.

+- **Manage Automation Assets** ΓÇô You can perform create, update, and delete operation against assets including certificates, variables, credentials, and connections.

+- **View properties** ΓÇô You can view the properties and select Hybrid worker group to execute hybrid jobs and view the recent last 10 jobs executed.

+- **Debug locally** - You can debug the PowerShell scripts locally.

+- **Runbook comparison** - You can compare the local runbook to the published or the draft runbook copy.

+

+## Key Features of v1.0.8

+- **Local directory configuration settings** - You can define the working directory that you want to save runbooks locally.

+ - **Change Directory:Base Path** - You use the changed directory path when you reopen Visual Studio code IDE. To change the directory using the Command Palette, use **Ctrl+Shift+P -> select Change Directory**. To change the base path from extension configuration settings, select **Manage** icon in the activity bar on the left and go to **Settings > Extensions > Azure Automation > Directory:Base Path**.

+ - **Change Directory:Folder Structure** - You can change the local directory folder structure from *vscodeAutomation/accHash* to *subscription/resourceGroup/automationAccount*. Select **Manage** icon in the activity bar on the left and go to **Settings > Extensions > Azure Automation > Directory:Folder Structure**. You can change the default configuration setting from *vscodeAutomation/accHash* to *subscription/resourceGroupe/automationAccount* format.

+ >[!NOTE]

+ >If your automation account is integrated with source control you can provide the runbook folder path of your GitHub repo as the directory path. For example: changing directory to *C:\abc* would store runbooks in *C:\abc\vscodeAutomation..* or *C:\abc//subscriptionName//resourceGroupName//automationAccountName//runbookname.ps1*.

+- **Runbook management operations** - You can create runbook, fetch draft runbook, fetch published runbook, open local runbook in the editor, compare local runbook with a published or draft runbook copy, upload as draft, publish runbook, and delete runbook from your Automation account.

+- **Runbook execution operations** - You can run a local version of Automation jobs such as, Start Automation jobs, Start Automation test job, view job outputs and run local version of the PowerShell Runbook in debug mode by allowing you to add breakpoints in the script.

+ >[!NOTE]

+ > Currently, we support the use of internal cmdlets likeΓÇ»`Get-AutomationVariable` only with non-encrypted assets.

+

+- **Work with schedules, assets and webhooks** - You can view the properties of a schedule, delete schedule, link schedule to link a schedule to a runbook.

+- **Add webhook** - You can add a webhook to the runbook.

+- **Update properties of assets** - You can create, update, view properties of assets such as Certificates, Connections, Credentials, Variables and Deletion of assets from the extension.

+## Limitations

+Currently, the following features aren't supported:

+- Creation of new schedules.

+- Adding new Certificates in Assets.

+- Upload Modules (PowerShell and Python) packages from the extension.

+- Auto-sync of local runbooks to Azure Automation account. You will have to perform the operation to **Fetch** or **Publish** runbook.

+- Management of Hybrid worker groups.

+- Graphical runbook and workflows.

+- For Python, we don't provide any debug options. We recommend that you install any debugger extension in your Python script.

+- Currently, we support only the unencrypted assets in local run.

+## Next steps

+- For Runbook management operations and to test runbook and jobs, see [Use Azure Automation extension for Visual Studio Code](../automation/how-to/runbook-authoring-extension-for-vscode.md)

+ Title: Azure Automation extension for Visual Studio Code

+description: Learn how to use the Azure Automation extension for Visual Studio Code to author runbooks.

+# Use Azure Automation extension for Visual Studio Code

+This article explains about the Visual Studio that you can use to create and manage runbooks. You can perform all runbook management operations such as creating runbooks, editing runbook, triggering a job, tracking recent jobs outputs, linking a schedule, asset management and local debugging.

+## Prerequisites

+The following items are required for completing the steps in this article:

+- An Azure subscription. If you don't have an Azure subscription, create a

+ [free account](https://azure.microsoft.com/free/)

+- [Visual Studio Code](https://code.visualstudio.com).

+- PowerShell modules and Python packages used by runbook must be locally installed on the machine to run the runbook locally.

+## Install and configure the Azure Automation extension

+After you meet the prerequisites, you can install the [Azure Automation extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=azure-automation.vscode-azureautomation&ssr=false#overview) by

+following these steps:

+1. Open Visual Studio Code.

+1. From the menu bar, go to **View** > **Extensions**.

+1. In the search box, enter **Azure Automation**.

+1. Select **Azure Automation** from the search results, and then select **Install**.

+1. Select **Reload** when necessary.

+## Using the Azure Automation extension

+The extension simplifies the process of creating and editing runbooks. You can now test them locally without logging into the Azure portal. The various actions that you can perform are listed below:

+### Create a runbook

+To create a runbook in the Automation account. Follow these steps:

+1. Sign in to Azure from the Azure Automation extension.

+1. Select **Runbooks**

+1. Right click and select **Create Runbook** to create a new Runbook in the Automation account.

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/create-runbook-inline.png" alt-text="Screenshot on how to create runbook using the Azure Automation extension." lightbox="media/runbook-authoring-extension-for-vscode/create-runbook-expanded.png":::

+### Publish a runbook

+To publish a runbook in the Automation account. Follow these steps:

+1. In Automation account, select the runbook.

+1. Right click and select **Publish runbook** to publish the runbook.

+ A notification appears that the runbook is successfully published.

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/publish-runbook-inline.png" alt-text="Screenshot on how to publish runbook using the Azure Automation extension." lightbox="media/runbook-authoring-extension-for-vscode/publish-runbook-expanded.png":::

+

+### Run local version of Automation job

+To run local version of Automation job, follow these steps:

+1. In Automation account, select the runbook.

+1. Right click and select **Run Local** to run local version of the Automation job.

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/run-local-job-inline.png" alt-text="Screenshot on how to run local version of job using the Azure Automation extension." lightbox="media/runbook-authoring-extension-for-vscode/run-local-job-expanded.png":::

+### Run Automation job

+To run the Automation job, follow these steps:

+1. In Automation account, select the runbook.

+1. Right click and select **Start Automation job** to run the Automation job.

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/start-automation-job-inline.png" alt-text="Screenshot on how to run Automation job using the Azure Automation extension." lightbox="media/runbook-authoring-extension-for-vscode/start-automation-job-expanded.png":::

+### Add new webhook

+To add a webhook to the runbook, follow these steps:

+1. In Automation account, select the runbook.

+1. Right click and select **Add New Webhook**.

+1. Select and copy the Webhook URI.

+1. Use the command palette and select **Azure Automation Trigger Webhook**

+1. Paste the Webhook URI.

+

+ A notification appears that JobId is created successfully.

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/add-new-webhook-inline.png" alt-text="Screenshot that shows the notification after successfully adding a new webhook." lightbox="media/runbook-authoring-extension-for-vscode/add-new-webhook-expanded.png":::

+

+### Link a schedule

+1. In Automation account, go to **Schedules** and select your schedule.

+1. Go to **Runbooks**, select your runbook.

+1. Right click and select **Link Schedule** and confirm the schedule.

+1. In the drop-down select **Azure**

+

+ A notification appears that the schedule is linked.

+### Manage Assets

+1. In Automation account, go to **Assets** > **fx Variables**.

+1. Right click and select **Create or Update**.

+1. Provide a name in the text box.

+ A notification appears that the variable is created, you can view the new variable in **fx Variables** option.

+### Run local in debug mode

+1. In Automation account, go to **Runbooks** and select a runbook.

+1. In the edit pane, add the break point.

+1. Right click on the runbook and select **Run local in Debug Mode**.

+

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/run-local-debug-mode-inline.png" alt-text="Screenshot that shows the running of local runbook in debug mode." lightbox="media/runbook-authoring-extension-for-vscode/add-new-webhook-expanded.png":::

+### Compare local runbook

+1. In Automation account, go to **Runbooks** and select a runbook

+1. Right click on the runbook and select **Compare local runbook**.

+1. In the edit pane, you will see the information in two layouts - runbook copy and published/draft copy.

+ >[!NOTE]

+ >If the runbook is **InEdit** mode, you will have to select either the Compare Published content or Compare Draft content to compare.

+

+ :::image type="content" source="media/runbook-authoring-extension-for-vscode/compare-local-runbook-inline.png" alt-text="Screenshot that shows how to compare local runbook." lightbox="media/runbook-authoring-extension-for-vscode/compare-local-runbook-expanded.png":::

+## Next steps

+- For information on key features and limitations of Azure Automation extension see, [Runbook authoring through VS Code in Azure Automation](../automation-runbook-authoring.md)

+1. Open *Program.cs* and add Azure App Configuration as an extra configuration source by calling the `AddAzureAppConfiguration` method.

+1. Add a *Settings.cs* file at the root of your project directory. It defines a strongly typed `Settings` class for the configuration you're going to use. Replace the namespace with the name of your project.

+ Update *Program.cs* with the following code and add the `TestAppConfig` namespace at the beginning of the file.

+ using TestAppConfig;

+1. Open *Index.cshtml.cs* in the *Pages* directory, and update the `IndexModel` class with the following code. Add the `using Microsoft.Extensions.Options` namespace at the beginning of the file, if it's not already there.

+1. The output of the `dotnet run` command contains two URLs. Open a browser and navigate to either one of these URLs to access your application. For example: `https://localhost:5001`.

+ :::image type="content" source="./media/quickstarts/cloud-shell-web-preview.png" alt-text="Screenshot of Azure Cloud Shell. Locate Web Preview.":::

+ The web page looks like this:

+ :::image type="content" source="./media/quickstarts/aspnet-core-app-launch-local-navbar.png" alt-text="Screenshot of the browser.Launching quickstart app locally.":::

+> [Enable dynamic configuration](./enable-dynamic-configuration-aspnet-core.md)

+The following features and services are not available for Azure Arc-enabled SQL Managed Instance.

+| **SQL Server Agent** | SQL Server agent is supported but the following specific capabilities are not supported: Subsystems (CmdExec, PowerShell, Queue Reader, SSIS, SSAS, SSRS), Alerts, Managed Backup

+| [Network isolation](cache-private-link.md) |Γ£ö|Γ£ö|Γ£ö|Γ£ö|Γ£ö|

+This article shows you how to create C# functions that run on .NET 6 [in the same process as the Functions host](functions-dotnet-class-library.md). These _in-process_ C# functions are only supported on [Long Term Support (LTS)](https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-core) .NET versions, such as .NET 6. When creating your project, you can choose to instead create a function that runs on .NET 6 in an [isolated worker process](dotnet-isolated-process-guide.md). [Isolated worker process](dotnet-isolated-process-guide.md) supports both LTS and Standard Term Support (STS) versions of .NET. For more information, see [Supported versions](dotnet-isolated-process-guide.md#supported-versions) in the .NET Functions isolated worker process guide. There's also a [CLI-based version](create-first-function-cli-csharp.md) of this article.

+For data intensive binding operations, you may want to use a separate storage account. For more information, see [Storage account guidance](storage-considerations.md#storage-account-guidance).

+PUT https://management.azure.com/{MOResourceId}/providers/microsoft.insights/datacollectionruleassociations/{associationName}?api-version=2021-09-01-preview

+PUT https://management.azure.com/providers/Microsoft.Insights/monitoredObjects/{AADTenantId}/providers/microsoft.insights/datacollectionruleassociations/{associationName}?api-version=2021-09-01-preview

+### Using PowerShell for onboarding

+$requestURL = "https://management.azure.com/providers/microsoft.insights/providers/microsoft.authorization/roleassignments/$newguid`?api-version=2021-04-01-preview"

+Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method PUT -Body $body

+$Location = "eastus" #Use your own loacation

+$requestURL = "https://management.azure.com/providers/Microsoft.Insights/monitoredObjects/$TenantID`?api-version=2021-09-01-preview"

+$body = @"

+ "location":`"$Location`"

+"@

+$Respond = Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method PUT -Body $body -Verbose

+#See reference documentation https://learn.microsoft.com/en-us/rest/api/monitor/data-collection-rule-associations/create?tabs=HTTP

+$associationName = "assoc01" #You can define your custom associationname, must change the association name to a unique name, if you want to associate multiple DCR to monitored object

+$DCRName = "dcr-WindowsClientOS" #Your Data collection rule name

+$requestURL = "https://management.azure.com$RespondId/providers/microsoft.insights/datacollectionruleassociations/$associationName`?api-version=2021-09-01-preview"

+$body = @"

+ {

+ "properties": {

+ "dataCollectionRuleId": "/subscriptions/$SubscriptionID/resourceGroups/$ResourceGroup/providers/Microsoft.Insights/dataCollectionRules/$DCRName"

+ }

+ }

+"@

+Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method PUT -Body $body

+#(Optional example). Associate another DCR to Monitored Object

+#See reference documentation https://learn.microsoft.com/en-us/rest/api/monitor/data-collection-rule-associations/create?tabs=HTTP

+$associationName = "assoc02" #You must change the association name to a unique name, if you want to associate multiple DCR to monitored object

+$DCRName = "dcr-PAW-WindowsClientOS" #Your Data collection rule name

+$requestURL = "https://management.azure.com$RespondId/providers/microsoft.insights/datacollectionruleassociations/$associationName`?api-version=2021-09-01-preview"

+Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method PUT -Body $body

+#4. (Optional) Get all the associatation.

+$requestURL = "https://management.azure.com$RespondId/providers/microsoft.insights/datacollectionruleassociations?api-version=2021-09-01-preview"

+(Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method get).value

+### Using PowerShell for offboarding

+```PowerShell

+#This will remove the monitor object

+$TenantID = "xxxxxxxxx-xxxx-xxx" #Your Tenant ID

+$SubscriptionID = "xxxxxx-xxxx-xxxxx" #Your Subscription ID

+$ResourceGroup = "rg-yourResourseGroup" #Your resroucegroup

+Connect-AzAccount -Tenant $TenantID

+#Select the subscription

+Select-AzSubscription -SubscriptionId $SubscriptionID

+#Delete monitored object

+$requestURL = "https://management.azure.com/providers/Microsoft.Insights/monitoredObjects/$TenantID`?api-version=2021-09-01-preview"

+#Invoke-RestMethod -Uri $requestURL -Headers $AuthenticationHeader -Method Delete

+```

+### Common installation issues

+#### Not AAD joined

+Error message: "Tenant and device ids retrieval failed"

+1. Run the command `dsregcmd /status`. This should produce the output as `AzureAdJoined : YES` in the 'Device State' section. If not, join the device with an AAD tenant and try installation again.

+### Post installation/Operational issues

+Once the agent is installed successfully (i.e. you see the agent service running but don't see data as expected), you can follow standard troubleshooting steps listed here for [Windows VM](./azure-monitor-agent-troubleshoot-windows-vm.md) and [Windows Arc-enabled server](azure-monitor-agent-troubleshoot-windows-arc.md) respectively.

+¹The frequency of notifications for stateless metric alerts differs based on the alert rule's configured frequency:

+- **Alert frequency of less than 5 minutes**: While the condition continues to be met, a notification is sent somewhere between one and six minutes.

+- **Alert frequency of more than 5 minutes**: While the condition continues to be met, a notification is sent between the configured frequency and double the frequency. For example, for an alert rule with a frequency of 15 minutes, a notification is sent somewhere between 15 to 30 minutes.

+For more information, see [Using Azure Monitor Application Insights with Spring Boot](./java-spring-boot.md).

+ A semi-colon delimited list of types that you don't want to be subject to sampling. Recognized types are: [`Dependency`](data-model-dependency-telemetry.md), [`Event`](data-model-event-telemetry.md), [`Exception`](data-model-exception-telemetry.md), [`PageView`](data-model-pageview-telemetry.md), [`Request`](data-model-request-telemetry.md), [`Trace`](data-model-trace-telemetry.md). All telemetry of the specified types is transmitted; the types that aren't specified will be sampled.

+ A semi-colon delimited list of types that you do want to subject to sampling. Recognized types are: [`Dependency`](data-model-dependency-telemetry.md), [`Event`](data-model-event-telemetry.md), [`Exception`](data-model-exception-telemetry.md), [`PageView`](data-model-pageview-telemetry.md), [`Request`](data-model-request-telemetry.md), [`Trace`](data-model-trace-telemetry.md). The specified types will be sampled; all telemetry of the other types will always be transmitted.

+- [Azure Monitor Logs API](/rest/api/loganalytics/): Retrieve log data from the workspace from any REST API client. The API request includes a query that's run against Azure Monitor to determine the data to retrieve.

+*.aadcdn.msftauthimages.net

+*.aadcdn.msauthimages.net

+ useGlobalConfig: false

+For the descriptions of the task inputs, see [Azure CLI task](/azure/devops/pipelines/tasks/reference/azure-cli-v2). When using the task on air-gapped cloud, you must set the `useGlobalConfig` property of the task to `true`. The default value is `false`.

+When using the [Azure CLI task](/azure/devops/pipelines/tasks/reference/azure-cli-v2) on air-gapped cloud, you must set the `useGlobalConfig` property of the task to `true`. The default value is `false`. See [CI/CD with Azure Pipelines and Bicep files](./add-template-to-azure-pipelines.md) for an example.

+To learn more about Azure pricing, see [Azure pricing overview](https://azure.microsoft.com/pricing/). There, you can estimate your costs by using the [pricing calculator](https://azure.microsoft.com/pricing/calculator/). You also can go to the pricing details page for a particular service, for example, [Windows VMs](https://azure.microsoft.com/pricing/details/virtual-machines/Windows/). For tips to help manage your costs, see [Prevent unexpected costs with Azure billing and cost management](../../cost-management-billing/cost-management-billing-overview.md).

+description: Learn how to create Azure NetApp Files-based NFS datastores for Azure VMware Solution hosts.

+- Work with your Microsoft representative to ensure that the Azure VMware Solution private cloud and the Azure NetApp Files volumes are deployed within the same [availability zone](../availability-zones/az-overview.md#availability-zones).

+Azure VMware Solution is compatible with AS-Path Prepend for redundant ExpressRoute configurations with the caveat of not honoring the outbound path selection from Azure towards on-premises. If you're running two or more ExpressRoute paths between on-premises and Azure, and the listed [Prerequisites](#prerequisites) are not met, you may experience impaired connectivity or no connectivity between your on-premises networks and Azure VMware Solution. The connectivity issue is caused when Azure VMware Solution doesn't see the AS-Path Prepend and uses equal cost multi-pathing (ECMP) to send traffic towards your environment over both ExpressRoute circuits. That action causes issues with stateful firewall inspection.

+The tools include [Pivotal Operations Manager](https://docs.pivotal.io/ops-manager/2-10/install/), a web application that simplifies deployment and management of a Cloud Foundry foundation, and [Pivotal Apps Manager](https://docs.pivotal.io/application-service/2-7/console/https://docsupdatetracker.net/index.html), a web application for managing users and applications.

+This Language service unifies the following previously available Cognitive

+The Language service also provides several new features as well, which can either be:

+ [Named entity recognition](./named-entity-recognition/overview.md) is a pre-configured feature that categorizes entities (words or phrases) in unstructured text across several pre-defined category groups. For example: people, events, places, dates, [and more](./named-entity-recognition/concepts/named-entity-categories.md).

+ [Entity linking](./entity-linking/overview.md) is a pre-configured feature that disambiguates the identity of entities (words or phrases) found in unstructured text and returns links to Wikipedia.

+ [Custom text classification](./custom-text-classification/overview.md) enables you to build custom AI models to classify unstructured text documents into custom classes you define.

+ [Custom NER](custom-named-entity-recognition/overview.md) enables you to build custom AI models to extract custom entity categories (labels for words or phrases), using unstructured text that you provide.

+This tutorial will walk you through using the Azure OpenAI [embeddings](../concepts/understand-embeddings.md) API to perform **document search** where you'll query a knowledge base to find the most relevant document.

+ Title: Azure Communication Services - Call Recording summary logs

+description: Learn about the properties of summary logs for the Call Recording feature.

+# Call Recording summary logs

+In Azure Communication Services, summary logs for the Call Recording feature provide details about:

+- Call duration.

+- Media content (for example, audio/video, unmixed, or transcription).

+- Format types used for the recording (for example, WAV or MP4).

+- The reason why the recording ended.

+A recording file is generated at the end of a call or meeting. The recording can be initiated and stopped by either a user or an app (bot). It can also end because of a system failure.

+Summary logs are published after a recording is ready to be downloaded. The logs are published within the standard latency time for Azure Monitor resource logs. See [Log data ingestion time in Azure Monitor](../../../azure-monitor/logs/data-ingestion-time.md#azure-metrics-resource-logs-activity-log).

+## Properties

+| Property name | Data type | Description |

+|`timeGenerated`|DateTime|Time stamp (UTC) of when the log was generated.|

+|`operationName`|String|Operation associated with a log record.|

+|`correlationId`|String|ID that's used to correlate events between tables.|

+|`recordingID`|String|ID for the recording that this log refers to.|

+|`category`|String|Log category of the event. Logs with the same log category and resource type have the same property fields.|

+|`resultType`|String| Status of the operation.|

+|`level`|String |Severity level of the operation.|

+|`chunkCount`|Integer|Total number of chunks created for the recording.|

+|`channelType`|String|Channel type of the recording, such as mixed or unmixed.|

+|`recordingStartTime`|DateTime|Time that the recording started.|

+|`contentType`|String|Content of the recording, such as audio only, audio/video, or transcription.|

+|`formatType`|String|File format of the recording.|

+|`recordingLength`|Double|Duration of the recording in seconds.|

+|`audioChannelsCount`|Integer|Total number of audio channels in the recording.|

+|`recordingEndReason`|String|Reason why the recording ended.|

+## Call Recording and example data

+A call can have one recording or many recordings, depending on how many times a recording event is triggered.

+For example, if an agent initiates an outbound call on a recorded line and the call drops because of a poor network signal, `callid` will have one `recordingid` value. If the agent calls back the customer, the system generates a new `callid` instance and a new `recordingid` value.

+#### Example: Call Recording for one call to one recording

+If the agent initiates a recording and then stops and restarts the recording multiple times while the call is still on, `callid` will have many `recordingid` values, depending on how many times the recording events were triggered.

+#### Example: Call Recording for one call to many recordings

+## Next steps

+For more information about Call Recording, see [Call Recording overview](../../../communication-services/concepts/voice-video-calling/call-recording.md).

+\** Applications from all other subscription types will be reviewed and approved on a case-by-case bases. Please reach out to acstns@microsoft.com for assistance with your application.

+ Title: Connect to IBM MQ

+description: Connect to an MQ server on premises or in Azure from a workflow in Azure Logic Apps.

+This article shows how to access an MQ server that's either on premises or in Azure from a workflow in Azure Logic Apps with the MQ connector. You can then create automated workflows that receive and send messages stored in your MQ server. For example, your workflow can browse for a single message in a queue and then run other actions. The MQ connector includes a Microsoft MQ client that communicates with a remote MQ server across a TCP/IP network.

+## Supported IBM WebSphere MQ versions

+## Connector technical reference

+The MQ connector has different versions, based on [logic app type and host environment](../logic-apps/logic-apps-overview.md#resource-environment-differences).

+| Logic app | Environment | Connection version |

+|--|-|--|

+| **Consumption** | Multi-tenant Azure Logic Apps | Managed connector, which appears in the designer under the **Enterprise** label. This connector provides only actions, not triggers. For more information, review the following documentation: <br><br>- [MQ managed connector reference](/connectors/mq) <br>- [Managed connectors in Azure Logic Apps](managed.md) |

+| **Consumption** | Integration service environment (ISE) | Managed connector, which appears in the designer under the **Enterprise** label. For more information, review the following documentation: <br><br>- [MQ managed connector reference](/connectors/mq) <br>- [Managed connectors in Azure Logic Apps](managed.md) |

+| **Standard** | Single-tenant Azure Logic Apps and App Service Environment v3 (Windows plans only) | Managed connector, which appears in the designer under the **Azure** label, and built-in connector, which appears in the designer under the **Built-in** label and is service provider based. The built-in version differs in the following ways: <br><br>- The built-in version includes actions *and* triggers. <br><br>- The built-in version can connect directly to an MQ server and access Azure virtual networks. You don't need an on-premises data gateway. <br><br>- The built-in version supports Transport Layer Security (TLS) encryption for data in transit, message encoding for both the send and receive operations, and Azure virtual network integration when your logic app uses the Azure Functions Premium plan <br><br>For more information, review the following documentation: <br><br>- [MQ managed connector reference](/connectors/mq) <br>- [MQ built-in connector reference](/azure/logic-apps/connectors/built-in/reference/mq/) <br>- [Built-in connectors in Azure Logic Apps](built-in.md) |

+## Limitations

+* The MQ connector doesn't support segmented messages.

+* The MQ connector doesn't use the message's **Format** field and doesn't make any character set conversions. The connector only puts whatever data appears in the message field into a JSON message and sends the message along.

+For more information, review the [MQ managed connector reference](/connectors/mq) or the [MQ built-in connector reference](/azure/logic-apps/connectors/built-in/reference/mq/).

+## Prerequisites

+* An Azure account and subscription. If you don't have an Azure subscription, [sign up for a free Azure account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+* If you're using an on-premises MQ server, [install the on-premises data gateway](../logic-apps/logic-apps-gateway-install.md) on a server within your network. For the MQ connector to work, the server with the on-premises data gateway also must have .NET Framework 4.6 installed.

+ After you install the gateway, you must also create a data gateway resource in Azure. The MQ connector uses this resource to access your MQ server. For more information, review [Set up the data gateway connection](../logic-apps/logic-apps-gateway-connection.md).

+ > [!NOTE]

+ >

+ > You don't need the gateway in the following scenarios:

+ >

+ > * Your MQ server is publicly available or available in Azure.

+ > * You're going to use the MQ built-in connector, not the managed connector.

+* The logic app workflow where you want to access your MQ server.

+ * If you're using the MQ managed connector, which doesn't provide any triggers, make sure that your workflow already starts with a trigger or that you first add a trigger to your workflow. For example, you can use the [Recurrence trigger](../connectors/connectors-native-recurrence.md).

+ * If you're using a trigger from the MQ built-in connector, make sure that you start with a blank workflow.

+ * If you're using the on-premises data gateway, your logic app resource must use the same location as your gateway resource in Azure.

+<a name="add-trigger"></a>

+## Add an MQ trigger (Standard logic app only)

+The following steps apply only to Standard logic app workflows, which can use triggers provided by the MQ built-in connector. The MQ managed connector doesn't include any triggers.

+These steps use the Azure portal, but with the appropriate Azure Logic Apps extension, you can also use [Visual Studio Code](../logic-apps/create-single-tenant-workflows-visual-studio-code.md) to create a Standard logic app workflow.

+1. In the [Azure portal](https://portal.azure.com), open your blank logic app workflow in the designer.

+1. On the designer, select **Choose an operation**, if not already selected.

+1. Under the **Choose an operation** search box, select **Built-in**. In the search box, enter **mq**.

+1. From the triggers list, select the [MQ trigger](/azure/logic-apps/connectors/built-in/reference/mq/#triggers) that you want to use.

+1. Provide the [information to authenticate your connection](/azure/logic-apps/connectors/built-in/reference/mq/#authentication). When you're done, select **Create**.

+1. When the trigger information box appears, provide the required [information for your trigger](/azure/logic-apps/connectors/built-in/reference/mq/#triggers).

+1. When you're done, save your workflow. On the designer toolbar, select **Save**.

+<a name="add-action"></a>

+## Add an MQ action

+A Consumption logic app workflow can use only the MQ managed connector. However, a Standard logic app workflow can use the MQ managed connector and the MQ built-in connector. Each version has multiple actions. For example, both managed and built-in connector versions have their own actions to browse a message.

+* Managed connector actions: These actions run in a Consumption or Standard logic app workflow.

+* Built-in connector actions: These actions run only in a Standard logic app workflow.

+The following steps use the Azure portal, but with the appropriate Azure Logic Apps extension, you can also use the following tools to create logic app workflows:

+* Consumption logic app workflows: [Visual Studio](../logic-apps/quickstart-create-logic-apps-with-visual-studio.md) or [Visual Studio Code](../logic-apps/quickstart-create-logic-apps-visual-studio-code.md)

+* Standard logic app workflows: [Visual Studio Code](../logic-apps/create-single-tenant-workflows-visual-studio-code.md)

+### [Consumption](#tab/consumption)

+1. In the [Azure portal](https://portal.azure.com/), open your logic app workflow in the designer.

+1. In your workflow where you want to add an MQ action, follow one of these steps:

+ * To add an action under the last step, select **New step**.

+ * To add an action between steps, move your mouse over the connecting arrow so that the plus sign (**+**) appears. Select the plus sign, and then select **Add an action**.

+1. Under the **Choose an operation** search box, select **Enterprise**. In the search box, enter **mq**.

+1. From the actions list, select the [MQ action](/connectors/mq/#actions) that you want to use.

+1. Provide the [information to authenticate your connection](/connectors/mq/#creating-a-connection). When you're done, select **Create**.

+1. When the action information box appears, provide the required [information for your action](/connectors/mq/#actions).

+1. When you're done, save your workflow. On the designer toolbar, select **Save**.

+### [Standard](#tab/standard)

+The steps to add and use an MQ action differ based on whether your workflow uses the built-in connector or the managed, Azure-hosted connector.

+* [Built-in connector](#add-built-in-action): Describes the steps to add an action for the MQ built-in connector.

+* [Managed connector](#add-managed-action): Describes the steps to add an action for the MQ managed connector.

+<a name="add-built-in-action"></a>

+#### Add an MQ built-in connector action

+1. In the [Azure portal](https://portal.azure.com/), open your logic app workflow in the designer.

+1. In your workflow where you want to add an MQ action, follow one of these steps:

+ * To add an action under the last step, select the plus sign (**+**), and then select **Add an action**.

+ * To add an action between steps, select the plus sign (**+**) between those steps, and then select **Add an action**.

+1. On the **Add an action** pane, under the **Choose an operation** search box, select **Built-in**. In the search box, enter **mq**.

+1. From the actions list, select the [MQ action](/azure/logic-apps/connectors/built-in/reference/mq/#actions) that you want to use.

+1. Provide the [information to authenticate your connection](/azure/logic-apps/connectors/built-in/reference/mq/#authentication). When you're done, select **Create**.

+1. When the action information box appears, provide the required [information for your action](/azure/logic-apps/connectors/built-in/reference/mq/#actions).

+1. When you're done, save your workflow. On the designer toolbar, select **Save**.

+<a name="add-managed-action"></a>

+#### Add an MQ managed connector action

+1. In the [Azure portal](https://portal.azure.com/), open your logic app workflow in the designer.

+1. In your workflow where you want to add an MQ action, follow one of these steps:

+ * To add an action under the last step, select **New step**.

+ * To add an action between steps, move your mouse over the connecting arrow between those steps, select the plus sign (**+**) that appears between those steps, and then select **Add an action**.

+1. Under the **Choose an operation** search box, select **Azure**. In the search box, enter **mq**.

+1. From the actions list, select the [MQ action](/connectors/mq/#actions) that you want to use.

+1. Provide the [information to authenticate your connection](/connectors/mq/#creating-a-connection). When you're done, select **Create**.

+1. When the action information box appears, provide the required [information for your action](/connectors/mq/#actions).

+1. When you're done, save your workflow. On the designer toolbar, select **Save**.

+## Test your workflow

+To check that your workflow returns the results that you expect, run your workflow and then review the outputs from your workflow's run history.

+1. Run your workflow.

+ * Consumption logic app: On the workflow designer toolbar, select **Run Trigger** > **Run**.

+ * Standard logic app: On workflow resource menu, select **Overview**. On the **Overview** pane toolbar, select **Run Trigger** > **Run**.

+ After the run finishes, the designer shows the workflow's run history along with the status for each step.

+

+When your workflow tries connecting to your on-premises MQ server, you might get the following error:

+* [Managed connectors in Azure Logic Apps](/connectors/connector-reference/connector-reference-logicapps-connectors)

+* [Built-in connectors in Azure Logic Apps](built-in.md)

+> [!div class="nextstepaction"]

+> [Push image to Azure Container Registry](container-instances-tutorial-prepare-acr.md)

+az acr repository show --name <acrName> --repository azuredocs/azure-vote-front

+ Title: Push and pull OCI artifact references

+description: Push and pull Open Container Initiative (OCI) artifacts using a container registry in Azure

+# Push and pull OCI artifacts using an Azure container registry

+You can use an [Azure container registry][acr-landing] to store and manage [Open Container Initiative (OCI) artifacts](container-registry-image-formats.md#oci-artifacts) as well as Docker and OCI container images.

+To demonstrate this capability, this article shows how to use the [OCI Registry as Storage (ORAS)][oras-cli] CLI to push a sample artifact - a text file - to an Azure container registry. Then, pull the artifact from the registry. You can manage various OCI artifacts in an Azure container registry using different command-line tools appropriate to each artifact.

+* **Azure container registry** - Create a container registry in your Azure subscription. For example, use the [Azure portal](container-registry-get-started-portal.md) or [az acr create][az-acr-create].

+* **Azure CLI** - Version `2.29.1` or later is required. See [Install Azure CLI][azure-cli-install] for installation and/or upgrade.

+* **ORAS CLI** - Version `v0.16.0` is required. See: [ORAS installation][oras-install-docs].

+* **Docker (Optional)** - While Docker Desktop isn't required, the `oras` CLI utilizes the Docker desktop credential store for storing credentials. If Docker Desktop is installed, it must be running for `oras login`.

+## Configure a registry

+Configure environment variables to easily copy/paste commands into your shell. The commands can be run locally or in the [Azure Cloud Shell](https://shell.azure.com/).

+```bash

+ACR_NAME=myregistry

+REGISTRY=$ACR_NAME.azurecr.io

+```

+## Sign in to a registry

+Authenticate with your [individual Azure AD identity](container-registry-authentication.md?tabs=azure-cli#individual-login-with-azure-ad) using an AD token. Always use "000..." for the `USER_NAME` as the token is parsed through the `PASSWORD` variable.

+# Login to Azure

+# Login to ACR, using a token based on your Azure identity

+USER_NAME="00000000-0000-0000-0000-000000000000"

+PASSWORD=$(az acr login --name $ACR_NAME --expose-token --output tsv --query accessToken)

+> ACR and ORAS support multiple authentication options for users and system automation. This article uses individual identity, using an Azure token. For more authentication options see [Authenticate with an Azure container registry][acr-authentication]

+Provide the credentials to `oras login`.

+## Push a root artifact

+A root artifact is an artifact that has no `subject` parent. Root artifacts can be anything from a container image, a helm chart, a readme file for the repository. Reference artifacts, described in [Attach, push, and pull supply chain artifacts](container-registry-oras-artifacts.md) are artifacts that refer to another artifact. Reference artifacts can be anything from a signature, software bill of materials, scan report or other evolving types.

+For this example, create content that represents a markdown file:

+echo 'Readme Content' > readme.md

+The following step pushes the `readme.md` file to `<myregistry>.azurecr.io/samples/artifact:readme`.

+- The registry is identified with the fully qualified registry name `<myregistry>.azurecr.io` (all lowercase), followed by the namespace and repo: `/samples/artifact`.

+- The artifact is tagged `:readme`, to identify it uniquely from other artifacts listed in the repo (`:latest, :v1, :v1.0.1`).

+- Setting `--artifact-type readme/example` differentiates the artifact from a container image, which uses `application/vnd.oci.image.config.v1+json`.

+- The `./readme.md` identifies the file uploaded, and the `:application/markdown` represents the [IANA `mediaType`][iana-mediatypes] of the file.

+ For more information, see [OCI Artifact Authors Guidance](https://github.com/opencontainers/artifacts/blob/main/artifact-authors.md).

+Use the `oras push` command to push the file to your registry.

+**Linux, WSL2 or macOS**

+oras push $REGISTRY/samples/artifact:readme \

+ --artifact-type readme/example \

+ ./readme.md:application/markdown

+.\oras.exe push $REGISTRY/samples/artifact:readme ^

+ --artifact-type readme/example ^

+ .\readme.md:application/markdown

+Output for a successful push is similar to the following output:

+Uploading 2fdeac43552b readme.md

+Uploaded 2fdeac43552b readme.md

+Pushed <myregistry>.azurecr.io/samples/artifact:readme

+Digest: sha256:e2d60d1b171f08bd10e2ed171d56092e39c7bac1aec5d9dcf7748dd702682d53

+## Push a multi-file root artifact

+When OCI artifacts are pushed to a registry with ORAS, each file reference is pushed as a blob. To push separate blobs, reference the files individually, or collection of files by referencing a directory.

+For more information how to push a collection of files, see [Pushing artifacts with multiple files][oras-push-multifiles]

+Create some documentation for the repository:

+```bash

+echo 'Readme Content' > readme.md

+mkdir details/

+echo 'Detailed Content' > details/readme-details.md

+echo 'More detailed Content' > details/readme-more-details.md

+Push the multi-file artifact:

+**Linux, WSL2 or macOS**

+oras push $REGISTRY/samples/artifact:readme \

+ --artifact-type readme/example\

+ ./readme.md:application/markdown\

+ ./details

+**Windows**

+```cmd

+.\oras.exe push $REGISTRY/samples/artifact:readme ^

+ --artifact-type readme/example ^

+ .\readme.md:application/markdown ^

+ .\details

+## Discover the manifest

+To view the manifest created as a result of `oras push`, use `oras manifest fetch`:

+oras manifest fetch --pretty $REGISTRY/samples/artifact:readme

+The output will be similar to:

+```json

+{

+ "mediaType": "application/vnd.oci.artifact.manifest.v1+json",

+ "artifactType": "readme/example",

+ "blobs": [

+ {

+ "mediaType": "application/markdown",

+ "digest": "sha256:2fdeac43552b71eb9db534137714c7bad86b53a93c56ca96d4850c9b41b777fc",

+ "size": 15,

+ "annotations": {

+ "org.opencontainers.image.title": "readme.md"

+ }

+ },

+ {

+ "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",

+ "digest": "sha256:0d6c7434a34f6854f971487621426332e6c0fda08040b9e6cc8a93f354cee0b1",

+ "size": 189,

+ "annotations": {

+ "io.deis.oras.content.digest": "sha256:11eceb2e7ac3183ec9109003a7389468ec73ad5ceaec0c4edad0c1b664c5593a",

+ "io.deis.oras.content.unpack": "true",

+ "org.opencontainers.image.title": "details"

+ }

+ }

+ ],

+ "annotations": {

+ "org.opencontainers.artifact.created": "2023-01-10T14:44:06Z"

+ }

+}

+## Pull a root artifact

+Create a clean directory for downloading

+mkdir ./download

+Run the `oras pull` command to pull the artifact from your registry.

+```bash

+oras pull -o ./download $REGISTRY/samples/artifact:readme

+### View the pulled files

+tree ./download

+## Remove the artifact (optional)

+To remove the artifact from your registry, use the `oras manifest delete` command.

+```bash

+ oras manifest delete $REGISTRY/samples/artifact:readme

+* Learn about [Artifact References](container-registry-oras-artifacts.md), associating signatures, software bill of materials and other reference types

+* Learn more about [the ORAS Project](https://oras.land/), including how to configure a manifest for an artifact

+[iana-mediatypes]: https://www.rfc-editor.org/rfc/rfc6838

+[oras-install-docs]: https://oras.land/cli/

+[oras-cli]: https://oras.land/cli_reference/

+[oras-push-multifiles]: https://oras.land/cli/1_pushing/#pushing-artifacts-with-multiple-files

+[acr-landing]: https://aka.ms/acr

+[acr-authentication]: /azure/container-registry/container-registry-authentication?tabs=azure-cli

+[az-acr-create]: /azure/container-registry/container-registry-get-started-azure-cli

+[azure-cli-install]: /cli/azure/install-azure-cli

+Use an Azure container registry to store and manage a graph of supply chain artifacts, including signatures, software bill of materials (SBOM), security scan results and other types.

+To demonstrate this capability, this article shows how to use the [OCI Registry as Storage (ORAS)](https://oras.land) CLI to `push`, `discover` and `pull` a graph of supply chain artifacts to an Azure container registry.

+Storing individual (root) OCI Artifacts are covered in [Push and pull OCI artifacts](container-registry-oci-artifacts.md).

+To store a graph of artifacts, a reference to a `subject` artifact is defined using the [OCI Artifact Manifest][oci-artifact-manifest], which is part of the [pre-release OCI 1.1 Distribution specification][oci-1_1-spec].

+OCI 1.1 Artifact Manifest support is an ACR preview feature and subject to [limitations](#preview-limitations).

+* **Azure container registry** - Create a container registry in your Azure subscription. For example, use the [Azure portal](container-registry-get-started-portal.md) or the [Azure CLI][az-acr-create].

+*See [Preview limitations](#preview-limitations) for Azure cloud support.*

+* **Azure CLI** - Version `2.29.1` or later is required. See [Install Azure CLI][azure-cli-install] for installation and/or upgrade.

+* **ORAS CLI** - Version `v0.16.0` is required. See: [ORAS installation][oras-install-docs].

+* **Docker (Optional)** - To complete the walkthrough, a container image is referenced.

+You can use [Docker installed locally][docker-install] to build and push a container image, or use [`acr build`][az-acr-build] to build remotely in Azure.

+While Docker Desktop isn't required, the `oras` cli utilizes the Docker desktop credential store for storing credentials. If Docker Desktop is installed, it must be running for `oras login`.

+OCI Artifact Manifest support ([OCI 1.1 specification][oci-1_1-spec]) is available in all Azure public regions. Azure China and government clouds aren't yet supported.

+Configure environment variables to easily copy/paste commands into your shell. The commands can be run locally or in the [Azure Cloud Shell](https://shell.azure.com/).

+Authenticate with your [individual Azure AD identity](container-registry-authentication.md?tabs=azure-cli#individual-login-with-azure-ad) using an AD token. Always use "000..." for the `USER_NAME` as the token is parsed through the `PASSWORD` variable.

+# Login to Azure

+# Login to ACR, using a token based on your Azure identity

+USER_NAME="00000000-0000-0000-0000-000000000000"

+PASSWORD=$(az acr login --name $ACR_NAME --expose-token --output tsv --query accessToken)

+> ACR and ORAS support multiple authentication options for users and system automation. This article uses individual identity, using an Azure token. For more authentication options see [Authenticate with an Azure container registry][acr-authentication]

+Provide the credentials to `oras login`.

+This example associates a graph of artifacts to a container image.

+Build and push a container image, or skip this step if `$IMAGE` references an existing image in the registry.

+az acr build -r $ACR_NAME -t $IMAGE https://github.com/wabbit-networks/net-monitor.git#main

+echo '{"artifact": "'${IMAGE}'", "signature": "jayden hancock"}' > signature.json

+The `oras attach` command creates a reference between the file (`./signature.json`) to the `$IMAGE`. The `--artifact-type` provides for differentiating artifacts, similar to file extensions that enable different file types. One or more files can be attached by specifying `[file]:[mediaType]`.

+ --artifact-type signature/example \

+ ./signature.json:application/json

+When OCI artifacts are pushed to a registry with ORAS, each file reference is pushed as a blob. To push separate blobs, reference the files individually, or collection of files by referencing a directory.

+For more information how to push a collection of files, see [Pushing artifacts with multiple files][oras-push-multifiles]

+Create some documentation around an artifact:

+mkdir details/

+echo 'Detailed Content' > details/readme-details.md

+echo 'More detailed Content' > details/readme-more-details.md

+Attach the multi-file artifact as a reference to `$IMAGE`:

+**Linux, WSL2 or macOS**

+ --artifact-type readme/example \

+ ./details

+```

+**Windows**

+```cmd

+.\oras.exe attach $IMAGE ^

+ --artifact-type readme/example ^

+ .\readme.md:application/markdown ^

+ .\details

+The [OCI v1.1 Specification][oci-spec] defines a [referrers API][oci-artifact-referrers] for discovering references to a `subject` artifact. The `oras discover` command can show the list of references to the container image.

+Γöé ΓööΓöÇΓöÇ sha256:555ea91f39e7fb30c06f3b7aa483663f067f2950dcb...

+The OCI v1.1 Specification enables deep graphs, enabling signed software bill of materials (SBOM) and other artifact types.

+### Create a sample SBOM

+### Attach a sample SBOM to the image in the registry

+**Linux, WSL2 or macOS**

+ --artifact-type sbom/example \

+ ./sbom.json:application/json

+**Windows**

+```cmd

+.\oras.exe attach $IMAGE ^

+ --artifact-type sbom/example ^

+ ./sbom.json:application/json

+```

+### Sign the SBOM

+Artifacts that are pushed as references, typically don't have tags as they're considered part of the `subject` artifact. To push a signature to an artifact that is a child of another artifact, use the `oras discover` with `--artifact-type` filtering to find the digest.

+Create a signature of an SBOM

+echo '{"artifact": "'$IMAGE@$SBOM_DIGEST'", "signature": "jayden hancock"}' > sbom-signature.json

+### Attach the SBOM signature

+Γö£ΓöÇΓöÇ sbom/example

+│   └── sha256:4f1843833c029ecf0524bc214a0df9a5787409fd27bed2160d83f8cc39fedef5

+│   └── signature/example

+│   └── sha256:3c43b8cb0c941ec165c9f33f197d7f75980a292400d340f1a51c6b325764aa93

+│   └── sha256:5fafd40589e2c980e2864a78818bff51ee641119cf96ebb0d5be83f42aa215af

+ΓööΓöÇΓöÇ signature/example

+ ΓööΓöÇΓöÇ sha256:00da2c1c3ceea087b16e70c3f4e80dbce6f5b7625d6c8308ad095f7d3f6107b5

+```

+## Promote the graph

+A typical DevOps workflow will promote artifacts from dev through staging, to the production environment

+Secure supply chain workflows promote public content to privately secured environments.

+In either case you'll want to promote the signatures, SBOMs, scan results and other related artifact with the root artifact to have a complete graph of dependencies.

+Using the [`oras copy`][oras-cli] command, you can promote a filtered graph of artifacts across registries.

+Copy the `net-monitor:v1` image, and it's related artifacts to `sample-staging/net-monitor:v1`:

+```bash

+TARGET_REPO=$REGISTRY/sample-staging/$REPO

+oras copy -r $IMAGE $TARGET_REPO:$TAG

+```

+The output of `oras copy`:

+```console

+Copying 6bdea3cdc730 sbom-signature.json

+Copying 78e159e81c6b sbom.json

+Copied 6bdea3cdc730 sbom-signature.json

+Copied 78e159e81c6b sbom.json

+Copying 7cf1385c7f4d signature.json

+Copied 7cf1385c7f4d signature.json

+Copying 3e797ecd0697 details

+Copying 2fdeac43552b readme.md

+Copied 3e797ecd0697 details

+Copied 2fdeac43552b readme.md

+Copied demo42.myregistry.io/net-monitor:v1 => myregistry.azurecr.io/sample-staging/net-monitor:v1

+Digest: sha256:ff858b2ea3cdf4373cba65d2ca6bcede4da1d620503a547cab5916614080c763

+```

+## Discover the promoted artifact graph

+```bash

+oras discover -o tree $TARGET_REPO:$TAG

+```

+Output of `oras discover`:

+```console

+myregistry.azurecr.io/sample-staging/net-monitor:v1

+Γö£ΓöÇΓöÇ sbom/example

+│   └── sha256:4f1843833c029ecf0524bc214a0df9a5787409fd27bed2160d83f8cc39fedef5

+│   └── signature/example

+│   └── sha256:3c43b8cb0c941ec165c9f33f197d7f75980a292400d340f1a51c6b325764aa93

+Γö£ΓöÇΓöÇ readme/example

+│   └── sha256:5fafd40589e2c980e2864a78818bff51ee641119cf96ebb0d5be83f42aa215af

+ΓööΓöÇΓöÇ signature/example

+ ΓööΓöÇΓöÇ sha256:00da2c1c3ceea087b16e70c3f4e80dbce6f5b7625d6c8308ad095f7d3f6107b5

+To pull a specific referenced artifact, the digest of reference is discovered with the `oras discover` command:

+ $TARGET_REPO:$TAG | jq -r ".manifests[0].digest")

+oras pull -o ./download $TARGET_REPO@$DOC_DIGEST

+tree ./download

+```

+The output of `tree`:

+```output

+./download

+Γö£ΓöÇΓöÇ details

+│   ├── readme-details.md

+│   └── readme-more-details.md

+ΓööΓöÇΓöÇ readme.md

+The OCI Artifact Manifest enables artifact graphs to be pushed, discovered, pulled and copied without having to assign tags. Artifact manifests enable a tag listing to focus on the artifacts users think about, as opposed to the signatures and SBOMs that are associated with the container images, helm charts and other artifacts.

+```bash

+oras repo tags $REGISTRY/$REPO

+A repository can have a list of manifests that are both tagged and untagged. Using the [`az acr manifest`][az-acr-manifest-metadata] CLI, view the full list of manifests:

+Note the container image manifests have `"tags"`, while the reference types (`"mediaType": "application/vnd.oci.artifact.manifest.v1+json"`) don't.

+In the output, the signature is untagged, but tracked as a `oci.artifact.manifest` reference to the container image:

+ "createdTime": "2023-01-10T17:58:28.4403142Z",

+ "digest": "sha256:00da2c1c3ceea087b16e70c3f4e80dbce6f5b7625d6c8308ad095f7d3f6107b5",

+ "imageSize": 80,

+ "lastUpdateTime": "2023-01-10T17:58:28.4403142Z",

+ "mediaType": "application/vnd.oci.artifact.manifest.v1+json"

+Support for the OCI v1.1 Specification enables deleting the graph of artifacts associated with the root artifact. Use the [`oras delete`][oras-cli] command to delete the graph of artifacts (signature, SBOM and the signature of the SBOM).

+oras manifest delete -f $REGISTRY/$REPO:$TAG

+oras manifest delete -f $REGISTRY/sample-staging/$REPO:$TAG

+By deleting the root artifact, all related artifacts are also deleted leaving a clean environment:

+ --registry $ACR_NAME -o jsonc

+Output:

+```output

+2023-01-10 18:38:45.366387 Error: repository "net-monitor" is not found.

+```

+## Summary

+In this article, a graph of supply chain artifacts is created, discovered, promoted and pulled providing lifecycle management of the artifacts you build and depend upon.

+[docker-install]: https://www.docker.com/get-started/

+[oci-artifact-manifest]: https://github.com/opencontainers/image-spec/blob/main/artifact.md/

+[oci-artifact-referrers]: https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers/

+[oci-spec]: https://github.com/opencontainers/distribution-spec/blob/main/spec.md/

+[oci-1_1-spec]: https://github.com/opencontainers/distribution-spec/releases/tag/v1.1.0-rc1

+[oras-docs]: https://oras.land/

+[oras-install-docs]: https://oras.land/cli/

+[oras-push-multifiles]: https://oras.land/cli/1_pushing/#pushing-artifacts-with-multiple-files

+[oras-cli]: https://oras.land/cli_reference/

+[acr-authentication]: /azure/container-registry/container-registry-authentication?tabs=azure-cli

+[az-acr-create]: /azure/container-registry/container-registry-get-started-azure-cli

+[az-acr-build]: /cli/azure/acr#az_acr_build

+[az-acr-manifest-metadata]: /cli/azure/acr/manifest/metadata

+[azure-cli-install]: /cli/azure/install-azure-cli

+* Azure Cosmos DB for MongoDB accounts with continuous backup do not support creating a unique index for an existing collection. For such an account, unique indexes must be created along with their collection; this is done using the create collection [extension commands](mongodb/custom-commands.md).

+> [!TIP]

+> If you are following along with these examples, you can use any of these properties (`age`, `firstName`, `lastName`) as a partition key when you create your graph. The `id` property is not supported as a partition key in a graph.

+ | **QueryRuntimeStatistics** | NoSQL | This table details query operations executed against an API for NoSQL account. By default, the query text and its parameters are obfuscated to avoid logging personal data with full text query logging available by request. | `databasename`, `partitionkeyrangeid`, `querytext` |

+ "familyName": "Merriam",

+ "givenName": "Jesse",

+ "gender": "female",

+ "grade": 1

+ "familyName": "Miller",

+ "givenName": "Lisa",

+ "gender": "female",

+ "grade": 8

+The Azure CLI approach is used in this example. Use the [`az cosmosdb sql database create`](/cli/azure/cosmosdb/sql/database#az-cosmosdb-sql-database-create) and [`az cosmosdb sql container create`](/cli/azure/cosmosdb/sql/container#az-cosmosdb-sql-container-create) commands to create a Cosmos DB NoSQL database and container.

+With **Time to Live** or TTL, Azure Cosmos DB provides the ability to delete items automatically from a container after a certain time period. By default, you can set time to live at the container level and override the value on a per-item basis. After you set the TTL at a container or at an item level, Azure Cosmos DB will automatically remove these items after the time period, since the time they were last modified. Time to live value is configured in seconds. When you configure TTL, the system will automatically delete the expired items based on the TTL value, without needing a delete operation that is explicitly issued by the client application. The maximum value for TTL is 2147483647 seconds, the approximate equivalent of 24,855 days or 68 years.

+# Conditional execution

+## Conditional paths

+| Name | Explanation |

+| | |

+| Upon Success | (Default Pass) Execute this path if the current activity succeeded |

+| Upon Failure | Execute this path if the current activity failed |

+| Upon Completion | Execute this path after the current activity completed, regardless if it succeeded or not |

+| Upon Skip | Execute this path if the activity itself didn't run |

+You may add multiple branches following an activity, with one exception: _Upon Completion_ path can't co-exist with either _Upon Success_ or _Upon Failure_ path. For each pipeline run, at most one path will be activated, based on the execution outcome of the activity.

+## Error Handling

+This article describes the various ways to cable your Azure Data Box for data transfer. For a full list of supported cables, see the [list of supported cables and switches from Mellanox](https://network.nvidia.com/pdf/firmware/ConnectX3-FW-2_42_5000-release_notes.pdf).

+Often, organizations have collections of machines that routinely run the same processes. Microsoft Defender for Cloud uses machine learning to analyze the applications running on your machines and create a list of the known-safe software. Allowlists are based on your specific Azure workloads, and you can further customize the recommendations using the following instructions.

+ :::image type="content" source="./media/adaptive-application/opening-adaptive-application-control-new.png" alt-text="Screenshot showing opening adaptive application controls from the Azure Dashboard." lightbox="./media/adaptive-application/opening-adaptive-application-control.png":::

+ > If you see a group name with the prefix "REVIEWGROUP", it contains machines with a partially consistent list of applications. Microsoft Defender for Cloud can't see a pattern but recommends reviewing this group to see whether _you_ can manually define some adaptive application controls rules as described in [Edit a group's adaptive application controls rule](#edit-a-groups-adaptive-application-controls-rule).

+ - AppLocker isn't available (Windows Server Core installations)

+1. Open the **Recommended** tab. The groups of machines with recommended allowlists appear.

+ 1. **Select machines** - By default, all machines in the identified group are selected. Unselect any to remove them from this rule.

+ > Applications are defined by their publishers; if an application doesn't have publisher information (it's unsigned), a path rule is created for the full path of the specific application.

+1. To view the details and settings of your group, select **Group settings**.

+ :::image type="content" source="./media/adaptive-application/adaptive-application-group-settings.png" alt-text="Screenshot showing the group settings page for adaptive application controls." lightbox="./media/adaptive-application/adaptive-application-group-settings.png":::

+ :::image type="content" source="./media/adaptive-application/recent-alerts.png" alt-text="Screenshot showing selecting a group the group settings page for adaptive application controls." lightbox="./media/adaptive-application/recent-alerts.png":::

+ :::image type="content" source="media/adaptive-application/adaptive-application-alerts-start-time.png" alt-text="Screenshot showing the start time of adaptive application controls alerts is the time that adaptive application controls created the alert.":::

+When you move a machine from one group to another, the application control policy applied to it changes to the settings of the group that you moved it to. You can also move a machine from a configured group to a non-configured group; doing so removes any application control rules that were applied to the machine.

+1. From the **Adaptive application controls** page, from the **Configured** tab, select the group containing the machine to be moved.

+The relevant API documentation is available in [the Adaptive application Controls section of Defender for Cloud's API docs](https://learn.microsoft.com/rest/api/defenderforcloud/adaptive-application-controls).

+Some of the functions available from the REST API include:

+* **Get** retrieves the JSON with the full recommendation data (list of machines, publisher/path rules, etc.).

+ > The **Put** function expects fewer parameters than the JSON returned by the **Get** command contains.

+ > Remove the following properties before using the JSON in the **Put** request: recommendationStatus, configurationStatus, issues, location, and sourceSystem.

+No enforcement options are currently available. Adaptive application controls are intended to provide **security alerts** if any application runs other than the ones you've defined as safe. They have a range of benefits ([What are the benefits of adaptive application controls?](#what-are-the-benefits-of-adaptive-application-controls)) and are customizable as shown on this page.

+An external attack surface is the entire area of an organization or system that is susceptible to an attack from an external source. An organization's attack surface is made up of all the points of access that an unauthorized person could use to enter their system. The larger your attack surface is, the harder it's to protect.

+EASM collects data for publicly exposed assets (ΓÇ£outside-inΓÇ¥). That data can be used by MDC CSPM (ΓÇ£inside-outΓÇ¥) to assist with internet-exposure validation and discovery capabilities to provide better visibility to customers.

+Defender for App Service also identifies any DNS entries remaining in your DNS registrar when an App Service website is decommissioned - these are known as dangling DNS entries. When you remove a website and don't remove its custom domain from your DNS registrar, the DNS entry is pointing to a non-existent resource, and your subdomain is vulnerable to a takeover. Defender for Cloud doesn't scan your DNS registrar for *existing* dangling DNS entries; it alerts you when an App Service website is decommissioned and its custom domain (DNS entry) isn't deleted.

+In this article, you learned about Microsoft Defender for App Service.

+ > [!Note]

+ > During the preview, the maximum number of repositories that can be onboarded to Microsoft Defender for Cloud is 2,000. If you try to connect more than 2,000 repositories, only the first 2,000 repositories, sorted alphabetically, will be onboarded.

+ >

+ > If your organization is interested in onboarding more than 2,000 repositories, please complete [this survey](https://aka.ms/dfd-forms/onboarding).

+To protect your DNS layer, enable Microsoft Defender for DNS for each of your subscriptions as described in [Enable enhanced protections](enable-enhanced-security.md).

+- An Azure subscription If you donΓÇÖt have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.

+1. Select a repository you want to configure the GitHub action to.

+description: Plan for agent deployment to protect Azure, AWS, GCP, and on-premises servers with Microsoft Defender for Servers.

+# Plan agents, extensions, and Azure Arc for Defender for Servers

+This article helps you plan your agents, extensions, and Azure Arc resources for your Microsoft Defender for Servers deployment.

+Defender for Servers is one of the paid plans provided by [Microsoft Defender for Cloud](defender-for-cloud-introduction.md).

+## Before you begin

+This article is the *fifth* article in the Defender for Servers planning guide. Before you begin, review the earlier articles:

+## Review Azure Arc requirements

+Azure Arc helps you onboard Amazon Web Services (AWS), Google Cloud Platform (GCP), and on-premises machines to Azure. Defender for Cloud uses Azure Arc to protect non-Azure machines.

+### Foundational cloud security posture management

+For free foundational cloud security posture management (CSPM) features, Azure Arc running on AWS or GCP machines isn't required. For full functionality, we recommend that you *do* have Azure Arc running on AWS or GCP machines.

+Azure Arc onboarding is required for on-premises machines.

+### Defender for Servers plan

+To use Defender for Servers, all AWS, GCP, and on-premises machines should be Azure Arc-enabled.

+You can onboard the Azure Arc agent to your AWS or GCP servers automatically with the AWS or GCP multicloud connector.

+To plan for Azure Arc deployment:

+1. Review the Azure Arc [planning recommendations](../azure-arc/servers/plan-at-scale-deployment.md) and [deployment prerequisites](../azure-arc/servers/prerequisites.md).

+1. Azure Arc installs the Connected Machine agent to connect to and manage machines that are hosted outside of Azure. Review the following information:

+## Log Analytics agent and Azure Monitor agent

+Defender for Cloud uses the Log Analytics agent and the Azure Monitor agent to collect information from compute resources. Then, it sends the data to a Log Analytics workspace for more analysis. Review the [differences and recommendations for both agents](../azure-monitor/agents/agents-overview.md).

+The following table describes the agents that are used in Defender for Servers:

+ | |

+Foundational CSPM recommendations (free) that depend on the agent: [OS baseline recommendation](apply-security-baseline.md) (Azure VMs) | :::image type="icon" source="./medi) is used.

+Foundational CSPM: [Antimalware/endpoint protection recommendations](endpoint-protection-recommendations-technical.md) (Azure VMs) | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported by the Log Analytics agent."::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported by the Azure Monitor agent.":::

+Attack detection at the OS level and network layer, including fileless attack detection<br/><br/> Plan 1 relies on Defender for Endpoint capabilities for attack detection. | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported by the Log Analytics agent. Plan 1 relies on Defender for Endpoint.":::<br/><br/> Plan 2| :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported by the Azure Monitor agent. Plan 1 relies on Defender for Endpoint.":::<br/><br/> Plan 2

+The Qualys extension is available in Defender for Servers Plan 2. The extension is deployed if you want to use Qualys for vulnerability assessment.

+Here's more information:

+ - If you're in a European Azure geography, data is processed in the Qualys European datacenter.

+ - For other regions, data is processed in the US datacenter.

+- To use Qualys on a machine, the extension must be installed and the machine must be able to communicate with the relevant network endpoint:

+ - Europe datacenter: `https://qagpublic.qg2.apps.qualys.eu`

+ - US datacenter: `https://qagpublic.qg3.apps.qualys.com`

+- If you're using the Azure Monitor Agent, Defender for Cloud uses this extension to analyze operating system security baseline settings on Windows and Linux machines.

+- Although Azure Arc-enabled servers and the guest configuration extension are free, more costs might apply if you use guest configuration policies on Azure Arc servers outside the scope of Defender for Cloud.

+Learn more about the Azure Policy [guest configuration extension](../virtual-machines/extensions/guest-configuration.md).

+- Windows machines extension: `MDE.Windows`

+- Linux machines extension: `MDE.Linux`

+- Some Windows Server versions have [specific requirements](/microsoft-365/security/defender-endpoint/configure-server-endpoints).

+Before you deploy Defender for Servers, verify operating system support for agents and extensions:

+- [Check requirements](../azure-arc/servers/prerequisites.md) for the Azure Arc Connect Machine agent.

+- Check operating system support for the [Log Analytics agent](../azure-monitor/agents/log-analytics-agent.md#supported-operating-systems) and [Azure Monitor agent](../azure-monitor/agents/agents-overview.md).

+When you enable Defender for Cloud plans, including Defender for Servers, you can choose to automatically provision some agents that are relevant for Defender for Servers:

+- Log Analytics agent and Azure Monitor agent for Azure VMs

+- Log Analytics agent and Azure Monitor agent for Azure Arc VMs

+- Qualys agent

+- Guest configuration agent

+When you enable Defender for Servers Plan 1 or Plan 2, the Defender for Endpoint extension is automatically provisioned on all supported machines in the subscription.

+## Provisioning considerations

+The following table describes provisioning considerations to be aware of:

+Provisioning | Details

+Defender for Endpoint sensor | If machines are running Microsoft Antimalware, also known as System Center Endpoint Protection (SCEP), the Windows extension automatically removes it from the machine.<br/><br/> If you deploy on a machine that already has the legacy Microsoft Monitoring agent (MMA) Defender for Endpoint sensor running, after the Defender for Cloud and Defender for Endpoint unified solution is successfully installed, the extension stops and it disables the legacy sensor. The change is transparent and the machineΓÇÖs protection history is preserved.

+AWS and GCP machines | Configure automatic provisioning when you set up the AWS or GCP connector.

+Manual installation | If you don't want Defender for Cloud to provision the Log Analytics agent and Azure Monitor agent, you can install agents manually.<br/><br/> You can connect the agent to the default Defender for Cloud workspace or to a custom workspace.<br/><br/> The workspace must have the *SecurityCenterFree* (for free foundational CSPM) or *Security* solution enabled (Defender for Servers Plan 2).

+[Log Analytics agent running directly](faq-data-collection-agents.yml#what-if-a-log-analytics-agent-is-directly-installed-on-the-machine-but-not-as-an-extension--direct-agent--) | If a Windows VM has the Log Analytics agent running but not as a VM extension, Defender for Cloud installs the extension. The agent reports to the Defender for Cloud workspace and to the existing agent workspace. <br/><br/> On Linux VMs, multi-homing isn't supported. If an existing agent exists, the Log Analytics agent isn't automatically provisioned.

+[Operations Manager agent](faq-data-collection-agents.yml#what-if-a-system-center-operations-manager-agent-is-already-installed-on-my-vm-) | The Log Analytics agent can work side by side with the Operations Manager agent. The agents share common runtime libraries that are updated when the Log Analytics agent is deployed.

+Removing the Log Analytics extension | If you remove the Log Analytics extension, Defender for Cloud can't collect security data and recommendations, and alerts will be missing. Within 24 hours, Defender for Cloud determines that the extension is missing and reinstalls it.

+## When to opt out of auto provisioning

+You might want to opt out of automatic provisioning in the circumstances that are described in the following table:

+You have critical VMs that shouldn't have agents installed | Log Analytics agent, Azure Monitor agent | Automatic provisioning is for an entire subscription. You can't opt out for specific machines.

+You're running the System Center Operations Manager agent version 2012 with Operations Manager 2012 | Log Analytics agent | With this configuration, don't turn on automatic provisioning. Management capabilities might be lost.

+You want to configure a custom workspace | Log Analytics agent, Azure Monitor agent | You have two options with a custom workspace:<br/><br/> - Opt out of automatic provisioning when you first set up Defender for Cloud. Then, configure provisioning on your custom workspace.<br/><br/>- Let automatic provisioning run to install the Log Analytics agents on machines. Set a custom workspace, and then reconfigure existing VMs with the new workspace setting.

+After you work through these planning steps, learn how to [scale your Defender for Servers deployment](plan-defender-for-servers-scale.md).

+ Title: Plan Defender for Servers data residency and workspaces

+description: Review data residency and workspace design for Microsoft Defender for Servers.

+# Plan data residency and workspaces for Defender for Servers

+This article helps you understand how your data is stored in Microsoft Defender for Servers and how Log Analytics workspaces are used in Defender for Servers.

+Defender for Servers is one of the paid plans provided by [Microsoft Defender for Cloud](defender-for-cloud-introduction.md).

+## Before you begin

+This article is the *second* article in the Defender for Servers planning guide series. Begin by [planning your deployment](plan-defender-for-servers.md).

+Data residency refers to the physical or geographic location of your organization's data.

+Before you deploy Defender for Servers, it's important for you to understand data residency for your organization:

+- Review [general Azure data residency considerations](https://azure.microsoft.com/blog/making-your-data-residency-choices-easier-with-azure/).

+- Review the table in the next section to understand where Defender for Cloud stores data.

+Understand where Defender for Cloud stores data and how you can work with your data:

+**Data** | **Location**

+**Security alerts and recommendations** | - Stored in the Defender for Cloud back end and accessible via the Azure portal, Azure Resource Graph, and REST APIs.<br/><br/> - You can export the data to a Log Analytics workspace by using [continuous export](continuous-export.md).

+**Machine information** | - Stored in a Log Analytics workspace.<br/><br/> - You can use either the default Defender for Cloud workspace or a custom workspace. Data is stored in accordance with the workspace location.

+## Workspace considerations

+In Defender for Cloud, you can store server data in the default Log Analytics workspace for your Defender for Cloud deployment or in a custom workspace.

+Here's more information:

+- By default, when you enable Defender for Cloud for the first time, a new resource group and a default workspace are created in the subscription region for each subscription that has Defender for Cloud enabled.

+- When you turn on a Defender for Cloud plan (including Defender for Servers), the plan is enabled for the default workspace, and the *Security* solution is enabled.

+- If you have virtual machines in multiple locations, Defender for Cloud creates multiple workspaces accordingly to ensure data compliance.

+- Default workspace names are in the format `[subscription-id]-[geo]`.

+## Default workspaces

+Defender for Cloud default workspaces are created in the following locations:

+United States, Canada, Europe, United Kingdom, Korea, India, Japan, China, Australia | The workspace is created in the matching location.

+You can store your server information in the default workspace or you can use a custom workspace. A custom workspace must meet these requirements:

+- You must enable the Defender for Servers plan in the custom workspace.

+- The custom workspace must be associated with the Azure subscription in which Defender for Cloud is enabled.

+- You must have at least read permissions for the workspace.

+- If the *Security & Audit* solution is installed in a workspace, Defender for Cloud uses the existing solution.

+Learn more about [Log Analytics workspace design strategy and criteria](../azure-monitor/logs/workspace-design.md).

+After you work through these planning steps, review [Defender for Server access roles](plan-defender-for-servers-roles.md).

+ Title: Plan Defender for Servers roles and permissions

+description: Review roles and permissions for Microsoft Defender for Servers.

+# Plan roles and permissions for Defender for Servers

+This article helps you understand how to control access to your Defender for Servers deployment.

+Defender for Servers is one of the paid plans provided by [Microsoft Defender for Cloud](defender-for-cloud-introduction.md).

+## Before you begin

+This article is the *third* article in the Defender for Servers planning guide. Before you begin, review the earlier articles:

+## Determine ownership and access

+In complex enterprises, different teams manage different [security functions](/azure/cloud-adoption-framework/organize/cloud-security) in the organization.

+It's critical that you identify ownership for server and endpoint security in your organization. Ownership that's undefined or hidden in organizational silos increases risk for the organization. Security operations (SecOps) teams that need to identify and follow threats across the enterprise are hindered. Deployments might be delayed or they might not be secure.

+Security leadership should identify the teams, roles, and individuals that are responsible for making and implementing decisions about server security.

+Responsibility usually is shared between a [central IT team](/azure/cloud-adoption-framework/organize/central-it) and a [cloud infrastructure and endpoint security team](/azure/cloud-adoption-framework/organize/cloud-security-infrastructure-endpoint). Individuals on these teams need Azure access rights to manage and use Defender for Cloud. As part of planning, determine the right level of access for individuals based on the Defender for Cloud role-based access control (RBAC) model.

+In addition to the built-in Owner, Contributor, and Reader roles for an Azure subscription and resource group, Defender for Cloud has built-in roles that control Defender for Cloud access:

+- **Security Reader**: Provides viewing rights to Defender for Cloud recommendations, alerts, security policy, and states. This role can't make changes to Defender for Cloud settings.

+- **Security Admin**: Provides Security Reader rights and the ability to update security policy, dismiss alerts and recommendations, and apply recommendations.

+Learn more about [allowed role actions](permissions.md#roles-and-allowed-actions).

+After you work through these planning steps, [decide which Defender for Servers plan](plan-defender-for-servers-select-plan.md) is right for your organization.

+description: Scale protection of Azure, AWS, GCP, and on-premises servers by using Microsoft Defender for Servers.

+This article helps you scale your Microsoft Defender for Servers deployment.

+Defender for Servers is one of the paid plans provided by [Microsoft Defender for Cloud](defender-for-cloud-introduction.md).

+## Before you begin

+This article is the *sixth* and final article in the Defender for Servers planning guide series. Before you begin, review the earlier articles:

+1. [Understand where your data is stored and Log Analytics workspace requirements](plan-defender-for-servers-data-workspace.md)

+1. [Review requirements for agents, extensions, and Azure Arc resources](plan-defender-for-servers-agents.md)

+When you enable a Defender for Cloud subscription, this process occurs:

+1. The *microsoft.security* resource provider is automatically registered on the subscription.

+1. At the same time, the Cloud Security Benchmark initiative that's responsible for creating security recommendations and calculating the security score is assigned to the subscription.

+1. After you enable Defender for Cloud on the subscription, you turn on Defender for Servers Plan 1 or Defender for Servers Plan 2, and then you enable auto provisioning.

+In the next sections, review considerations for specific steps as you scale your deployment:

+- Scale a Cloud Security Benchmark deployment

+- Scale a Defender for Servers plan

+- Scale auto provisioning

+## Scale a Cloud Security Benchmark deployment

+In a scaled deployment, you might want the Cloud Security Benchmark (formerly the Azure Security Benchmark) to be automatically assigned.

+The assignment is inherited for every existing and future subscription in the management group. To set up your deployment to automatically apply the benchmark, assign the policy initiative to your management group (root) instead of to each subscription.

+You can get the *Azure Security Benchmark* policy definition on [GitHub](https://github.com/Azure/azure-policy/blob/master/built-in-policies/policySetDefinitions/Security%20Center/AzureSecurityCenter.json).

+[Learn more](onboard-management-group.md) about using a built-in policy definition to register a resource provider.

+## Scale a Defender for Servers plan

+You can use a policy definition to enable Defender for Servers at scale:

+- To get the built-in *Configure Defender for Servers to be enabled* policy definition, in the Azure portal for your deployment, go to **Azure Policy** > **Policy Definitions**.

+- Alternatively, you can use a [custom policy](https://github.com/Azure/Microsoft-Defender-for-Cloud/tree/main/Policy/Enable%20Defender%20for%20Servers%20plans) to enable Defender for Servers and select the plan at the same time.

+- You can enable only one Defender for Servers plan on each subscription. You can't enable both Defender for Servers Plan 1 and Plan 2 at the same time.

+- If you want to use both plans in your environment, divide your subscriptions into two management groups. On each management group, assign a policy to enable the respective plan on each underlying subscription.

+## Scale auto provisioning

+You can set up auto provisioning by assigning the built-in policy definitions to an Azure management group to cover underlying subscriptions. The following table summarizes the definitions:

+Log Analytics agent (default workspace) | *Enable Security Center's autoprovisioning of the Log Analytics agent on your subscriptions with default workspaces*

+Log Analytics agent (custom workspace) | *Enable Security Center's autoprovisioning of the Log Analytics agent on your subscriptions with custom workspaces*

+Azure Monitor agent (default data collection rule) | *[Preview]: Configure Arc machines to create the default Microsoft Defender for Cloud pipeline using Azure Monitor Agent*<br/><br/> *[Preview]: Configure virtual machines to create the default Microsoft Defender for Cloud pipeline using Azure Monitor Agent*

+Azure Monitor agent (custom data collection rule) | *[Preview]: Configure Arc machines to create the Microsoft Defender for Cloud user-defined pipeline using Azure Monitor Agent*<br/><br/> *[Preview]: Configure machines to create the Microsoft Defender for Cloud user-defined pipeline using Azure Monitor Agent*

+Qualys vulnerability assessment | *Configure machines to receive a vulnerability assessment provider*

+To review policy definitions, in the Azure portal, go to **Policy** > **Definitions**.

+Begin a deployment for your scenario:

+- [Enable a Defender for Servers plan](enable-enhanced-security.md)

+- [Connect an on-premises machine to Azure](quickstart-onboard-aws.md)

+- [Connect an AWS account to Defender for Cloud](quickstart-onboard-aws.md)

+- [Connect a GCP project to Defender for Cloud](quickstart-onboard-gcp.md)

+ Title: Select a Defender for Servers plan in Microsoft Defender for Cloud

+description: Select a Microsoft Defender for Servers plan in Microsoft Defender for Cloud to protect Azure, AWS, and GCP servers and on-premises machines.

+This article helps you select the Microsoft Defender for Servers plan that's right for your organization.

+Defender for Servers is one of the paid plans provided by [Microsoft Defender for Cloud](defender-for-cloud-introduction.md).

+## Before you begin

+This article is the *fourth* article in the Defender for Servers planning guide. Before you begin, review the earlier articles:

+1. [Understand where your data is stored and Log Analytics workspace requirements](plan-defender-for-servers-data-workspace.md)

+You can choose from two Defender for Servers paid plans:

+- **Defender for Servers Plan 1** is entry-level and must be enabled at the subscription level. Features include:

+ - [Foundational cloud security posture management (CSPM)](concept-cloud-security-posture-management.md#defender-cspm-plan-options), which is provided free by Defender for Cloud.

+ - For Azure virtual machines and Amazon Web Services (AWS) and Google Cloud Platform (GCP) machines, you don't need a Defender for Cloud plan enabled to use foundational CSPM features.

+ - For on-premises server, to receive configuration recommendations machines must be onboarded to Azure with Azure Arc, and Defender for Servers must be enabled.

+ - Endpoint detection and response (EDR) features that are provided by [Microsoft Defender for Endpoint Plan 2](/microsoft-365/security/defender-endpoint/defender-endpoint-plan-1-2).

+- **Defender for Servers Plan 2** provides all features. The plan must be enabled at the subscription level and at the workspace level to get full feature coverage. Features include:

+ - All the functionality that's provided by Defender for Servers Plan 1.

+ - More extended detection and response (XDR) capabilities.

+| **Defender for Endpoint integration** | Defender for Servers integrates with Defender for Endpoint and protects servers with all the features, including:<br/><br/>- [Attack surface reduction](/microsoft-365/security/defender-endpoint/overview-attack-surface-reduction) to lower the risk of attack.<br/><br/> - [Next-generation protection](/microsoft-365/security/defender-endpoint/next-generation-protection), including real-time scanning and protection and [Microsoft Defender Antivirus](/microsoft-365/security/defender-endpoint/next-generation-protection).<br/><br/> - EDR, including [threat analytics](/microsoft-365/security/defender-endpoint/threat-analytics), [automated investigation and response](/microsoft-365/security/defender-endpoint/automated-investigations), [advanced hunting](/microsoft-365/security/defender-endpoint/advanced-hunting-overview), and [Microsoft Defender Experts](/microsoft-365/security/defender-endpoint/endpoint-attack-notifications).<br/><br/> - Vulnerability assessment and mitigation provided by Defender for Endpoint integration with [Microsoft Defender Vulnerability Management](/microsoft-365/security/defender-vulnerability-management/defender-vulnerability-management-capabilities). | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 1."::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Licensing** | Defender for Servers covers licensing for Defender for Endpoint. Licensing is charged per hour instead of per seat, lowering costs by protecting virtual machines only when they're in use.| :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 1."::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Defender for Endpoint provisioning** | Defender for Servers automatically provisions the Defender for Endpoint sensor on every supported machine that's connected to Defender for Cloud.| :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 1."::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Unified view** | Defender for Endpoint alerts appear in the Defender for Cloud portal. You can get detailed information in the Defender for Endpoint portal.| :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 1."::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Threat detection for OS-level (agent-based)** | Defender for Servers and Defender for Endpoint detect threats at the OS level, including virtual machine behavioral detections and *fileless attack detection*, which generates detailed security alerts that accelerate alert triage, correlation, and downstream response time.<br>[Learn more](alerts-reference.md#alerts-windows) | :::image type="icon" source="./media/icons/yes-icon.png"::: | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Threat detection for network-level (agentless)** | Defender for Servers detects threats that are directed at the control plane on the network, including network-based detections for Azure virtual machines. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **[Qualys vulnerability assessment](deploy-vulnerability-assessment-vm.md)** | As an alternative to Defender Vulnerability Management, Defender for Cloud integrates with the Qualys scanner to identify vulnerabilities. You don't need a Qualys license or account. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2.":::|

+**[Adaptive application controls](adaptive-application-controls.md)** | Adaptive application controls define allowlists of known safe applications for machines. To use this feature, Defender for Cloud must be enabled on the subscription. | Not supported in Plan 1 |:::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **Free data ingestion (500 MB) in workspaces** | Free data ingestion is available for [specific data types](faq-defender-for-servers.yml#what-data-types-are-included-in-the-daily-allowance-). Data ingestion is calculated per node, per reported workspace, and per day. It's available for every workspace that has a *Security* or *AntiMalware* solution installed. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **[Just-in-time virtual machine access](just-in-time-access-overview.md)** | Just-in-time virtual machine access locks down machine ports to reduce the attack surface. To use this feature, Defender for Cloud must be enabled on the subscription. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **[Adaptive network hardening](adaptive-network-hardening.md)** | Network hardening filters traffic to and from resources by using network security groups (NSGs) to improve your network security posture. Further improve security by hardening the NSG rules based on actual traffic patterns. To use this feature, Defender for Cloud must be enabled on the subscription. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **[File integrity monitoring](file-integrity-monitoring-overview.md)** | File integrity monitoring examines files and registries for changes that might indicate an attack. A comparison method is used to determine whether suspicious modifications have been made to files. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+| **[Docker host hardening](harden-docker-hosts.md)** | Assesses containers hosted on Linux machines running Docker containers, and then compares them with the Center for Internet Security (CIS) Docker Benchmark. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2."::: |

+[Agentless scanning](concept-agentless-data-collection.md) | Scans Azure virtual machines by using cloud APIs to collect data. | Not supported in Plan 1 | :::image type="icon" source="./media/icons/yes-icon.png" alt-text="Icon that shows it's supported in Plan 2.":::

+A couple vulnerability assessment options are available in Defender for Servers:

+ - Available in Defender for Servers Plan 1 and Defender for Servers Plan 2.

+ - Defender Vulnerability Management is enabled by default on machines that are onboarded to Defender for Endpoint if [Defender for Endpoint has Defender Vulnerability Management](/microsoft-365/security/defender-vulnerability-management/get-defender-vulnerability-management) enabled.

+ - Has the same [Windows](/microsoft-365/security/defender-endpoint/configure-server-endpoints#prerequisites), [Linux](/microsoft-365/security/defender-endpoint/microsoft-defender-endpoint-linux#prerequisites), and [network](/microsoft-365/security/defender-endpoint/configure-proxy-internet#enable-access-to-microsoft-defender-for-endpoint-service-urls-in-the-proxy-server) prerequisites as Defender for Endpoint.

+ - No extra software is required.

+- [Qualys vulnerability scanner](deploy-vulnerability-assessment-vm.md): Provided by Defender for Cloud Qualys integration.

+ - Available only in Defender for Servers Plan 2.

+ - A great fit if you're using a third-party EDR solution or a Fanotify-based solution. In these scenarios, you might not be able to deploy the Defender for Endpoint for vulnerability assessment.

+ - The integrated Defender for Cloud Qualys solution doesn't support a proxy configuration, and it can't connect to an existing Qualys deployment. Vulnerability findings are available only in Defender for Cloud.

+## Next steps

+After you work through these planning steps, [review Azure Arc and agent and extension requirements](plan-defender-for-servers-agents.md).

+description: Design a solution to protect on-premises and multicloud servers with Microsoft Defender for Servers.

+# Plan your Defender for Servers deployment

+Microsoft Defender for Servers extends protection to your Windows and Linux machines that run in Azure, Amazon Web Services (AWS), Google Cloud Platform (GCP), and on-premises. Defender for Servers integrates with Microsoft Defender for Endpoint to provide endpoint detection and response (EDR) and other threat protection features.

+This guide helps you design and plan an effective Defender for Servers deployment. [Microsoft Defender for Cloud](defender-for-cloud-introduction.md) offers two paid plans for Defender for Servers.

+The intended audience of this guide is cloud solution and infrastructure architects, security architects and analysts, and anyone who's involved in protecting cloud and hybrid servers and workloads.

+The guide answers these questions:

+- What does Defender for Servers do and how is it deployed?

+- Where will my data be stored and what Log Analytics workspaces do I need?

+- Who needs access to my Defender for Servers resources?

+- Which Defender for Servers plan should I choose and which vulnerability assessment solution should I use?

+- When do I need to use Azure Arc and which agents and extensions are required?

+Before you review the series of articles in the Defender for Servers planning guide:

+- Review Defender for Servers [pricing details](https://azure.microsoft.com/pricing/details/defender-for-cloud/).

+- If you're deploying for AWS machines or GCP projects, review the [multicloud planning guide](plan-multicloud-security-get-started.md).

+The following diagram shows an overview of the Defender for Servers deployment process:

+You've begun the Defender for Servers planning process. Review the next article in the planning guide to [understand how your data is stored and the Log Analytics workspace requirements](plan-defender-for-servers-data-workspace.md).

+In this quickstart, you'll learn how to use an Azure Resource Manager template (ARM template) or a Bicep file to create a workflow automation. The workflow automation will trigger a logic app when specific security alerts are received by Microsoft Defender for Cloud.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Search for and select **Microsoft Defender for Cloud**.

+1. Select **filter**.

+1. Select the specific subscription on which you deployed the new workflow automation.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Search for and select **Microsoft Defender for Cloud**.

+1. Select **filter**.

+1. Select the specific subscription on which you deployed the new workflow automation.

+ - **automationName**: Replace **\<automation-name\>** with the name of the automation. It has a minimum length of three characters and a maximum length of 24 characters.

+ - **logicAppName**: Replace **\<logic-name\>** with the name of the logic app. It has a minimum length of three characters.

+ - **logicAppResourceGroupName**: Replace **\<group-name\>** with the name of the resource group in which the resources are located. It has a minimum length of three characters.

+## January 2023

+Updates in January include:

+- [New version of the recommendation to find missing system updates (Preview)](#new-version-of-the-recommendation-to-find-missing-system-updates-preview)

+### New version of the recommendation to find missing system updates (Preview)

+You no longer need an agent on your Azure VMs and Azure Arc machines to make sure the machines have all of the latest security or critical system updates.

+The new system updates recommendation, "System updates should be installed on your machines (powered by Update management center)" in the "Apply system updates" control, is based on the [Update management center (preview)](../update-center/overview.md) and relies on a native agent embedded in every Azure VM and Azure Arc machines instead of an installed agent. The Quick Fix in the new recommendation leads you to a one-time installation of the missing updates in the Update management center portal.

+To use the new recommendation you need to:

+- Connect your non-Azure machines to Arc

+- Turn on the [periodic assessment property](../update-center/assessment-options.md#periodic-assessment). For this, you can use the Quick Fix in a new recommendation, "Machines should be configured to periodically check for missing system updates".

+The existing "System updates should be installed on your machines" recommendation, which relies on the Log Analytics agent, is still available under the same control.

+Your [secure score is calculated](secure-score-security-controls.md?branch=main#how-your-secure-score-is-calculated) based on the security recommendations that you've implemented. In order to increase your score and improve your security posture, you have to find recommendations with unhealthy resources and [remediate those recommendations](implement-security-recommendations.md).

+When you [remediate](implement-security-recommendations.md) all of the recommendations in the security control, your secure score increases by the percentage point listed for the control.

+[Security teams can assign a recommendation](governance-rules.md) to a specific person and assign a due date to drive your organization towards increased security. If you have recommendations assigned to you, you're accountable to remediate the resources affected by the recommendations to help your organization be compliant with the security policy.

+Recommendations are listed as **On time** until their due date is passed, when they're changed to **Overdue**. Before the recommendation is overdue, the recommendation doesn't impact the secure score. The security team can also apply a grace period during which overdue recommendations continue to not impact the secure score.

+ - The insights column indicates the recommendations that are in a grace period, so they currently don't impact your secure score until they become overdue.

+ The owner of the resource gets a weekly email listing the recommendations that they're assigned to.

+The due date for the recommendation doesn't change, but the security team can see that you plan to update the resources by the specified ETA date.

+> Versions 22.2.x of the sensor are incompatible with Hyper-V. Until the issue has been resolved, we recommend using either version 22.3.x or 22.1.7.

+1. Select **Specify Generation** > **Generation 1** or **Generation 2**.

+- [Update your sensors from the Azure portal](update-ot-software.md#update-your-sensors)

+> If you are updating from software versions earlier than [22.1.x](whats-new.md#update-to-version-221x), note that [version 22.1.x](release-notes.md#2223) has a large update with more complicated background processes. Expect this update to take more time than earlier updates have required.

+### Prerequisites

+If you're using an on-premises management console to manage your sensors, make sure to update your on-premises management console software *before* you update your sensor software.

+On-premises management software is backwards compatible, and can connect to sensors with earlier versions installed, but not later versions. If you update your sensor software before updating your on-premises management console, the updated sensor will be disconnected from the on-premises management console.

+For more information, see [Update an on-premises management console](#update-an-on-premises-management-console).

+# [From the Azure portal (Public preview)](#tab/portal)

+This procedure describes how to send a software version update to one or more OT sensors, and then run the updates remotely from the Azure portal. Bulk updates are supported for up to 10 sensors at a time.

+> [!TIP]

+> Sending your version update and running the update process are two separate steps, which can be done one right after the other or at different times.

+> For example, you might want to first send the update to your sensor and then an administrator to run the installation during a planned maintenance window.

+**Prerequisites**: A cloud-connected sensor with a software version equal to or higher than [22.2.3](release-notes.md#2223), but not yet the latest version available.

+**To send the software update to your OT sensor**:

+1. In the Azure portal, go to **Defender for IoT** > **Sites and sensors** and identify the sensors that have legacy versions installed.

+ If you know your site and sensor name, you can browse or search for it directly. Alternately, filter the sensors listed to show only cloud-connected, OT sensors that have *Remote updates supported*, and have legacy software version installed. For example:

+ :::image type="content" source="media/update-ot-software/filter-remote-update.png" alt-text="Screenshot of how to filter for OT sensors that are ready for remote update." lightbox="media/update-ot-software/filter-remote-update.png":::

+1. Select one or more sensors to update, and then select **Update (Preview)** > **Send package**. For a specific sensor, you can also access the **Send package** option from the **...** options menu to the right of the sensor row. For example:

+ :::image type="content" source="media/update-ot-software/send-package.png" alt-text="Screenshot of the Send package option." lightbox="media/update-ot-software/send-package.png":::

+1. In the **Send package** pane that appears on the right, check to make sure that you're sending the correct software to the sensor you want to update. For more information, see [Legacy version updates vs. recent version updates](#legacy-version-updates-vs-recent-version-updates).

+ To jump to the release notes for the new version, select **Learn more** at the top of the pane.

+1. When you're ready, select **Send package**. The software transfer to your sensor machine is started, and you can see the progress in the **Sensor version** column.

+ When the transfer is complete, the **Sensor version** column changes to :::image type="icon" source="media/update-ot-software/ready-to-update.png" border="false"::: **Ready to update**.

+ Hover over the **Sensor version** value to see the source and target version for your update.

+**To run your sensor update from the Azure portal**:

+When the **Sensor version** column for your sensors reads :::image type="icon" source="media/update-ot-software/ready-to-update.png" border="false"::: **Ready to update**, you're ready to run your update.

+1. As in the previous step, either select multiple sensors that are ready to update, or select one sensor at a time.

+1. Select either **Update (Preview)** > **Update sensor** from the toolbar, or for an individual sensor, select the **...** options menu > **Update sensor**. For example:

+ :::image type="content" source="media/update-ot-software/update-sensor.png" alt-text="Screenshot of the Update sensor option." lightbox="media/update-ot-software/update-sensor.png":::

+1. In the **Update sensor (Preview)** pane that appears on the right, verify your update details.

+ When you're ready, select **Update now** > **Confirm update**. In the grid, the **Sensor version** value changes to :::image type="icon" source="media/update-ot-software/installing.png" border="false"::: **Installing** until the update is complete, when the value switches to the new sensor version number instead.

+If a sensor fails to update for any reason, the software reverts back to the previous version installed, and a sensor health alert is triggered. For more information, see [Understand sensor health (Public preview)](how-to-manage-sensors-on-the-cloud.md#understand-sensor-health-public-preview) and [Sensor health message reference](sensor-health-messages.md).

+> After upgrading to version 22.1.x or higher, the new upgrade log is accessible by the *cyberx_host* user on the sensor at the following path: `/opt/sensor/logs/legacy-upgrade.log`. To access the update log, sign into the sensor via SSH with the *cyberx_host* user.

+|**OT networks** | - **Sensor version 22.3.4**: [Azure connectivity status shown on OT sensors](#azure-connectivity-status-shown-on-ot-sensors)<br>- **Sensor version 22.2.3**: [Update sensor software from the Azure portal](#update-sensor-software-from-the-azure-portal-public-preview) |

+### Update sensor software from the Azure portal (Public preview)

+For cloud-connected sensor versions [22.2.3](release-notes.md#2223) and higher, now you can update your sensor software directly from the new **Sites and sensors** page on the Azure portal.

+For more information, see [Update your sensors from the Azure portal](update-ot-software.md#update-your-sensors).

+|**OT networks** | [New purchase experience for OT plans](#new-purchase-experience-for-ot-plans) |

+1. Select **New pipeline**.

+1. In the **Select a template** pane, select **Empty job**.

+1. Close the **Stage** pane.

+### Add an artifact

+1. On the new release pipeline page, on the **Pipeline** tab, select **Add an artifact**.

+1. On the **Add an artifact pane**, select **Azure Repo**.

+1. In the **Project** list, select your DevOps project.

+1. In the **Source (repository)** list, select your source repo.

+1. In the **Default branch** list, select the branch to check out.

+1. Select **Add**.

+ - **Virtual Machine Name**: the variable you specified for your virtual machine name: *$vmName*.

+ - **Parameters File**: If you checked a parameters file into your repository, browse to and select it.

+1. On the release pipeline **Tasks** tab, select the plus sign **+** next to **Agent job**.

+1. On the release pipeline **Tasks** tab, select the plus sign **+** next to **Agent job**.

+1. On the release pipeline **Tasks** tab, select the plus sign **+** next to **Agent job**.

+- **Cause**: You've specified a logical file name that isn't in the database backup. Another potential cause of this error is an incorrect storage account container name.

+Event Hubs provides a Kafka endpoint that can be used by your existing Kafka based applications as an alternative to running your own Kafka cluster. Event Hubs works with many of your existing Kafka applications. For more information, see [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md)

+Azure Event Hubs exposes an Apache Kafka endpoint, which enables you to connect to Event Hubs using the Kafka protocol. By making minimal changes to your existing Kafka application, you can connect to Azure Event Hubs and reap the benefits of the Azure ecosystem. Event Hubs works with many of your existing Kafka applications, including MirrorMaker. For more information, see [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md)

+> - This content applies to both Event Hubs and Event Hubs for Apache Kafka. For more information on Event Hubs for Kafka support, see [Event Hubs for Kafka - security and authentication](azure-event-hubs-kafka-overview.md#security-and-authentication).

+> This article applies to both Event Hubs and [Apache Kafka](azure-event-hubs-kafka-overview.md) scenarios.

+ Title: Use Azure Event Hubs from an Apache Kafka app

+description: This article provides you the information on using Azure Event Hubs to stream data from Apache Kafka applications without setting up a Kafka cluster.

+# Use Azure Event Hubs from Apache Kafka applications

+This article provides information about using Azure Event Hubs to stream data from [Apache Kafka](https://kafka.apache.org) applications without setting up a Kafka cluster on your own.

+> [!NOTE]

+> Event Hubs supports Apache Kafka's producer and consumer APIs clients at version 1.0 and above.

+## Azure Event Hubs for Apache Kafka overview

+The Event Hubs for Apache Kafka feature provides a protocol head on top of Azure Event Hubs that is protocol compatible with Apache Kafka clients built for Apache Kafka server versions 1.0 and later and supports for both reading from and writing to Event Hubs, which are equivalent to Apache Kafka topics.

+You can often use the Event Hubs Kafka endpoint from your applications without code changes and only modify the configuration: Update the connection string in configurations to point to the Kafka endpoint exposed by your event hub instead of pointing to your Kafka cluster. Then, you can start streaming events from your applications that use the Kafka protocol into Event Hubs.

+Conceptually, Kafka and Event Hubs are very similar: they're both partitioned logs built for streaming data, whereby the client controls which part of the retained log it wants to read. The following table maps concepts between Kafka and Event Hubs.

+### Kafka and Event Hubs conceptual mapping

+| Kafka Concept | Event Hubs Concept|

+| | |

+| Cluster | Namespace |

+| Topic | An event hub |

+| Partition | Partition|

+| Consumer Group | Consumer Group |

+| Offset | Offset|

+### Key differences between Apache Kafka and Event Hubs

+While [Apache Kafka](https://kafka.apache.org/) is software you typically need to install and operate, Event Hubs is a fully managed, cloud-native service. There are no servers, disks, or networks to manage and monitor and no brokers to consider or configure, ever. You create a namespace, which is an endpoint with a fully qualified domain name, and then you create Event Hubs (topics) within that namespace.

+For more information about Event Hubs and namespaces, see [Event Hubs features](event-hubs-features.md#namespace). As a cloud service, Event Hubs uses a single stable virtual IP address as the endpoint, so clients don't need to know about the brokers or machines within a cluster. Even though Event Hubs implements the same protocol, this difference means that all Kafka traffic for all partitions is predictably routed through this one endpoint rather than requiring firewall access for all brokers of a cluster.

+Scale in Event Hubs is controlled by how many [throughput units (TUs)](event-hubs-scalability.md#throughput-units) or [processing units](event-hubs-scalability.md#processing-units) you purchase. If you enable the [Auto-Inflate](event-hubs-auto-inflate.md) feature for a standard tier namespace, Event Hubs automatically scales up TUs when you reach the throughput limit. This feature work also works with the Apache Kafka protocol support. For a premier tier namespace, you can increase the number of processing units assigned to the namespace.

+### Is Apache Kafka the right solution for your workload?

+Coming from building applications using Apache Kafka, it's also useful to understand that Azure Event Hubs is part of a fleet of services, which also includes [Azure Service Bus](../service-bus-messaging/service-bus-messaging-overview.md), and [Azure Event Grid](../event-grid/overview.md).

+While some providers of commercial distributions of Apache Kafka might suggest that Apache Kafka is a one-stop-shop for all your messaging platform needs, the reality is that Apache Kafka doesn't implement, for instance, the [competing-consumer](/azure/architecture/patterns/competing-consumers) queue pattern, doesn't have support for [publish-subscribe](/azure/architecture/patterns/publisher-subscriber) at a level that allows subscribers access to the incoming messages based on server-evaluated rules other than plain offsets, and it has no facilities to track the lifecycle of a job initiated by a message or sidelining faulty messages into a dead-letter queue, all of which are foundational for many enterprise messaging scenarios.

+To understand the differences between patterns and which pattern is best covered by which service, see the [Asynchronous messaging options in Azure](/azure/architecture/guide/technology-choices/messaging) guidance. As an Apache Kafka user, you may find that communication paths you have so far realized with Kafka, can be realized with far less basic complexity and yet more powerful capabilities using either Event Grid or Service Bus.

+If you need specific features of Apache Kafka that aren't available through the Event Hubs for Apache Kafka interface or if your implementation pattern exceeds the [Event Hubs quotas](event-hubs-quotas.md), you can also run a [native Apache Kafka cluster in Azure HDInsight](../hdinsight/kafk).

+## Security and authentication

+Every time you publish or consume events from an Event Hubs for Kafka, your client is trying to access the Event Hubs resources. You want to ensure that the resources are accessed using an authorized entity. When using Apache Kafka protocol with your clients, you can set your configuration for authentication and encryption using the SASL mechanisms. When using Event Hubs for Kafka requires the TLS-encryption (as all data in transit with Event Hubs is TLS encrypted), it can be done specifying the SASL_SSL option in your configuration file.

+Azure Event Hubs provides multiple options to authorize access to your secure resources.

+- OAuth 2.0

+- Shared access signature (SAS)

+### OAuth 2.0

+Event Hubs integrates with Azure Active Directory (Azure AD), which provides an **OAuth 2.0** compliant centralized authorization server. With Azure AD, you can use Azure role-based access control (Azure RBAC) to grant fine grained permissions to your client identities. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **OAUTHBEARER** for the mechanism. For details about Azure roles and levels for scoping access, see [Authorize access with Azure AD](authorize-access-azure-active-directory.md).

+```properties

+bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093

+security.protocol=SASL_SSL

+sasl.mechanism=OAUTHBEARER

+sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;

+sasl.login.callback.handler.class=CustomAuthenticateCallbackHandler

+```

+> [!NOTE]

+> The above configuration properties are for the Java programming language. For **samples** that show how to use OAuth with Event Hubs for Kafka using different programming languages, see [samples on GitHub](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth).

+### Shared Access Signature (SAS)

+Event Hubs also provides the **Shared Access Signatures (SAS)** for delegated access to Event Hubs for Kafka resources. Authorizing access using OAuth 2.0 token-based mechanism provides superior security and ease of use over SAS. The built-in roles can also eliminate the need for ACL-based authorization, which has to be maintained and managed by the user. You can use this feature with your Kafka clients by specifying **SASL_SSL** for the protocol and **PLAIN** for the mechanism.

+```properties

+bootstrap.servers=NAMESPACENAME.servicebus.windows.net:9093

+security.protocol=SASL_SSL

+sasl.mechanism=PLAIN

+sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="{YOUR.EVENTHUBS.CONNECTION.STRING}";

+```

+> [!IMPORTANT]

+> Replace `{YOUR.EVENTHUBS.CONNECTION.STRING}` with the connection string for your Event Hubs namespace. For instructions on getting the connection string, see [Get an Event Hubs connection string](event-hubs-get-connection-string.md). Here's an example configuration: `sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=XXXXXXXXXXXXXXXX";`

+> [!NOTE]

+> When using SAS authentication with Kafka clients, established connections aren't disconnected when the SAS key is regenerated.

+> [!NOTE]

+> [Generated shared access signature tokens](authenticate-shared-access-signature.md#generate-a-shared-access-signature-token) are not supported when using the Event Hubs for Apache Kafka endpoint.

+## Samples

+For a **tutorial** with step-by-step instructions to create an event hub and access it using SAS or OAuth, see [Quickstart: Data streaming with Event Hubs using the Kafka protocol](event-hubs-quickstart-kafka-enabled-event-hubs.md).

+## Other Event Hubs features

+The Event Hubs for Apache Kafka feature is one of three protocols concurrently available on Azure Event Hubs, complementing HTTP and AMQP. You can write with any of these protocols and read with any another, so that your current Apache Kafka producers can continue publishing via Apache Kafka, but your reader can benefit from the native integration with Event Hubs' AMQP interface, such as Azure Stream Analytics or Azure Functions. Conversely, you can readily integrate Azure Event Hubs into AMQP routing networks as a target endpoint, and yet read data through Apache Kafka integrations.

+Additionally, Event Hubs features such as [Capture](event-hubs-capture-overview.md), which enables extremely cost efficient long-term archival via Azure Blob Storage and Azure Data Lake Storage, and [Geo Disaster-Recovery](event-hubs-geo-dr.md) also work with the Event Hubs for Kafka feature.

+## Idempotency

+Azure Event Hubs for Apache Kafka supports both idempotent producers and idempotent consumers.

+One of the core tenets of Azure Event Hubs is the concept of **at-least once** delivery. This approach ensures that events will always be delivered. It also means that events can be received more than once, even repeatedly, by consumers such as a function. For this reason, it's important that the consumer supports the [idempotent consumer](https://microservices.io/patterns/communication-style/idempotent-consumer.html) pattern.

+## Apache Kafka feature differences

+The goal of Event Hubs for Apache Kafka is to provide access to Azure Event Hubs capabilities to applications that are locked into the Apache Kafka API and would otherwise have to be backed by an Apache Kafka cluster.

+As explained [above](#is-apache-kafka-the-right-solution-for-your-workload), the Azure Messaging fleet provides rich and robust coverage for a multitude of messaging scenarios, and although the following features aren't currently supported through Event Hubs' support for the Apache Kafka API, we point out where and how the desired capability is available.

+### Transactions

+[Azure Service Bus](../service-bus-messaging/service-bus-transactions.md) has robust transaction support that allows receiving and settling messages and sessions while sending outbound messages resulting from message processing to multiple target entities under the consistency protection of a transaction. The feature set not only allows for exactly once processing of each message in a sequence, but also avoids the risk of another consumer inadvertently reprocessing the same messages as it would be the case with Apache Kafka. Service Bus is the recommended service for transactional message workloads.

+### Compression

+The client-side [compression](https://cwiki.apache.org/confluence/display/KAFKA/Compression) feature of Apache Kafka compresses a batch of multiple messages into a single message on the producer side and decompresses the batch on the consumer side. The Apache Kafka broker treats the batch as a special message.

+This feature is fundamentally at odds with Azure Event Hubs' multi-protocol model, which allows for messages, even those sent in batches, to be individually retrievable from the broker and through any protocol.

+The payload of any Event Hubs event is a byte stream and the content can be compressed with an algorithm of your choosing. The Apache Avro encoding format supports compression natively.

+### Kafka Streams

+Kafka Streams is a client library for stream analytics that is part of the Apache Kafka open-source project, but is separate from the Apache Kafka event stream broker.

+The most common reason Azure Event Hubs customers ask for Kafka Streams support is because they're interested in Confluent's "ksqlDB" product. "ksqlDB" is a proprietary shared source project that is [licensed such](https://github.com/confluentinc/ksql/blob/master/LICENSE) that no vendor "offering software-as-a-service, platform-as-a-service, infrastructure-as-a-service, or other similar online services that compete with Confluent products or services" is permitted to use or offer "ksqlDB" support. Practically, if you use ksqlDB, you must either operate Kafka yourself or you must use ConfluentΓÇÖs cloud offerings. The licensing terms might also affect Azure customers who offer services for a purpose excluded by the license.

+Standalone and without ksqlDB, Kafka Streams has fewer capabilities than many alternative frameworks and services, most of which have built-in streaming SQL interfaces, and all of which integrate with Azure Event Hubs today:

+- [Azure Stream Analytics](../stream-analytics/stream-analytics-introduction.md)

+- [Azure Synapse Analytics (via Event Hubs Capture)](../event-grid/event-grid-event-hubs-integration.md)

+- [Azure Databricks](/azure/databricks/scenarios/databricks-stream-from-eventhubs)

+- [Apache Samza](https://samza.apache.org/learn/documentation/latest/connectors/eventhubs)

+- [Apache Storm](event-hubs-storm-getstarted-receive.md)

+- [Apache Spark](event-hubs-kafka-spark-tutorial.md)

+- [Apache Flink](event-hubs-kafka-flink-tutorial.md)

+- [Akka Streams](event-hubs-kafka-akka-streams-tutorial.md)

+The listed services and frameworks can generally acquire event streams and reference data directly from a diverse set of sources through adapters. Kafka Streams can only acquire data from Apache Kafka and your analytics projects are therefore locked into Apache Kafka. To use data from other sources, you're required to first import data into Apache Kafka with the Kafka Connect framework.

+If you must use the Kafka Streams framework on Azure, [Apache Kafka on HDInsight](../hdinsight/kafk) will provide you with that option. Apache Kafka on HDInsight provides full control over all configuration aspects of Apache Kafka, while being fully integrated with various aspects of the Azure platform, from fault/update domain placement to network isolation to monitoring integration.

+## Next steps

+This article provided an introduction to Event Hubs for Kafka. To learn more, see [Apache Kafka developer guide for Azure Event Hubs](apache-kafka-developer-guide.md).

+For a **tutorial** with step-by-step instructions to create an event hub and access it using SAS or OAuth, see [Quickstart: Data streaming with Event Hubs using the Kafka protocol](event-hubs-quickstart-kafka-enabled-event-hubs.md).

+Also, see the [OAuth samples on GitHub](https://github.com/Azure/azure-event-hubs-for-kafka/tree/master/tutorials/oauth).

+> For Apache Kafka clients, an **event hub** maps to a **Kafka topic**. For more mappings between Azure Event Hubs and Apache Kafka, see [Kafka and Event Hubs conceptual mapping](azure-event-hubs-kafka-overview.md#kafka-and-event-hubs-conceptual-mapping)

+Event Hubs is a fully managed Platform-as-a-Service (PaaS) with little configuration or management overhead, so you focus on your business solutions. [Event Hubs for Apache Kafka ecosystems](azure-event-hubs-kafka-overview.md) gives you the PaaS Kafka experience without having to manage, configure, or run your clusters.

+[Event Hubs for Apache Kafka ecosystems](azure-event-hubs-kafka-overview.md) furthermore enables [Apache Kafka (1.0 and later)](https://kafka.apache.org/) clients and applications to talk to Event Hubs. You don't need to set up, configure, and manage your own Kafka and Zookeeper clusters or use some Kafka-as-a-Service offering not native to Azure.

+ > Azure Event Hubs provides you with a Kafka endpoint. This endpoint enables your Event Hubs namespace to natively understand [Apache Kafka](https://kafka.apache.org/intro) message protocol and APIs. With this capability, you can communicate with your event hubs as you would with Kafka topics without changing your protocol clients or running your own clusters. Event Hubs supports [Apache Kafka versions 1.0](https://kafka.apache.org/10/documentation.html) and later. For more information, see [Use Event Hubs from Apache Kafka applications](azure-event-hubs-kafka-overview.md).

+- [Use Azure Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md)

+* [Learn about Event Hubs for Kafka](azure-event-hubs-kafka-overview.md)

+> [The protocol support for **Apache Kafka** clients](azure-event-hubs-kafka-overview.md) (versions >=1.0) provides network endpoints that enable applications built to use Apache Kafka with any client to use Event Hubs. Most existing Kafka applications can simply be reconfigured to point to an Event Hub namespace instead of a Kafka cluster bootstrap server.

+* Read through the [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md) article.

+- Read through the [Event Hubs for Apache Kafka](./azure-event-hubs-kafka-overview.md) introduction article

+> - Debezium will auto-create a topic per table and a bunch of metadata topics. Kafka **topic** corresponds to an Event Hubs instance (event hub). For Apache Kafka to Azure Event Hubs mappings, see [Kafka and Event Hubs conceptual mapping](azure-event-hubs-kafka-overview.md#kafka-and-event-hubs-conceptual-mapping).

+- Read through the [Event Hubs for Apache Kafka](./azure-event-hubs-kafka-overview.md) introduction article

+This tutorial shows you how to connect Apache Flink to an event hub without changing your protocol clients or running your own clusters. For more information on Event Hubs' support for the Apache Kafka consumer protocol, see [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md).

+* Read through the [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md) article.

+* Read through the [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md) article.

+* Read through the [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md) article.

+* Read through the [Event Hubs for Apache Kafka](azure-event-hubs-kafka-overview.md) article.

+ Title: 'Locations and connectivity providers for Azure ExpressRoute'

+description: This article provides a detailed overview of available providers and services per each ExpressRoute location to connect to Azure regions.

+# ExpressRoute peering locations and connectivity partners

+ExpressRoute locations (sometimes referred to as peering locations or meet-me-locations) are co-location facilities where Microsoft Enterprise edge (MSEE) devices are located. ExpressRoute locations are the entry point to Microsoft's network ΓÇô and are globally distributed, providing customers the opportunity to connect to Microsoft's network around the world. These locations are where ExpressRoute partners and ExpressRoute Direct customers issue cross connections to Microsoft's network. In general, the ExpressRoute location doesn't need to match the Azure region. For example, a customer can create an ExpressRoute circuit with the resource location *East US*, in the *Seattle* Peering location.

+You'll have access to Azure services across all regions within a geopolitical region if you connected to at least one ExpressRoute location within the geopolitical region.

+* **Local Azure Regions** refers to the regions that can be accessed by [ExpressRoute Local](expressroute-faqs.md#expressroute-local) at each peering location. **n/a** indicates that ExpressRoute Local isn't available at that peering location.

+* **ER Direct** refers to [ExpressRoute Direct](expressroute-erdirect-about.md) support at each peering location. If you want to view the available bandwidth at a location, see [Determine available bandwidth](expressroute-howto-erdirect.md#resources)

+| **Los Angeles** | [CoreSite LA1](https://www.coresite.com/data-centers/locations/los-angeles/one-wilshire) | 1 | n/a | Supported | CoreSite, Equinix*, Megaport, Neutrona Networks, NTT, Zayo</br></br> **New ExpressRoute circuits are no longer supported with Equinix in Los Angeles. Create new circuits in Los Angeles2.* |

+Azure national clouds are isolated from each other and from global commercial Azure. ExpressRoute for one Azure cloud can't connect to the Azure regions in the others.

+If your connectivity provider isn't listed in previous sections, you can still create a connection.

+* Check with your connectivity provider to see if they're connected to any of the exchanges in the table above. You can check the following links to gather more information about services offered by exchange providers. Several connectivity providers are already connected to Ethernet exchanges.

+If you're remote and don't have fiber connectivity or want to explore other connectivity options, you can check the following satellite operators.

+* Ensure that all prerequisites are met. For more information, see [ExpressRoute prerequisites & checklist](expressroute-prerequisites.md).

+Insight Priorities are determined by MicrosoftΓÇÖs assessment of the potential impact of each insight. For instance, high severity insights may include vulnerabilities that are new, exploited frequently, particularly damaging, or easily exploited by hackers with a lower skill level. Low severity insights may include use of deprecated technology that is no longer supported, infrastructure that will soon expire, or compliance issues that do not align with security best practices. Each insight contains suggested remediation actions to protect against potential exploits.

+Some insights will be flagged with "Potential" in the title. A "Potential" insight occurs when Defender EASM is unable to confirm that an asset is impacted by a vulnerability. This is common when our scanning system detects the presence of a specific service but cannot detect the version number; for example, some services enable administrators to hide version information. Vulnerabilities are often associated with specific versions of the software, so manual investigation is required to determine whether the asset is impacted. Other vulnerabilities can be remediated by steps that Defender EASM is unable to detect. For instance, users can make recommended changes to service configurations or run backported patches. If an insight is prefaced with "Potential", the system has reason to believe that the asset is impacted by the vulnerability but is unable to confirm it for one of the above listed reasons. To manually investigate, please click the insight name to review remediation guidance that can help you determine whether your assets are impacted.

+A user will usually decide to first investigate any High Severity Observations. You can click the top-listed observation to be directly routed to a list of impacted assets, or instead select ΓÇ£View All __ InsightsΓÇ¥ to see a comprehensive, expandable list of all potential observations within that severity group.

+The Observations page features a list of all potential insights in the left-hand column. This list is sorted by the number of assets that are impacted by each security risk, displaying the issues that impact the greatest number of assets first. To view the details of any security risk, simply click on it from this list.

+This section helps users understand how their IP space is managed, detecting services that are exposed on the open internet. Attackers commonly scan ports across the internet to look for known exploits related to service vulnerabilities or misconfigurations. Microsoft identifies these open ports to complement vulnerability assessment tools, flagging observations for review to ensure they are properly managed by your information technology team.

+ Alternatively, users can manually input their seeds. Defender EASM accepts organization names, domains, IP blocks, hosts, email contacts, ASNs, and WhoIs organizations as seed values. You can also specify entities to exclude from asset discovery to ensure they are not added to your inventory if detected. For example, this is useful for organizations that have subsidiaries that will likely be connected to their central infrastructure, but do not belong to your organization.

+The Discovery page defaults to a list view of Discovery Groups, but users can also view lists of all seeds and excluded entities from this page. Simply click either tab to view a list of all the seeds or exclusions that power your discovery groups.

+- [Understanding dashboards](understanding-dashboards.md)

+- Organization Names

+- Domains

+- IP Blocks

+- Hosts

+- Email Contacts

+- ASNs

+- Whois organizations

+For IP Group limits, see [Azure subscription and service limits, quotas, and constraints](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-firewall-limits)

+- [Tutorial: Secure your virtual WAN using Azure Firewall Manager](secure-cloud-network.md)

+For IP Group limits, see [Azure subscription and service limits, quotas, and constraints](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-firewall-limits)

+| 2.7.206 |Stable |January 2021 |Configuration through REST API (orchestrated mode), supports Samples telemetry format and PubSub format - [Release notes](https://github.com/Azure/Industrial-IoT/releases/tag/2.7.206)|

+ Title: Connect an IoT Edge transparent gateway to an application

+description: How to connect devices through an IoT Edge transparent gateway to an IoT Central application. The article shows how to use the IoT Edge 1.4 runtime.

+This article shows how to implement the scenario by using the IoT Edge 1.4 runtime.

+- [Transparent gateway manifest (EdgeTransparentGatewayManifest.json)](https://raw.githubusercontent.com/Azure-Samples/iot-central-docs-samples/main/transparent-gateway-1-4/EdgeTransparentGatewayManifest.json) - this file is the IoT Edge deployment manifest for the gateway device.

+> To learn how to deploy the IoT Edge runtime to a physical device, see [Create an IoT Edge device](../../iot-edge/how-to-create-iot-edge-device.md) in the IoT Edge documentation.

+To try out the transparent gateway scenario, select the following button to deploy two Linux virtual machines. One virtual machine has the IoT Edge 1.4 runtime installed and is the transparent IoT Edge gateway. The other virtual machine is a downstream device where you run code to send simulated thermostat telemetry:

+[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fiot-central-docs-samples%2Fmain%2Ftransparent-gateway-1-4%2FDeployGatewayVMs.json)

+ :::image type="content" source="media/how-to-connect-iot-edge-transparent-gateway/iot-edge-runtime-1-4.png" alt-text="Screenshot showing the $edgeAgent and $edgeHub version 1.4 modules running on the IoT Edge gateway." lightbox="media/how-to-connect-iot-edge-transparent-gateway/iot-edge-runtime-1-4.png":::

+ wget https://raw.githubusercontent.com/Azure-Samples/iot-central-docs-samples/main/transparent-gateway-1-4/provision_device.py

+ wget https://raw.githubusercontent.com/Azure-Samples/iot-central-docs-samples/main/transparent-gateway-1-4/simple_thermostat.py

+ Title: Transform data for an IoT Central application

+description: IoT devices send data in various formats that you may need to transform. This article describes how to transform data both on the way in and out of IoT Central.

+1. Save a copy of the deployment manifest to your local development machine: [moduledeployment.json](https://raw.githubusercontent.com/Azure-Samples/iot-central-docs-samples/main/iotedge/moduledeployment.json).

+1. Find the `registryCredentials` section and replace the placeholders with the values you made a note of when you created the Azure container registry. The `address` value looks like `{your username}.azurecr.io`.

+1. Find the `settings` section for the `transformmodule`. Replace `{your username}` with the same value you used in the previous step. Save the changes.

+1. Enter *IoT Edge gateway device* as the device template name. Select **This is a gateway device**.

+1. In the model, select **Relationships**. Don't select **Relationships** in the **transformmodule** module.

+1. Select **Add relationship**.

+1. Enter *Downstream Sensor* as the display name, *sensor* as the name, and select **Any** as the target. Select **Save**.

+1. Select **IoT Edge gateway device** and select **+ New**. Enter *IoT Edge gateway device* as the device name, enter *gateway-01* as the device ID, make sure **IoT Edge gateway device** is selected as the device template and **No** is selected as **Simulate this device?**. Select **Transformer** as the edge manifest. Select **Create**.

+1. In the list of devices, click on the **Downstream 01** device, then select **Manage device > Attach to gateway**.

+1. In the **Attach to a gateway** dialog, select the **IoT Edge gateway device** device template, and the **IoT Edge gateway device** device instance. Select **Attach**.

+1. On the **Downstream 01** device, select **Connect**.

+For convenience, this article uses Azure virtual machines to run the gateway and downstream devices. To create the two Azure virtual machines, select the **Deploy to Azure** button shown after the following table. Use the information in the table to complete the **Custom deployment** form:

+[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fiot-central-docs-samples%2Fmain%2Ftransparent-gateway-1-4%2FDeployGatewayVMs.json)

+1. Open the *config.toml* file in a text editor. For example:

+ sudo nano /etc/aziot/config.toml

+1. Uncomment and modify the certificate settings as follows:

+ trust_bundle_cert = "file:///home/AzureUser/certs/certs/azure-iot-test-only.root.ca.cert.pem"

+ ...

+ [edge_ca]

+ cert = "file:///home/AzureUser/certs/certs/iot-edge-device-ca-mycacert-full-chain.cert.pem"

+ pk = "file:///home/AzureUser/certs/private/iot-edge-device-ca-mycacert.key.pem"

+ The previous example assumes you're signed in as **AzureUser** and created a device CA certificated called "mycacert".

+1. Save the changes and run the following command to verify that the *config.toml* file is correct:

+ sudo iotedge config apply

+If the runtime doesn't start, check the changes you made in *config.toml* and see [Troubleshoot your IoT Edge device](../../iot-edge/troubleshoot.md).

+This scenario uses the same Azure Functions deployment as the IoT Central device bridge. To deploy the device bridge, select the **Deploy to Azure** button shown after the following table. Use the information in the table to complete the **Custom deployment** form:

+| .NET|[NuGet](https://www.nuget.org/packages/Microsoft.Azure.Management.DeviceProvisioningServices) |[GitHub](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/deviceprovisioningservices/Microsoft.Azure.Management.DeviceProvisioningServices)| [Reference](/dotnet/api/overview/azure/resourcemanager.deviceprovisioningservices-readme) |

+| Node | [npm](https://www.npmjs.com/package/azure-iothub) | [GitHub](https://github.com/Azure/azure-iot-sdk-node) | | [Reference](/javascript/api/azure-iothub/) |

+- Change the size of the IoT hub. For example, migrate a hub from the B1 tier to the B2 tier to increase the number of messages that each unit can support per day from 400,000 to 6 million.

+Both these changes can occur without interrupting existing operations.

+- * Upgrade to a higher tier. For example, upgrade a hub from the B1 tier to the S1 tier for access to advanced features with the same messaging capacity.

+> When you are upgrading your IoT Hub to a higher tier, some messages may be received out of order for a short period of time. If your business logic relies on the order of messages, we recommend upgrading during non-business hours.

+> In the following table, replace `<storage>` with the name of the default storage account for your Azure Machine Learning workspace. Replace `<region>` with the region of your workspace.

+| Integrated notebook | \*.\<region\>.notebooks.azure.net | TCP | 443 |

+These Datastore URIs are a known implementation of [Filesystem spec](https://filesystem-spec.readthedocs.io/en/latest/https://docsupdatetracker.net/index.html) (`fsspec`): A unified pythonic interface to local, remote and embedded file systems and bytes storage.

+- [Access data in a job](how-to-read-write-data-v2.md)

+* Azure Machine Learning studio

+ **Download the file**:

+ 1. Sign in to [Azure Machine Learning studio](https://ml.azure.com)

+ 1. In the upper right Azure Machine Learning studio toolbar, select your workspace name.

+ 1. Select the **Download config file** link.

+ :::image type="content" source="media/how-to-configure-environment/configure.png" alt-text="Screenshot shows how to download your config file." lightbox="media/how-to-configure-environment/configure.png":::

+* Some operations may be executed directly using the MLflow fluent API (`mlflow.<method>`). However, others may require to create an MLflow client, which allows to communicate with Azure Machine Learning in the MLflow protocol. You can create an `MlflowClient` object as follows. This tutorial will use the object `client` to refer to such MLflow client.

+ ```python

+ using mlflow

+ client = mlflow.tracking.MlflowClient()

+ ```

+> [!TIP]

+> We recommend to register models from runs or using the method `mlflow.<flavor>.log_model` from inside the run as it keeps lineage from the job that generated the asset.

+You can query all the registered models in the registry using the MLflow client. The following sample prints all the model's names:

+ * __AzureFrontDoor.frontend outbound__: Azure Front Door is used by the Azure Machine Learning studio UI and AutoML. Instead of allowing outbound to the service tag (AzureFrontDoor.frontend), switch to the following fully qulified domain names (FQDN). Switching to these FQDNs removes unnecessary outbound traffic included in the service tag and allows only what is needed for Azure Machine Learning studio UI and AutoML.

+ __Allow__ outbound traffic over __TCP port 443__ to the following FQDNs:

+To restrict communication between a deployment and external resources, including the Azure resources it uses, set the deployment's `egress_public_network_access` flag to `disabled`. Use this flag to ensure that the download of the model, code, and images needed by your deployment are secured with a private endpoint. Note that disabling the flag alone is not enough ΓÇö your workspace must also have a private link that allows access to Azure resources via a private endpoint. See the [Prerequisites](#prerequisites) for more details.

+> [!NOTE]

+> For online deployments with `egress_public_network_access` flag set to `disabled`, access from the deployments to Microsoft Container Registry (MCR) is restricted. If you want to leverage container images from MCR (such as when using curated environment or mlflow no-code deployment), recommendation is to push the images into the Azure Container Registry (ACR) which is attached with the workspace. The images in this ACR is accessible to secured deployments via the private endpoints which are automatically created on behalf of you when you set `egress_public_network_access` flag to `disabled`. For a quick example, please refer to this [custom container example](https://github.com/Azure/azureml-examples/tree/main/cli/endpoints/online/custom-container/minimal/single-model).

+Azure Machine Learning supports submission of standalone machine learning jobs, and creation of [machine learning pipelines](./concept-ml-pipelines.md), that involve multiple machine learning workflow steps. Azure Machine Learning handles both standalone Spark job creation, and creation of reusable Spark components that Azure Machine Learning pipelines can use. In this article, you'll learn how to submit Spark jobs using:

+- Azure Machine Learning Studio UI

+# [Studio UI](#tab/ui)

+These prerequisites cover the submission of a Spark job from Azure Machine Learning Studio UI:

+- An Azure subscription; if you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free) before you begin.

+- An Azure Machine Learning workspace. See [Create workspace resources](./quickstart-create-resources.md).

+- To enable this feature:

+ 1. Navigate to Azure Machine Learning Studio UI.

+ 2. Select **Manage preview features** (megaphone icon) from the icons on the top right side of the screen.

+ 3. In **Managed preview feature** panel, toggle on **Run notebooks and jobs on managed Spark** feature.

+ :::image type="content" source="media/interactive-data-wrangling-with-apache-spark-azure-ml/how_to_enable_managed_spark_preview.png" alt-text="Screenshot showing option for enabling Managed Spark preview.":::

+- [(Optional): An attached Synapse Spark pool in the Azure Machine Learning workspace](./how-to-manage-synapse-spark-pool.md).

+Spark jobs can use either user identity passthrough, or a managed identity, to access data and other resources. The following table summarizes the different mechanisms for resource access while using Azure Machine Learning Managed (Automatic) Spark compute and attached Synapse Spark pool.

+If the CLI or SDK code defines an option to use managed identity, Azure Machine Learning Managed (Automatic) Spark compute uses user-assigned managed identity attached to the workspace. You can attach a user-assigned managed identity to an existing Azure Machine Learning workspace using Azure Machine Learning CLI v2, or with `ARMClient`.

+1. Create a YAML file that defines the user-assigned managed identity that should be attached to the workspace:

+1. With the `--file` parameter, use the YAML file in the `az ml workspace update` command to attach the user assigned managed identity:

+1. Install [DMClient](https://github.com/projectkudu/ARMClient), a simple command line tool that invokes the Azure Resource Manager API.

+1. Create a JSON file that defines the user-assigned managed identity that should be attached to the workspace:

+1. Execute the following command in the PowerShell prompt or the command prompt, to attach the user-assigned managed identity to the workspace.

+> - To ensure successful execution of the Spark job, assign the **Contributor** and **Storage Blob Data Contributor** roles, on the Azure storage account used for data input and output, to the identity that the Spark job uses

+> - If an [attached Synapse Spark pool](./how-to-manage-synapse-spark-pool.md) points to a Synapse Spark pool, in an Azure Synapse workspace that has a managed virtual network associated with it, [a managed private endpoint to storage account should be configured](../synapse-analytics/security/connect-to-a-secure-storage-account.md) to ensure data access.

+A Python script developed by [interactive data wrangling](./interactive-data-wrangling-with-apache-spark-azure-ml.md) can be used to submit a batch job to process a larger volume of data, after making necessary changes for Python script parameterization. A simple data wrangling batch job can be submitted as a standalone Spark job.

+A Spark job requires a Python script that takes arguments, which can be developed with modification of the Python code developed from [interactive data wrangling](./interactive-data-wrangling-with-apache-spark-azure-ml.md). A sample Python script is shown here.

+To create a job, a standalone Spark job can be defined as a YAML specification file, which can be used in the `az ml job create` command, with the `--file` parameter. Define these properties in the YAML file as follows:

+- `args` - the command line arguments that should be passed to the job entry point Python script or class. See the YAML specification file provided here for an example.

+> To use an attached Synapse Spark pool, define the `compute` property in the sample YAML specification file shown above, instead of the `resources` property.

+- `driver_memory`: the allocated memory for the Spark driver, with a size unit suffix `k`, `m`, `g` or `t` (for example, `512m`, `2g`).

+- `executor_memory`: the allocated memory for the Spark executor, with a size unit suffix `k`, `m`, `g` or `t` (for example, `512m`, `2g`).

+- `args` - the command line arguments that should be passed to the job entry point Python script or class. See the sample code provided here for an example.

+> To use an attached Synapse Spark pool, define the `compute` parameter in the `azure.ai.ml.spark` function, instead of `resources`.

+# [Studio UI](#tab/ui)

+This functionality isn't available in the Studio UI. The Studio UI doesn't support this feature.

+### Submit a standalone Spark job from Azure Machine Learning Studio UI

+To submit a standalone Spark job using the Azure Machine Learning Studio UI:

+ 1. To add any another Python file(s) required by the standalone job at runtime, select **+ Add file** under **Py files** and input the name of the `.zip`, `.egg`, or `.py` file to be placed in the `PYTHONPATH` for successful execution of the job. Multiple files can be added.

+ 1. Enter an **Input name**. The input should refer to this name later in the **Arguments**.

+ - For **URI**, enter a storage data URI (for example, `abfss://` or `wasbs://` URI), or enter a data asset `azureml://`.

+ 1. Enter an **Output name**. The output should refer to this name later to in the **Arguments**.

+ 1. For **Output URI destination**, enter a storage data URI (for example, `abfss://` or `wasbs://` URI) or enter a data asset `azureml://`.

+A Spark component offers the flexibility to use the same component in multiple [Azure Machine Learning pipelines](./concept-ml-pipelines.md), as a pipeline step.

+- `inputs` - this property is similar to `inputs` property described in [YAML syntax for Spark job specification](#yaml-properties-in-the-spark-job-specification), except that it doesn't define the `path` property. This code snippet shows an example of the Spark component `inputs` property:

+- `outputs` - this property is similar to the `outputs` property described in [YAML syntax for Spark job specification](#yaml-properties-in-the-spark-job-specification), except that it doesn't define the `path` property. This code snippet shows an example of the Spark component `outputs` property:

+> A Spark component does not define `identity`, `compute` or `resources` properties. The pipeline YAML specification file defines these properties.

+The Spark component defined in the above YAML specification file can be used in an Azure Machine Learning pipeline job. See [pipeline job YAML schema](./reference-yaml-job-pipeline.md) to learn more about the YAML syntax that defines a pipeline job. This example shows a YAML specification file for a pipeline job, with a Spark component, and an Azure Machine Learning Managed (Automatic) Spark compute:

+> To use an attached Synapse Spark pool, define the `compute` property in the sample YAML specification file shown above, instead of `resources` property.

+To create an Azure Machine Learning pipeline with a Spark component, you should have familiarity with creation of [Azure Machine Learning pipelines from components, using Python SDK](./tutorial-pipeline-python-sdk.md#create-the-pipeline-from-components). A Spark component is created using `azure.ai.ml.spark` function. The function parameters are defined almost the same way as for the [standalone Spark job](#standalone-spark-job-using-python-sdk). These parameters are defined differently for the Spark component:

+> A Spark component created using `azure.ai.ml.spark` function does not define `identity`, `compute` or `resources` parameters. The Azure Machine Learning pipeline defines these parameters.

+This Python code snippet shows use of a managed identity, together with the creation of an Azure Machine Learning pipeline job. Additionally, it shows use of a Spark component and an Azure Machine Learning Managed (Automatic) Synapse compute:

+> To use an attached Synapse Spark pool, define the `compute` parameter in the `azure.ai.ml.spark` function, instead of `resources` parameter. For example, in the code sample shown above, define `spark_step.compute = "<ATTACHED_SPARK_POOL_NAME>"` instead of defining `spark_step.resources`.

+# [Studio UI](#tab/ui)

+This functionality isn't available in the Studio UI. The Studio UI doesn't support this feature.

+> Batch Endpoints can consume data stored in storage accounts instead of Azure Machine Learning Data Stores or Data Assets. However, you may need to configure additional permissions for the identity of the compute where the batch endpoint runs on. See [Security considerations when reading data](how-to-access-data-batch-endpoints-jobs.md#security-considerations-when-reading-data).

+1. Select __Pipeline__ > __Import from pipeline template__

+1. You will be prompted to select a `zip` file. Uses [the following template if using managed identities](https://azuremlexampledata.blob.core.windows.net/data/templates/batch-inference/Run-BatchEndpoint-MI.zip) or [the following one if using a service principal](https://azuremlexampledata.blob.core.windows.net/data/templates/batch-inference/Run-BatchEndpoint-SP.zip).

+1. A preview of the pipeline will show up in the portal. Click __Use this template__.

+1. The pipeline will be created for you with the name __Run-BatchEndpoint__.

+1. Configure the parameters of the batch deployment you are using:

+ > For best reusability, use the created pipeline as a template and call it from within other Azure Data Factory pipelines by leveraging the [Execute pipeline activity](../data-factory/control-flow-execute-pipeline-activity.md). In that case, do not configure the parameters in the inner pipeline but pass them as parameters from the outer pipeline as shown in the following image:

+ :::image type="content" source="media/quickstart-create-resources/compute-section.png" alt-text="Screenshot: shows Compute section on left hand side of screen." lightbox="media/quickstart-create-resources/compute-section.png":::

+ Title: "Quickstart: Submit Apache Spark jobs in Azure Machine Learning (preview)"

+> [!NOTE]

+> This document is deprecated as the API keys feature has been replaced by a new feature in Grafana 9.1. Go to [Service accounts](./how-to-service-accounts.md) to access the current recommended method to create and manage API keys.

+> [!TIP]

+> To switch to using service accounts, in Grafana instances created before the release of Grafana 9.1, go to **Configuration > API keys and select Migrate to service accounts now**. Select **Yes, migrate now**. Each existing API keys will be automatically migrated into a service account with a token. The service account will be created with the same permission as the API Key and current API keys will continue to work as before.

+API keys are disabled by default in Azure Managed Grafana. You can enable this feature during the creation of the instance in the Azure portal, or you can activate it on an existing instance, using the Azure portal or the CLI.

+ :::image type="content" source="media/create-api-keys/form.png" alt-text="Screenshot of the Grafana dashboard. API creation form is filled out.":::

+In this how-to guide, you learned how to create an API key for Azure Managed Grafana. When you're ready, start using service accounts as the new way to authenticate applications that interact with Grafana:

+> [User service accounts](how-to-service-accounts.md)

+ Title: How to use service accounts in Azure Managed Grafana

+description: In this guide, learn how to use service accounts in Azure Managed Grafana.

+# How to use service accounts in Azure Managed Grafana

+In this guide, learn how to use service accounts. Service accounts are used to run automated operations and authenticate applications in Grafana with the Grafana API.

+Common use cases include:

+- Provisioning or configuring dashboards

+- Scheduling reports

+- Defining alerts

+- Setting up an external SAML authentication provider

+- Interacting with Grafana without signing in as a user

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free).

+- An Azure Managed Grafana instance. If you don't have one yet, [create an Azure Managed Grafana instance](./quickstart-managed-grafana-portal.md).

+## Enable service accounts

+Service accounts are disabled by default in Azure Managed Grafana. If your existing Grafana workspace doesn't have service accounts enabled, you can enable them by updating the preference settings of your Grafana instance.

+### [Portal](#tab/azure-portal)

+ 1. In the Azure portal, under **Settings**, select **Configuration**, and then under **API keys and service accounts**, select **Enable**.

+ :::image type="content" source="media/service-accounts/enable.png" alt-text="Screenshot of the Azure platform. Enable service accounts.":::

+ 1. Select **Save** to confirm that you want to enable API keys and service accounts in Azure Managed Grafana.

+### [Azure CLI](#tab/azure-cli)

+1. Azure Managed Grafana CLI extension 0.3.0 or above is required. To update your extension, run `az extension update --name amg`.

+1. Run the [az grafana update](/cli/azure/grafana#az-grafana-update) command to enable the creation of API keys and service accounts in an existing Azure Managed Grafana instance. In the command below, replace `<azure-managed-grafana-name>` with the name of the Azure Managed Grafana instance to update.

+```azurecli-interactive

+az grafana update --name <azure-managed-grafana-name> service-account Enabled

+```

+## Create a service account

+Follow the steps below to create a new Grafana service account and list existing service accounts:

+### [Portal](#tab/azure-portal)

+1. Go to your Grafana instance endpoint, and under **Configuration**, select **Service accounts**.

+1. Select **Add service account**, and enter a **Display name** and a **Role** for your new Grafana service account: *Viewer*, *Editor* or *Admin* and select **Create**.

+ :::image type="content" source="media/service-accounts/service-accounts.png" alt-text="Screenshot of Grafana. Add service account page.":::

+1. The page displays the notification *Service account successfully created* and some information about your new service account.

+1. Select the back arrow sign to view a list of all the service accounts of your Grafana instance.

+### [Azure CLI](#tab/azure-cli)

+Run the `az grafana service-account create` command to create a service account. Replace the placeholders `<azure-managed-grafana-name>`, `<service-account-name>` and `<role>` with your own information.

+Available roles: `Admin`, `Editor`, `Viewer`.

+```azurecli-interactive

+az grafana service-account create --name <azure-managed-grafana-name> --service-account <service-account-name> --role <role>

+```

+#### List service accounts

+Run the `az grafana service-account list` command to get a list of all service accounts that belong to a given Azure Managed Grafana instance. Replace `<azure-managed-grafana-name>` with the name of your Azure Managed Grafana workspace.

+```azurecli-interactive

+az grafana service-account list --name <azure-managed-grafana-name> --output table

+```

+Example of output:

+```output

+AvatarUrl IsDisabled Login Name OrgId Role Tokens

+-- -- - - --

+/avatar/abc12345678 False sa-account1 account1 1 Viewer 0

+```

+#### Display service account details

+Run the `az grafana service-account show` command to get the details of a service account. Replace `<azure-managed-grafana-name>` and `<service-account-name>` with your own information.

+```azurecli-interactive

+az grafana service-account show --name <azure-managed-grafana-name> --service-account <service-account-name>

+```

+## Add a service account token and review tokens

+Once you've created a service account, add one or more access tokens. Access tokens are generated strings used to authenticate to the Grafana API.

+### [Portal](#tab/azure-portal)

+1. To create a service account token, select **Add token**.

+1. Use the automatically generated **Display name** or enter a name of your choice, and optionally select an **Expiration date** or keep the default option to set no expiry date.

+ :::image type="content" source="media/service-accounts/add-service-account-token.png" alt-text="Screenshot of the Azure platform. Add service account token page.":::

+1. Select **Generate token**, and take note of the token generated. This token will only be shown once, so make sure you save it, as loosing a token requires creating a new one.

+1. Select the service account to access information about your service account, including a list of all associated tokens.

+### [Azure CLI](#tab/azure-cli)

+#### Create a new token

+1. Create a Grafana service account token with `az grafana service-account token create`. Replace the placeholders `<azure-managed-grafana-name>`, `<service-account-name>` and `<token-name>` with your own information.

+ Optionally set an expiry time:

+ | Parameter | Description | Example |

+ ||-|-|

+ | `--time-to-live` | Tokens have an unlimited expiry date by default. Set an expiry time to disable the token after a given time. Use `s` for seconds, `m` for minutes, `h` for hours, `d` for days, `w` for weeks, `M` for months or `y` for years. | `15d` |

+ ```azurecli-interactive

+ az grafana service-account token create --name <azure-managed-grafana-name> --service-account <service-account-name> --token <token-name> --time-to-live 15d

+ ```

+1. Take note of the generated token. This token will only be shown once, so make sure you save it, as loosing a token requires creating a new one.

+#### List service account tokens

+Run the `az grafana service-account token list` command to get a list of all tokens that belong to a given service account. Replace the placeholders `<azure-managed-grafana-name>` and `<service-account-name>` with your own information.

+```azurecli-interactive

+az grafana service-account token list --name <azure-managed-grafana-name> --service-account <service-account-name> --output table

+```

+Example of output:

+```output

+Created Expiration HasExpired Name SecondsUntilExpiration

+-- --

+2022-12-07T11:40:45Z 2022-12-08T11:40:45Z False token1 85890.870731556

+2022-12-07T11:42:35Z 2022-12-22T11:42:35Z False token2 0

+```

+## Edit a service account

+In this section, learn how to update a Grafana service account in the following ways:

+- Edit the name of a service account

+- Edit the role of a service account

+- Disable a service account

+- Enable a service account

+### [Portal](#tab/azure-portal)

+Actions:

+- To edit the name, select the service account and under **Information** select **Edit**.

+- To edit the role, select the service account and under **Information**, select the role and choose another role name.

+- To disable a service account, select a service account and at the top of the page select **Disable service account**, then select **Disable service account** to confirm. Disabled service accounts can be re-enabled by selecting **Enable service account**.

+The notification *Service account updated* is instantly displayed.

+### [Azure CLI](#tab/azure-cli)

+Edit a service account with `az grafana service-account update`. Replace the placeholders `<azure-managed-grafana-name>`, and `<service-account-name>` with your own information and use one or several of the following parameters:

+| Parameter | Description |

+|--|-|

+| `--is-disabled` | Enter `--is-disabled true` disable a service account, or `--is-disabled false` to enable a service account. |

+| `--name` | Enter another name for your service account. |

+| `--role` | Enter another role for your service account. Available roles: `Admin`, `Editor`, `Viewer`. |

+```azurecli-interactive

+az grafana service-account update --name <azure-managed-grafana-name> --service-account <service-account-name> --role <role> --enabled false

+```

+To disable a service account run the `az grafana update` command and use the option `--is-disabled true`. To enable a service account, use `--is-disabled false`.

+```azurecli-interactive

+az grafana update --service-account Disabled --name <service-account-name>

+```

+## Delete a service account

+### [Portal](#tab/azure-portal)

+To delete a Grafana service account, select a service account and at the top of the page select **Delete service account**, then select **Delete service account** to confirm. Deleting a service account is final and a service account can't be recovered once deleted.

+### [Azure CLI](#tab/azure-cli)

+To delete a service account, run the `az grafana service-account delete` command. Replace the placeholders `<azure-managed-grafana-name>` and `<service-account-name>` with your own information.

+```azurecli-interactive

+az grafana service-account delete --name <azure-managed-grafana-name> --service-account <service-account-name>

+```

+## Delete a service account token

+### [Portal](#tab/azure-portal)

+To delete a service account token, select a service account and under **Tokens**, select **Delete (x)**. Select **Delete** to confirm.

+### [Azure CLI](#tab/azure-cli)

+To delete a service account, run the `az grafana service-account token delete` command. Replace the placeholders `<azure-managed-grafana-name>`, `<service-account-name>` and `<token-name>` with your own information.

+```azurecli-interactive

+az grafana service-account token delete --name <azure-managed-grafana-name> --service-account <service-account-name> --toke <token-name>

+```

+## Next steps

+In this how-to guide, you learned how to create and manage service accounts and tokens to run automated operations in Azure Managed Grafana. When you're ready, explore more articles:

+> [!div class="nextstepaction"]

+> [Enable zone redundancy](how-to-enable-zone-redundancy.md)

+ > [!NOTE]

+ > Customers can find their billing account ID in 2 ways. 1) In the [Azure portal](https://aka.ms/PrivateOfferAzurePortal) under **Cost Management + Billing** > **Properties** >**ID**. A user in the customer organization should have access to the billing account to see the ID in Azure Portal. 2) If the customer knows the subscription they plan to use for the purchase, click on **Subscriptions**, click on the relevant subscription **Properties** (or Billing Properties) **Billing Account ID**. See [Billing account scopes in the Azure portal](/azure/cost-management-billing/manage/view-all-accounts).

+

+- **Private offer terms** ΓÇô Specify the duration, accept-by date, and terms:

+ - **Start date** ΓÇô Choose **Accepted date** if you want the private offer to start as soon as the customer accepts it. If a private offer is extended to an existing customer of a Pay-as-you-go product, this will make the private price applicable for the entire month. To have your private offer start in an upcoming month, select **Specific month** and choose one. The start date for this option will always be the first day of the selected month.

+> Offers will be billed to customers in the customersΓÇÖ agreement currency, using the local market price that was published at the time the offer was created. The amount that customers pay, and that ISVs are paid, depends on the Foreign Exchange rates at the time the customer transacts the offer. Learn more on ["How we convert currency?"](/azure/marketplace/marketplace-geo-availability-currencies#how-we-convert-currency).

+If the price change was configured within the last 2 days, it can be canceled using the cancel button next to the price change expected on date and then publishing the changes. For a price change configured more than 2 days ago that has not yet taken effect, [submit a support request](https://partner.microsoft.com/support/?stage=1) that includes the Plan ID, price, and the market (if the change was market specific) in the request.

+ Title: Create a Power Automate flow with Azure Database for MySQL Flexible Server

+description: Create a Power Automate flow with Azure Database for MySQL Flexible Server

+# Tutorial: Create a Power Automate flow app with Azure Database for MySQL Flexible Server

+Power Automate is a service that helps you create automated workflows between your favorite apps and services to synchronize files, get notifications, collect data, and more. Here are a few examples of what you can do with Power Automate.

+- Automate business processes

+- Move business data between systems on a schedule

+- Connect to more than 500 data sources or any publicly available API

+- Perform CRUD (create, read, update, delete) operations on data

+In this quickstart shows how to create an automated workflow usingPower automate flow with [Azure database for MySQL connector](/connectors/azuremysql/).

+## Prerequisites

+* An account on [flow.microsoft.com](https://flow.microsoft.com).

+* An Azure account and subscription. If you don't have a subscription, [sign up for a free Azure account](https://azure.microsoft.com/free).

+- Create an Azure Database for MySQL Flexible server using [Azure portal](./quickstart-create-server-portal.md) <br/> or [Azure CLI](./quickstart-create-server-cli.md) if you don't have one.

+- Populate the database server with this [sample data](https://raw.githubusercontent.com/Azure-Samples/mysql-database-samples/main/mysqltutorial.org/mysql-classicmodesl.sql).

+[Having issues? Let us know](https://github.com/MicrosoftDocs/azure-docs/issues)

+## Overview of cloud flows

+Create a cloud flow when you want your automation to be triggered either automatically, instantly, or via a schedule. Here are types of flows you can create and then use with Azure database for MySQL connector.

+| **Flow type** | **Use case** | **Automation target** |

+|-|--|-|

+| Automated cloud flows | Create an automation that is triggered by an event such as arrival of an email from a specific person, or a mention of your company in social media.| Connectors for cloud or on-premises services connect your accounts and enable them to talk to each other. |

+| Instant cloud flows | Start an automation with a click of a button. You can automate for repetitive tasks from your Desktop or Mobile devices. For example, instantly send a reminder to the team with a push of a button from your mobile device. | Wide range of tasks such as requesting an approval, an action in Teams or SharePoint. |

+| Scheduled flows | Schedule an automation such as daily data upload to SharePoint or a database. |Tasks that need to be automated on a schedule.

+For this tutorial, we'll use **instant cloud flow* that can be triggered manually from any device, easy-to-share instant flows automate tasks so you donΓÇÖt have to repeat yourself.

+## Specify an event to start the flow

+Follow the steps to create an instant cloud flow with a manual trigger.

+1. In [Power Automate](https://flow.microsoft.com), select **Create** from the navigation bar on the left.

+2. Under **Start from blank*, select **Instant cloud flow**.

+3. Give your flow a name in the **Flow name" field and select **Manually trigger a flow**.

+ :::image type="content" source="./media/tutorial-power-automate-with-mysql/create-instant-cloud-flow.png" alt-text="Screenshot that shows how to create instant cloud flow app.":::

+

+4. Select the **Create** button at the bottom of the screen.

+## Create a MySQL operation

+An operation is an action. Power Automate flow allows you to add one or more advanced options and multiple actions for the same trigger. For example, add an advanced option that sends an email message as high priority. In addition to sending mail when an item is added to a list created in Microsoft Lists, create a file in Dropbox that contains the same information.

+1. Once the flow app is created, select **Next Step** to create an operation.

+2. In the box that shows Search connectors and actions, enter **Azure database for MySQL**.

+3. Select **Azure database for MySQL** connector and then select **Get Rows** operation. Get rows operation allows you to get all the rows from a table or query.

+ :::image type="content" source="./media/tutorial-power-automate-with-mysql/azure-mysql-connector-add-action.png" alt-text="Screenshot that shows how to view all the actions for Azure database for MySQL connector.":::

+5. Add a new MySQL connection and enter the **authentication type**,**server name**, **database name**, **username**, **password**. Select **encrypt connection** if SSL is enabled on your MySQL server.

+ :::image type="content" source="./media/tutorial-power-automate-with-mysql/add-mysql-connection-information.png" alt-text="Screenshot that adding a new MySQL connection for Azure Database for MySQL server.":::

+ > [!NOTE]

+ > If you get an error **Test connection failed. Details: Authentication to host `'servername'` for user `'username'` using method 'mysql_native_password' failed with message: Access denied for user `'username'@'IP address'`(using password: YES)**, please update the firewall rules on MySQL server in [Azure protal](https://portal.azure.com) with this IP address.

+

+5. After the connection is successfully added, provide the **servername, database name and table name** parameters for **Get Rows** operation using the newly added connection. Select **advanced options** to add more filters or limit the number of rows returned.

+

+ :::image type="content" source="./media/tutorial-power-automate-with-mysql/get-rows-from-table.png" alt-text="Screenshot that shows configuring Get Rows operation.":::

+6. Select **Save**.

+## Test and run your flow

+After saving the flow, we need to test it and run the flow app.

+1. Select **Flow checker** to see if there are any errors that need to be resolved.

+2. Select **Test** and then select **Manually** to test the trigger.

+3. Select **Run flow**.

+4. When the flow is successfully executed, you can select **click to download** in the output section to see the JSON response received.

+ :::image type="content" source="./media/tutorial-power-automate-with-mysql/run-flow-to-get-rows-from-table.png" alt-text="Screenshot that shows output of the run.":::

+## Next steps

+[Azure database for MySQL connector](/connectors/azuremysql/) reference

+# QuickStart: Get started with Datadog - An Azure Native ISV Service by creating new instance

+In this quickstart, you'll create a new instance of Datadog - An Azure Native ISV Service. You can either create a new Datadog organization or [link to an existing Datadog organization](link-to-existing-organization.md).

+- By default, metrics are collected for all resources, except virtual machines, Virtual Machine Scale Sets, and App Service plans.

+- Virtual machines, Virtual Machine Scale Sets, and App Service plan with _Include_ tags send metrics to Datadog.

+- Virtual machines, Virtual Machine Scale Sets, and App Service plan with _Exclude_ tags don't send metrics to Datadog.

+For example, the following screenshot shows a tag rule where only those virtual machines, Virtual Machine Scale Sets, and App Service plan tagged as _Datadog = True_ send metrics to Datadog.

+- [Manage the Datadog resource](manage.md)

+# Get support for Datadog - An Azure Native ISV Service

+This article describes how to contact support when working with Datadog - An Azure Native ISV Service. Before contacting support, see [Fix common errors](troubleshoot.md).

+To contact support about the Datadog - An Azure Native ISV Service, select **New Support request** in the left pane. Select the link to the Datadog portal.

+- For potential solutions, see [Fix common errors](troubleshoot.md).

+- To learn about making changes to your existing Datadog resource, see [Manage the Datadog resource](manage.md).

+Before creating your first instance of Datadog - An Azure Native ISV Service, [configure your environment](prerequisites.md). These steps must be completed before continuing with the next steps in this quickstart.

+Use the Azure portal to find Datadog - An Azure Native ISV Service.

+1. In the Marketplace, search for **Datadog - An Azure Native ISV Service**.

+If you're linking to an existing Datadog organization, select **Create** under the **Link Azure subscription to an existing Datadog organization**

+By default, Azure links your current Datadog organization to your Datadog resource. If you would like to link to a different organization, select the appropriate organization in the authentication window.

+- By default, metrics are collected for all resources, except **Virtual Machines, Virtual Machine Scale Sets, and App Service Plans**.

+- **Virtual Machines, Virtual Machine Scale Sets, and App Service Plans** with *Include* tags send metrics to Datadog.

+- **Virtual Machines, Virtual Machine Scale Sets, and App Service Plans** with *Exclude* tags don't send metrics to Datadog.

+For example, the screenshot shows a tag rule where only those **Virtual Machines, Virtual Machine Scale Sets, and App Service Plans** tagged as *Datadog = True* send metrics to Datadog.

+- [Manage the Datadog resource](manage.md)

+# Manage the Datadog - An Azure Native ISV Service resource

+This article shows how to manage the settings for your Datadog - An Azure Native ISV Service.

+The portal retrieves the appropriate Datadog application from Azure Active Directory. The app comes from the enterprise app name you selected when setting up integration. Select the Datadog app name:

+The portal retrieves all the available Datadog plans for your tenant. Select the appropriate plan and select on **Change Plan**.

+To enable the Azure integration with Datadog, go to **Overview**. Select **Enable** and **OK**. Selecting **Enable** retrieves any previous configuration for metrics and logs. The configuration determines which Azure resources emit metrics and logs to Datadog. After you complete this step, metrics and logs are sent to Datadog.

+- For help with troubleshooting, see [Troubleshooting Datadog solutions](troubleshoot.md).

+# What is Datadog - An Azure Native ISV Service?

+Datadog's offering in the Azure Marketplace enables you to manage Datadog in the Azure console as an integrated service. This availability means you can implement Datadog as a monitoring solution for your cloud workloads through a streamlined workflow. The workflow covers everything from procurement to configuration. The onboarding experience simplifies how you start monitoring the health and performance of your applications, whether they're based entirely in Azure or spread across hybrid or multicloud environments.

+You create the Datadog resources through a resource provider named `Microsoft.Datadog`. You can create and manage Datadog organization resources through the [Azure portal](https://portal.azure.com/). Datadog owns and runs the software as a service (SaaS) application including the organization and API keys.

+Datadog - An Azure Native ISV Service provides the following capabilities:

+- **Integrated onboarding** - Datadog is an integrated service on Azure. You can create a Datadog resource and manage the integration through the Azure portal.

+For more help using the Datadog - An Azure Native ISV service, see the following links to the [Datadog website](https://www.datadoghq.com/):

+- To create an instance of Datadog, see [QuickStart: Get started with Datadog - An Azure Native ISV Service](create.md).

+# Configure environment before Datadog - An Azure Native ISV Service deployment

+This article describes how to set up your environment before deploying your first instance of Datadog - An Azure Native ISV Service. These conditions are prerequisites for completing the quickstarts.

+To set up the Datadog - An Azure Native ISV Service, you must have **Owner** access on the Azure subscription. [Confirm that you have the appropriate access](../../role-based-access-control/check-access.md) before starting the setup.

+To use the Security Assertion Markup Language (SAML) single sign-on (SSO) feature within the Datadog resource, you must set up an enterprise application. To add an enterprise application, you need one of these roles: Global Administrator, Cloud Application Administrator, Application Administrator, or owner of the service principal.

+To create an instance of Datadog, see [QuickStart: Get started with Datadog - An Azure Native ISV Service](create.md).

+# Fix common errors for Datadog - An Azure Native ISV Service

+This document contains information about troubleshooting your solutions that use Datadog - An Azure Native ISV Service.

+ Use a different Azure subscription. Or, add or update the credit card or payment method for the subscription. For more information, see [updating the credit and payment method](/azure/cost-management-billing/manage/change-credit-card).

+ Use a different subscription. Or, check if your EA subscription is enabled for Marketplace purchase. For more information, see [Enable Marketplace purchases](/azure/cost-management-billing/manage/ea-azure-marketplace.md#enabling-azure-marketplace-purchases). If those options don't solve the problem, contact [Datadog support](https://www.datadoghq.com/support).

+## Unable to create Datadog - An Azure Native ISV Service resource

+- **Unable to save Single sign-on settings**

+ - This error happens where there's another Enterprise app that is using the Datadog SAML identifier. To find which app is using it, select **Edit** on the Basic SAML Configuration section.

+ To resolve this issue, either disable the other app or use the other app as the Enterprise app to set up SAML SSO with Datadog. If you decide to use the other app, ensure the app has the [required settings](create.md#configure-single-sign-on).

+- **App not showing in Single sign-on setting page**

+ - First, search for the application ID. If no result is shown, check the SAML settings of the app. The grid only shows apps with correct SAML settings.

+ The Identifier URL must be `https://us3.datadoghq.com/account/saml/metadata.xml`.

+

+ The reply URL must be `https://us3.datadoghq.com/account/saml/assertion`.

+

+ The following image shows the correct values.

+ :::image type="content" source="media/troubleshoot/troubleshooting.png" alt-text="Check SAML settings for the Datadog application in Azure A D." border="true":::

+- **Guest users invited to the tenant are unable to access Single sign-on**

+ - Some users have two email addresses in Azure portal. Typically, one email is the user principal name (UPN) and the other email is an alternative email.

+ When inviting guest user, use the home tenant UPN. By using the UPN, you keep the email address in-sync during the Single sign-on process. You can find the UPN by looking for the email address in the top-right corner of the user's Azure portal.

+- Only resources listed in the Azure Monitor resource log categories emit logs to Datadog.

+ To verify whether the resource is emitting logs to Datadog:

+ 1. Navigate to Azure diagnostic setting for the specific resource.

+ 1. Verify that there's a Datadog diagnostic setting.

+ :::image type="content" source="media/troubleshoot/diagnostic-setting.png" alt-text="Datadog diagnostic setting on the Azure resource" border="true":::

+- Resource doesn't support sending logs. Only resource types with monitoring log categories can be configured to send logs. For more information, see [supported categories](/azure/azure-monitor/essentials/resource-logs-categories).

+- Limit of five diagnostic settings reached. Each Azure resource can have a maximum of five diagnostic settings. For more information, see [diagnostic settings](/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal).

+- Export of Metrics data isn't supported currently by the partner solutions under Azure Monitor diagnostic settings.

+To verify the resource has the correct role assignment, open the Azure portal and select the subscription. In the left pane, select **Access Control (IAM)**. Search for the Datadog resource name. Confirm that the Datadog resource has the **Monitoring Reader** role assignment.

+The Azure Datadog integration provides you the ability to install Datadog agent on a virtual machine or app service. The API key selected as **Default Key** in the API Keys screen is used to configure the Datadog agent. If a default key isn't selected, the Datadog agent installation fails.

+- Learn about [managing your instance](manage.md) of Datadog.

+- Create fails because Last Name is empty. The issue happens when the user info in Azure AD is incomplete and doesn't contain Last Name. Contact your Azure tenant's global administrator to rectify the issue and try again.

+### Logs not being emitted

+- Resource doesn't support sending logs. Only resource types with monitoring log categories can be configured to send logs. For more information, see [supported categories](/azure/azure-monitor/essentials/resource-logs-categories).

+- Limit of five diagnostic settings reached. Each Azure resource can have a maximum of five diagnostic settings. For more information, see [diagnostic settings](/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal)

+- Export of Metrics data isn't supported currently by the partner solutions under Azure Monitor diagnostic settings.

+- **Single sign-on configuration indicates lack of permissions**

+ - Occurs when the user that is trying to configure single sign-on doesn't have Manage users permissions for the Dynatrace account. For a description of how to configure this permission, see [here](https://www.dynatrace.com/support/help/shortlink/azure-native-integration#setup).

+- **Unable to save single sign-on settings**

+ - Error happens when there's another Enterprise app that is using the Dynatrace SAML identifier. To find which app is using it, select **Edit** on the Basic **SAML** configuration section.

+ To resolve this issue, either disable the other app or use the other app as the Enterprise app to set up SAML SSO.

+Elastic integration with Azure can only be set up by users who have *Owner* or *Contributor* access on the Azure subscription. [Confirm that you have the appropriate access](/azure/role-based-access-control/check-access).

+- Only resources listed in [Azure Monitor resource log categories](../../azure-monitor/essentials/resource-logs-categories.md) emit logs to Elastic. To verify whether the resource is emitting logs to Elastic:

+ 1. Navigate to [Azure diagnostic setting](../../azure-monitor/essentials/diagnostic-settings.md) for the resource.

+ 1. Verify that there's a diagnostic setting option available.

+ :::image type="content" source="media/troubleshoot/check-diagnostic-setting.png" alt-text="Verify diagnostic setting":::

+- Resource doesn't support sending logs. Only resource types with monitoring log categories can be configured to send logs. For more information, see [supported categories](/azure/azure-monitor/essentials/resource-logs-categories).

+- Limit of five diagnostic settings reached. Each Azure resource can have a maximum of five diagnostic settings. For more information, see [diagnostic settings](/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal)

+- Export of Metrics data is not supported currently by the partner solutions under Azure Monitor diagnostic settings.

+ Use a different Azure subscription. Or, add or update the credit card or payment method for the subscription. For more information, see [updating the credit and payment method](/azure/cost-management-billing/manage/change-credit-card).

+ Use a different subscription. Or, check if your EA subscription is enabled for Marketplace purchase. For more information, see [Enable Marketplace purchases](/azure/cost-management-billing/manage/ea-azure-marketplace#enabling-azure-marketplace-purchases).

+To set up Logz.io, you must be assigned the [Owner role](/azure/role-based-access-control/rbac-and-directory-admin-roles) in the Azure subscription. Before you begin this integration, [check your access](/azure/role-based-access-control/check-access).

+### Logs not being sent to Logz.io

+- Only resources listed in [Azure Monitor resource log categories](/azure/azure-monitor/essentials/resource-logs-categories) send logs to Logz.io. To verify whether a resource is sending logs to Logz.io:

+ 1. Go to [Azure diagnostic setting](/azure/azure-monitor/essentials/diagnostic-settings) for the specific resource.

+ 1. Verify that there's a Logz.io diagnostic setting.

+ :::image type="content" source="media/troubleshoot/diagnostics.png" alt-text="Screenshot of the Azure monitoring diagnostic settings for Logz.io.":::

+- Limit of five diagnostic settings reached. Each Azure resource can have a maximum of five diagnostic settings. For more information, see [diagnostic settings](/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal).

+- Export of Metrics data isn't supported currently by the partner solutions under Azure Monitor diagnostic settings.

+You must register `Microsoft.Logz` in the Azure subscription that contains the Logz.io resource, and any subscriptions with resources that send data to Logz.io. For more information about troubleshooting resource provider registration, see [Resolve errors for resource provider registration](/azure/azure-resource-manager/troubleshooting/error-register-resource-provider).

+- Add or update the subscription's credit card or payment method. For more information, see [Add or update a credit card for Azure](/azure/cost-management-billing/manage/change-credit-card).

+|Azure| [Azure Databricks](register-scan-azure-databricks.md)

+|| [Azure Databricks](register-scan-azure-databricks.md) | [Yes](register-scan-azure-databricks.md#register) | [Yes](register-scan-azure-databricks.md#scan) | [Yes](register-scan-azure-databricks.md#lineage) | No | No |

+ Title: Connect to and manage Azure Databricks

+description: This guide describes how to connect to Azure Databricks in Microsoft Purview, and how to use Microsoft Purview to scan and manage your Azure Databricks source.

+# Connect to and manage Azure Databricks in Microsoft Purview (Preview)

+This article outlines how to register Azure Databricks, and how to authenticate and interact with Azure Databricks in Microsoft Purview. For more information about Microsoft Purview, read the [introductory article](overview.md).

+## Supported capabilities

+|**Metadata Extraction**| **Full Scan** |**Incremental Scan**|**Scoped Scan**|**Classification**|**Access Policy**|**Lineage**|**Data Sharing**|

+|||||||||

+| [Yes](#register)| [Yes](#scan)| No | No | No | No| [Yes](#lineage) | No |

+When scanning Azure Databricks source, Microsoft Purview supports:

+- Extracting technical metadata including:

+ - Azure Databricks workspace

+ - Hive server

+ - Databases

+ - Tables including the columns, foreign keys, unique constraints, and storage description

+ - Views including the columns and storage description

+- Fetching relationship between external tables and Azure Data Lake Storage Gen2/Azure Blob assets.

+- Fetching static lineage on assets relationships among tables and views.

+This connector brings metadata from Databricks metastore. Comparing to scan via [Hive Metastore connector](register-scan-hive-metastore-source.md) in case you use it to scan Azure Databricks earlier:

+- You can directly set up scan for Azure Databricks workspaces without direct HMS access. It uses Databricks personal access token for authentication and connects to a cluster to perform scan.

+- The Databricks workspace info is captured.

+- The relationship between tables and storage assets is captured.

+## Prerequisites

+* You must have an Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+* You must have an active [Microsoft Purview account](create-catalog-portal.md).

+* You need Data Source Administrator and Data Reader permissions to register a source and manage it in the Microsoft Purview governance portal. For more information about permissions, see [Access control in Microsoft Purview](catalog-permissions.md).

+* Set up the latest [self-hosted integration runtime](https://www.microsoft.com/download/details.aspx?id=39717). For more information, see [Create and configure a self-hosted integration runtime](manage-integration-runtimes.md). The minimal supported elf-hosted Integration Runtime version is 5.20.8227.2.

+ * Ensure [JDK 11](https://www.oracle.com/java/technologies/downloads/#java11) is installed on the machine where the self-hosted integration runtime is installed. Restart the machine after you newly install the JDK for it to take effect.

+ * Ensure that Visual C++ Redistributable for Visual Studio 2012 Update 4 is installed on the machine where the self-hosted integration runtime is running. If you don't have this update installed, [download it now](https://www.microsoft.com/download/details.aspx?id=30679).

+* In your Azure Databricks workspace:

+ * [Generate a personal access token](/azure/databricks/dev-tools/auth#--azure-databricks-personal-access-tokens), and store it as a secret in Azure Key Vault.

+ * [Create a cluster](/azure/databricks/clusters/create-cluster). Note down the cluster ID - you can find it in Azure Databricks workspace -> Compute -> your cluster -> Tags -> Automatically added tags -> `ClusterId`.

+## Register

+This section describes how to register an Azure Databricks workspace in Microsoft Purview by using [the Microsoft Purview governance portal](https://web.purview.azure.com/).

+1. Go to your Microsoft Purview account.

+1. Select **Data Map** on the left pane.

+1. Select **Register**.

+1. In **Register sources**, select **Azure Databricks** > **Continue**.

+1. On the **Register sources (Azure Databricks)** screen, do the following:

+ 1. For **Name**, enter a name that Microsoft Purview will list as the data source.

+ 1. For **Azure subscription** and **Databricks workspace name**, select the subscription and workspace that you want to scan from the dropdown. The Databricks workspace URL will be automatically populated.

+ 1. For **Select a collection**, choose a collection from the list or create a new one. This step is optional.

+ :::image type="content" source="media/register-scan-azure-databricks/configure-sources.png" alt-text="Screenshot of registering Azure Databricks source." border="true":::

+1. Select **Finish**.

+## Scan

+> [!TIP]

+> To troubleshoot any issues with scanning:

+> 1. Confirm you have followed all [**prerequisites**](#prerequisites).

+> 1. Review our [**scan troubleshooting documentation**](troubleshoot-connections.md).

+Use the following steps to scan Azure Databricks to automatically identify assets. For more information about scanning in general, see [Scans and ingestion in Microsoft Purview](concept-scans-and-ingestion.md).

+1. In the Management Center, select integration runtimes. Make sure that a self-hosted integration runtime is set up. If it isn't set up, use the steps in [Create and manage a self-hosted integration runtime](./manage-integration-runtimes.md).

+1. Go to **Sources**.

+1. Select the registered Azure Databricks.

+1. Select **+ New scan**.

+1. Provide the following details:

+ 1. **Name**: Enter a name for the scan.

+ 1. **Connect via integration runtime**: Select the configured self-hosted integration runtime.

+ 1. **Credential**: Select the credential to connect to your data source. Make sure to:

+ * Select **Access Token Authentication** while creating a credential.

+ * Provide secret name of the personal access token that you created in [Prerequisites](#prerequisites) in the appropriate box.

+ For more information, see [Credentials for source authentication in Microsoft Purview](manage-credentials.md).

+ 1. **Cluster ID**: Specify the cluster ID that Microsoft Purview will connect to and perform the scan. You can find it in Azure Databricks workspace -> Compute -> your cluster -> Tags -> Automatically added tags -> `ClusterId`.

+ 1. **Mount points**: Provide the mount point and Azure Storage source location string when you have external storage manually mounted to Databricks. Use the format `/mnt/<path>=abfss://<container>@<adls_gen2_storage_account>.dfs.core.windows.net/;/mnt/<path>=wasbs://<container>@<blob_storage_account>.blob.core.windows.net` It will be used to capture the relationship between tables and the corresponding storage assets in Microsoft Purview. This setting is optional, if it's not specified, such relationship won't be retrieved.

+

+ You can get the list of mount points in your Databricks workspace by running the following Python command in a notebook:

+ ```

+ dbutils.fs.mounts()

+ ```

+ It will print all the mount points like below:

+ ```

+ [MountInfo(mountPoint='/databricks-datasets', source='databricks-datasets', encryptionType=''),

+ MountInfo(mountPoint='/mnt/ADLS2', source='abfss://samplelocation1@azurestorage1.dfs.core.windows.net/', encryptionType=''),

+ MountInfo(mountPoint='/databricks/mlflow-tracking', source='databricks/mlflow-tracking', encryptionType=''),

+ MountInfo(mountPoint='/mnt/Blob', source='wasbs://samplelocation2@azurestorage2.blob.core.windows.net', encryptionType=''),

+ MountInfo(mountPoint='/databricks-results', source='databricks-results', encryptionType=''),

+ MountInfo(mountPoint='/databricks/mlflow-registry', source='databricks/mlflow-registry', encryptionType=''), MountInfo(mountPoint='/', source='DatabricksRoot', encryptionType='')]ΓÇ»

+ ```

+ In this example, specify the following as mount points:

+ `/mnt/ADLS2=abfss://samplelocation1@azurestorage1.dfs.core.windows.net/;/mnt/Blob=wasbs://samplelocation2@azurestorage2.blob.core.windows.net`

+ 1. **Maximum memory available**: Maximum memory (in gigabytes) available on the customer's machine for the scanning processes to use. This value is dependent on the size of Hive Metastore database to be scanned.

+ :::image type="content" source="media/register-scan-azure-databricks/scan.png" alt-text="Screenshot of setting up Azure Databricks scan." border="true":::

+1. Select **Continue**.

+1. For **Scan trigger**, choose whether to set up a schedule or run the scan once.

+1. Review your scan and select **Save and Run**.

+Once the scan successfully completes, see how to [browse and search Azure Databricks assets](#browse-and-search-assets).

+## Browse and search assets

+After scanning your Azure Databricks, you can [browse data catalog](how-to-browse-catalog.md) or [search data catalog](how-to-search-catalog.md) to view the asset details.

+From the Databricks workspace asset, you can find the associated Hive Metastore and the tables/views, reversed applies too.

+## Lineage

+Refer to the [supported capabilities](#supported-capabilities) section on the supported Azure Databricks scenarios. For more information about lineage in general, see [data lineage](concept-data-lineage.md) and [lineage user guide](catalog-lineage-user-guide.md).

+Go to the Hive table/view asset -> lineage tab, you can see the asset relationship when applicable. For relationship between table and external storage assets, you'll see Hive Table asset and the storage asset are directly connected bi-directionally, as they mutually impact each other.

+## Next steps

+Now that you've registered your source, use the following guides to learn more about Microsoft Purview and your data:

+- [Data Estate Insights in Microsoft Purview](concept-insights.md)

+- [Lineage in Microsoft Purview](catalog-lineage-user-guide.md)

+- [Search the data catalog](how-to-search-catalog.md)

+| Canada Central | Germany West Central | UAE North | | Central India |

+| East US | Norway East | | | East Asia |

+| East US 2 | Sweden Central | | | Japan East |

+| South Central US | Switzerland North | | | Southeast Asia |

+| West US 2 | UK South | | | |

+| West US 3 | West Europe | | | |

+You can use a free service for this quickstart.

+ Title: Connect with API keys

+description: Learn how to use an admin or query API key for inbound access to an Azure Cognitive Search service endpoint.

+# Connect to Cognitive Search using key authentication

+Cognitive Search offers key-based authentication that you can use on connections to your search service. An API key is a unique string composed of 52 randomly generated numbers and letters. A request made to a search service endpoint will be accepted if both the request and the API key are valid.

+API keys are frequently used when making REST API calls to a search service. You can also use them in search solutions if Azure Active Directory isn't an option.

+> A quick note about "key" terminology in Cognitive Search. An "API key", which is described in this article, refers to a GUID used for authenticating a request. A "document key" refers to a unique string in your indexed content that's used to uniquely identify documents in a search index. API keys and document keys are unrelated.

+## What's an API key?

+There are two kinds of keys used for authenticating a request:

+| Type | Permission level | Maximum | How created|

+|||||

+| Admin | Full access (read-write) for all content operations | 2 ¹| Two admin keys, referred to as *primary* and *secondary* keys in the portal, are generated when the service is created and can be individually regenerated on demand. |

+| Query | Read-only access, scoped to the documents collection of a search index | 50 | One query key is generated with the service. More can be created on demand by a search service administrator. |

+¹ Having two allows you to roll over one key while using the second key for continued access to the service.

+Visually, there's no distinction between an admin key or query key. Both keys are strings composed of 52 randomly generated alpha-numeric characters. If you lose track of what type of key is specified in your application, you can [check the key values in the portal](#find-existing-keys).

+## Use API keys on connections

+API keys are specified on client requests to a search service. Passing a valid API key on the request is considered proof that the request is from an authorized client. If you're creating, modifying, or deleting objects, you'll need an admin API key. Otherwise, query keys are typically distributed to client applications that issue queries.

+You can specify API keys in a request header for REST API calls, or in code that calls the azure.search.documents client libraries in the Azure SDKs. If you're using the Azure portal to perform tasks, your role assignment determines the level of access.

+Best practices for using hard-coded in source files include:

+### [**REST**](#tab/rest-use)

+ Alternatively, you can pass a query key as a parameter on a URL if you're using GET: `GET /indexes/hotels/docs?search=*&$orderby=lastRenovationDate desc&api-version=2020-06-30&api-key=[query key]`

+### [**Azure PowerShell**](#tab/azure-ps-use)

+A script example showing API key usage can be found at [Quickstart: Create an Azure Cognitive Search index in PowerShell using REST APIs](search-get-started-powershell.md).

+### [**.NET**](#tab/dotnet-use)

+In search solutions, a key is often specified as a configuration setting and then passed as an [AzureKeyCredential](/dotnet/api/azure.azurekeycredential). See [How to use Azure.Search.Documents in a C# .NET Application](search-howto-dotnet-sdk.md) for an example.

+> It's considered a poor security practice to pass sensitive data such as an `api-key` in the request URI. For this reason, Azure Cognitive Search only accepts a query key as an `api-key` in the query string. As a general rule, we recommend passing your `api-key` as a request header.

+You can view and manage API keys in the [Azure portal](https://portal.azure.com), or through [PowerShell](/powershell/module/az.search), [Azure CLI](/cli/azure/search), or [REST API](/rest/api/searchmanagement/).

+### [**Azure portal**](#tab/portal-find)

+1. Sign in to the [Azure portal](https://portal.azure.com) and [find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices).

+1. Under **Settings**, select **Keys** to view admin and query keys.

+### [**REST**](#tab/rest-find)

+Use [ListAdminKeys](/rest/api/searchmanagement/2020-08-01/admin-keys) or [ListQueryKeys](/rest/api/searchmanagement/2020-08-01/query-keys/list-by-search-service) in the Management REST API to return API keys.

+You must have a [valid role assignment](#permissions-to-view-or-manage-api-keys) to return or update API keys. See [Manage your Azure Cognitive Search service with REST APIs](search-manage-rest.md) for guidance on meeting role requirements using the REST APIs.

+```rest

+POST https://management.azure.com/subscriptions/{{subscriptionId}}/resourceGroups/{{resource-group}}/providers//Microsoft.Search/searchServices/{{search-service-name}}/listAdminKeys?api-version=2021-04-01-preview

+```

+### [**Azure portal**](#tab/portal-query)

+1. Sign in to the [Azure portal](https://portal.azure.com) and [find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices).

+1. Under **Settings**, select **Keys** to view API keys.

+1. Under **Manage query keys**, use the query key already generated for your service, or create new query keys. The default query key isn't named, but other generated query keys can be named for manageability.

+ :::image type="content" source="media/search-security-overview/create-query-key.png" alt-text="Screenshot of the query key management options." border="true":::

+### [**Azure CLI**](#tab/azure-cli-query)

+A script example showing query key usage can be found at [Create or delete query keys](search-manage-azure-cli.md#create-or-delete-query-keys).

+### [**.NET**](#tab/dotnet-query)

+A code example showing query key usage can be found in [DotNetHowTo](https://github.com/Azure-Samples/search-dotnet-getting-started/tree/master/DotNetHowTo).

+Two admin keys are created for each service so that you can rotate a primary key while using the secondary key for business continuity.

+1. In the **Settings** > **Keys** page, copy the secondary key.

+1. For all applications, update the API key settings to use the secondary key.

+1. Regenerate the primary key.

+1. Update all applications to use the new primary key.

+If you inadvertently regenerate both keys at the same time, all client requests using those keys will fail with HTTP 403 Forbidden. However, content isn't deleted and you aren't locked out permanently.

+You can still access the service through the portal or programmatically. Management functions are operative through a subscription ID not a service API key, and are thus still available even if your API keys aren't.

+After you create new keys via portal or management layer, access is restored to your content (indexes, indexers, data sources, synonym maps) once you provide those keys on requests.

+## Permissions to view or manage API keys

+Permissions for viewing and managing API keys is conveyed through [role assignments](search-security-rbac.md). Members of the following roles can view and regenerate keys:

+The following roles don't have access to API keys:

+## Secure API key access

+Use role assignments to restrict access to API keys.

+Note that it's not possible to use [customer-managed key encryption](search-security-manage-encryption-keys.md) to encrypt API keys. Only sensitive data within the search service itself (for example, index content or connection strings in data source object definitions) can be CMK-encrypted.

+1. In the **Role** filter, select the roles that have permission to view or manage keys (Owner, Contributor, Search Service Contributor). The resulting security principals assigned to those roles have key permissions on your search service.

+1. As a precaution, also check the **Classic administrators** tab for administrators and co-administrators.

+ * For Linux: **useManagedDataDisk:true** and **dataPath: '/mnt/sfroot'**.

+- **App artifact location (output location)**: Lists the value for `output_location`, which is the [folder for built versions of application files](build-configuration.md).

+| [Hugo](https://gohugo.io/) | `public` | n/a |

+4 virtual CPU cores at 2.7 GHz each and 8 GiB of memory (RAM) is the minimum specification for an Azure Storage Mover agent.

+8 virtual CPU cores at 2.7 GHz each and 16 GiB of memory (RAM) is the minimum specification for an Azure Storage Mover agent.

+ Title: Permitted scope for copy operations (preview)

+## About Permitted scope for copy operations (preview)

+The **AllowedCopyScope** property of a storage account is used to specify the environments from which data can be copied to the destination account. It is displayed in the Azure portal as configuration setting **Permitted scope for copy operations (preview)**. The property is not set by default and does not return a value until you explicitly set it. It has three possible values:

+## Restrict the Permitted scope for copy operations (preview)

+### Permissions for changing the Permitted scope for copy operations (preview)

+### Configure the Permitted scope for copy operations (preview)

+| Instance details | Performance | Required | Select **Standard** performance for general-purpose v2 storage accounts (default). This type of account is recommended by Microsoft for most scenarios. For more information, see [Types of storage accounts](storage-account-overview.md#types-of-storage-accounts).<br /><br />Select **Premium** for scenarios requiring low latency. After selecting **Premium**, select the type of premium storage account to create. The following types of premium storage accounts are available: <ul><li>[Block blobs](./storage-account-overview.md)</li><li>[File shares](../files/storage-files-planning.md#management-concepts)</li><li>[Page blobs](../blobs/storage-blob-pageblob-overview.md)</li></ul><br />Microsoft recommends creating a general-purpose v2, premium block blob, or premium file share account for most scenarios. To select a legacy account type, use the link provided beneath **Instance details**. For more information about legacy account types, see [Legacy storage account types](storage-account-overview.md#legacy-storage-account-types). |

+| Security | Permitted scope for copy operations (preview) | Required | Select the scope of storage accounts from which data can be copied to the new account. The default value is `From any storage account`. When set to the default value, users with the appropriate permissions can copy data from any storage account to the new account.<br /><br />Select `From storage accounts in the same Azure AD tenant` to only allow copy operations from storage accounts within the same Azure AD tenant.<br />Select `From storage accounts that have a private endpoint to the same virtual network` to only allow copy operations from storage accounts with private endpoints on the same virtual network.<br /><br /> For more information, see [Restrict the source of copy operations to a storage account](security-restrict-copy-operations.md). |

+- [Download and unzip the latest version of the AzFilesHybrid module](https://github.com/Azure-Samples/azure-files-samples/releases). Note that AES-256 Kerberos encryption is supported on v0.2.2 or above, and is the default encryption method beginning in v0.2.5. If you've enabled the feature with an AzFilesHybrid version below v0.2.2 and want to update to support AES-256 Kerberos encryption, see [this article](./storage-troubleshoot-windows-file-connection-problems.md#azure-files-on-premises-ad-ds-authentication-support-for-aes-256-kerberos-encryption).

+- Install and execute the module on a device that is domain joined to on-premises AD DS with AD DS credentials that have permissions to create a computer account or service logon account in the target AD (such as domain admin).

+The `Join-AzStorageAccount` cmdlet performs the equivalent of an offline domain join on behalf of the specified storage account. The script below uses this cmdlet to create a [computer account](/windows/security/identity-protection/access-control/active-directory-accounts#manage-default-local-accounts-in-active-directory) in your AD domain. If for whatever reason you can't use a computer account, you can alter the script to create a [service logon account](/windows/win32/ad/about-service-logon-accounts) instead. Using AES-256 encryption with service logon accounts is supported beginning with AzFilesHybrid version 0.2.5.

+The AD DS credential must also have permissions to create a computer account or service logon account in the target AD. Replace the placeholder values with your own before executing the script.

+# and preference. Run Get-Help Join-AzStorageAccountForAuth for more details on this cmdlet.

+> In order to enable AES-256 encryption, the domain object that represents your storage account must be a computer account or service logon account in the on-premises AD domain. If your domain object doesn't meet this requirement, delete it and create a new domain object that does.

+> SMB 2.1 support was added to Linux kernel version 3.7. If you're using a version of the Linux kernel after 3.7, it should support SMB 2.1.

+To mount an Azure file share on Linux, use the storage account name as the username of the file share, and the storage account key as the password. Because the storage account credentials may change over time, you should store the credentials for the storage account separately from the mount configuration.

+## Mount a file share snapshot

+If you want to mount a specific snapshot of an SMB Azure file share, you must supply the `snapshot` option as part of the `mount` command, where `snapshot` is the time that the particular snapshot was created in a format such as @GMT-2023.01.05-00.08.20. The `snapshot` option has been supported in the Linux kernel since version 4.19.

+After you've created the file share snapshot, following these instructions to mount it.

+1. In the Azure portal, navigate to the storage account that contains the file share that you want to mount a snapshot of.

+2. Select **Data storage > File shares** and select the file share.

+3. Select **Operations > Snapshots** and take note of the name of the snapshot you want to mount. The snapshot name will be a GMT timestamp, such as in the screenshot below.

+

+ :::image type="content" source="media/storage-how-to-use-files-linux/mount-snapshot.png" alt-text="Screenshot showing how to locate a file share snapshot name and timestamp in the Azure portal." border="true" :::

+

+4. Convert the timestamp to the format expected by the `mount` command, which is **@GMT-year.month.day-hour.minutes.seconds**. In this example, you'd convert **2023-01-05T00:08:20.0000000Z** to **@GMT-2023.01.05-00.08.20**.

+5. Run the `mount` command using the GMT time to specify the `snapshot` value. Be sure to replace `<storage-account-name>`, `<file-share-name>`, and the GMT timestamp with your values. The .cred file contains the credentials to be used to mount the share (see [Automatically mount file shares](#automatically-mount-file-shares)).

+

+ ```bash

+ sudo mount -t cifs //<storage-account-name>.file.core.windows.net/<file-share-name> /mnt/<file-share-name>/snapshot1 -o credentials=/etc/smbcredentials/snapshottestlinux.cred,snapshot=@GMT-2023.01.05-00.08.20

+ ```

+

+6. If you're able to browse the snapshot under the path `/mnt/<file-share-name>/snapshot1`, then the mount succeeded.

+If the mount fails, see [Troubleshoot Azure Files problems in Linux (SMB)](storage-troubleshoot-linux-file-connection-problems.md).

+- You're using a Linux distribution with an outdated SMB client. See [Use Azure Files with Linux](storage-how-to-use-files-linux.md) for more information on common Linux distributions available in Azure that have compatible clients.

+- SMB utilities (cifs-utils) aren't installed on the client.

+- The minimum SMB version, 2.1, isn't available on the client.

+- SMB 3.x encryption isn't supported on the client. The preceding table provides a list of Linux distributions that support mounting from on-premises and cross-region using encryption. Other distributions require kernel 4.11 and later versions.

+- You're trying to connect to an Azure file share from an Azure VM, and the VM isn't in the same region as the storage account.

+<a id="mounterror22"></a>

+## "Mount error(22): Invalid argument" when trying to mount an Azure file share snapshot

+### Cause

+If the `snapshot` option for the `mount` command isn't passed in a recognized format, the `mount` command can fail with this error. To confirm, check kernel log messages (dmesg), and dmesg will show a log entry such as **cifs: Bad value for 'snapshot'**.

+### Solution

+Make sure you're passing the `snapshot` option for the `mount` command in the correct format. Refer to the mount.cifs manual page (e.g. `man mount.cifs`). A common error is passing the GMT timestamp in the wrong format, such as using hyphens or colons in place of periods. For more information, see [Mount a file share snapshot](storage-how-to-use-files-linux.md#mount-a-file-share-snapshot).

+<a id="badsnapshottoken"></a>

+## "Bad snapshot token" when trying to mount an Azure file share snapshot

+### Cause

+If the snapshot `mount` option is passed starting with @GMT, but the format is still wrong (such as using hyphens and colons instead of periods), the `mount` command can fail with this error.

+### Solution

+Make sure you're passing the GMT timestamp in the correct format, which is **@GMT-year.month.day-hour.minutes.seconds**. For more information, see [Mount a file share snapshot](storage-how-to-use-files-linux.md#mount-a-file-share-snapshot).

+<a id="mounterror2"></a>

+## "Mount error(2): No such file or directory" when trying to mount an Azure file share snapshot

+### Cause

+If the snapshot that you're attempting to mount doesn't exist, the `mount` command can fail with this error. To confirm, check kernel log messages (dmesg), and dmesg will show a log entry such as:

+```bash

+[Mon Dec 12 10:34:09 2022] CIFS: Attempting to mount \\snapshottestlinux.file.core.windows.net\snapshot-test-share1

+[Mon Dec 12 10:34:09 2022] CIFS: VFS: cifs_mount failed w/return code = -2

+```

+### Solution

+Make sure the snapshot you're attempting to mount exists. For more information on how to list the available snapshots for a given Azure file share, see [Mount a file share snapshot](storage-how-to-use-files-linux.md#mount-a-file-share-snapshot).

+#### Workarounds

+- Separate the file share into multiple file shares within the same storage account.

+- Add a virtual hard disk (VHD) on the file share and mount the VHD from the client to perform file operations against the data. This approach works for single writer/reader scenarios or scenarios with multiple readers and no writers. Because the file system is owned by the client rather than Azure Files, this allows metadata operations to be local. The setup offers performance similar to that of local directly attached storage. However, because the data is in a VHD, it can't be accessed via any other means other than the SMB mount, such as REST API or through the Azure portal.

+ 1. From the machine which needs to access the Azure file share, mount the file share using the storage account key and map it to an available network drive (for example, Z:).

+ 1. Go to **Disk Management** and select **Action > Create VHD**.

+ 1. Set **Location** to the network drive that the Azure file share is mapped to, set **Virtual hard disk size** as needed, and select **Fixed size**.

+ 1. Select **OK**. Once the VHD creation is complete, it will automatically mount, and a new unallocated disk will appear.

+ 1. Right-click the new unknown disk and select **Initialize Disk**.

+ 1. Right-click the unallocated area and create a **New Simple Volume**.

+ 1. You should see a new drive letter appear in **Disk Management** representing this VHD with read/write access (for example, E:). In **File Explorer**, you should see the new VHD on the mapped Azure file share's network drive (Z: in this example). To be clear, there should be two drive letters present: the standard Azure file share network mapping on Z:, and the VHD mapping on the E: drive.

+ 1. There should be much better performance on heavy metadata operations against files on the VHD mapped drive (E:) versus the Azure file share mapped drive (Z:). If desired, it should be possible to disconnect the mapped network drive (Z:) and still access the mounted VHD drive (E:).

+ - To mount a VHD on a Windows client, you can also use the [`Mount-DiskImage`](/powershell/module/storage/mount-diskimage) PowerShell cmdlet.

+ - To mount a VHD on Linux, consult the documentation for your Linux distribution. [Here's an example](https://man7.org/linux/man-pages/man5/nfs.5.html).

+You can learn more from the [how to query delta lake tables video](https://www.youtube.com/watch?v=LSIVX0XxVfc).

+Apache Spark pools in Azure Synapse enable data engineers to modify Delta Lake files using Scala, PySpark, and .NET. Serverless SQL pools help data analysts to create reports on Delta Lake files created by data engineers.

+> [!IMPORTANT]

+> Querying Delta Lake format using the serverless SQL pool is **Generally available** functionality. However, querying Spark Delta tables is still in public preview and not production ready. There are known issues that might happen if you query Delta tables created using the Spark pools. See the known issues in the [self-help page](resources-self-help-sql-on-demand.md#delta-lake).

+If your users have both RDP Shortpath for managed network and public networks available to them, then the first-found algorithm will be used. The user will use whichever connection gets established first for that session.

+If your users have both RDP Shortpath for managed network and public networks available to them, then the first-found algorithm will be used. The user will use whichever connection gets established first for that session.

+If your users are in a scenario where RDP Shortpath for both managed network and public networks is available to them, then the first algorithm found will be used. The user will use whichever connection gets established first for that session.

+## Example scenarios

+Here are some example scenarios to show how connections are evaluated to decide whether RDP Shortpath is used across different network topologies.

+### Scenario 1

+A UDP connection can only be established between the client device and the session host over a public network (internet). A direct connection, such as a VPN, is not available.

+### Scenario 2

+A UDP connection can be established between the client device and the session host over a public network or over a direct VPN connection, but RDP Shortpath for managed networks is not enabled. When the client initiates the connection, the ICE/STUN protocol can see multiple routes and will evaluate each route and choose the one with the lowest latency.

+In this example, a UDP connection using RDP Shortpath for public networks over the direct VPN connection will be made as it has the lowest latency, as shown by the green line.

+### Scenario 3

+Both RDP Shortpath for public networks and managed networks are enabled. A UDP connection can be established between the client device and the session host over a public network or over a direct VPN connection. When the client initiates the connection, there are simultaneous attempts to connect using RDP Shortpath for managed networks through port 3390 (by default) and RDP Shortpath for public networks through the ICE/STUN protocol. The first-found algorithm will be used and the user will use whichever connection gets established first for that session.

+Since going over a public network has additional steps, for example a NAT device, a load balancer, or a STUN server, it is likely that the first-found algorithm will select the connection using RDP Shortpath for managed networks and be established first.

+### Scenario 4

+A UDP connection can be established between the client device and the session host over a public network or over a direct VPN connection, but RDP Shortpath for managed networks is not enabled. To prevent ICE/STUN from using a particular route, an admin can block one of the routes for UDP traffic. Blocking a route would ensure the remaining path is always used.

+In this example, UDP is blocked on the direct VPN connection and the ICE/STUN protocol establishes a connection over the public network.

+### Scenario 5

+Both RDP Shortpath for public networks and managed networks are configured, however a UDP connection could not be established. In this instance, RDP Shortpath will fail and the connection will fall back to TCP-based reverse connect transport.

+* [Manage users, SSH, and check or repair disks on Azure Linux VMs using the 'VMAccess' Extension](/azure/virtual-machines/extensions/vmaccess)

+#Customer intent: As an IT administrator, I want to learn more about using a dedicated host for my Azure virtual machines

+Azure Dedicated Host is a service that provides physical servers able to host one or more virtual machines assigned to one Azure subscription. Dedicated hosts are the same physical servers used in our data centers, provided instead as a directly accessible hardware resource. You can provision dedicated hosts within a region, availability zone, and fault domain. You can then place VMs directly into your provisioned hosts in whatever configuration best meets your needs.

+Reserving the entire host provides several benefits beyond those of a standard shared virtual machine host:

+- Cost Optimization: With the Azure hybrid benefit, you can bring your own licenses for Windows and SQL to Azure. For more information, see [Azure Hybrid Benefit](https://azure.microsoft.com/pricing/hybrid-benefit/).

+- Reliability: You have near complete control over maintenance events initiated by the Azure platform. While most maintenance events have little to no impact on your virtual machines, there are some sensitive workloads where each second of pause can have an impact. With dedicated hosts, you can opt in to a maintenance window to reduce the impact to your service.

+- Performance Efficiency: Because you have control over a physical host, you can choose which applications share physical resources such as memory and storage. This can speed up certain workloads that benefit from low latency and high throughput on the host machine.

+- Security: Hardware isolation at the physical server level allows for sensitive memory data to remain isolated within a physical host. No other customer's VMs will be placed on your hosts. Dedicated hosts are deployed in the same data centers and share the same network and underlying storage infrastructure as other, non-isolated hosts.

+Availability zones are unique physical locations within an Azure region. Each zone is made up of one or more datacenters equipped with independent power, cooling, and networking. A host group is created in a single availability zone. Once created, all hosts will be placed within that zone. To achieve high availability across zones, you need to create multiple host groups (one per zone) and spread your hosts between them accordingly.

+A host can be created in a specific fault domain. Just like VM in a scale set or availability set, hosts in different fault domains will be placed on different physical racks in the data center. When you create a host group, you're required to specify the fault domain count. When creating hosts within the host group, you assign fault domain for each host. The VMs don't require any fault domain assignment.

+Fault domains aren't the same as colocation. Having the same fault domain for two hosts doesn't mean they are in proximity with each other.

+Fault domains are scoped to the host group. You shouldn't make any assumption on anti-affinity between two host groups (unless they are in different availability zones).

+You can use both capabilities together to achieve even more fault isolation. To use both, specify the availability zone and fault domain count in for each host group, assign a fault domain to each host in the group, then assign an availability zone to each VM.

+Host groups that are enabled for automatic placement don't require all the VMs to be automatically placed. You'll still be able to explicitly pick a host, even when automatic placement is selected for the host group.

+- You won't be able to redeploy your VM.

+- You won't be able to use DCv2, Lsv2, NVasv4, NVsv3, Msv2, or M-series VMs with dedicated hosts.

+## Virtual Machine Scale Set support

+Virtual Machine Scale Sets let you treat a group of virtual machines as a single resource, and apply availability, management, scaling and orchestration policies as a group. Your existing dedicated hosts can also be used for Virtual Machine Scale Sets.

+When creating a Virtual Machine Scale Set, you can specify an existing host group to have all of the VM instances created on dedicated hosts.

+The following requirements apply when creating a Virtual Machine Scale Set in a dedicated host group:

+- Overprovisioning isn't recommended, and it's disabled by default. You can enable overprovisioning, but the scale set allocation will fail if the host group doesn't have capacity for all of the VMs, including the overprovisioned instances.

+- Don't use proximity placement groups for co-location

+The infrastructure supporting your virtual machines may occasionally be updated to improve reliability, performance, security, and to launch new features. The Azure platform tries to minimize the impact of platform maintenance whenever possible, however customers with *maintenance sensitive* workloads can't tolerate even few seconds that the VM needs to be shut down for maintenance.

+**Maintenance Control** provides customers with an option to skip regular platform updates scheduled on their dedicated hosts, then apply it at the time of their choice within a 35-day rolling window. Within the maintenance window, you can apply maintenance directly at the host level, in any order. Once the maintenance window is over, Microsoft will move forward and apply the pending maintenance to the hosts in an order that may not follow the user defined fault domains.

+Once a dedicated host is provisioned, Azure assigns it to physical server. Doing so guarantees the availability of the capacity when you need to provision your VM. Azure uses the entire capacity in the region (or zone) to pick a physical server for your host. It also means that customers can expect to be able to grow their dedicated host footprint without the concern of running out of space in the cluster.

+Provisioning a dedicated host will consume both dedicated host vCPU and the VM family vCPU quota, but it won't consume the regional vCPU. VMs placed on a dedicated host won't count against VM family vCPU quota. Should a VM be moved off a dedicated host into a multi-tenant environment, the VM will consume VM family vCPU quota.

+Free trial and MSDN subscriptions don't have quota for Azure Dedicated Hosts.

+Users are charged per dedicated host, regardless how many VMs are deployed. In your monthly statement, you'll see a new billable resource type of hosts. The VMs on a dedicated host will still be shown in your statement, but will carry a price of 0.

+Software licensing, storage and network usage are billed separately from the host and VMs. There's no change to those billable items.

+A SKU represents the VM size series and type on a given host. You can mix multiple VMs of different sizes within a single host as long as they are of the same size series.

+| Host Under Investigation | WeΓÇÖre having some issues with the host that weΓÇÖre looking into. This transitional state is required for Azure to try to identify the scope and root cause for the issue identified. Virtual machines running on the host may be impacted. |

+| Host deallocated | All virtual machines have been removed from the host. You're no longer being charged for this host since the hardware was taken out of rotation. |

+- There's a [sample template](https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.compute/vm-dedicated-hosts/README.md) that uses both zones and fault domains for maximum resiliency in a region.

+ Title: Create an encrypted image version with customer-managed keys

+# Create an encrypted image version with customer-managed keys

+When you're using customer-managed keys for encrypting images in an Azure Compute Gallery, these limitations apply:

+- VM image version source doesn't currently support customer-managed key encryption.

+Once you connect to your VM, find the disk. In this example, we're using `lsblk` to list the disks.

+Here, `sdc` is the disk that we want, because it's 50G. If you add multiple disks, and aren't sure which disk it's based on size alone, you can go to the VM page in the portal, select **Disks**, and check the LUN number for the disk under **Data disks**. Compare the LUN number from the portal to the last number of the **HTCL** portion of the output, which is the LUN.

+Format the disk with `parted`, if the disk size is two tebibytes (TiB) or larger then you must use GPT partitioning, if it is under 2TiB, then you can use either MBR or GPT partitioning.

+The following example uses `parted` on `/dev/sdc`, which is where the first data disk will typically be on most VMs. Replace `sdc` with the correct option for your disk. We're also formatting it using the [XFS](https://xfs.wiki.kernel.org/) filesystem.

+To ensure that the drive is remounted automatically after a reboot, it must be added to the */etc/fstab* file. It's also highly recommended that the UUID (Universally Unique Identifier) is used in */etc/fstab* to refer to the drive rather than just the device name (such as, */dev/sdc1*). If the OS detects a disk error during boot, using the UUID avoids the incorrect disk being mounted to a given location. Remaining data disks would then be assigned those same device IDs. To find the UUID of the new drive, use the `blkid` utility:

+Next, open the **/etc/fstab** file in a text editor. Add a line to the end of the file, using the UUID value for the `/dev/sdc1` device that was created in the previous steps, and the mountpoint of `/datadrive`. Using the example from this article, the new line would look like the following:

+When you're done editing the file, save and close the editor.

+# [Ubuntu](#tab/ubuntu)

+```bash

+sudo apt-get install util-linux

+sudo fstrim /datadrive

+```

+# [RHEL](#tab/rhel)

+```bash

+sudo yum install util-linux

+sudo fstrim /datadrive

+```

+# [SUSE](#tab/suse)

+```bash

+sudo fstrim /datadrive

+```

+* Expand your storage capacity by adding more disks and [configure RAID](/previous-versions/azure/virtual-machines/linux/configure-raid) for extra performance.

+1. When you're done, select **Save** at the top of the page to create the managed disk and update the VM configuration.

+1. Select the drop-down menu for **Disk name** and select a disk from the list of available managed disks.

+1. Select **Save** to attach the existing managed disk and update the VM configuration:

+Once connected to your VM, you need to find the disk. In this example, we're using `lsblk` to list the disks.

+In this example, the disk that was added was `sdc`. It's a LUN 0 and is 4GB.

+For a more complex example, here's what multiple data disks look like in the portal:

+Here's what that might look like using `lsblk`:

+> The following instructions will delete data on the disk.

+If you're attaching a new disk, you need to partition the disk.

+- Use the latest version `parted` that is available for your distro.

+The following example uses `parted` on `/dev/sdc`, which is where the first data disk will typically be on most VMs. Replace `sdc` with the correct option for your disk. We're also formatting it using the [XFS](https://xfs.wiki.kernel.org/) filesystem.

+To ensure that the drive is remounted automatically after a reboot, it must be added to the */etc/fstab* file. It's also highly recommended that the UUID (Universally Unique Identifier) is used in */etc/fstab* to refer to the drive rather than just the device name (such as, */dev/sdc1*). If the OS detects a disk error during boot, using the UUID avoids the incorrect disk being mounted to a given location. Remaining data disks would then be assigned those same device IDs. To find the UUID of the new drive, use the `blkid` utility:

+> Improperly editing the **/etc/fstab** file could result in an unbootable system. If unsure, refer to the distribution's documentation for information on how to properly edit this file. You should create a backup of the **/etc/fstab** file is created before editing.

+Next, open the **/etc/fstab** file in a text editor. Add a line to the end of the file, using the UUID value for the `/dev/sdc1` device that was created in the previous steps, and the mountpoint of `/datadrive`. Using the example from this article, the new line would look like the following:

+When you're done editing the file, save and close the editor.

+Some Linux kernels support TRIM/UNMAP operations to discard unused blocks on the disk. This feature is primarily useful to inform Azure that deleted pages are no longer valid and can be discarded. This feature can save money on disks that are billed based on the amount of consumed storage, such as unmanaged standard disks and disk snapshots.

+# [Ubuntu](#tab/ubuntu)

+```bash

+sudo apt-get install util-linux

+sudo fstrim /datadrive

+```

+# [RHEL](#tab/rhel)

+```bash

+sudo yum install util-linux

+sudo fstrim /datadrive

+```

+# [SUSE](#tab/suse)

+```bash

+sudo fstrim /datadrive

+```

+Open the **/etc/fstab** file in a text editor and remove the line containing the UUID of your disk. Using the example values in this article, the line would look like the following:

+Save and close the file when you're done.

+Next, use `umount` to unmount the disk. The following example unmounts the */dev/sdc1* partition from the */datadrive* mount point:

+# [SUSE](#tab/suse)

+ sudo -i

+ zypper install growpart

+ growpart /dev/sda 4

+ lsblk

+ lsblk -f

+ xfs_growfs /

+ resize2fs /dev/sda4

+ df -Thl

+ df -Thl

+ sudo -i

+ lsblk -f

+ vgdisplay rootvg

+ yum install cloud-utils-growpart gdisk

+ pvscan

+ lsblk /dev/sda4

+ growpart /dev/sda 4

+ lsblk /dev/sda4

+ pvresize /dev/sda4

+ pvscan

+ lvresize -r -L +10G /dev/mapper/rootvg-rootlv

+ lvresize -r -L +10G /dev/mapper/rootvg-rootlv

+ df -Th /

+ sudo -i

+ yum install cloud-utils-growpart gdisk

+ lsblk -f

+ gdisk -l /dev/sda

+ growpart /dev/sda 2

+ gdisk -l /dev/sda

+ xfs_growfs /

+ df -hl

+**Applies to:** :heavy_check_mark: Linux VMs :heavy_check_mark: Windows VMs :heavy_check_mark: Flexible scale sets

+* [Migrating Virtual Machines from Amazon AWS to Microsoft Azure](/shows/it-ops-talk/migrate-your-aws-vms-to-azure-with-azure-migrate)

+To see scheduled events in action, watch Azure Friday - [Using Azure Scheduled Events to prepare for VM maintenance](https://youtu.be/ApsoXLVg_0U).

+If you'd like a video going into more detail on managed disks, check out: [Better Azure VM Resiliency with Managed Disks](/shows/azure/managed-disks-azure-resiliency).

+ - [Azure Reservations in Partner Center Cloud Solution Provider (CSP) program](/partner-center/azure-reservations)

+* [Migrating Virtual Machines from Amazon AWS to Microsoft Azure](/shows/it-ops-talk/migrate-your-aws-vms-to-azure-with-azure-migrate)

+To see scheduled events in action, watch Azure Friday - [Using Azure Scheduled Events to prepare for VM maintenance](https://youtu.be/ApsoXLVg_0U).

+ :::image type="content" source="./media/create-virtual-network-manager-portal/connectivity-configuration-dropdown.png" alt-text="Screenshot of configuration drop-down menu.":::

+In this article, you'll learn to block high risk network ports using [Azure Virtual Network Manager](overview.md) and Security Admin Rules. You'll walk through the creation of an Azure Virtual Network Manager instance, group your virtual networks (VNets) with [network groups](concept-network-groups.md), and create & deploy security admin configurations for your organization. You'll deploy a general block rule for high risk ports. Then you'll create an exception for managing a specific application's VNet using network security groups.

+While this article focuses on a single port, SSH, you can protect any high-risk ports in your environment with the same steps. To learn more, review this list of [high risk ports](concept-security-admins.md#protect-high-risk-ports)

+You'll need a virtual network environment that includes virtual networks that can be segregated for allowing and blocking specific network traffic. You may use the following table or your own configuration of virtual networks:

+With your virtual network manager created, you now create a network group containing all of the VNets in the organization. You'll manually add all of the VNets.

+1. Select **Security configuration** from the drop-down menu.

+ :::image type="content" source="./media/create-virtual-network-manager-portal/security-admin-dropdown.png" alt-text="Screenshot of add a configuration drop-down.":::

+In this section, the rules created will take effect when you deploy the security admin configuration.

+With traffic blocked across all of your VNets, you need an exception to allow traffic to specific virtual networks. You'll create a network group specifically for the VNets needing exclusion from the other security admin rule.

+In this section, you create a new rule collection and security admin rule that will allow high-risk traffic to the subset of virtual networks you've defined as exceptions. Next, you'll add it to your existing security admin configuration.

+1. Select **Security configuration** from the drop-down menu.

+ :::image type="content" source="./media/create-virtual-network-manager-portal/security-admin-dropdown.png" alt-text="Screenshot of add a configuration drop-down.":::

+ |**Destination**| |

+1. Select **Connectivity configuration** from the drop-down menu to begin creating a connectivity configuration.

+ :::image type="content" source="./media/create-virtual-network-manager-portal/connectivity-configuration-dropdown.png" alt-text="Screenshot of configuration drop-down menu.":::

+ "field": "Name",

+ "contains": "myVNet01"

+ {

+ "field": "Name",

+ "contains": "myVNet01"

+ }

+ :::image type="content" source="./media/create-virtual-network-manager-portal/connectivity-configuration-dropdown.png" alt-text="Screenshot of configuration drop-down menu.":::

+In regions without availability zones, all public IP addresses are created as non-zonal. Public IP addresses created in a region that is later upgraded to have availability zones remain non-zonal. A public IP's availability zone can't be changed after the public IP's creation.

+1. Select **Automation** > **Export template**.

+- [Move Azure virtual machines to another region](../site-recovery/azure-to-azure-tutorial-migrate.md)

+- Resources in one virtual network can't communicate with the front-end IP address of a Basic Load Balancer (internal or public) in a globally peered virtual network. Support for Basic Load Balancer only exists within the same region. Support for Standard Load Balancer exists for both, VNet Peering and Global VNet Peering. Some services that use a Basic load balancer don't work over global virtual network peering. For more information, see [Constraints related to Global VNet Peering and Load Balancers](virtual-networks-faq.md#what-are-the-constraints-related-to-global-vnet-peering-and-load-balancers).

+* Resources in one virtual network can't communicate with the front-end IP address of a Basic Load Balancer (internal or public) in a globally peered virtual network.

+ Title: 'Configure BGP for VPN Gateway: CLI'

+# How to configure BGP for Azure VPN Gateway: CLI

+This article helps you enable BGP on cross-premises site-to-site (S2S) VPN connections and VNet-to-VNet connections using Azure CLI. You can also create this configuration using the [Azure portal](bgp-howto.md) or [PowerShell](vpn-gateway-bgp-resource-manager-ps.md) steps.

+BGP is the standard routing protocol commonly used in the Internet to exchange routing and reachability information between two or more networks. BGP enables the Azure VPN gateways and your on-premises VPN devices, called BGP peers or neighbors, to exchange "routes" that will inform both gateways on the availability and reachability for those prefixes to go through the gateways or routers involved. BGP can also enable transit routing among multiple networks by propagating routes a BGP gateway learns from one BGP peer to all other BGP peers.

+For more information about the benefits of BGP and to understand the technical requirements and considerations of using BGP, see [About BGP and Azure VPN Gateway](vpn-gateway-bgp-overview.md).

+Each part of this article helps you form a basic building block for enabling BGP in your network connectivity. If you complete all three parts (configure BGP on the gateway, S2S connection, and VNet-to-VNet connection) you build the topology as shown in Diagram 1.

+**Diagram 1**

+## <a name ="enablebgp"></a>Enable BGP for the VPN gateway

+This section is required before you perform any of the steps in the other two configuration sections. The following configuration steps set up the BGP parameters of the Azure VPN gateway as shown in Diagram 2.

+**Diagram 2**

+### Create and configure TestVNet1

+#### 1. Create a resource group

+```azurecli-interactive

+az group create --name TestRG1 --location eastus

+#### 2. Create TestVNet1

+```azurecli-interactive

+az network vnet create -n TestVNet1 -g TestRG1 --address-prefix 10.11.0.0/16 --subnet-name FrontEnd --subnet-prefix 10.11.0.0/24

+```

+```azurecli-interactive

+az network vnet update -n TestVNet1 --address-prefixes 10.11.0.0/16 10.12.0.0/16 -g TestRG1

+az network vnet subnet create --vnet-name TestVNet1 -n BackEnd -g TestRG1 --address-prefix 10.12.0.0/24

+az network vnet subnet create --vnet-name TestVNet1 -n GatewaySubnet -g TestRG1 --address-prefix 10.12.255.0/27

+### Create the VPN gateway for TestVNet1 with BGP parameters

+```azurecli-interactive

+az network public-ip create -n GWPubIP -g TestRG1 --allocation-method Dynamic 

+If you run this command by using the `--no-wait` parameter, you don't see any feedback or output. The `--no-wait` parameter allows the gateway to be created in the background. It doesn't mean that the VPN gateway is created immediately.

+```azurecli-interactive

+az network vnet-gateway create -n VNet1GW -l eastus --public-ip-address GWPubIP -g TestRG1 --vnet TestVNet1 --gateway-type Vpn --sku HighPerformance --vpn-type RouteBased --asn 65010 --no-wait

+After the gateway is created, you can use this gateway to establish a cross-premises connection or a VNet-to-VNet connection with BGP.

+Run the following command.

+```azurecli-interactive

+az network vnet-gateway list -g TestRG1

+```

+Make a note of the `bgpSettings` section at the top of the output. You'll use this

+```azurecli-interactive

+If you don't see the BgpPeeringAddress displayed as an IP address, your gateway is still being configured. Try again when the gateway is complete.

+## Establish a cross-premises connection with BGP

+To establish a cross-premises connection, you need to create a local network gateway to represent your on-premises VPN device. Then you connect the Azure VPN gateway with the local network gateway. Although these steps are similar to creating other connections, they include the additional properties required to specify the BGP configuration parameter, as shown in Diagram 3.

+**Diagram 3**

+### Create and configure the local network gateway

+* As a reminder, you must use different BGP ASNs between your on-premises networks and the Azure virtual network. If they're the same, you need to change your VNet ASN if your on-premises VPN devices already use the ASN to peer with other BGP neighbors.

+Before you proceed, make sure that you've completed the [Enable BGP for your VPN gateway](#enablebgp) section of this exercise. Notice that in this example, you create a new resource group. Also, notice the two additional parameters for the local network gateway: `Asn` and `BgpPeerAddress`.

+```azurecli-interactive

+az group create -n TestRG5 -l westus 

+az network local-gateway create --gateway-ip-address 23.99.221.164 -n Site5 -g TestRG5 --local-address-prefixes 10.51.255.254/32 --asn 65050 --bgp-peering-address 10.51.255.254

+### Connect the VNet gateway and local network gateway

+In this step, you create the connection from TestVNet1 to Site5. You must specify the `--enable-bgp` parameter to enable BGP for this connection.

+```azurecli-interactive

+az network vnet-gateway show -n VNet1GW -g TestRG1

+  "id": "/subscriptions/<subscription ID>/resourceGroups/TestRG1/providers/Microsoft.Network/virtualNetworkGateways/VNet1GW",

+```azurecli-interactive

+az network local-gateway show -n Site5 -g TestRG5

+In this step, you create the connection from TestVNet1 to Site5. As discussed earlier, it's possible to have both BGP and non-BGP connections for the same Azure VPN gateway. Unless BGP is enabled in the connection property, Azure won't enable BGP for this connection, even though BGP parameters are already configured on both gateways. Replace the subscription IDs with your own.

+```azurecli-interactive

+az network vpn-connection create -n VNet1ToSite5 -g TestRG1 --vnet-gateway1 /subscriptions/<subscription ID>/resourceGroups/TestRG1/providers/Microsoft.Network/virtualNetworkGateways/VNet1GW --enable-bgp -l eastus --shared-key "abc123" --local-gateway2 /subscriptions/<subscription ID>/resourceGroups/TestRG5/providers/Microsoft.Network/localNetworkGateways/Site5

+#### On-premises device configuration

+The following example lists the parameters you enter into the BGP configuration section on your on-premises VPN device for this exercise:

+- Site5 ASN : 65050

+- Site5 BGP IP : 10.51.255.254

+- Prefixes to announce : (for example) 10.51.0.0/16

+- Azure VNet ASN : 65010

+- Azure VNet BGP IP : 10.12.255.30

+- Static route : Add a route for 10.12.255.30/32, with nexthop being the VPN tunnel interface on your device

+- eBGP Multihop : Ensure the "multihop" option for eBGP is enabled on your device if needed

+## Establish a VNet-to-VNet connection with BGP

+This section adds a VNet-to-VNet connection with BGP, as shown in Diagram 4.

+**Diagram 4**

+### Create TestVNet2 and the VPN gateway

+It's important to make sure that the IP address space of the new virtual network, TestVNet2, doesn't overlap with any of your VNet ranges.

+```azurecli-interactive

+az group create -n TestRG2 -l eastus

+```azurecli-interactive

+az network vnet create -n TestVNet2 -g TestRG2 --address-prefix 10.21.0.0/16 --subnet-name FrontEnd --subnet-prefix 10.21.0.0/24

+```

+```azurecli-interactive

+az network vnet update -n TestVNet2 --address-prefixes 10.21.0.0/16 10.22.0.0/16 -g TestRG2

+az network vnet subnet create --vnet-name TestVNet2 -n BackEnd -g TestRG2 --address-prefix 10.22.0.0/24

+az network vnet subnet create --vnet-name TestVNet2 -n GatewaySubnet -g TestRG2 --address-prefix 10.22.255.0/27

+```azurecli-interactive

+az network public-ip create -n GWPubIP2 -g TestRG2 --allocation-method Dynamic

+```azurecli-interactive

+az network vnet-gateway create -n VNet2GW -l eastus --public-ip-address GWPubIP2 -g TestRG2 --vnet TestVNet2 --gateway-type Vpn --sku Standard --vpn-type RouteBased --asn 65020 --no-wait

+### Connect the TestVNet1 and TestVNet2 gateways

+In the following example, the virtual network gateway and local network gateway are in different resource groups. When the gateways are in different resource groups, you must specify the entire resource ID of the two gateways to set up a connection between the virtual networks.

+```azurecli-interactive

+az network vnet-gateway show -n VNet1GW -g TestRG1

+```

+Example value for the gateway resource:

+```

+"/subscriptions/<subscripion ID value>/resourceGroups/TestRG2/providers/Microsoft.Network/virtualNetworkGateways/VNet2GW"

+```azurecli-interactive

+az network vnet-gateway show -n VNet2GW -g TestRG2

+Create the connection from TestVNet1 to TestVNet2, and the connection from TestVNet2 to TestVNet1. These commands use the resource IDs. For this exercise, most of the resource ID is already in the example. Be sure to replace the subscription ID values with your own. The subscription ID is used in multiple places in the same command. When using this command for production, you'll replace the entire resource ID for each object you are referencing.

+```azurecli-interactive

+az network vpn-connection create -n VNet1ToVNet2 -g TestRG1 --vnet-gateway1 /subscriptions/<subscription ID>/resourceGroups/TestRG1/providers/Microsoft.Network/virtualNetworkGateways/VNet1GW --enable-bgp -l eastus --shared-key "abc123" --vnet-gateway2 /subscriptions/<subscription ID>/resourceGroups/TestRG2/providers/Microsoft.Network/virtualNetworkGateways/VNet2GW

+```azurecli-interactive

+az network vpn-connection create -n VNet2ToVNet1 -g TestRG2 --vnet-gateway1 /subscriptions/<subscription ID>/resourceGroups/TestRG2/providers/Microsoft.Network/virtualNetworkGateways/VNet2GW --enable-bgp -l eastus --shared-key "abc123" --vnet-gateway2 /subscriptions/<subscription ID>/resourceGroups/TestRG1/providers/Microsoft.Network/virtualNetworkGateways/VNet1GW

+>

+For more information about BGP, see [About BGP and VPN Gateway](vpn-gateway-bgp-overview.md).

+description: Learn how to configure BGP for Azure VPN Gateway using the Azure portal.

+This article helps you enable BGP on cross-premises site-to-site (S2S) VPN connections and VNet-to-VNet connections using the Azure portal. This article helps you enable BGP on cross-premises site-to-site (S2S) VPN connections and VNet-to-VNet connections using Azure PowerShell. You can also create this configuration using the [Azure portal](bgp-howto.md) or [PowerShell](vpn-gateway-bgp-resource-manager-ps.md) steps.

+BGP is the standard routing protocol commonly used in the Internet to exchange routing and reachability information between two or more networks. BGP enables the VPN gateways and your on-premises VPN devices, called BGP peers or neighbors, to exchange "routes" that will inform both gateways on the availability and reachability for those prefixes to go through the gateways or routers involved. BGP can also enable transit routing among multiple networks by propagating routes a BGP gateway learns from one BGP peer to all other BGP peers.

+## <a name ="config"></a>Enable BGP for the VPN gateway

+This section is required before you perform any of the steps in the other two configuration sections. The following configuration steps set up the BGP parameters of the VPN gateway as shown in Diagram 2.

+ > * By default, Azure assigns a private IP address from the GatewaySubnet prefix range automatically as the Azure BGP IP address on the VPN gateway. The custom Azure APIPA BGP address is needed when your on premises VPN devices use an APIPA address (169.254.0.1 to 169.254.255.254) as the BGP IP. VPN Gateway will choose the custom APIPA address if the corresponding local network gateway resource (on-premises network) has an APIPA address as the BGP peer IP. If the local network gateway uses a regular IP address (not APIPA), VPN Gateway will revert to the private IP address from the GatewaySubnet range.

+ > * The APIPA BGP addresses must not overlap between the on-premises VPN devices and all connected VPN gateways.

+ > * When APIPA addresses are used on VPN gateways, the gateways do not initiate BGP peering sessions with APIPA source IP addresses. The on-premises VPN device must initiate BGP peering connections.

+Once the gateway is created, you can obtain the BGP Peer IP addresses on the VPN gateway. These addresses are needed to configure your on-premises VPN devices to establish BGP sessions with the VPN gateway.

+On the virtual network gateway **Configuration** page, you can view the BGP configuration information on your VPN gateway: ASN, Public IP address, and the corresponding BGP peer IP addresses on the Azure side (default and APIPA). You can also make the following configuration changes:

+To establish a cross-premises connection, you need to create a *local network gateway* to represent your on-premises VPN device, and a *connection* to connect the VPN gateway with the local network gateway as explained in [Create site-to-site connection](tutorial-site-to-site-portal.md). The following sections contain the additional properties required to specify the BGP configuration parameters, as shown in Diagram 3.

+Before proceeding, make sure you have enabled BGP for the VPN gateway.

+* You can leave the **Address space** empty only if you're using BGP to connect to this network. Azure VPN gateway will internally add a route of your BGP peer IP address to the corresponding IPsec tunnel. If you're **NOT** using BGP between the VPN gateway and this particular network, you **must** provide a list of valid address prefixes for the **Address space**.

+* You can optionally use an **APIPA IP address** (169.254.x.x) as your on-premises BGP peer IP if needed. But you'll also need to specify an APIPA IP address as described earlier in this article for your VPN gateway, otherwise the BGP session can't establish for this connection.

+In this step, you create a new connection that has BGP enabled. If you already have a connection and you want to enable BGP on it, you can update it.

+#### To update an existing connection

+#### On-premises device configuration

+The following example lists the parameters you enter into the BGP configuration section on your on-premises VPN device for this exercise:

+```

+- Site5 ASN : 65050

+- Site5 BGP IP : 10.51.255.254

+- Prefixes to announce : (for example) 10.51.0.0/16

+- Azure VNet ASN : 65010

+- Azure VNet BGP IP : 10.12.255.30

+- Static route : Add a route for 10.12.255.30/32, with nexthop being the VPN tunnel interface on your device

+- eBGP Multihop : Ensure the "multihop" option for eBGP is enabled on your device if needed

+```

+## Enable BGP on VNet-to-VNet connections

+> [!NOTE]

+> A VNet-to-VNet connection without BGP will limit the communication to the two connected VNets only. Enable BGP to allow transit routing capability to other S2S or VNet-to-VNet connections of these two VNets.

+If you completed all three parts of this exercise, you have established the following network topology:

+For context, referring to **Diagram 4**, if BGP were to be disabled between TestVNet2 and TestVNet1, TestVNet2 wouldn't learn the routes for the on-premises network, Site5, and therefore couldn't communicate with Site 5. Once you enable BGP, as shown in the Diagram 4, all three networks will be able to communicate over the S2S IPsec and VNet-to-VNet connections.

+ Title: 'Configure BGP for VPN Gateway: PowerShell'

+# How to configure BGP for VPN Gateway: PowerShell

+This article helps you enable BGP on cross-premises site-to-site (S2S) VPN connections and VNet-to-VNet connections using Azure PowerShell. You can also create this configuration using the [Azure portal](bgp-howto.md) or [CLI](bgp-how-to-cli.md) steps.

+BGP is the standard routing protocol commonly used in the Internet to exchange routing and reachability information between two or more networks. BGP enables the VPN gateways and your on-premises VPN devices, called BGP peers or neighbors, to exchange "routes" that will inform both gateways on the availability and reachability for those prefixes to go through the gateways or routers involved. BGP can also enable transit routing among multiple networks by propagating routes a BGP gateway learns from one BGP peer to all other BGP peers.

+For more information about the benefits of BGP and to understand the technical requirements and considerations of using BGP, see [About BGP and Azure VPN Gateway](vpn-gateway-bgp-overview.md).

+## Getting started

+Each part of this article helps you form a basic building block for enabling BGP in your network connectivity. If you complete all three parts (configure BGP on the gateway, S2S connection, and VNet-to-VNet connection) you build the topology as shown in Diagram 1. You can combine these sections to build a more complex multihop transit network that meets your needs.

+**Diagram 1**

+## <a name ="enablebgp"></a>Enable BGP for the VPN gateway

+This section is required before you perform any of the steps in the other two configuration sections. The following configuration steps set up the BGP parameters of the VPN gateway as shown in Diagram 2.

+**Diagram 2**

+You can run the steps for this exercise using Azure Cloud Shell in your browser. If you want to use PowerShell directly from your computer instead, install the Azure Resource Manager PowerShell cmdlets. For more information about installing the PowerShell cmdlets, see [How to install and configure Azure PowerShell](/powershell/azure/).

+### Create and configure VNet1

+For this exercise, we start by declaring variables. The following example declares the variables using the values for this exercise. You can use the example variables (with the exception of subscription name) if you're running through the steps to become familiar with this type of configuration. Modify any variables, and then copy and paste into your PowerShell console. Be sure to replace the values with your own when configuring for production.

+```azurepowershell-interactive

+$RG1 = "TestRG1"

+If you use Azure Cloud Shell, you automatically connect to your account. If you use PowerShell from your computer, open your PowerShell console and connect to your account. Use the following sample to help you connect:

+```azurepowershell-interactive

+Next, create a new resource group.

+```azurepowershell-interactive

+New-AzResourceGroup -Name $RG1 -Location $Location1

+```

+```azurepowershell-interactive

+### Create the VPN gateway with BGP enabled

+Request a public IP address to be allocated to the gateway you'll create for your VNet. You'll also define the required subnet and IP configurations.

+```azurepowershell-interactive

+Create the virtual network gateway for TestVNet1. BGP requires a Route-Based VPN gateway, and also an additional parameter *-Asn* to set the ASN (AS Number) for TestVNet1. If you don't set the ASN parameter, ASN 65515 is assigned. Creating a gateway can take a while (45 minutes or more to complete).

+```azurepowershell-interactive

+Once the gateway is created, you can use this gateway to establish cross-premises connection or VNet-to-VNet connection with BGP.

+#### 3. Get the Azure BGP Peer IP address

+Once the gateway is created, you need to obtain the BGP Peer IP address on the VPN gateway. This address is needed to configure the VPN gateway as a BGP Peer for your on-premises VPN devices.

+If you are using CloudShell, you may need to reestablish your variables if the session timed out while creating your gateway.

+Reestablish variables if necessary:

+```azurepowershell-interactive

+$RG1 = "TestRG1"

+$GWName1 = "VNet1GW"

+```

+Run the following command and note the "BgpPeeringAddress" value from the output.

+```azurepowershell-interactive

+Example output:

+```PowerShell

+If you don't see the BgpPeeringAddress displayed as an IP address, your gateway is still being configured. Try again when the gateway is complete.

+## Establish a cross-premises connection with BGP

+To establish a cross-premises connection, you need to create a *local network gateway* to represent your on-premises VPN device, and a *connection* to connect the VPN gateway with the local network gateway as explained in [Create site-to-site connection](tutorial-site-to-site-portal.md). The following sections contain the properties required to specify the BGP configuration parameters, shown in Diagram 3.

+**Diagram 3**

+Before proceeding, make sure you enabled BGP for the VPN gateway in the previous section.

+This exercise continues to build the configuration shown in the diagram. Be sure to replace the values with the ones that you want to use for your configuration. For example, you need to the IP address for your VPN device. For this exercise, you can substitute a valid IP address if you don't plan on connecting to your VPN device at this time. You can later replace the IP address.

+```azurepowershell-interactive

+$RG5 = "TestRG5"

+$Location5 = "West US"

+$LNGPrefix50 = "10.51.255.254/32"

+$LNGIP5 = "4.3.2.1"

+$BGPPeerIP5 = "10.51.255.254"

+* The prefix you need to declare for the local network gateway is the host address of your BGP Peer IP address on your VPN device. In this case, it's a /32 prefix of "10.51.255.254/32".

+* As a reminder, you must use different BGP ASNs between your on-premises networks and Azure VNet. If they're the same, you need to change your VNet ASN if your on-premises VPN device already uses the ASN to peer with other BGP neighbors.

+Create the resource group before you create the local network gateway.

+```azurepowershell-interactive

+```

+Create the local network gateway. Notice the two additional parameters for the local network gateway: Asn and BgpPeerAddress.

+```azurepowershell-interactive

+```azurepowershell-interactive

+In this step, you create the connection from TestVNet1 to Site5. You must specify "-EnableBGP $True" to enable BGP for this connection. As discussed earlier, it's possible to have both BGP and non-BGP connections for the same VPN gateway. Unless BGP is enabled in the connection property, Azure won't enable BGP for this connection even though BGP parameters are already configured on both gateways.

+Redeclare your variables if necessary:

+```azurepowershell-interactive

+$Connection15 = "VNet1toSite5"

+$Location1 = "East US"

+```

+Then run the following command:

+```azurepowershell-interactive

+#### On-premises device configuration

+- Site5 BGP IP : 10.51.255.254

+- Prefixes to announce : (for example) 10.51.0.0/16

+## Establish a VNet-to-VNet connection with BGP

+This section adds a VNet-to-VNet connection with BGP, as shown in the Diagram 4.

+**Diagram 4**

+The following instructions continue from the previous steps. You must first complete the steps in the [Enable BGP for the VPN gateway](#enablebgp) section to create and configure TestVNet1 and the VPN gateway with BGP.

+It's important to make sure that the IP address space of the new virtual network, TestVNet2, doesn't overlap with any of your VNet ranges.

+```azurepowershell-interactive

+$RG2 = "TestRG2"

+$Location2 = "East US"

+```azurepowershell-interactive

+Request a public IP address to be allocated to the gateway you'll create for your VNet and define the required subnet and IP configurations.

+Declare your variables.

+```azurepowershell-interactive

+Create the VPN gateway with the AS number. You must override the default ASN on your VPN gateways. The ASNs for the connected VNets must be different to enable BGP and transit routing.

+```azurepowershell-interactive

+Reestablish variables if necessary:

+```azurepowershell-interactive

+$GWName1 = "VNet1GW"

+$GWName2 = "VNet2GW"

+$RG1 = "TestRG1"

+$RG2 = "TestRG2"

+$Connection12 = "VNet1toVNet2"

+$Connection21 = "VNet2toVNet1"

+$Location1 = "East US"

+$Location2 = "East US"

+```

+Get both gateways.

+```azurepowershell-interactive

+TestVNet1 to TestVNet2 connection.

+```azurepowershell-interactive

+```

+TestVNet2 to TestVNet1 connection.

+```azurepowershell-interactive

+If you completed all three parts of this exercise, you've established the following network topology:

+**Diagram 4**

+For context, referring to **Diagram 4**, if BGP were to be disabled between TestVNet2 and TestVNet1, TestVNet2 wouldn't learn the routes for the on-premises network, Site5, and therefore couldn't communicate with Site 5. Once you enable BGP, as shown in the Diagram 4, all three networks will be able to communicate over the S2S IPsec and VNet-to-VNet connections.

+For more information about BGP, see [About BGP and VPN Gateway](vpn-gateway-bgp-overview.md).

+> [!Note]

+> VPN gateways do not reply to ICMP on their local address.