+ <DataUri>urn:com:microsoft:aad:b2c:elements:contract:unifiedssp:2.1.7</DataUri>

+ <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.1.14</DataUri>

+ <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.1.14</DataUri>

+ <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.1.14</DataUri>

+ <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.1.14</DataUri>

+If you accidentally deleted the `b2c-extensions-app`, you have 30 days to recover it.

+> [!NOTE]

+> An application can only be restored if it has been deleted within the last 30 days. If it has been more than 30 days, data will be permanently lost. For more assistance, file a support ticket.

+### Recover the extensions app using the Azure portal

+1. Sign in to your Azure AD B2C tenant.

+2. Search for and open **App registrations**.

+1. Select the **Deleted applications** tab and identify the `b2c-extensions-app` from the list of recently deleted applications.

+1. Select **Restore app registration**.

+You should now be able to [see the restored app](#verifying-that-the-extensions-app-is-present) in the Azure portal.

+### Recover the extensions app using Microsoft Graph

+To restore the app using Microsoft Graph, you must restore both the application and the service principal.

+To restore the application:

+1. Issue an HTTP GET against the URL `https://graph.microsoft.com/v1.0/directory/deleteditems/microsoft.graph.application`. This operation will list all of the applications that have been deleted within the past 30 days. You can also use the URL `https://graph.microsoft.com/v1.0/directory/deletedItems/microsoft.graph.application?$filter=displayName eq 'b2c-extensions-app. Do not modify. Used by AADB2C for storing user data.'` to filter by the app's **displayName** property.

+1. Find the application in the list where the name begins with `b2c-extensions-app` and copy its `id` property value.

+1. Issue an HTTP POST against the URL `https://graph.microsoft.com/v1.0/directory/deleteditems/{id}/restore`. Replace the `{id}` portion of the URL with the `id` from the previous step.]

+To restore the service principal:

+1. Issue an HTTP GET against the URL `https://graph.microsoft.com/v1.0/directory/deleteditems/microsoft.graph.servicePrincipal`. This operation will list all of the service principals that have been deleted within the past 30 days. You can also use the URL `https://graph.microsoft.com/v1.0/directory/deletedItems/microsoft.graph.servicePrincipal?$filter=displayName eq 'b2c-extensions-app. Do not modify. Used by AADB2C for storing user data.'` to filter by the app's **displayName** property.

+1. Find the service principal in the list where the name begins with `b2c-extensions-app` and copy its `id` property value.

+1. Issue an HTTP POST against the URL `https://graph.microsoft.com/v1.0/directory/deleteditems/{id}/restore`. Replace the `{id}` portion of the URL with the `id` from the previous step.

+You should now be able to [see the restored app](#verifying-that-the-extensions-app-is-present) in the Azure portal.

+## Application extension (directory extension) properties

+Application extension properties are also known as directory or Azure AD extensions. To manage them in Azure AD B2C, use the [identityUserFlowAttribute resource type](/graph/api/resources/identityuserflowattribute) and its associated methods.

+- [Create user flow attribute](/graph/api/identityuserflowattribute-post)

+- [List user flow attributes](/graph/api/identityuserflowattribute-list)

+- [Get a user flow attribute](/graph/api/identityuserflowattribute-get)

+- [Update a user flow attribute](/graph/api/identityuserflowattribute-update)

+- [Delete a user flow attribute](/graph/api/identityuserflowattribute-delete)

+You can store up to 100 directory extension values per user. To manage the directory extension properties for a user, use the following [User APIs](/graph/api/resources/user) in Microsoft Graph.

+- [Update user](/graph/api/user-update): To write or remove the value of the directory extension property from the user object.

+- [Get a user](/graph/api/user-get): To retrieve the value of the directory extension for the user. The property will be returned by default through the `beta` endpoint, but only on `$select` through the `v1.0` endpoint.

+For user flows, these extension properties are [managed by using the Azure portal](user-flow-custom-attributes.md). For custom policies, Azure AD B2C creates the property for you, the first time the policy writes a value to the extension property.

+> [!NOTE]

+> In Azure AD, directory extensions are managed through the [extensionProperty resource type](/graph/api/resources/extensionproperty) and its associated methods. However, because they are used in B2C through the `b2c-extensions-app` app which should not be updated, they are managed in Azure AD B2C using the [identityUserFlowAttribute resource type](/graph/api/resources/identityuserflowattribute) and its associated methods.

+## Retrieve or restore deleted users and applications

+Deleted items can only be restored if they were deleted within the last 30 days.

+- [List deleted items](/graph/api/directory-deleteditems-list)

+- [Get a deleted item](/graph/api/directory-deleteditems-get)

+- [Restore a deleted item](/graph/api/directory-deleteditems-restore)

+- [Permanently delete a deleted item](/graph/api/directory-deleteditems-delete)

+Azure AD B2C allows you to extend the set of attributes stored on each user account. You can also read and write these attributes by using the [Microsoft Graph API](microsoft-graph-operations.md#application-extension-directory-extension-properties).

+Once you've created a new user using the user flow, you can use the [Run user flow](./tutorial-create-user-flows.md) feature on the user flow to verify the customer experience. You should now see **ShoeSize** in the list of attributes collected during the sign-up journey, and see it in the token sent back to your application.

+|`extension_<b2c-extensions-app-guid>_loyaltyId` | [Microsoft Graph API](microsoft-graph-operations.md#application-extension-directory-extension-properties)|

+You can use Microsoft Graph to create and manage the custom attributes then set the values for a user. Extension attributes are also called directory or Azure AD extensions.

+Custom attributes (directory extensions) in the Microsoft Graph API are named by using the convention `extension_{appId-without-hyphens}_{extensionProperty-name}` where `{appId-without-hyphens}` is the stripped version of the **appId** (called Client ID on the Azure AD B2C portal) for the `b2c-extensions-app` with only characters 0-9 and A-Z. For example, if the **appId** of the `b2c-extensions-app` application is `25883231-668a-43a7-80b2-5685c3f874bc` and the attribute name is `loyaltyId`, then the custom attribute will be named `extension_25883231668a43a780b25685c3f874bc_loyaltyId`.

+Learn how to [manage extension attributes in your Azure AD B2C tenant](microsoft-graph-operations.md#application-extension-directory-extension-properties) using the Microsoft Graph API.

+Unlike built-in attributes, custom attributes can be removed. The extension attributes' values can also be removed.

+> Before you remove the custom attribute, for each account in the directory, set the extension attribute value to `null`. In this way you explicitly remove the extension attributesΓÇÖs values. Then continue to remove the extension attribute itself. Custom attributes can be queries using Microsoft Graph API.

+Use the following steps to remove a custom attribute from a user flow in your:

+Use the [Microsoft Graph API](microsoft-graph-operations.md#application-extension-directory-extension-properties) to manage the custom attributes.

+[ms-graph-api]: /graph/api/overview

+> [!NOTE]

+> Email notifications from the SSPR service will be sent from the following addresses based on the Azure cloud you are working with:

+> - Public: msonlineservicesteam@microsoft.com

+> - China: msonlineservicesteam@oe.21vianet.com

+> - Government: msonlineservicesteam@azureadnotifications.us

+> If you observe issues in receiving notifications, please check your spam settings.

+> [!NOTE]

+> Email notifications from the SSPR service will be sent from the following addresses based on the Azure cloud you are working with:

+> - Public: msonlineservicesteam@microsoft.com

+> - China: msonlineservicesteam@oe.21vianet.com

+> - Government: msonlineservicesteam@azureadnotifications.us

+> If you observe issues in receiving notifications, please check your spam settings.

+### Custom controls (preview)

+Custom controls is a preview capability of the Azure Active Directory. When using custom controls, your users are redirected to a compatible service to satisfy authentication requirements outside of Azure Active Directory. For more information, check out the [Custom controls](controls.md) article.

+- **Phase 1**: Collect session details

+- **Phase 2**: Enforcement

+ - [Multi-factor authenticationΓÇï](concept-conditional-access-grant.md#require-multifactor-authentication)

+ - [Device to be marked as compliant](./concept-conditional-access-grant.md#require-device-to-be-marked-as-compliant)

+ - [Hybrid Azure AD joined device](./concept-conditional-access-grant.md#require-hybrid-azure-ad-joined-device)

+ - [Approved client app](./concept-conditional-access-grant.md#require-approved-client-app)

+ - [App protection policy](./concept-conditional-access-grant.md#require-app-protection-policy)

+ - [Password change](./concept-conditional-access-grant.md#require-password-change)

+ - [Terms of use](concept-conditional-access-grant.md#terms-of-use)

+ - [Custom controls](./concept-conditional-access-grant.md#custom-controls-preview)

+The software the user is employing to access the cloud app. For example, 'Browser', and 'Mobile apps and desktop clients'. By default, all newly created Conditional Access policies will apply to all client app types even if the client apps condition isn't configured.

+- Require multi-factor authentication

+ - Passes device information to allow control of experience granting full or limited access.

+- Customize continuous access evaluation

+- Disable resilience defaults

+ - This selection includes any [B2B guests and external users](../external-identities/external-identities-overview.md) including any user with the `user type` attribute set to `guest`. This selection also applies to any external user signed-in from a different organization like a Cloud Solution Provider (CSP).

+ - Allows administrators to select specific [built-in Azure AD directory roles](../roles/permissions-reference.md) used to determine policy assignment. For example, organizations may create a more restrictive policy on users assigned the global administrator role. Other role types aren't supported, including administrative unit-scoped roles and custom roles.

+ - Signals integration with [Azure AD Identity Protection](../identity-protection/overview-identity-protection.md) allows Conditional Access policies to identify risky sign-in behavior. Policies can then force users to change their password, do multi-factor authentication to reduce their risk level, or block access until an administrator takes manual action.

+- [Microsoft Defender for Cloud Apps](/defender-cloud-apps/what-is-defender-for-cloud-apps)

+1. Search for and select **Azure Active Directory**. From the menu on the left-hand side select **Security**.

+1. Select **Conditional Access**.

+Don't include a `Subject` element. Azure AD doesn't support specifying a subject in `AuthnRequest` and will return an error if one is provided.

+A subject can instead be provided by adding a `login_hint` parameter to the HTTP request to the single sign-on URL, with the subject's NameID as the parameter value.

+This article describes cross-tenant access settings, which are used to manage B2B collaboration and B2B direct connect with external Azure AD organizations, including across Microsoft clouds. More settings are available for B2B collaboration with non-Azure AD identities (for example, social identities or non-IT managed external accounts). These [external collaboration settings](external-collaboration-settings-configure.md) include options for restricting guest user access, specifying who can invite guests, and allowing or blocking domains.

+The external identities cross-tenant access settings manage how you collaborate with other Azure AD organizations. These settings determine both the level of inbound access users in external Azure AD organizations have to your resources, and the level of outbound access your users have to external organizations.

+The following diagram shows the cross-tenant access inbound and outbound settings. The **Resource Azure AD tenant** is the tenant containing the resources to be shared. In the case of B2B collaboration, the resource tenant is the inviting tenant (for example, your corporate tenant, where you want to invite the external users to). The **User's home Azure AD tenant** is the tenant where the external users are managed.

+

+- **B2B collaboration**: All your internal users are enabled for B2B collaboration by default. This setting means your users can invite external guests to access your resources and they can be invited to external organizations as guests. MFA and device claims from other Azure AD organizations aren't trusted.

+- Use B2B collaboration to [share Power BI content to a user in the partner tenant](/power-bi/enterprise/service-admin-azure-ad-b2b#cross-cloud-b2b).

+- To configure trust settings or apply access settings to specific users, groups, or applications, you'll need an Azure AD Premium P1 license. The license is required on the tenant that you configure. For B2B direct connect, where mutual trust relationship with another Azure AD organization is required, you'll need an Azure AD Premium P1 license in both tenants.

+ - **Select external users and groups**: Lets you apply the action you chose under **Access status** to specific users and groups within the external organization. An Azure AD Premium P1 license is required on the tenant that you configure.

+ > When targeting your users and groups, you won't be able to select users who have configured [SMS-based authentication](../authentication/howto-authentication-sms-signin.md). This is because users who have a "federated credential" on their user object are blocked to prevent external users from being added to outbound access settings. As a workaround, you can use the [Microsoft Graph API](/graph/api/resources/crosstenantaccesspolicy-overview) to add the user's object ID directly or target a group the user belongs to.

+mail | This user has an email address that doesn't match with verified Azure AD or SAML/WS-Fed domains, and isn't a Gmail address or a Microsoft account.

+phone | This user has an email address that doesn't match a verified Azure AD domain or a SAML/WS-Fed domain, and isn't a Gmail address or Microsoft account.

+## Guest user permissions

+Guest users have [default restricted directory permissions](../fundamentals/users-default-permissions.md). They can manage their own profile, change their own password, and retrieve some information about other users, groups, and apps. However, they can't read all directory information.

+There may be cases where you want to give your guest users higher privileges. You can add a guest user to any role and even remove the default guest user restrictions in the directory to give a user the same privileges as members. It's possible to turn off the default limitations so that a guest user in the company directory has the same permissions as a member user. For more information, check out the [Restrict guest access permissions in Azure Active Directory](../enterprise-users/users-restrict-guest-permissions.md) article.

+* Some of the [limits on Guest functionality can be removed](../external-identities/user-properties.md#guest-user-permissions).

+* If a mail-contact object exists for the same mail address as the invited guest user, the guest user will be created, but is NOT mail enabled.

+Some organizations use the mail-contact object to show users in the GAL. This approach integrates a GAL without providing other permissions as mail-contacts aren't security principals.

+A mail-contact object can't be converted to a user object. Therefore, any properties associated with a mail-contact object can't be transferred. For example, group memberships and other resource access aren't transferred.

+* **Azure AD Self-service group management (SSGM)** ΓÇô Mail-contact objects aren't eligible to be members in groups using the SSGM feature. More tools may be needed to manage groups with recipients represented as contacts instead of user objects.

+More considerations when configuring access control.

+If an application has the [User assignment required?] property set to [No], guest users can access the application. Application admins must understand access control impacts, especially if the application contains sensitive information. For more information, see [How to restrict your Azure AD app to a set of users](../develop/howto-restrict-your-app-to-a-set-of-users.md).

+Azure AD External Identities (guest user) pricing is based on monthly active users (MAU). The number of active users is the count of unique users with authentication activity within a calendar month. MAU billing helps you reduce costs by offering a free tier and flexible, predictable pricing. In addition, the first 50,000 MAUs per month are free for both Premium P1 and Premium P2 features. Premium features include Conditional Access Policies and Azure AD Multi-Factor Authentication for guest users.

+* A guest user can be assigned an Exchange Online license. However, they're prevented from being issued a token for Exchange Online. The results are that they aren't able to access the resource.

+* Guest users need to be unhidden in order to be included in the GAL. By default, they're hidden.

+As shown above, you can use the [Set-MailUser](/powershell/module/exchange/set-mailuser?view=exchange-ps&preserve-view=true) PowerShell cmdlet for mail-specific properties. More user properties you can modify with the [Set-User](/powershell/module/exchange/set-user?view=exchange-ps&preserve-view=true) PowerShell cmdlet. Most of the properties can also be modified using the Azure AD Graph APIs.

+When using Azure B2B with Office 365 workloads, there are some key considerations. There are instances in which guest accounts don't have the same experience as a member account.

+* Users that are licensed in their home tenant may access resources in another tenant within the same legal entity. The access is granted using **External Members** setting with no extra licensing fees. The setting applies for SharePoint, OneDrive for Business, Teams, and Groups.

+ * Engineering work is underway to automatically check the license status of a user in their home tenant and enable them to participate as a Member with no extra license assignment or configuration. However, for customers who wish to use External Members now, there's a licensing workaround that requires the Account Executive to work with the Microsoft Business Desk.

+### Limitations

+OIDC issuer is only supported in Azure Public regions now.

+[aks-add-np-containerd]: ./learn/quick-windows-container-deploy-cli.md#add-a-windows-server-node-pool-with-containerd

+| * / 6380 | Inbound & Outbound | TCP | VirtualNetwork / VirtualNetwork | Access external Azure Cache for Redis service for [caching](api-management-caching-policies.md) policies between machines (optional) | External & Internal |

+| * / 6381 - 6383 | Inbound & Outbound | TCP | VirtualNetwork / VirtualNetwork | Access internal Azure Cache for Redis service for [caching](api-management-caching-policies.md) policies between machines (optional) | External & Internal |

+ version = "~> 3.0.0"

+resource "azurerm_service_plan" "appserviceplan" {

+ os_type = "Linux"

+ sku_name = "F1"

+# Create the web app, pass in the App Service Plan ID

+resource "azurerm_linux_web_app" "webapp" {

+ name = "webapp-${random_integer.ri.result}"

+ location = azurerm_resource_group.rg.location

+ resource_group_name = azurerm_resource_group.rg.name

+ service_plan_id = azurerm_service_plan.appserviceplan.id

+ https_only = true

+ site_config {

+ minimum_tls_version = "1.2"

+# Deploy code from a public GitHub repo

+resource "azurerm_app_service_source_control" "sourcecontrol" {

+ app_id = azurerm_linux_web_app.webapp.id

+ repo_url = "https://github.com/Azure-Samples/nodejs-docs-hello-world"

+ branch = "master"

+ use_manual_integration = true

+ use_mercurial = false

+}

+Four Azure resources are defined in the template. Links to the [Azure Provider Terraform Registry](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs) are given below for further details and usage information:

+ * [azurerm_service_plan](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/service_plan)

+* [**Microsoft.Web/sites**](/azure/templates/microsoft.web/sites): create a Linux App Service app.

+ * [azurerm_linux_web_app](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/linux_web_app)

+ * [azurerm_app_service_source_control](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/app_service_source_control)

+> [Terraform samples for Azure App Service](./samples-terraform.md)

+In this quickstart, you'll learn how to create and deploy your first [WordPress](https://www.wordpress.org) site to [Azure App Service on Linux](overview.md#app-service-on-linux) using the [WordPress on the Azure Marketplace](https://azuremarketplace.microsoft.com/marketplace/apps/WordPress.WordPress?tab=Overview). It uses the **Basic** tier and [**incurs a cost**](https://azure.microsoft.com/pricing/details/app-service/linux/) for your Azure subscription. The WordPress installation comes with pre-installed plugins for performance improvements, [W3TC](https://wordpress.org/plugins/w3-total-cache/) for caching and [Smush](https://wordpress.org/plugins/wp-smushit/) for image compression.

+To complete this quickstart, you need an Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs).

+> - [After November 28, 2022, PHP will only be supported on App Service on Linux.](https://github.com/Azure/app-service-linux-docs/blob/master/Runtime_Support/php_support.md#end-of-life-for-php-74).

+> - The MySQL Flexible Server is created behind a private [Virtual Network](/virtual-network/virtual-networks-overview) and can't be accessed directly. To access the database, use phpMyAdmin that's deployed with the WordPress site. It can be found at the URL : https://`<sitename>`.azurewebsites.net/phpmyadmin

+> If you have feedback to improve this WordPress offering on App Service, submit your ideas at [Web Apps Community](https://feedback.azure.com/d365community/forum/b09330d1-c625-ec11-b6e6-000d3a4f0f1c).

+1. To start creating the WordPress site, browse to [https://ms.portal.azure.com/#create/WordPress.WordPress](https://ms.portal.azure.com/#create/WordPress.WordPress).

+ :::image type="content" source="./media/quickstart-wordpress/01-portal-create-wordpress-on-app-service.png?text=WordPress from Azure Marketplace" alt-text="Screenshot of Create a WordPress site.":::

+ :::image type="content" source="./media/quickstart-wordpress/04-wordpress-basics-project-details.png?text=Azure portal WordPress Project Details" alt-text="Screenshot of WordPress project details.":::

+ :::image type="content" source="./media/quickstart-wordpress/05-wordpress-basics-instance-details.png?text=WordPress basics instance details" alt-text="Screenshot of WordPress instance details.":::

+ :::image type="content" source="./media/quickstart-wordpress/06-wordpress-basics-wordpress-settings.png?text=Azure portal WordPress settings" alt-text="Screenshot of WordPress settings.":::

+ :::image type="content" source="./media/quickstart-wordpress/08-wordpress-advanced-settings.png" alt-text="Screenshot of WordPress Advanced Settings.":::

+ :::image type="content" source="./media/quickstart-wordpress/09-wordpress-create.png?text=WordPress create button" alt-text="Screenshot of WordPress create button.":::

+ :::image type="content" source="./media/quickstart-wordpress/wordpress-sample-site.png?text=WordPress sample site" alt-text="Screenshot of WordPress site.":::

+ :::image type="content" source="./media/quickstart-wordpress/wordpress-admin-login.png?text=WordPress admin login" alt-text="Screenshot of WordPress admin login.":::

+ :::image type="content" source="./media/quickstart-wordpress/resource-group.png" alt-text="Resource group in App Service overview page.":::

+ :::image type="content" source="./media/quickstart-wordpress/delete-resource-group.png" alt-text="Delete resource group.":::

+## Check version of Hybrid Worker

+To check the version of agent-based Linux Hybrid Runbook Worker, go to the following path:

+`vi/opt/microsoft/omsconfig/modules/nxOMSAutomationWorker/VERSION`

+The file *VERSION* has the version number of Hybrid Runbook Worker.

+## Check version of Hybrid Worker

+To check version of agent-based Windows Hybrid Runbook Worker, go to the following path:

+`C:\ProgramFiles\Microsoft Monitoring Agent\Agent\AzureAutomation\`

+The *AzureAutomation* folder has a sub-folder with the version number as the name of the sub-folder.

+You can use the user Hybrid Runbook Worker feature of Azure Automation to run runbooks directly on an Azure machine or a non-Azure machine through servers registered with [Azure Arc-enabled servers](../azure-arc/servers/overview.md). From the machine or server that's hosting the role, you can run runbooks directly against it and against resources in the environment to manage those local resources.

+## Check version of Hybrid Worker

+To check the version of the extension-based Hybrid Runbook Worker:

+|OS types | Paths | Description|

+| | | |

+|**Windows** |`C:\Packages\Plugins\Microsoft.Azure.Automation.HybridWorker.HybridWorkerForWindows\`| The path has *version* folder that has the version information. |

+|**Linux** | `/var/lib/waagent/Microsoft.Azure.Automation.HybridWorker.HybridWorkerForLinux-<version>` | The folder name ends with *version* information. |

+The following high-level steps are involved in [connecting a Kubernetes cluster to Azure Arc](quickstart-connect-cluster.md):

+> [!NOTE]

+> To translate the `*.servicebus.windows.net` wildcard into specific endpoints, use the command `\GET /urls/allowlist?api-version=2020-01-01&location=<location>`. Within this command, the region must be specified for the <location> placeholder.

+Some Linux versions use TCP settings that are too high by default. The higher TCP settings can create a situation where a client connection to a cache cannot be reestablished for a long time when a Redis server stops responding. The client waits too long before closing the connection gracefully.

+The failure to reestablish a connection can happen if the primary node of your Azure Cache For Redis becomes unavailable, for example, for unplanned maintenance.

+| [Persistence](#persistence) | Periodic data saving to storage account. | 99.9% (see [details](https://azure.microsoft.com/support/legal/sla/cache/v1_1/)) |-|Γ£ö|Preview|

+Applicable tiers: **Premium**, **Enterprise (preview)**, **Enterprise Flash (preview)**

+Update the Python code file *__init__.py*, based on the interface that your framework uses. The following example shows either an ASGI handler approach or a WSGI wrapper approach for Flask:

+|Latest GA SDK | No newer supported stable version | **NO UPDATE NECESSARY** |

+|Stable minor version of a GA SDK | Newer supported stable version | **UPDATE RECOMMENDED** |

+|Unsupported ([support policy](/lifecycle/faq/azure)) | Any supported version | **UPDATE REQUIRED** |

+|Preview | Stable version | **UPDATE REQUIRED** |

+|Preview | Older stable version | **UPDATE RECOMMENDED** |

+|Preview | Newer preview version, no older stable version | **UPDATE RECOMMENDED** |

+To see the current version of Application Insights SDKs and previous versions release dates, reference the [release notes](release-notes.md).

+Container insights includes a predefined set of metrics and inventory items collected that are written as log data in your Log Analytics workspace. All metrics listed below are collected every one minute.

+For more information about how to understand what the costs are likely to be based on recent usage patterns from data collected with Container insights, see [Analyze usage in Log Analytics workspace](../logs/analyze-usage.md).

+ Title: Structure of transformation in Azure Monitor (preview)

+> [!TIP]

+> For more information on generating a new Service Principal, refer to the section [Enable communication with Storage](azacsnap-installation.md?tabs=azure-netapp-files%2Csap-hana#enable-communication-with-storage) in the [Install Azure Application Consistent Snapshot tool](azacsnap-installation.md) guide.

+Several features of Azure NetApp Files require that you have an Active Directory connection. For example, you need to have an Active Directory connection before you can create an [SMB volume](azure-netapp-files-create-volumes-smb.md), a [NFSv4.1 Kerberos volume](configure-kerberos-encryption.md), or a [dual-protocol volume](create-volumes-dual-protocol.md). This article shows you how to create and manage Active Directory connections for Azure NetApp Files.

+The reset Active Directory computer account password feature is currently in public preview. If you're using this feature for the first time, you need to register the feature first.

+Alternately, navigate to the **Volumes** menu. Identify the volume for which you want to reset the Active Directory account and select the three dots (`...`) at the end of the row. Select **Reset Active Directory Account**.

+| The SMB or dual-protocol volume creation fails with the following error: <br> `{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"InternalServerError", "message":"Error when creating - Could not query DNS server. Verify that the network configuration is correct and that DNS servers are available."}]}` | This error indicates that the DNS is not reachable. <br> Consider the following solutions: <ul><li>Check if AD DS and the volume are being deployed in same region.</li> <li>Check if AD DS and the volume are using the same VNet. If they're using different VNETs, make sure that the VNets are peered with each other. See [Guidelines for Azure NetApp Files network planning](azure-netapp-files-network-topologies.md). </li> <li>The DNS server might have network security groups (NSGs) applied. As such, it does not allow the traffic to flow. In this case, open the NSGs to the DNS or AD to connect to various ports. For port requirements, see [Requirements for Active Directory connections](create-active-directory-connections.md#requirements-for-active-directory-connections). </li></ul> <br>The same solutions apply for Azure AD DS. Azure AD DS should be deployed in the same region. The VNet should be in the same region or peered with the VNet used by the volume. |

+| The SMB or dual-protocol volume creation fails with the following error: <br> `Failed to create the Active Directory machine account \"SMB-ANF-VOL. Reason: LDAP Error: Local error occurred Details: Error: Machine account creation procedure failed. [nnn] Loaded the preliminary configuration. [nnn] Successfully connected to ip 10.x.x.x, port 88 using TCP [nnn] Successfully connected to ip 10.x.x.x, port 389 using [nnn] Entry for host-address: 10.x.x.x not found in the current source: FILES. Ignoring and trying next available source [nnn] Source: DNS unavailable. Entry for host-address:10.x.x.x found in any of the available sources\n*[nnn] FAILURE: Unable to SASL bind to LDAP server using GSSAPI: local error [nnn] Additional info: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Cannot determine realm for numeric host address) [nnn] Unable to connect to LDAP (Active Directory) service on contoso.com (Error: Local error) [nnn] Unable to make a connection (LDAP (Active Directory):contosa.com, result: 7643. ` | The pointer (PTR) record of the AD host machine might be missing on the DNS server. You need to create a reverse lookup zone on the DNS server, and then add a PTR record of the AD host machine in that reverse lookup zone. |

+| SMB volume creation fails with the following error: <br> `Failed to create the Active Directory machine account. Reason: LDAP Error: Intialization of LDAP library failed Details: Error: Machine account creation procedure failed` | This error occurs because the service or user account used in the Azure NetApp Files Active Directory connections does not have sufficient privilege to create computer objects or make modifications to the newly created computer object. <br> To solve the issue, you should grant the account being used greater privilege. You can apply a default role with sufficient privilege. You can also delegate additional privilege to the user or service account or to a group it's part of. |

+| Dual-protocol volume creation fails with the error `Could not query DNS server. Verify that the network configuration is correct and that DNS servers are available`. | This error indicates that DNS is not reachable. The reason might be because DNS IP is incorrect, or there's a networking issue. Check the DNS IP entered in AD connection and make sure that the IP is correct. <br> Also, make sure that the AD and the volume are in same region and in same VNet. If they are in different VNETs, ensure that VNet peering is established between the two VNets. <br> See [Guidelines for Azure NetApp Files network planning](azure-netapp-files-network-topologies.md#azure-native-environments) for details. |

+|`mount.nfs: access denied by server when mounting volume <SMB_SERVER_NAME-XXX.DOMAIN_NAME>/<VOLUME_NAME>` <br> Example: `smb-test-64d9.contoso.com:/nfs41-vol101` | <ol><li> Ensure that the A/PTR records are properly set up and exist in the Active Directory for the server name `smb-test-64d9.contoso.com`. <br> In the NFS client, if `nslookup` of `smb-test-64d9.contoso.com` resolves to IP address IP1 (that is, `10.1.1.68`), then `nslookup` of IP1 must resolve to only one record (that is, `smb-test-64d9.contoso.com`). `nslookup` of IP1 *must* not resolve to multiple names. </li> <li>Set AES-256 for the NFS machine account of type `NFS-<Smb NETBIOS NAME>-<few random characters>` on AD using either PowerShell or the UI. <br> Example commands: <ul><li>`Set-ADComputer <NFS_MACHINE_ACCOUNT_NAME> -KerberosEncryptionType AES256` </li><li>`Set-ADComputer NFS-SMB-TEST-64 -KerberosEncryptionType AES256` </li></ul> </li> <li>Ensure that the time of the NFS client, AD, and Azure NetApp Files storage software is synchronized with each other and is within a five-minute skew range. </li> <li>Get the Kerberos ticket on the NFS client using the command `kinit <administrator>`.</li> <li>Reduce the NFS client hostname to fewer than 15 characters and perform the realm join again. </li><li>Restart the NFS client and the `rpcgssd` service as follows. The command might vary depending on the OS.<br> RHEL 7: <br> `service nfs restart` <br> `service rpcgssd restart` <br> CentOS 8: <br> `systemctl enable nfs-client.target && systemctl start nfs-client.target` <br> Ubuntu: <br> (Restart the `rpc-gssd` service.) <br> `sudo systemctl start rpc-gssd.service` </ul>|

+| Error when creating an LDAP-enabled NFS volume: <br> `Could not query DNS server` <br> `Sample error message:` <br> `"log": time="2020-10-21 05:04:04.300" level=info msg=Res method=GET url=/v2/Volumes/070d0d72-d82c-c893-8ce3-17894e56cea3 x-correlation-id=9bb9e9fe-abb6-4eb5-a1e4-9e5fbb838813 x-request-id=c8032cb4-2453-05a9-6d61-31ca4a922d85 xresp="200: {\"created\":\"2020-10-21T05:02:55.000Z\",\"lifeCycleState\":\"error\",\"lifeCycleStateDetails\":\"Error when creating - Could not query DNS server. Verify that the network configuration is correct and that DNS servers are available.\",\"name\":\"smb1\",\"ownerId\ \":\"8c925a51-b913-11e9-b0de-9af5941b8ed0\",\"region\":\"westus2stage\",\"volumeId\":\"070d0d72-d82c-c893-8ce3-` | This error occurs because DNS is unreachable. <br> <ul><li> Check if you've configured the correct site (site scoping) for Azure NetApp Files. </li><li> The reason that DNS is unreachable might be an incorrect DNS IP address or networking issues. Check the DNS IP address entered in the AD connection to make sure that it is correct. </li><li> Make sure that the AD and the volume are in the same region and the same VNet. If they are in different VNets, ensure that VNet peering is established between the two VNets.</li></ul> |

+ "artifacts-parameters": {

+ "level": "warning"

+ },

+For Azure CLI, use [az deployment group create](/cli/azure/deployment/group#az-deployment-group-create). The following example deploys a template to create a resource group. The resource group you specify in the `--resource-group` parameter is the **target resource group**.

+For the PowerShell deployment command, use [New-AzResourceGroupDeployment](/powershell/module/az.resources/new-azresourcegroupdeployment). The following example deploys a template to create a resource group. The resource group you specify in the `-ResourceGroupName` parameter is the **target resource group**.

+ Title: Linter rule - artifacts parameters

+description: Linter rule - artifacts parameters

+# Linter rule - artifacts parameters

+This rule verifies whether the artifacts parameters are defined correctly. The following conditions must be met to pass the test:

+- If you provide one parameter (either `_artifactsLocation` or `_artifactsLocationSasToken`), you must provide the other.

+- `_artifactsLocation` must be a string.

+- If `_artifactsLocation` has a default value, it must be either `deployment().properties.templateLink.uri` or a raw URL for its default value.

+- `_artifactsLocationSasToken` must be a secure string.

+- If `_artifactsLocationSasToken` has a default value, it must be an empty string.

+- If a referenced module has an `_artifactsLocation` or `_artifactsLocationSasToken` parameter, a value must be passed in for those parameters, even if they have default values in the module.

+## Linter rule code

+Use the following value in the [Bicep configuration file](bicep-config-linter.md) to customize rule settings:

+`artifacts-parameters`

+## Solution

+The following example fails this test because `_artifactsLocationSasToken` is missing:

+```bicep

+@description('The base URI where artifacts required by this template are located including a trailing \'/\'')

+param _artifactsLocation string = deployment().properties.templateLink.uri

+...

+```

+The next example fails this test because `_artifactsLocation` must be either `deployment().properties.templateLink.uri` or a raw URL when the default value is provided, and the default value of `_artifactsLocationSasToken` is not an empty string.

+```bicep

+@description('The base URI where artifacts required by this template are located including a trailing \'/\'')

+param _artifactsLocation string = 'something'

+@description('SAS Token for accessing script path')

+@secure()

+param _artifactsLocationSasToken string = 'something'

+...

+````

+This example passes this test.

+```bicep

+@description('The base URI where artifacts required by this template are located including a trailing \'/\'')

+param _artifactsLocation string = deployment().properties.templateLink.uri

+@description('SAS Token for accessing script path')

+@secure()

+param _artifactsLocationSasToken string = ''

+...

+```

+## Next steps

+For more information about the linter, see [Use Bicep linter](./linter.md).

+- [artifacts-parameters](./linter-rule-artifacts-parameters.md)

+ If your ARM template is stored in a `.jsonc` file, comments using the `//` syntax are supported, as shown here.

+ ```javascript

+ "resources": [

+ {

+ // This storage account is used to store the VM disks.

+ "name": "[variables('storageAccountName')]",

+ "type": "Microsoft.Storage/storageAccounts",

+ "apiVersion": "2019-06-01",

+ "location": "[resourceGroup().location]",

+ ...

+ }

+ ]

+ ```

+ For more details about comments and metadata see [Understand the structure and syntax of ARM templates](/azure/azure-resource-manager/templates/syntax#comments-and-metadata).

+## Comments

+In addition to the `comments` property, comments using the `//` syntax are supported. For more details about comments and metadata see [Understand the structure and syntax of ARM templates](/azure/azure-resource-manager/templates/syntax#comments-and-metadata). You may choose to save JSON files that contain `//` comments using the `.jsonc` file extension, to indicate the JSON file contains comments. The ARM service will also accept comments in any JSON file including parameters files.

+## Visual Studio Code ARM Tools

+Working with ARM templates is much easier with the Azure Resource Manager (ARM) Tools for Visual Studio Code. This extension provides language support, resource snippets, and resource auto-completion to help you create and validate Azure Resource Manager templates. To learn more and install the extension, see [Azure Resource Manager (ARM) Tools](https://marketplace.visualstudio.com/items?itemName=msazurermtools.azurerm-vscode-tools).

+The value of the `--template-file` parameter must be a Bicep file or a `.json` or `.jsonc` file. The `.jsonc` file extension indicates the file can contain `//` style comments. The ARM system accepts `//` comments in `.json` files. It does not care about the file extension. For more details about comments and metadata see [Understand the structure and syntax of ARM templates](/azure/azure-resource-manager/templates/syntax#comments-and-metadata).

+## Comments and the extended JSON format

+You can include `//` style comments in your parameter file, but you must name the file with a `.jsonc` extension.

+```azurecli-interactive

+az deployment group create \

+ --name ExampleDeployment \

+ --resource-group ExampleGroup \

+ --template-file storage.json \

+ --parameters '@storage.parameters.jsonc'

+```

+For more details about comments and metadata see [Understand the structure and syntax of ARM templates](/azure/azure-resource-manager/templates/syntax#comments-and-metadata).

+If you are using Azure CLI with version 2.3.0 or older, you can deploy a template with multi-line strings or comments using the `--handle-extended-json-format` switch. For example:

+For Azure CLI, use [az deployment group create](/cli/azure/deployment/group#az-deployment-group-create). The following example deploys a template to create a resource group. The resource group you specify in the `--resource-group` parameter is the **target resource group**.

+For the PowerShell deployment command, use [New-AzResourceGroupDeployment](/powershell/module/az.resources/new-azresourcegroupdeployment). The following example deploys a template to create a resource group. The resource group you specify in the `-ResourceGroupName` parameter is the **target resource group**.

+## Limited access features

+This section talks about limited access features in Azure Video Indexer.

+|When did I create the account?|Trial Account (Free)| Paid Account <br/>(classic or ARM-based)|

+||||

+|Existing VI accounts <br/><br/>created before June 21, 2022|Able to access face identification, customization and celebrities recognition till June 2023. <br/><br/>**Recommended**: Move to a paid account and afterward fill in the [intake form](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUMkZIOUE1R0YwMkU0M1NMUTA0QVNXVDlKNiQlQCN0PWcu) and based on the eligibility criteria we will enable the features also after the grace period. |Able to access face identification, customization and celebrities recognition till June 2023\*.<br/><br/>**Recommended**: fill in the [intake form](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUMkZIOUE1R0YwMkU0M1NMUTA0QVNXVDlKNiQlQCN0PWcu) and based on the eligibility criteria we will enable the features also after the grace period. <br/><br/>We proactively sent emails to these customers + AEs.|

+|New VI accounts <br/><br/>created after June 21, 2022 |Not able the access face identification, customization and celebrities recognition as of today. <br/><br/>**Recommended**: Move to a paid account and afterward fill in the [intake form](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUMkZIOUE1R0YwMkU0M1NMUTA0QVNXVDlKNiQlQCN0PWcu). Based on the eligibility criteria we will enable the features (after max 10 days).|Azure Video Indexer disables the access face identification, customization and celebrities recognition as of today by default, but gives the option to enable it. <br/><br/>**Recommended**: Fill in the [intake form](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUMkZIOUE1R0YwMkU0M1NMUTA0QVNXVDlKNiQlQCN0PWcu) and based on the eligibility criteria we will enable the features (after max 10 days).|

+\*In Brazil South we also disabled the face detection.

+**Precondition** for the matching is that the person that showing in the observed faces was detected and can be found in the People insight.

+**Obstructions**: There is no match between faces and observed people where there are obstruction (people or faces overlapping each other).

+**Spatial allocation per frame**: There is no match where different people appear in the same spatial position relatively to the frame in a short time.

+### No NAT rule for specific address ranges

+3. Log in to **VMWare NSX-T** and then select **NAT Rules**.

+1. Select the T1 Router and then select **ADD NAT RULE**.

+1. The **Source IP** is **Any** and **Destination IP** is the Azure VMware Solution reserved Public IP.

+# Billing model for Azure Web PubSub service

+The billing model for Azure Web PubSub service is based on the number of units allocated and the message count of outbound traffic. This article explains how units and outbound traffic (message count) are defined and counted for billing.

+## Terms used in billing

+### Connection

+A *connection*, also known as a client or a client connection, represents an individual WebSocket connection connected to the Web PubSub service.

+### Unit

+A *unit* is an abstract concept of the capability of Web PubSub service. Each unit supports up to 1,000 concurrent connections. Each Web PubSub service instance can have 1, 2, 5, 10, 20, 50 or 100 units. The unit count * 1000 equals the maximum number of connections your Web PubSub service instance can accept.

+In production, it's recommended to plan for no more than 80% unit utilization before scaling up to more units to maintain acceptable system performance. For more information, see [Performance guide for Azure Web PubSub service](concept-performance.md).

+### Message count

+The *message count* is an abstract concept for billing purposes. It's defined as the size of outbound traffic (bytes) in 2-KB increments, with each increment counting as one message for billing. For example, 100 KB of traffic is counted as 50 messages.

+### Outbound traffic

+The *outbound traffic* is the messages sent out of Web PubSub service.

+- The resource logs with live trace tool.

+### Inbound traffic

+The *inbound traffic* is the messages sent to the Azure Web PubSub service.

+For more information, see [Metrics in Azure Web PubSub service](concept-metrics.md).

+## How units are counted for billing

+The units are counted based on the number of units and the usage time in seconds, and billed daily.

+For example, imagine you have one Web PubSub Enterprise tier instance with five units allocated. You've added a custom scale condition to scale up to 10 units from 10:00 AM to 16:00 PM and then scale back to five units after 16:00 PM. Total usage for the day is 5 units for 18 hours and 10 units for 6 hours.

+> Total units are used for billing = (5 units * 18 hours + 10 units * 6 hours) / 24 hours = 6.25 Unit/Day

+## How outbound traffic is counted for billing

+Only the outbound traffic is counted for billing.

+For example, imagine you have an application with Web PubSub service and Azure Functions. One user broadcast 4 KB of data to 10 connections in a group. Total data is 4 KB upstream from service to function, and 40 KB from the service broadcast to 10 connections * 4 KB each.

+> Outbound traffic for billing = 4 KB (upstream traffic to Azure Functions) + 4 KB * 10 (from service broadcasting to clients) = 44 KB

+> Equivalent message count = 44 KB / 2 KB = 22

+The Web PubSub service also offers a daily free quota of outbound traffic (message count) based on the usage of the units. The outbound traffic beyond the free quota is the outbound traffic not included in the base quota. Consider standard tier as example: the free quota is 2,000,000-KB outbound traffic (1,000,000 messages) per unit per day.

+For example, an application that uses 6.25 units per day has a daily free quota of 12,500,000-KB outbound traffic or 6.25 million messages. Assuming that the actual daily outbound traffic is 30,000,000 KB (15 million messages), the extra messages above the free quota is 17,500,000-KB outbound traffic, which counts as 8.75 million messages for billing.

+As a result, you'll be billed with 6.25 standard units and 8.75 additional message units for the day.

+## Pricing

+The Web PubSub service offers multiple tiers with different pricing. For more information about Web PubSub pricing, see [Azure Web PubSub service pricing](https://azure.microsoft.com/pricing/details/web-pubsub).

+ Title: Metrics in Azure Web PubSub service

+description: Metrics in Azure Web PubSub service.

+Azure Web PubSub service has some built-in metrics and you and sets up [alerts](../azure-monitor/alerts/alerts-overview.md) base on metrics.

+|Connection Count|Count|Max / Avg|The number of connections to the service.|No Dimensions|

+|Connection Quota Utilization|Percent|Max / Avg|The percentage of connections relative to connection quota.|No Dimensions|

+|Inbound Traffic|Bytes|Sum|The inbound traffic to the service.|No Dimensions|

+|Outbound Traffic|Bytes|Sum|The outbound traffic from the service.|No Dimensions|

+|Server Load|Percent|Max / Avg|The percentage of server load.|No Dimensions|

+ - ServiceTransientError: Internal server error.

+ - BadRequest: Caused by an invalid hub name, wrong payload, etc.

+ - ServiceReload: Triggered when a connection is dropped due to an internal service component reload. This event doesn't indicate a malfunction and is part of normal service operation.

+ - Unauthorized: The connection is unauthorized.

+- Clients can be written in any language that has Websocket support.

+- Both text and binary messages are supported within one connection.

+- There's a simple protocol for clients to do direct client-to-client message publishing.

+- The service manages the WebSocket connections for you.

+Workflow as shown in the above graph:

+1. A *client* connects to the service `/client` endpoint using WebSocket transport. Service forward every WebSocket frame to the configured upstream(server). The WebSocket connection can connect with any custom subprotocol for the server to handle, or it can connect with the service-supported subprotocol `json.webpubsub.azure.v1`, which empowers the clients to do pub/sub directly. Details are described in [client protocol](#client-protocol).

+1. The service invokes the server using **CloudEvents HTTP protocol** on different client events. [**CloudEvents**](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) is a standardized and protocol-agnostic definition of the structure and metadata description of events hosted by the Cloud Native Computing Foundation (CNCF). Details are described in [server protocol](#server-protocol).

+1. The Web PubSub server can invoke the service using the REST API to send messages to clients or to manage the connected clients. Details are described in [server protocol](#server-protocol)

+- One is called [the simple WebSocket client](#the-simple-websocket-client)

+- The other is called [the PubSub WebSocket client](#the-pubsub-websocket-client)

+For example, in JS, a simple WebSocket client can be created using the following code.

+1. When the client starts a WebSocket handshake, the service tries to invoke the `connect` event handler (the server) for WebSocket handshake. Developers can use this handler to handle the WebSocket handshake, determine the subprotocol to use, authenticate the client, and join the client to groups.

+1. When the client is successfully connected, the service invokes a `connected` event handler. It works as a notification and doesn't block the client from sending messages. Developers can use this handler to do data storage and can respond with messages to the client.

+1. When the client sends messages, the service triggers the `message` event to the event handler (the server) to handle the messages sent. This event is a general event containing the messages sent in a WebSocket frame. Your code needs to dispatch the messages inside this event handler.

+1. When the client disconnects, the service tries to trigger the `disconnected` event to the event handler (the server) once it detects the disconnect.

+* Synchronous events (blocking)

+* Asynchronous events (non-blocking)

+#### Scenarios

+These connections can be used in a typical client-server architecture where the client sends messages to the server and the server handles incoming messages using [Event Handlers](#event-handler). It can also be used when customers apply existing [subprotocols](https://www.iana.org/assignments/websocket/websocket.xml) in their application logic.

+The service also supports a specific subprotocol called `json.webpubsub.azure.v1`, which empowers the clients to do publish/subscribe directly instead of a round trip to the upstream server. We call the WebSocket connection with `json.webpubsub.azure.v1` subprotocol a PubSub WebSocket client. For more information, see the [Web PubSub client specification](https://github.com/Azure/azure-webpubsub/blob/main/protocols/client/client-spec.md) on GitHub.

+For example, in JS, a PubSub WebSocket client can be created using the following code.

+You may have noticed that for a [simple WebSocket client](#the-simple-websocket-client), the *server* is a **must have** role to handle the events from clients. A simple WebSocket connection always triggers a `message` event when it sends messages, and always relies on the server-side to process messages and do other operations. With the help of the `json.webpubsub.azure.v1` subprotocol, an authorized client can join a group and publish messages to a group directly. It can also route messages to different upstream (event handlers) by customizing the *event* the message belongs.

+### Client authentication

+#### Authentication workflow

+Client uses a signed JWT token to connect to the service. The upstream can also reject the client when it's `connect` event handler of the incoming client. The event handler authenticates the client by specifying the `userId` and the `role`s the client has in the webhook response, or decline the client with 401. [Event handler](#event-handler) section describes it in detail.

+The following graph describes the workflow.

+As you may have noticed when we describe the PubSub WebSocket clients, that a client can publish to other clients only when it's *authorized* to. The `role`s of the client determines the *initial* permissions the client have:

+The server-side can also grant or revoke permissions of the client dynamically through [server protocol](#connection-manager) as to be illustrated in a later section.

+## Server protocol

+1. [Event handler](#event-handler)

+1. [Connection manager](#connection-manager)

+The event handler handles the incoming client events. Event handlers are registered and configured in the service through the portal or Azure CLI. When a client event is triggered, the service can identify if the event is to be handled or not. Now we use `PUSH` mode to invoke the event handler. The event handler on the server side exposes a publicly accessible endpoint for the service to invoke when the event is triggered. It acts as a **webhook**.

+Web PubSub service delivers client events to the upstream webhook using the [CloudEvents HTTP protocol](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md).

+For every event, the service formulates an HTTP POST request to the registered upstream and expects an HTTP response.

+The data sent from the service to the server is always in CloudEvents `binary` format.

+Event handlers need to be registered and configured in the service through the portal or Azure CLI before first use. When a client event is triggered, the service can identify if the event must be handled or not. For public preview, we use `PUSH` mode to invoke the event handler. The event handler on the server side exposes publicly accessible endpoint for the service to invoke when the event is triggered. It acts as a **webhook** **upstream**.

+The URL can use `{event}` parameter to define a URL template for the webhook handler. The service calculates the value of the webhook URL dynamically when the client request comes in. For example, when a request `/client/hubs/chat` comes in, with a configured event handler URL pattern `http://host.com/api/{event}` for hub `chat`, when the client connects, it will first POST to this URL: `http://host.com/api/connect`. This behavior can be useful when a PubSub WebSocket client sends custom events, that the event handler helps dispatch different events to different upstream. The `{event}` parameter isn't allowed in the URL domain name.

+When doing the validation, the `{event}` parameter is resolved to `validate`. For example, when trying to set the URL to `http://host.com/api/{event}`, the service tries to **OPTIONS** a request to `http://host.com/api/validate` and only when the response is valid the configuration can be set successfully.

+For now, we don't support [WebHook-Request-Rate](https://github.com/cloudevents/spec/blob/v1.0/http-webhook.md#414-webhook-request-rate) and [WebHook-Request-Callback](https://github.com/cloudevents/spec/blob/v1.0/http-webhook.md#413-webhook-request-callback).

+- Simple authentication that `code` is provided through the configured Webhook URL.

+- Use Azure Active Directory (Azure AD) authentication. For more information, see [how to use managed identity](howto-use-managed-identity.md) for details.

+ - Step2: Select from existing Azure AD application that stands for your webhook web app

+ - Add clients authenticated as the same user to a group

+ - Remove clients authenticated as the same user from a group

+ - Grant publish/join permissions to some specific group or to all groups

+ - Revoke publish/joinh permissions for some specific group or for all groups

+ - Check if the client has permission to join or publish to some specific group or to all groups

+The service provides REST APIs for the server to do connection management.

+You may have noticed that the *event handler role* handles communication from the service to the server while *the manager role* handles communication from the server to the service. After combining the two roles, the data flow between service and server looks similar to the following diagram using HTTP protocol.

+If you're using an [event handler](concept-service-internals.md#event-handler) in Azure Web PubSub Service, you might have outbound traffic to an upstream. Upstream such as

+Private endpoints of secured resources that are created through Azure Web PubSub Service APIs are referred to as *shared private link resources*. This term is used because you're "sharing" access to a resource, such as an Azure Function that has been integrated with the [Azure Private Link service](https://azure.microsoft.com/services/private-link/). These private endpoints are created inside Azure Web PubSub Service execution environment and aren't directly visible to you.

+1. Select **Add shared private endpoint**.

+1. Select **Add**.

+The process of creating an outbound private endpoint is a long-running (asynchronous) operation. As in all asynchronous Azure operations, the `PUT` call returns an `Azure-AsyncOperation` header value that looks like the following example.

+1. In the Azure portal, select the **Networking** tab of your Function App and navigate to **Private endpoint connections**. Select **Configure your private endpoint connections**. After the asynchronous operation has succeeded, there should be a request for a private endpoint connection with the request message from the previous API call.

+This command would return JSON, where the connection state would show up as "status" under the "properties" section.

+If the "Provisioning State" (`properties.provisioningState`) of the resource is `Succeeded` and "Connection State" (`properties.status`) is `Approved`, it means that the shared private link resource is functional, and Azure Web PubSub Service can communicate over the private endpoint.

+description: Use service tags to allow outbound traffic to your Azure Web PubSub service.

+You can use [Service Tags](../virtual-network/network-security-groups-overview.md#service-tags) for Azure Web PubSub service when configuring [Network Security Group](../virtual-network/network-security-groups-overview.md#network-security-groups). It allows you to define inbound/outbound network security rule for Azure Web PubSub Service endpoints without need to hardcode IP addresses.

+Azure Web PubSub service manages these service tags. You can't create your own service tag or modify an existing tag. Microsoft manages the address prefixes that match to the service tag and automatically updates the service tag as addresses change.

+> Starting from 15 August 2021, Azure Web PubSub service supports bidirectional Service Tag for both inbound and outbound traffic.

+You can allow outbound traffic to Azure Web PubSub service by adding a new outbound network security rule:

+If you're using [event handler](concept-service-internals.md#event-handler), you can also allow inbound traffic from Azure Web PubSub service by adding a new inbound network security rule:

+description: Understand the basic concepts and terms used in Azure Web PubSub.

+Azure Web PubSub service helps you build real-time messaging web applications. The clients connect to the service using the [standard WebSocket protocol](https://datatracker.ietf.org/doc/html/rfc6455), and the service exposes [REST APIs](/rest/api/webpubsub) and SDKs for you to manage these clients.

+1. A *client* connects to the service `/client` endpoint using WebSocket transport. Service forward every WebSocket frame to the configured upstream(server). The WebSocket connection can connect with any custom subprotocol for the server to handle, or it can connect with the service-supported subprotocol `json.webpubsub.azure.v1`, which empowers the clients to do pub/sub directly. Details are described in [client protocol](concept-service-internals.md#client-protocol).

+2. The service invokes the server using **CloudEvents HTTP protocol** on different client events. [**CloudEvents**](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) is a standardized and protocol-agnostic definition of the structure and metadata description of events hosted by the Cloud Native Computing Foundation (CNCF). Details are described in [server protocol](concept-service-internals.md#server-protocol).

+3. Server can invoke the service using REST API to send messages to clients or to manage the connected clients. Details are described in [server protocol](concept-service-internals.md#server-protocol)

+ Title: Reference - Azure Web PubSub Client Specification

+description: This article provides a summary of the Web PubSub client specification.

+# Azure Web PubSub client specification

+## Summary

+The client specification outlines the complete feature set available to an Azure Web PubSub client. It's highly recommended that client library developers familiarize themselves with the client specification. The specification defines the behaviors and features a client library **must** implement to work properly with Web PubSub service.

+For more information, see the [entire client specification](https://github.com/Azure/azure-webpubsub/blob/main/protocols/client/client-spec.md), located on GitHub.

+## Next steps

+> [!div class="nextstepaction"]

+> [Learn about Web PubSub service internals](concept-service-internals.md)

+> [!div class="nextstepaction"]

+> [Explore the REST API](reference-rest-api-data-plane.md)

+> [!div class="nextstepaction"]

+> [Explore Azure Web PubSub code samples](https://aka.ms/awps/samples)

+## <a name="host"></a>Bastion FAQs

+## <a name="vm"></a>VM features and connection FAQs

+## <a name="peering"></a>VNet peering FAQs

+## Next steps

+For more information, see [What is Azure Bastion](bastion-overview.md).

+ Title: 'About Azure Bastion'

+* [Quickstart: Deploy Bastion using default settings](quickstart-host-portal.md).

+* [Tutorial: Deploy Bastion using specified settings](tutorial-create-host-portal.md).

+The text moderation response can optionally return the text with basic auto-correction applied. It can fix some spelling errors, for example.

+1. [Create a project](how-to-custom-speech-create-project.md) and choose a model. Use a <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesSpeechServices" title="Create a Speech resource" target="_blank">Speech resource</a> that you create in the Azure portal. If you will train a custom model with audio data, choose a Speech resource region with dedicated hardware for training audio data. See footnotes in the [regions](regions.md#speech-service) table for more information.

+ > If you plan to use neural voices, make sure that you create your resource in [a region that supports neural voices](regions.md#speech-service).

+ > [!IMPORTANT]

+ > If you will train a custom model with audio data, choose a Speech resource region with dedicated hardware for training audio data. See footnotes in the [regions](regions.md#speech-service) table for more information.

+If you will train a custom model with audio data, choose a Speech resource region with dedicated hardware for training audio data. See footnotes in the [regions](regions.md#speech-service) table for more information. In regions with dedicated hardware for Custom Speech training, the Speech service will use up to 20 hours of your audio training data, and can process about 10 hours of data per day. In other regions, the Speech service uses up to 8 hours of your audio data, and can process about 1 hour of data per day. After the model is trained, you can copy the model to another region as needed with the [CopyModelToSubscription](https://eastus.dev.cognitive.microsoft.com/docs/services/speech-to-text-api-v3-0/operations/CopyModelToSubscription) REST API.

+> If a base model doesn't support customization with audio data, only the transcription text will be used for training. If you switch to a base model that supports customization with audio data, the training time may increase from several hours to several days. The change in training time would be most noticeable when you switch to a base model in a [region](regions.md#speech-service) without dedicated hardware for training. If the audio data is not required, you should remove it to decrease the training time.

+A large training dataset is required to improve recognition. Generally, we recommend that you provide word-by-word transcriptions for 1 to 20 hours of audio. However, even as little as 30 minutes can help improve recognition results. Although creating human-labeled transcription can take time, improvements in recognition will only be as good as the data that you provide. You should upload only high-quality transcripts.

+|`//`|Comments follow a double slash (`//`).|Not applicable|

+Even if a base model supports training with audio data, the service might use only part of the audio. In [regions](regions.md#speech-service) with dedicated hardware available for training audio data, the Speech service will use up to 20 hours of your audio training data. In other regions, the Speech service uses up to 8 hours of your audio data.

+> [!NOTE]

+> You pay to use Custom Speech models, but you are not charged for training a model.

+> [!IMPORTANT]

+> If you will train a custom model with audio data, choose a Speech resource region with dedicated hardware for training audio data. After a model is trained, you can [copy it to a Speech resource](#copy-a-model) in another region as needed.

+>

+> In regions with dedicated hardware for Custom Speech training, the Speech service will use up to 20 hours of your audio training data, and can process about 10 hours of data per day. In other regions, the Speech service uses up to 8 hours of your audio data, and can process about 1 hour of data per day. See footnotes in the [regions](regions.md#speech-service) table for more information.

+You can copy a model to another project that uses the same locale. For example, after a model is trained with audio data in a [region](regions.md#speech-service) with dedicated hardware for training, you can copy it to a Speech resource in another region as needed.

+> Custom Neural Voice training is only available in some regions. But you can easily copy a neural voice model from these regions to other regions. For more information, see the [regions for Custom Neural Voice](regions.md#speech-service).

+[Custom Neural Voice](https://aka.ms/customvoice) is a set of online tools that you use to create a recognizable, one-of-a-kind voice for your brand. All it takes to get started are a handful of audio files and the associated transcriptions. See if Custom Neural Voice supports your [language](language-support.md#custom-neural-voice) and [region](regions.md#speech-service).

+> Custom neural voice training is only available in the these regions: East US, Southeast Asia, and UK South. But you can copy a neural voice model from those regions to other regions. For more information, see the [regions for custom neural voice](regions.md#speech-service).

+> For public voices, neural types are available only for specific regions. For more information, see [Speech service supported regions](./regions.md#speech-service).

+> For information about availability of pronunciation assessment, see [supported languages](language-support.md#pronunciation-assessment) and [available regions](regions.md#speech-service).

+> You can view a list of regions that support the **Advanced** model type in the [keyword recognition region support](regions.md#speech-service) documentation.

+> Voices and styles in public preview are only available in three service [regions](regions.md): East US, West Europe, and Southeast Asia.

+> For pronunciation assessment, `en-US` and `en-GB` are available inΓÇ»[all regions](regions.md#speech-service), `zh-CN` is available in East Asia and Southeast Asia regions, `de-DE`, `es-ES`, and `fr-FR` are available in West Europe region, and `en-AU` is available in Australia East region.

+* If your application uses a [Speech SDK](speech-sdk.md), you provide the region identifier, such as `westus`, when you create a `SpeechConfig`. Make sure the region matches the region of your subscription.

+## Speech service

+The following regions are supported for Speech service features such as speech-to-text, text-to-speech, pronunciation assessment, and translation. The geographies are listed in alphabetical order.

+| Geography | Region | Region identifier |

+| -- | -- | -- |

+| Africa | South Africa North | `southafricanorth` ⁶|

+| Asia Pacific | East Asia | `eastasia` ⁵|

+| Asia Pacific | Southeast Asia | `southeastasia` ^1,2,3,4,5|

+| Asia Pacific | Australia East | `australiaeast` ^1,2,3,4|

+| Asia Pacific | Central India | `centralindia` ^1,2,3,4,5|

+| Asia Pacific | Japan East | `japaneast` ^2,5|

+| Asia Pacific | Japan West | `japanwest` |

+| Asia Pacific | Korea Central | `koreacentral` ²|

+| Canada | Canada Central | `canadacentral` ¹|

+| Europe | North Europe | `northeurope` ^1,2,4,5|

+| Europe | West Europe | `westeurope` ^1,2,3,4,5|

+| Europe | France Central | `francecentral` |

+| Europe | Germany West Central | `germanywestcentral` |

+| Europe | Norway East | `norwayeast` |

+| Europe | Switzerland North | `switzerlandnorth` ⁶|

+| Europe | Switzerland West | `switzerlandwest` |

+| Europe | UK South | `uksouth` ^1,2,3,4|

+| Middle East | UAE North | `uaenorth` ⁶|

+| South America | Brazil South | `brazilsouth` ⁶|

+| US | Central US | `centralus` |

+| US | East US | `eastus` ^1,2,3,4,5|

+| US | East US 2 | `eastus2` ^1,2,4,5|

+| US | North Central US | `northcentralus` ^1,4,6|

+| US | South Central US | `southcentralus` ^1,2,3,4,5,6|

+| US | West Central US | `westcentralus` ⁵|

+| US | West US | `westus` ^2,5|

+| US | West US 2 | `westus2` ^1,2,4,5|

+| US | West US 3 | `westus3` |

+¹ The region has dedicated hardware for Custom Speech training. If you plan to train a custom model with audio data, use one of the regions with dedicated hardware for faster training. Then you can [copy the trained model](how-to-custom-speech-train-model.md#copy-a-model) to another region.

+² The region is available for Custom Neural Voice training. You can copy a trained neural voice model to other regions for deployment.

+³ The Long Audio API is available in the region.

+⁴ The region supports custom keyword advanced models.

+⁵ The region supports keyword verification.

+⁶ The region does not support Speaker Recognition.

+## Intent recognition

+## Voice assistants

+Replace `<REGION_IDENTIFIER>` with the identifier that matches the [region](regions.md) of your subscription.

+- For information about regional availability, see [Speech service supported regions](regions.md#speech-service).

+| What Azure regions are supported? | See [Speaker recognition region support](regions.md#speech-service).|

+You'll need to set Swagger to the region of your Speech resource. You can confirm the region in the **Overview** part of your Speech resource settings in Azure portal. The complete list of supported regions is available [here](regions.md#speech-service).

+1. In a browser, go to the Swagger specification for your [region](regions.md#speech-service):

+Ensure that you use a [supported Azure region](regions.md#voice-assistants). The Direct Line Speech channel uses the text-to-speech service, which has neural and standard voices. Neural voices are used at [these Azure regions](regions.md#speech-service), and standard voices (retiring) are used at [these Azure regions](how-to-migrate-to-prebuilt-neural-voice.md).

+|Error (ConnectionFailure) : Connection was closed by the remote host. Error code: 1011. Error details: Response status code does not indicate success: 500 (InternalServerError)| Your bot specified a neural voice in the [speak](https://github.com/microsoft/botframework-sdk/blob/master/specs/botframework-activity/botframework-activity.md#speak) field of its output activity, but the Azure region associated with your subscription key doesn't support neural voices. See [neural voices](./regions.md#speech-service) and [standard voices](how-to-migrate-to-prebuilt-neural-voice.md).|

+* [Deploy to an Azure region that supports high-quality neural text-to-speech voices](./regions.md#speech-service).

+ * [C#](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample8_RecognizeCustomEntities.md)

+ * [JavaScript](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js)

+|C# (Runtime) | [C# documentation](/dotnet/api/azure.ai.textanalytics?view=azure-dotnet-preview&preserve-view=true) | [C# samples](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample8_RecognizeCustomEntities.md) |

+|JavaScript (Runtime) | [JavaScript documentation](/javascript/api/overview/azure/ai-text-analytics-readme?view=azure-node-preview&preserve-view=true) | [JavaScript samples](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js) |

+ * [C#](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample9_SingleLabelClassify.md)

+ * [JavaScript](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js)

+ * [Python](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/textanalytics/azure-ai-textanalytics/samples/sample_single_label_classify.py)

+ * [C#](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample10_MultiLabelClassify.md)

+ * [JavaScript](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js)

+ * [Python](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/textanalytics/azure-ai-textanalytics/samples/sample_multi_label_classify.py)

+|C# (Runtime) | [C# documentation](/dotnet/api/azure.ai.textanalytics?view=azure-dotnet-preview&preserve-view=true) | [C# samples - Single label classification](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample9_SingleLabelClassify.md) [C# samples - Multi label classification](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample10_MultiLabelClassify.md) |

+|JavaScript (Runtime) | [JavaScript documentation](/javascript/api/overview/azure/ai-text-analytics-readme?view=azure-node-preview&preserve-view=true) | [JavaScript samples - Single label classification](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js) [JavaScript samples - Multi label classification](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/ai-text-analytics_6.0.0-beta.1/sdk/textanalytics/ai-text-analytics/samples/v5/javascript/customText.js) |

+|Python (Runtime)| [Python documentation](/python/api/azure-ai-textanalytics/azure.ai.textanalytics?view=azure-python-preview&preserve-view=true) | [Python samples - Single label classification](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/textanalytics/azure-ai-textanalytics/samples/sample_single_label_classify.py) [Python samples - Multi label classification](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/textanalytics/azure-ai-textanalytics/samples/sample_multi_label_classify.py) |

+| Email | Access Key or Azure AD authentication |

+* **Data encryption at rest** - Your analytical store encryption is enabled by default.

+### readWrite

+## PO number management

+PO number management functionality in the EA portal is getting deprecated. It is currently read-only in the EA portal. Instead, an EA administrator can use the Azure portal to manage PO numbers. For more information, see [Update a PO number](direct-ea-azure-usage-charges-invoices.md#update-a-po-number-for-an-upcoming-overage-invoice).

+- **System-assigned:** You can enable a managed identity directly on a service instance. When you allow a system-assigned managed identity during the creation of the service, an identity is created in Azure AD tied to that service instance's lifecycle. By design, only that Azure resource can use this identity to request tokens from Azure AD. So when the resource is deleted, Azure automatically deletes the identity for you.

+## July 2022

+<br>

+<table>

+<tr><td><b>Service category</b></td><td><b>Service improvements</b></td><td><b>Details</b></td></tr>

+<tr><td rowspan=5><b>Data flow</b></td><td>Asana connector added as source</td><td>WeΓÇÖve added a new REST-based connector to mapping data flows! Users can now read their tables from Asana. Note: This connector is only available when using inline datasets.<br><a href="connector-asana.md">Learn more</a></td></tr>

+<tr><td>3 new data transformation functions are supported</td><td>3 new data transformation functions have been added to mapping data flows in Azure Data Factory and Azure Synapse Analytics. Now, users are able to use collectUnique(), to create a new collection of unique values in an array, substringIndex(), to extract the substring before n occurrences of a delimiter, and topN(), to return the top n results after sorting your data.<br><a href=https://techcommunity.microsoft.com/t5/azure-data-factory-blog/3-new-data-transformation-functions-in-adf/ba-p/3582738>Learn more</a></td></tr>

+<tr><td>Refetch from source available in Refresh for data source change scenarios</td><td>When building and debugging a data flow, your source data can change. There is now a new easy way to refetch the latest updated source data from the refresh button in the data preview pane.<br><a href="concepts-data-flow-debug-mode.md#data-preview">Learn more</a></td></tr>

+<tr><td>User defined functions (GA) </td><td>Create reusable and customized expressions to avoid building complex logic over and over<br><a href="concepts-data-flow-debug-mode.md#data-preview">Learn more</a></td></tr>

+<tr><td>Easier configuration on data flow runtime ΓÇô choose compute size among Small, Medium, and Large to pre-configure all integration runtime settings</td><td>Azure Data Factory has made it easier for users to configure Azure Integration Runtime for mapping data flows by choosing compute size among Small, Medium, and Large to pre-configure all integration runtime settings. You can still set your own custom configurations.<br><a href=https://techcommunity.microsoft.com/t5/azure-data-factory-blog/adf-makes-it-easy-to-select-azure-ir-size-for-data-flows/ba-p/3578033>Learn more</a></td></tr>

+<tr><td rowspan=1><b>Continuous integration and continuous delivery (CI/CD)</b></td><td>Include Global parameters supported in ARM template</td><td>WeΓÇÖve added a new mechanism to include Global Parameters in the ARM templates. This helps to solve an earlier issue, which overrode some configurations during deployments when users included global parameters in ARM templates.<br><a href=https://techcommunity.microsoft.com/t5/azure-data-factory-blog/ci-cd-improvement-using-global-parameters-in-azure-data-factory/ba-p/3557265#M665>Learn more</a></td></tr>

+<tr><td><b>Developer productivity</b></td><td>Azure Data Factory studio preview experience</td><td>Be a part of Azure Data Factory Preview features to experience latest Azure Data Factory capabilities, and be the first to share your feedback<br><a href=https://techcommunity.microsoft.com/t5/azure-data-factory-blog/introducing-the-azure-data-factory-studio-preview-experience/ba-p/3563880>Learn more</a></td></tr>

+</table>

+* After that, you can [integrate Power BI with Azure Synapse Analytics](/power-bi/connect-data/service-azure-sql-data-warehouse-with-direct-connect) to create visual representation of the data.

+[pvk2pfx]: /windows-hardware/drivers/devtest/pvk2pfx

+For an additional example to run, see [this one on GitHub](https://github.com/hdinsight/storm-performance-automation).

+This article applies to the **Azure Stack Edge 2207** release, which maps to software version number **2.2.2037.5375**. This software can be applied to your device if you're running at least Azure Stack Edge 2106 (2.2.1636.3457) software.

+- Device software version - **2.2.2037.5375**

+- Device Kubernetes version - **2.2.2037.5375**

+ 

+ 

+ Title: Exclude storage accounts from Microsoft Defender for Storage

+description: Learn how to exclude specific Azure Storage accounts from Microsoft Defender for Storage protections.

+# Exclude a storage account from Microsoft Defender for Storage protections

+When you [enable Microsoft Defender for Storage](../storage/common/azure-defender-storage-configure.md#set-up-microsoft-defender-for-cloud) on a subscription, all current and future Azure Storage accounts in that subscription are protected. If you have specific accounts that you want to exclude from the Defender for Storage protections, you can exclude them using the Azure portal, PowerShell, or the Azure CLI.

+We don't recommend that you exclude storage accounts from Defender for Storage because attackers can use any opening in order to compromise your environment. If you want to optimize your Azure costs and remove storage accounts that you feel are low risk from Defender for Storage, you can use the [Price Estimation Workbook](https://ms.portal.azure.com/#view/Microsoft_Azure_Security/SecurityMenuBlade/~/28) in the Azure portal to evaluate the cost savings.

+## Exclude an Azure Storage account

+To exclude an Azure Storage account from Microsoft Defender for Storage:

+**To exclude an active Databricks workspace**:

+> [!Note]

+1. Follow [these steps](/azure/databricks/scenarios/quickstart-create-Databricks-workspace-portal?tabs=azure-portal) to create a new Azure Databricks workspace.

+1. In the Tags tab, enter a tag named `AzDefenderPlanAutoEnable`.

+1. Enter the value `off`.

+1. Use the following command to install the package into the local Maven store. This enables you to add it as a reference in the Storm project in a later step.

+1. In Eclipse, create a new Maven project (click **File**, then **New**, then **Project**).

+1. Select **Use default Workspace location**, then click **Next**

+1. Select the **maven-archetype-quickstart** archetype, then click **Next**

+1. Insert a **GroupId** and **ArtifactId**, then click **Finish**

+1. In **pom.xml**, add the following dependencies in the `<dependency>` node.

+1. In the **src** folder, create a file called **Config.properties** and copy the following content, substituting the `receive rule key` and `event hub name` values:

+1 . Create a new class called **LoggerBolt** with the following code:

+|Audit Process Creation<br />_{(CCE-36059-4)} |**Description**: This subcategory reports the creation of a process and the name of the program or user that created it. Events for this subcategory include: - 4688: A new process has been created. - 4696: A primary token was assigned to process. Refer to Microsoft Knowledge Base article 947226: Description of security events in Windows Vista and in Windows Server 2008 for the most recent information about this setting. The recommended state for this setting is: `Success`.<br />**Key Path**: {0CCE922B-69AE-11D9-BED3-505054503030}<br />**OS**: WS2008, WS2008R2, WS2012, WS2012R2, WS2016, WS2019, WS2022<br />**Server Type**: Domain Controller, Domain Member, Workgroup Member |\>\= Success<br />_(Audit) |Critical |

+The cluster type determines the workload your HDInsight cluster is configured to run. Types include [Apache Hadoop](./hadoop/apache-hadoop-introduction.md), [Apache Kafka](./kafk#cluster-types-in-hdinsight). Each cluster type has a specific deployment topology that includes requirements for the size and number of nodes.

+ |`--type`| Type of HDInsight cluster, like: hadoop, interactivehive, hbase, kafka, spark, rserver, mlservices. This article uses the variable `clusterType` as the value passed to `--type`. See also: [Cluster types and configuration](./hdinsight-hadoop-provision-linux-clusters.md#cluster-type).|

+ > HDInsight clusters come in various types, which correspond to the workload or technology that the cluster is tuned for. There is no supported method to create a cluster that combines multiple types, such as HBase on one cluster.

+description: Learn how to create Apache Hadoop, Apache HBase, or Apache Spark clusters on Linux for HDInsight by using Azure PowerShell.

+* [Apache Spark with Machine Learning: Use Spark in HDInsight to predict food inspection results](spark/apache-spark-machine-learning-mllib-ipython.md)

+* [Tips for HDInsight clusters on Linux](hdinsight-hadoop-linux-information.md)

+In a streaming application, one or more data sources are generating events (sometimes in the millions per second) that need to be ingested quickly without dropping any useful information. The incoming events are handled with *stream buffering*, also called *event queuing*, by a service such as [Apache Kafka](kafk) or [Event Hubs](https://azure.microsoft.com/services/event-hubs/). After you collect the events, you can then analyze the data using a real-time analytics system within the *stream processing* layer. The processed data can be stored in long-term storage systems, like [Azure Data Lake Storage](https://azure.microsoft.com/services/storage/data-lake-storage/), and displayed in real time on a business intelligence dashboard, such as [Power BI](https://powerbi.microsoft.com), Tableau, or a custom web page.

+Spark Streaming is an extension to Spark, which allows you to reuse the same code that you use for batch processing. You can combine both batch and interactive queries in the same application. Unlike Spark Streaming provides stateful exactly once processing semantics. When used in combination with the [Kafka Direct API](https://spark.apache.org/docs/latest/streaming-kafka-integration.html), which ensures that all Kafka data is received by Spark Streaming exactly once, it's possible to achieve end-to-end exactly once guarantees. One of Spark Streaming's strengths is its fault-tolerant capabilities, recovering faulted nodes rapidly when multiple nodes are being used within the cluster.

+Apache Spark Streaming support adding worker nodes to their clusters, even while data is being processed.

+* [Start with Apache Kafka on HDInsight](kafk)

+description: Troubleshoot Azure HDInsight. Step-by-step documentation shows you how to use HDInsight to solve common problems with Apache Hive, Apache Spark, Apache YARN, Apache HBase, and HDFS.

+This article will show you how to get started with the Azure MedTech service in the [Azure Health Data Services](../healthcare-apis-overview.md). There are six steps you need to follow to be able to deploy and process MedTech service to ingest health data from a medical device using Azure Event Hubs service, persist the data to Azure Fast Healthcare Interoperability Resources (FHIR&#174;) service as Observation resources, and link FHIR service Observations to patient and device resources. This article provides an architecture overview to help you follow the six steps of the implementation process.

+## Architecture overview of MedTech service

+The following diagram outlines the basic architectural path that enables the MedTech service to receive data from a medical device and send it to the FHIR service. This diagram shows how the six-step implementation process is divided into three key development stages: deployment, post-deployment, and data processing.

+[](media/iot-get-started/get-started-with-iot.png#lightbox)

+### Deployment

+- Step 1 introduces the subscription and permissions prerequisites required.

+- Step 2 shows how Azure services are provisioned for MedTech services.

+- Step 3 presents the configuration process.

+### Post-deployment

+- Step 4 outlines how to connect to other services.

+### Data processing

+- Step 5 represents the data flow from a device to an event hub and is processed through the five parts of the MedTech service.

+- Step 6 demonstrates the path to verify processed data sent from MedTech service to the FHIR service.

+## Get started implementing MedTech service

+Follow these six steps to set up and start using MedTech service.

+## Step 1: Prerequisites for deployment

+In order to begin deployment, you need to determine if you have: an Azure subscription and correct Azure RBAC (Role-Based Access Control) role assignments. If you already have the appropriate subscription and roles, you can skip this step.

+## Step 2: Provision services for deployment

+After obtaining the required prerequisites, the next phase of deployment is to create a workspace and provision instances of the Event Hubs service, FHIR service, and MedTech service. You must also give the Event Hubs permission to read data from your device and give the MedTech service permission to read and write to the FHIR service. There are four parts of this provisioning process.

+By design, the MedTech service retrieves data from the specified event hub using the system-assigned managed identity. For more information on how to assign the role to the MedTech service from [Event Hubs](deploy-iot-connector-in-azure.md#granting-access-to-the-device-message-event-hub).

+## Step 3: Configure MedTech for deployment

+After you have fulfilled the prerequisites and provisioned your services, the next phase of deployment is to configure MedTech services to ingest data, set up device mappings, and set up destination mappings. These configuration settings will ensure that the data can be translated from your device to Observations in the FHIR service. There are four parts in this configuration process.

+### Configuring MedTech service to ingest data

+MedTech service must be configured to ingest data it will receive from an event hub. First you must begin the official deployment process at the Azure portal. For more information about configuring MedTech service using the Azure portal, see [Deployment using the Azure portal](deploy-iot-connector-in-azure.md#prerequisites).

+Once you have starting using the portal and added MedTech service to your workspace, you must then configure MedTech service to ingest data from an event hub. For more information about configuring MedTech service to ingest data, see [Configure the MedTech service to ingest data](deploy-iot-connector-in-azure.md#configure-the-medtech-service-to-ingest-data).

+### Configuring device mappings

+You must configure MedTech to map it to the device you want to receive data from. Each device has unique settings that MedTech service must use. For more information on how to use Device mappings, see [How to use Device mappings](./how-to-use-device-mappings.md).

+- Azure Health Data Services provides an open source tool you can use called [IoMT Connector Data Mapper](https://github.com/microsoft/iomt-fhir/tree/main/tools/data-mapper) that will help you map your device's data structure to a form that MedTech can use. For more information on device content mapping, see [Device Content Mapping](https://github.com/microsoft/iomt-fhir/blob/main/docs/Configuration.md#device-content-mapping).

+- When you are deploying MedTech service, you must set specific device mapping properties. For more information on device mapping properties, see [Configure the Device mapping properties](deploy-iot-connector-in-azure.md#configure-the-device-mapping-properties).

+### Configuring destination mappings

+Once your device's data is properly mapped to your device's data format, you must then map it to an Observation in the FHIR service. For an overview of FHIR destination mappings, see [How to use the FHIR destination mappings](how-to-use-fhir-mappings.md).

+For step-by-step destination property mapping, see [Configure destination properties](deploy-iot-connector-in-azure.md#configure-destination-properties

+).

+### Create and deploy the MedTech service

+If you have completed the prerequisites, provisioning, and configuration, you are now ready to deploy the MedTech service. Create and deploy your MedTech service by following deployment the procedure at [Create your MedTech service](deploy-iot-connector-in-azure.md#create-your-medtech-service).

+## Step 4: Connect to required services (post deployment)

+When you complete the final [deployment procedure](deploy-iot-connector-in-azure.md#create-your-medtech-service) and don't get any errors, you must link MedTech service to an Event Hubs and the FHIR service. This will enable a connection from MedTech service to an Event Hubs instance and the FHIR service, so that data can flow smoothly from device to FHIR Observation. In order to do this, the Event Hubs instance for device message flow must be granted access via role assignment, so MedTech service can receive Event Hubs data. You must also grant access to The FHIR service via role assignments in order for MedTech to receive the data. There are two parts of the process to connect to required services.

+For more information about granting access via role assignments, see [Granting the MedTech service access to the device message event hub and FHIR service](deploy-iot-connector-in-azure.md#granting-the-medtech-service-access-to-the-device-message-event-hub-and-fhir-service).

+### Granting access to the device message event hub

+The Event Hubs instance for device message event hub must be granted access using managed identity in order for the MedTech service to receive data sent to the event hub from a device. The step-by-step procedure for doing this is at [Granting access to the device message event hub](deploy-iot-connector-in-azure.md#granting-access-to-the-device-message-event-hub).

+For more information about authorizing access to Event Hubs resources, see [Authorize access with Azure Active Directory](../../event-hubs/authorize-access-azure-active-directory.md).

+For more information about application roles, see [Authentication and Authorization for Azure Health Data Services](../authentication-authorization.md).

+### Granting access to FHIR service

+You must also grant access via role assignments to the FHIR service. This will enable FHIR service to receive data from the MedTech service by granting access using managed identity. The step-by-step procedure for doing this is at [Granting access to the FHIR service](deploy-iot-connector-in-azure.md#granting-access-to-the-fhir-service).

+For more information about assigning roles to the FHIR services, see [Configure Azure RBAC role for Azure Health Data Services](../configure-azure-rbac.md).

+For more information about application roles, see [Authentication and Authorization for Azure Health Data Services](../authentication-authorization.md).

+## Step 5: Send the data for processing

+When MedTech service is deployed and connected to the Event Hubs and FHIR services, it is ready to process data from a device and translate it into a FHIR service Observation. There are three parts of the sending process.

+### Data sent from Device to Event Hubs

+The data is sent to an Event Hub instance so that it can wait until MedTech service is ready to receive it. The data transfer needs to be asynchronous because it is sent over the Internet and delivery times cannot be precisely measured. Normally the data won't sit on an event hub longer than 24 hours.

+For more information about Event Hubs, see [Event Hubs](../../event-hubs/event-hubs-about.md).

+For more information on Event Hubs data retention, see [Event Hubs quotas](../../event-hubs/event-hubs-quotas.md)

+### Data Sent from Event Hubs to MedTech

+MedTech requests the data from the Event Hubs instance and the data is sent from the event hub to MedTech. This procedure is called ingestion.

+### MedTech processes the data

+MedTech processes the data in five steps:

+- Ingest

+- Normalize

+- Group

+- Transform

+- Persist

+If the processing was successful and you did not get any error messages, your device data is now a FHIR service [Observation](http://hl7.org/fhir/observation.html) resource.

+For more details on the data flow through MedTech, see [MedTech service data flow](iot-data-flow.md).

+## Step 6: Verify the processed data

+You can verify that the data was processed correctly by checking to see if there is now a new Observation resource in the FHIR service. If the data isn't mapped or if the mapping isn't authored properly, the data will be skipped. If there are any problems, check the [device mapping](how-to-use-device-mappings.md) or the [FHIR destination mapping](how-to-use-fhir-mappings.md).

+ Title: How to use custom functions in the MedTech service - Azure Health Data Services

+description: This article describes how to use custom functions with MedTech service device mapping.

+# How to use custom functions

+Many functions are available when using **JmesPath** as the expression language. Besides the functions available as part of the JmesPath specification, many more custom functions may also be used. This article describes MedTech service-specific custom functions for use with the MedTech service device mapping during the device message normalization process.

+> [!NOTE]

+>

+>[!TIP]

+>

+> Check out the [IoMT Connector Data Mapper](https://github.com/microsoft/iomt-fhir/tree/master/tools/data-mapper) tool for editing, testing, and troubleshooting the MedTech service Device and FHIR destination mappings. Export mappings for uploading to the MedTech service in the Azure portal or use with the [open-source version](https://github.com/microsoft/iomt-fhir) of the MedTech service.

+In this article, you learned how to use the MedTech service custom functions. To learn how to use custom functions with the MedTech service device mapping, see

+>[How to use device mappings](how-to-use-device-mappings.md)

+FHIR&#174; is a registered trademark of Health Level Seven International, registered in the U.S. Trademark Office and is used with their permission.

+ | FQDN (`*` = wildcard) | Outbound TCP Ports | Usage |

+ | `*.data.mcr.microsoft.com` | 443 | Data endpoint providing content delivery |

+ | `*.azurecr.io` | 443 | Personal and third-party container registries |

+ | `*.blob.core.windows.net` | 443 | Download Azure Container Registry image deltas from blob storage |

+ | `*.azure-devices.net` | 5671, 8883, 443¹ | IoT Hub access |

+ | `*.docker.io` | 443 | Docker Hub access (optional) |

+* The sample applications you run in this article are written using C# with .NET Core.

+ :::image type="content" source="./media/iot-hub-csharp-csharp-file-upload/view-uploaded-file.png" alt-text="Screenshot of selecting the uploaded file in the Azure portal." lightbox="./media/iot-hub-csharp-csharp-file-upload/view-uploaded-file.png":::

+* If you want to route your device-to-cloud messages to custom endpoints, see [Use message routes and custom endpoints for device-to-cloud messages](iot-hub-devguide-messages-read-custom.md).

+ [!INCLUDE [iot-hub-include-find-service-connection-string](../../includes/iot-hub-include-find-service-connection-string.md)]

+ :::image type="content" source="./media/iot-hub-node-node-file-upload/view-uploaded-file.png" alt-text="Screenshot of viewing the uploaded file in the Azure portal." lightbox="./media/iot-hub-node-node-file-upload/view-uploaded-file.png":::

+* An IoT Hub. Create one with the [CLI](iot-hub-create-using-cli.md) or the [Azure portal](iot-hub-create-through-portal.md).

+* A registered device. Register one in the [Azure portal](iot-hub-create-through-portal.md#register-a-new-device-in-the-iot-hub).

+* Make sure that port 8883 is open in your firewall. The device sample in this article uses MQTT protocol, which communicates over port 8883. This port may be blocked in some corporate and educational network environments. For more information and ways to work around this issue, see [Connecting to IoT Hub (MQTT)](iot-hub-mqtt-support.md#connecting-to-iot-hub).

+ :::image type="content" source="./media/iot-hub-python-python-file-upload/run-device-app.png" alt-text="Screenshot showing output from running the FileUpload app." border="true" lightbox="./media/iot-hub-python-python-file-upload/run-device-app.png":::

+ :::image type="content" source="./media/iot-hub-python-python-file-upload/view-blob.png" alt-text="Screenshot of the container in the Azure portal that shows the uploaded file." border="true" lightbox="./media/iot-hub-python-python-file-upload/view-blob.png":::

+## Integration with Azure services

+In the diagrams below, you see how IP address mapping works before and after enabling Floating IP:

+* ensure the system can send/receive packets on interfaces that don't have the IP address assigned to that interface (on Windows, this requires setting interfaces to use the "weak host" model; on Linux this model is normally used by default)

+You can use [Standard Load Balancer](./load-balancer-overview.md) to create a more predictable application behavior for your scenarios by enabling TCP Reset on Idle for a given rule. Load Balancer's default behavior is to silently drop flows when the idle timeout of a flow is reached. Enabling TCP reset will cause Load Balancer to send bidirectional TCP Resets (TCP RST packet) on idle timeout. This will inform your application endpoints that the connection has timed out and is no longer usable. Endpoints can immediately establish a new connection if needed.

+For many scenarios, TCP reset may reduce the need to send TCP (or application layer) keepalives to refresh the idle timeout of a flow.

+If your idle durations exceed configuration limits or your application shows an undesirable behavior with TCP Resets enabled, you may still need to use TCP keepalives, or application layer keepalives, to monitor the liveness of the TCP connections. Further, keepalives can also remain useful for when the connection is proxied somewhere in the path, particularly application layer keepalives.

+By carefully examining the entire end to end scenario, you can determine the benefits from enabling TCP Resets and adjusting the idle timeout. Then you decide if more steps may be required to ensure the desired application behavior.

+By default, it's set to 4 minutes. If a period of inactivity is longer than the timeout value, there's no guarantee that the TCP or HTTP session is maintained between the client and your cloud service.

+- TCP idle timeout doesn't affect load balancing rules on UDP protocol.

+- TCP reset isn't supported for ILB HA ports when a network virtual appliance is in the path. A workaround could be to use outbound rule with TCP reset from NVA.

+The _parameters_ provide fine grained control over the outbound NAT algorithm.

+Each extra IP address provided by a frontend provides another 64,000 ephemeral ports for load balancer to use as SNAT ports.

+Each of the IP addresses within public IP prefix provides an extra 64,000 ephemeral ports per IP address for load balancer to use as SNAT ports.

+To use a different public IP or prefix than what is used by a load-balancing rule:

+5. Configure an outbound rule on the public load balancer to enable outbound NAT for the VMs using the frontend. It isn't recommended to use a load-balancing rule for outbound, disable outbound SNAT on the load-balancing rule.

+Load balancer gives [SNAT](load-balancer-outbound-connections.md) ports in multiples of 8. If you provide a value not divisible by 8, the configuration operation is rejected. Each load balancing rule and inbound NAT rule will consume a range of eight ports. If a load balancing or inbound NAT rule shares the same range of 8 as another, no extra ports will be consumed.

+If you attempt to give out more [SNAT](load-balancer-outbound-connections.md) ports than are available (based on the number of public IP addresses), the configuration operation is rejected. For example, if you give 10,000 ports per VM and seven VMs in a backend pool share a single public IP, the configuration is rejected. Seven multiplied by 10,000 exceeds the 64,000 port limit. Add more public IP addresses to the frontend of the outbound rule to enable the scenario.

+Use a public standard load balancer to provide outbound NAT for a group of VMs. In this scenario, use an outbound rule by itself, without configuring extra rules.

+With a public standard load balancer, the automatic outbound NAT provided matches the transport protocol of the load-balancing rule.

+ Title: Scenarios for VNET deployment

+description: Learn about the scenarios for deploying Azure Load Testing in a virtual network (VNET). This deployment enables you to load test private application endpoints and hybrid deployments.

+# Scenarios for deploying Azure Load Testing in a virtual network

+In this article, you'll learn about the scenarios for deploying Azure Load Testing Preview in a virtual network (VNET). This deployment is sometimes called VNET injection.

+This functionality enables the following usage scenarios:

+- Generate load to an [endpoint hosted in an Azure virtual network](#scenario-load-test-an-azure-hosted-private-endpoint).

+- Generate load to a [public endpoint with access restrictions](#scenario-load-test-a-public-endpoint-with-access-restrictions), such as restricting client IP addresses.

+- Generate load to an [on-premises service, not publicly accessible, that is connected to Azure via ExpressRoute (hybrid application deployment)](#scenario-load-test-an-on-premises-hosted-service-connected-via-azure-expressroute).

+When you deploy Azure Load Testing in a virtual network, the load test engine virtual machines are attached to the virtual network in your subscription. The load test engines can then communicate with the other resources in the virtual network, such as the private application endpoint. You are not billed for the test engine compute resources.

+> [!IMPORTANT]

+> When you deploy Azure Load Testing in a virtual network, you'll incur additional charges. Azure Load Testing deploys an [Azure Load Balancer](https://azure.microsoft.com/pricing/details/load-balancer/) and a [Public IP address](https://azure.microsoft.com/pricing/details/ip-addresses/) in your subscription and there might be a cost for generated traffic. For more information, see the [Virtual Network pricing information](https://azure.microsoft.com/pricing/details/virtual-network).

+The following diagram provides a technical overview:

+> [!IMPORTANT]

+> Azure Load Testing is currently in preview. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

+## Scenario: Load test an Azure-hosted private endpoint

+In this scenario, you've deployed an application endpoint in a virtual network on Azure, which isn't publicly accessible. For example, the endpoint could be behind an internal load balancer, or running on a VM with a private IP address.

+When you deploy Azure Load Testing in the virtual network, the load test engines can now communicate with the application endpoint. If you've used separate subnets for the application endpoint and Azure Load Testing, make sure that communication between the subsets isn't blocked, for example by a network security group (NSG). Learn how [network security groups filter network traffic](/azure/virtual-network/network-security-group-how-it-works).

+## Scenario: Load test a public endpoint with access restrictions

+In this scenario, you've deployed a publicly available web service in Azure, or any other location. Access to the endpoint is restricted to specific client IP addresses. For example, the service could be running behind an [Azure Application Gateway](/azure/application-gateway/overview), hosted on [Azure App Service with access restrictions](/azure/app-service/app-service-ip-restrictions), or deployed behind a web application firewall.

+To restrict access to the endpoint for the load test engines, you need a range of public IP addresses for the test engine virtual machines. You deploy a [NAT Gateway resource](/azure/virtual-network/nat-gateway/nat-gateway-resource) in the virtual network, and then create and run a load test in the virtual network. A NAT gateway is a fully managed Azure service that provides source network address translation (SNAT).

+Attach the NAT gateway to the subnet in which the load test engines are injected. You can configure the public IP addresses used by the NAT gateway. These load test engine VMs use these IP addresses for generating load. You can then allowlist these IP addresses for restricting access to your application endpoint.

+## Scenario: Load test an on-premises hosted service, connected via Azure ExpressRoute

+In this scenario, you have an on-premises application endpoint, which isn't publicly accessible. The on-premises environment is connected to Azure by using Azure ExpressRoute.

+ExpressRoute lets you extend your on-premises networks into the Microsoft cloud over a private connection with the help of a connectivity provider. Deploy Azure Load Testing in an Azure virtual network and then [connect the network to your ExpressRoute circuit](/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager). After you've set up the connection, the load test engines can connect to the on-premises hosted application endpoint.

+## Next steps

+- Learn how to [load test a private application endpoint](./how-to-test-private-endpoint.md).

+- Start using Azure Load Testing with the [Tutorial: Use a load test to identify performance bottlenecks](./tutorial-identify-bottlenecks-azure-portal.md).

+ Title: Load test private endpoints

+description: Learn how to deploy Azure Load Testing in a virtual network (VNET injection) to test private application endpoints and hybrid deployments.

+# Test private endpoints by deploying Azure Load Testing in an Azure virtual network

+In this article, learn how to test private application endpoints with Azure Load Testing Preview. You'll create an Azure Load Testing resource and enable it to generate load from within your virtual network (VNET injection).

+This functionality enables the following usage scenarios:

+- Generate load to an endpoint that is deployed in an Azure virtual network.

+- Generate load to a public endpoint with access restrictions, such as restricting client IP addresses.

+- Generate load to an on-premises service, not publicly accessible, that is connected to Azure via ExpressRoute.

+Learn more about the scenarios for [deploying Azure Load Testing in your virtual network](./concept-azure-load-testing-vnet-injection.md).

+The following diagram provides a technical overview:

+When you start the load test, Azure Load Testing service injects the following Azure resources in the virtual network that contains the application endpoint:

+- The test engine virtual machines. These VMs will invoke your application endpoint during the load test.

+- A public IP address.

+- A network security group (NSG).

+- An Azure Load Balancer.

+These resources are ephemeral and exist only for the duration of the load test run. If you restrict access to your virtual network, you need to [configure your virtual network](#configure-your-virtual-network) to enable communication between these Azure Load Testing and the injected VMs.

+> [!NOTE]

+> Virtual network support for Azure Load Testing is available in the following Azure regions: Australia East, East US, East US 2, and North Europe.

+> [!IMPORTANT]

+> Azure Load Testing is currently in preview. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

+## Prerequisites

+- An existing virtual network and a subnet to use with Azure Load Testing.

+- The virtual network must be in the same subscription as the Azure Load Testing resource.

+- The subnet you use for Azure Load Testing must have enough unassigned IP addresses to accommodate the number of load test engines for your test. Learn more about [configuring your test for high-scale load](./how-to-high-scale-load.md).

+- The subnet shouldn't be delegated to any other Azure service. For example, it shouldn't be delegated to Azure Container Instances (ACI). Learn more about [subnet delegation](/azure/virtual-network/subnet-delegation-overview).

+## Configure your virtual network

+To test private endpoints, you need an Azure virtual network and at least one subnet. In this section, you'll configure your virtual network and subnet.

+### Create a subnet

+When you deploy Azure Load Testing in your virtual network, it's recommended to use different subnets for Azure Load Testing and the application endpoint. This approach enables you to configure network traffic access specifically for each purpose. Learn more about how to [add a subnet to a virtual network](/azure/virtual-network/virtual-network-manage-subnet#add-a-subnet).

+### Configure traffic access

+Azure Load Testing requires both inbound and outbound access for the injected VMs in your virtual network. If you plan to restrict traffic access to your virtual network, or are already using a network security group, configure the network security group for the subnet in which you deploy the load test.

+1. If you don't have an NSG yet, create one in the same region as your virtual network and associate it with your subnet. Follow these steps to [create a network security group](/azure/virtual-network/manage-network-security-group#create-a-network-security-group).

+1. Go to the [Azure portal](https://portal.azure.com) to view your network security groups. Search for and select **Network security groups**.

+1. Select the name of your network security group.

+1. Select **Inbound security rules** in the left navigation.

+1. Select **+ Add**, to add a new inbound security rule. Enter the following information to create a new rule, and then select **Add**.

+ | Field | Value |

+ | -- | -- |

+ | **Source** | *Service Tag* |

+ | **Source service tag** | *BatchNodeManagement* |

+ | **Source port ranges** | *\** |

+ | **Destination** | *Any* |

+ | **Destination port ranges** | *29876-29877* |

+ | **Name** | *batch-node-management-inbound* |

+ | **Description**| *Create, update, and delete of Azure Load Testing compute instances.* |

+1. Add a second inbound security rule using the following information:

+ | Field | Value |

+ | -- | -- |

+ | **Source** | *Service Tag* |

+ | **Source service tag** | *AzureLoadTestingInstanceManagement* |

+ | **Source port ranges** | *\** |

+ | **Destination** | *Any* |

+ | **Destination port ranges** | *8080* |

+ | **Name** | *azure-load-testing-inbound* |

+ | **Description**| *Create, update, and delete of Azure Load Testing compute instances.* |

+1. Select **Outbound security rules** in the left navigation.

+1. Select **+ Add**, to add a new outbound security rule. Enter the following information to create a new rule, and then select **Add**.

+ | Field | Value |

+ | -- | -- |

+ | **Source** | *Any* |

+ | **Source port ranges** | *\** |

+ | **Destination** | *Any* |

+ | **Destination port ranges** | *\** |

+ | **Name** | *azure-load-testing-outbound* |

+ | **Description**| *Used for various operations involved in orchestrating a load tests.* |

+## Configure your load test script

+The test engine VMs, which run the JMeter script, are injected in the virtual network that contains the application endpoint. You can now refer directly to the endpoint in the JMX file by using the private IP address or use [name resolution in your network](/azure/virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances).

+For example, for an endpoint with IP address 10.179.0.7, in a virtual network with subnet range 10.179.0.0/18, the JMX file could have this information:

+```xml

+<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Internal service homepage" enabled="true">

+ <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="Service homepage" enabled="true">

+ <collectionProp name="Arguments.arguments"/>

+ </elementProp>

+ <stringProp name="HTTPSampler.domain">10.179.0.7</stringProp>

+ <stringProp name="HTTPSampler.port">8081</stringProp>

+ <stringProp name="HTTPSampler.protocol"></stringProp>

+ <stringProp name="HTTPSampler.contentEncoding"></stringProp>

+ <stringProp name="HTTPSampler.path"></stringProp>

+ <stringProp name="HTTPSampler.method">GET</stringProp>

+</HTTPSamplerProxy>

+```

+## Set up a load test

+To create a load test for testing your private endpoint, you have to specify the virtual network details in the test creation wizard.

+1. Sign in to the [Azure portal](https://portal.azure.com) by using the credentials for your Azure subscription.

+

+1. Go to your Azure Load Testing resource, select **Tests** from the left pane, and then select **+ Create new test**.

+ :::image type="content" source="media/how-to-test-private-endpoint/create-new-test.png" alt-text="Screenshot that shows the Azure Load Testing page and the button for creating a new test.":::

+1. On the **Basics** tab, enter the **Test name** and **Test description** information. Optionally, you can select the **Run test after creation** checkbox.

+ :::image type="content" source="media/how-to-test-private-endpoint/create-new-test-basics.png" alt-text="Screenshot that shows the 'Basics' tab for creating a test.":::

+1. On the **Test plan** tab, select your Apache JMeter script, and then select **Upload** to upload the file to Azure.

+ You can select and upload other Apache JMeter configuration files or other files that are referenced in the JMX file. For example, if your test script uses CSV data sets, you can upload the corresponding *.csv* file(s).

+1. On the **Load** tab, select **Private** traffic mode, and then select your virtual network and subnet.

+ :::image type="content" source="media/how-to-test-private-endpoint/create-new-test-load-vnet.png" alt-text="Screenshot that shows the 'Load' tab for creating a test.":::

+ > [!IMPORTANT]

+ > When you deploy Azure Load Testing in a virtual network, you'll incur additional charges. Azure Load Testing deploys an [Azure Load Balancer](https://azure.microsoft.com/pricing/details/load-balancer/) and a [Public IP address](https://azure.microsoft.com/pricing/details/ip-addresses/) in your subscription and there might be a cost for generated traffic. For more information, see the [Virtual Network pricing information](https://azure.microsoft.com/pricing/details/virtual-network).

+

+1. Select **Review + create**. Review all settings, and then select **Create** to create the load test.

+## Troubleshooting

+### Starting the load test fails with `Test cannot be started`

+To start a load test, you must have sufficient permissions to deploy Azure Load Testing to the virtual network. You require the [Network Contributor](/azure/role-based-access-control/built-in-roles#network-contributor) role, or a parent of this role, on the virtual network. See [Check access for a user to Azure resources](/azure/role-based-access-control/check-access) to verify your permissions.

+If you're using the [Azure Load Testing REST API](/rest/api/loadtesting/) to start a load test, check that you're using a valid subnet ID. The subnet must be in the same Azure region as your Azure Load Testing resource.

+### The load test is stuck in `Provisioning` state and then goes to `Failed`

+1. Verify that your subscription is registered with `Microsoft.Batch`.

+ Run the following Azure CLI command to verify the status. The result should be `Registered`.

+ ```azurecli

+ az provider show --namespace Microsoft.Batch --query registrationState

+ ```

+1. Verify that Microsoft Batch node management and the Azure Load Testing IPs can make inbound connections to the test engine VMs.

+ 1. Enable [Network Watcher](/azure/network-watcher/network-watcher-monitoring-overview) for the virtual network region.

+ ```azurecli

+ az network watcher configure \

+ --resource-group NetworkWatcherRG \

+ --locations eastus \

+ --enabled

+ ```

+ 1. Create a temporary VM with a Public IP in the subnet you're using for the Azure Load Testing service. You'll only use this VM to diagnose the network connectivity and delete it afterwards. The VM can be of any type.

+ ```azurecli

+ az vm create \

+ --resource-group myResourceGroup \

+ --name myVm \

+ --image UbuntuLTS \

+ --generate-ssh-keys \

+ --subnet mySubnet

+ ```

+ 1. Test the inbound connectivity to the temporary VM from the `BatchNodeManagement` service tag.

+ 1. In the [Azure portal](https://portal.azure.com), go to **Network Watcher**.

+ 1. On the left pane, select **NSG Diagnostic**.

+ 1. Enter the details of the VM you created in the previous step.

+ 1. Select **Service Tag** for the **Source type**, and then select **BatchNodeManagement** for the **Service tag**.

+ 1. The **Destination IP address** is the IP address of the VM you created in previous step.

+ 1. For **Destination port**, you have to validate two ports: *29876* and *29877*. Enter one value at a time and move to the next step.

+ 1. Press **Check** to verify that the network security group isn't blocking traffic.

+ :::image type="content" source="media/how-to-test-private-endpoint/test-network-security-group-connectivity.png" alt-text="Screenshot that shows the NSG Diagnostic page to test network connectivity.":::

+ If the traffic status is **Denied**, [configure your virtual network](#configure-your-virtual-network) to allow traffic for the **BatchNodeManagement** service tag.

+ 1. Test the inbound connectivity to the temporary VM from the `AzureLoadTestingInstanceManagement` service tag.

+ 1. In the [Azure portal](https://portal.azure.com), go to **Network Watcher**.

+ 1. On the left pane, select **NSG Diagnostic**.

+ 1. Enter the details of the VM you created in the previous step.

+ 1. Select **Service Tag** for the **Source type**, and then select **AzureLoadTestingInstanceManagement** for the **Service tag**.

+ 1. The **Destination IP address** is the IP address of the VM you created in previous step.

+ 1. For **Destination port**, enter *8080*.

+ 1. Press **Check** to verify that the network security group isn't blocking traffic.

+ If the traffic status is **Denied**, [configure your virtual network](#configure-your-virtual-network) to allow traffic for the **AzureLoadTestingInstanceManagement** service tag.

+ 1. Delete the temporary VM you created earlier.

+### The test executes and results in a 100% error rate

+Possible cause: there are connectivity issues between the subnet in which you deployed Azure Load Testing and the subnet in which the application endpoint is hosted.

+1. You might deploy a temporary VM in the subnet used by Azure Load Testing and then use the [curl](https://curl.se/) tool to test connectivity to the application endpoint. Verify that there are no firewall or NSG rules that are blocking traffic.

+1. Verify the [Azure Load Testing results file](./how-to-export-test-results.md) for error response messages:

+ |Response message | Action |

+ |||

+ | **Non http response code java.net.unknownhostexception** | Possible cause is a DNS resolution issue. If youΓÇÖre using Azure Private DNS, verify that the DNS is set up correctly for the subnet in which Azure Load Testing instances are injected, and for the application subnet. |

+ | **Non http response code SocketTimeout** | Possible cause is when thereΓÇÖs a firewall blocking connections from the subnet in which Azure Load Testing instances are injected to your application subnet. |

+## Next steps

+- Learn more about the [scenarios for deploying Azure Load Testing in a virtual network](./concept-azure-load-testing-vnet-injection.md).

+- Learn how to [Monitor server-side application metrics](./how-to-monitor-server-side-metrics.md).

+Azure Load Testing enables you to test private application endpoints or applications that you host on-premises. For more information, see the [scenarios for deploying Azure Load Testing in a virtual network](./concept-azure-load-testing-vnet-injection.md).

+The application can be hosted anywhere: in Azure, on-premises, or in other clouds. To load test services that have no public endpoint, [deploy Azure Load Testing in a virtual network](./how-to-test-private-endpoint.md).

+During the load test, the service collects the following resource metrics and displays them in a dashboard:

+A model in MLflow is also an artifact. However, we make stronger assumptions about this type of artifacts. Such assumptions provide a clear contract between the saved files and what they mean. When you log your models as artifacts (simple files), you need to know what the model builder meant for each of them in order to know how to load the model for inference. On the contrary, MLflow models can be loaded using the contract specified in the [The MLModel format](concept-mlflow-models.md#the-mlmodel-format).

+In Azure Machine Learning, logging models has the following advantages:

+> * You can deploy them on real-time or batch endpoints without providing an scoring script nor an environment.

+> * When deployed, Model's deployments have a Swagger generated automatically and the __Test__ feature can be used in Azure ML studio.

+> * You can use the [Responsible AI dashbord (preview)](how-to-responsible-ai-dashboard.md).

+import mlflow

+> [!IMPORTANT]

+> Azure ML SDK v1 doesn't have the *model* concept.

+## The MLmodel format

+MLflow adopts the MLmodel format as a way to create a contract between the artifacts and what they represent. The MLmodel format stores assets in a folder. Among them, there is a particular file named MLmodel. This file is the single source of truth about how a model can be loaded and used.

+

+* **Column-based signature** corresponding to signatures that operate to tabular data. For models with this signature, MLflow supplies `pandas.DataFrame` objects as inputs.

+* **Tensor-based signature:** corresponding to signatures that operate with n-dimensional arrays or tensors. For models with this signature, MLflow supplies `numpy.ndarray` as inputs (or a dictionary of `numpy.ndarray` in the case of named-tensors).

+> Azure Machine Learning generates Swagger for model's deployment in MLflow format with a signature available. This makes easier to test deployed endpoints using the Azure ML studio.

+* **Loading back the same object and types that were logged:** You can load models using MLflow SDK and obtain an instance of the model with types belonging to the training library. For instance, an ONNX model will return a `ModelProto` while a decision tree trained with Scikit-Learn model will return a `DecisionTreeClassifier` object. Use `mlflow.<flavor>.load_model()` to do so.

+* **Loading back a model for running inference:** You can load models using MLflow SDK and obtain a wrapper where MLflow warranties there will be a `predict` function. It doesn't matter which flavor you are using, every MLflow model needs to implement this contract. Furthermore, MLflow warranties that this function can be called using arguments of type `pandas.DataFrame`, `numpy.ndarray` or `dict[string, numpyndarray]` (depending on the signature of the model). MLflow handles the type conversion to the input type the model actually expects. Use `mlflow.pyfunc.load_model()` to do so.

+> Azure Machine Learning workspaces are MLflow-compatible, meaning that you can use Azure Machine Learning workspaces in the same way you use an MLflow Tracking Server. Such compatibility has the following advantages:

+Azure Machine Learning uses MLflow Tracking for metric logging and artifact storage for your experiments, whether you created the experiment via the Azure Machine Learning Python SDK, Azure Machine Learning CLI or the Azure Machine Learning studio. We recommend using MLflow for tracking experiments. To get you started, see [Log & view metrics and log files with MLflow](how-to-log-view-metrics.md).

+With MLflow Tracking you can connect Azure Machine Learning as the backend of your MLflow experiments. The workspace provides a centralized, secure, and scalable location to store training metrics and models. This includes:

+* [Track ML experiments and models running locally or in the cloud](how-to-use-mlflow-cli-runs.md) with MLflow in Azure Machine Learning.

+* [Track Azure Databricks ML experiments](how-to-use-mlflow-azure-databricks.md) with MLflow in Azure Machine Learning.

+* [Track Azure Synapse Analytics ML experiments](how-to-use-mlflow-azure-databricks.md) with MLflow in Azure Machine Learning.

+> - MLflow in R support is limited to tracking experiment's metrics and parameters on Azure Machine Learning jobs. RStudio or Jupyter Notebooks with R kernels are not supported. Artifacts and models can't be tracked using the MLflow R SDK. As an alternative, you can save them locally using [`mlflow_save_model.crate`](https://mlflow.org/docs/latest/R-api.html#mlflow-save-model-crate) in the `outputs` folder. Then, use Azure ML CLI or Azure ML studio for model registration. View the following [R example about using the MLflow tracking client with Azure Machine Learning](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/single-step/r).

+> - MLflow in Java support is limited to tracking experiment's metrics and parameters on Azure Machine Learning jobs. Artifacts and models can't be tracked using the MLflow Java SDK. View the following [Java example about using the MLflow tracking client with the Azure Machine Learning](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/single-step/java/iris).

+To learn how to use MLflow to query experiments and runs in Azure Machine Learning, see [Manage experiments and runs with MLflow](how-to-track-experiments-mlflow.md)

+The societal implications of AI and the responsibility of organizations to anticipate and mitigate unintended consequences of AI technology are significant. Organizations are finding the need to create internal policies, practices, and tools to guide their AI efforts, whether they're deploying third-party AI solutions or developing their own. At Microsoft, we've recognized six principles that we believe should guide AI development and use: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. For us, these principles are the cornerstone of a responsible and trustworthy approach to AI, especially as intelligent technology becomes more prevalent in the products and services we use every day. Azure Machine Learning currently supports various tools for these principles, making it seamless for ML developers and data scientists to implement Responsible AI in practice.

+- `custom` is a type that refers to a model file or folder trained with a custom standard not currently supported by Azure ML.

+> During development, consider using the [python-dotenv](https://pypi.org/project/python-dotenv/) package to set these environment variables. Python-dotenv loads environment variables from `.env` files. The standard `.gitignore` file for Python automatically excludes `.env` files, so they shouldn't be checked into any GitHub repos during development.

+Azure Machine Learning is still in development in Airgap Regions.

+| [Reinforcement Learning](./v1/how-to-use-reinforcement-learning.md) | Public Preview | NO | NO |

+| `timeout` | integer | The maximum time in seconds the entire sweep job is allowed to run. Once this limit is reached the system will cancel the sweep job, including all its trials. | `604800` |

+ Title: Train and deploy a reinforcement learning model (preview)

+description: Learn how to use Azure Machine Learning reinforcement learning (preview) to train an RL agent to play Pong.

+# Reinforcement learning (preview) with Azure Machine Learning

+> [!WARNING]

+> Azure Machine Learning reinforcement learning via the [`azureml.contrib.train.rl`](/python/api/azureml-contrib-reinforcementlearning/azureml.contrib.train.rl) package will no longer be supported after June 2022. We recommend customers use the [Ray on Azure Machine Learning library](https://github.com/microsoft/ray-on-aml) for reinforcement learning experiments with Azure Machine Learning. For an example, see the notebook [Reinforcement Learning in Azure Machine Learning - Pong problem](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/reinforcement-learning/atari-on-distributed-compute/pong_rllib.ipynb).

+In this article, you learn how to train a reinforcement learning (RL) agent to play the video game Pong. You use the open-source Python library [Ray RLlib](https://docs.ray.io/en/master/rllib/) with Azure Machine Learning to manage the complexity of distributed RL.

+In this article you learn how to:

+> [!div class="checklist"]

+> * Set up an experiment

+> * Define head and worker nodes

+> * Create an RL estimator

+> * Submit an experiment to start a job

+> * View results

+This article is based on the [RLlib Pong example](https://aka.ms/azureml-rl-pong) that can be found in the Azure Machine Learning notebook [GitHub repository](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/reinforcement-learning/README.md).

+## Prerequisites

+Run this code in either of these environments. We recommend you try Azure Machine Learning compute instance for the fastest start-up experience. You can quickly clone and run the reinforcement sample notebooks on an Azure Machine Learning compute instance.

+ - Azure Machine Learning compute instance

+ - Learn how to clone sample notebooks in [Tutorial: Train and deploy a model](../tutorial-train-deploy-notebook.md).

+ - Clone the **how-to-use-azureml** folder instead of **tutorials**

+ - Run the virtual network setup notebook located at `/how-to-use-azureml/reinforcement-learning/setup/devenv_setup.ipynb` to open network ports used for distributed reinforcement learning.

+ - Run the sample notebook `/how-to-use-azureml/reinforcement-learning/atari-on-distributed-compute/pong_rllib.ipynb`

+

+ - Your own Jupyter Notebook server

+ - Install the [Azure Machine Learning SDK](/python/api/overview/azure/ml/install).

+ - Install the [Azure Machine Learning RL SDK](/python/api/azureml-contrib-reinforcementlearning/): `pip install --upgrade azureml-contrib-reinforcementlearning`

+ - Create a [workspace configuration file](../how-to-configure-environment.md#workspace).

+ - Run the virtual network to open network ports used for distributed reinforcement learning.

+## How to train a Pong-playing agent

+Reinforcement learning (RL) is an approach to machine learning that learns by doing. While other machine learning techniques learn by passively taking input data and finding patterns within it, RL uses **training agents** to actively make decisions and learn from their outcomes.

+Your training agents learn to play Pong in a **simulated environment**. Training agents make a decision every frame of the game to move the paddle up, down, or stay in place. It looks at the state of the game (an RGB image of the screen) to make a decision.

+RL uses **rewards** to tell the agent if its decisions are successful. In this example, the agent gets a positive reward when it scores a point and a negative reward when a point is scored against it. Over many iterations, the training agent learns to choose the action, based on its current state, that optimizes for the sum of expected future rewards. It's common to use **deep neural networks** (DNN) to perform this optimization in RL.

+Training ends when the agent reaches an average reward score of 18 in a training epoch. This means that the agent has beaten its opponent by an average of at least 18 points in matches up to 21.

+The process of iterating through simulation and retraining a DNN is computationally expensive, and requires a lot of data. One way to improve performance of RL jobs is by **parallelizing work** so that multiple training agents can act and learn simultaneously. However, managing a distributed RL environment can be a complex undertaking.

+Azure Machine Learning provides the framework to manage these complexities to scale out your RL workloads.

+## Set up the environment

+Set up the local RL environment by:

+1. Loading the required Python packages

+1. Initializing your workspace

+1. Creating an experiment

+1. Specifying a configured virtual network.

+### Import libraries

+Import the necessary Python packages to run the rest of this example.

+```python

+# Azure ML Core imports

+import azureml.core

+from azureml.core import Workspace

+from azureml.core import Experiment

+from azureml.core.compute import AmlCompute

+from azureml.core.compute import ComputeTarget

+from azureml.core.runconfig import EnvironmentDefinition

+from azureml.widgets import RunDetails

+from azureml.tensorboard import Tensorboard

+# Azure ML Reinforcement Learning imports

+from azureml.contrib.train.rl import ReinforcementLearningEstimator, Ray

+from azureml.contrib.train.rl import WorkerConfiguration

+```

+### Initialize a workspace

+Initialize a [workspace](../concept-workspace.md) object from the `config.json` file created in the [prerequisites section](#prerequisites). If you are executing this code in an Azure Machine Learning Compute Instance, the configuration file has already been created for you.

+```Python

+ws = Workspace.from_config()

+```

+### Create a reinforcement learning experiment

+Create an [experiment](/python/api/azureml-core/azureml.core.experiment.experiment) to track your reinforcement learning job. In Azure Machine Learning, experiments are logical collections of related trials to organize job logs, history, outputs, and more.

+```python

+experiment_name='rllib-pong-multi-node'

+exp = Experiment(workspace=ws, name=experiment_name)

+```

+### Specify a virtual network

+For RL jobs that use multiple compute targets, you must specify a virtual network with open ports that allow worker nodes and head nodes to communicate with each other.

+The virtual network can be in any resource group, but it should be in the same region as your workspace. For more information on setting up your virtual network, see the workspace setup notebook in the prerequisites section. Here, you specify the name of the virtual network in your resource group.

+```python

+vnet = 'your_vnet'

+```

+## Define head and worker compute targets

+This example uses separate compute targets for the Ray head and workers nodes. These settings let you scale your compute resources up and down depending on your workload. Set the number of nodes, and the size of each node, based on your needs.

+### Head computing target

+You can use a GPU-equipped head cluster to improve deep learning performance. The head node trains the neural network that the agent uses to make decisions. The head node also collects data points from the worker nodes to train the neural network.

+The head compute uses a single [`STANDARD_NC6` virtual machine](../../virtual-machines/nc-series.md) (VM). It has 6 virtual CPUs to distribute work across.

+```python

+from azureml.core.compute import AmlCompute, ComputeTarget

+# choose a name for the Ray head cluster

+head_compute_name = 'head-gpu'

+head_compute_min_nodes = 0

+head_compute_max_nodes = 2

+# This example uses GPU VM. For using CPU VM, set SKU to STANDARD_D2_V2

+head_vm_size = 'STANDARD_NC6'

+if head_compute_name in ws.compute_targets:

+ head_compute_target = ws.compute_targets[head_compute_name]

+ if head_compute_target and type(head_compute_target) is AmlCompute:

+ print(f'found head compute target. just use it {head_compute_name}')

+else:

+ print('creating a new head compute target...')

+ provisioning_config = AmlCompute.provisioning_configuration(vm_size = head_vm_size,

+ min_nodes = head_compute_min_nodes,

+ max_nodes = head_compute_max_nodes,

+ vnet_resourcegroup_name = ws.resource_group,

+ vnet_name = vnet_name,

+ subnet_name = 'default')

+ # create the cluster

+ head_compute_target = ComputeTarget.create(ws, head_compute_name, provisioning_config)

+

+ # can poll for a minimum number of nodes and for a specific timeout.

+ # if no min node count is provided it will use the scale settings for the cluster

+ head_compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)

+

+ # For a more detailed view of current AmlCompute status, use get_status()

+ print(head_compute_target.get_status().serialize())

+```

+### Worker computing cluster

+This example uses four [`STANDARD_D2_V2` VMs](../../virtual-machines/nc-series.md) for the worker compute target. Each worker node has 2 available CPUs for a total of 8 available CPUs.

+GPUs aren't necessary for the worker nodes since they aren't performing deep learning. The workers run the game simulations and collect data.

+```python

+# choose a name for your Ray worker cluster

+worker_compute_name = 'worker-cpu'

+worker_compute_min_nodes = 0

+worker_compute_max_nodes = 4

+# This example uses CPU VM. For using GPU VM, set SKU to STANDARD_NC6

+worker_vm_size = 'STANDARD_D2_V2'

+# Create the compute target if it hasn't been created already

+if worker_compute_name in ws.compute_targets:

+ worker_compute_target = ws.compute_targets[worker_compute_name]

+ if worker_compute_target and type(worker_compute_target) is AmlCompute:

+ print(f'found worker compute target. just use it {worker_compute_name}')

+else:

+ print('creating a new worker compute target...')

+ provisioning_config = AmlCompute.provisioning_configuration(vm_size = worker_vm_size,

+ min_nodes = worker_compute_min_nodes,

+ max_nodes = worker_compute_max_nodes,

+ vnet_resourcegroup_name = ws.resource_group,

+ vnet_name = vnet_name,

+ subnet_name = 'default')

+ # create the cluster

+ worker_compute_target = ComputeTarget.create(ws, worker_compute_name, provisioning_config)

+

+ # can poll for a minimum number of nodes and for a specific timeout.

+ # if no min node count is provided it will use the scale settings for the cluster

+ worker_compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)

+

+ # For a more detailed view of current AmlCompute status, use get_status()

+ print(worker_compute_target.get_status().serialize())

+```

+## Create a reinforcement learning estimator

+Use the [ReinforcementLearningEstimator](/python/api/azureml-contrib-reinforcementlearning/azureml.contrib.train.rl.reinforcementlearningestimator) to submit a training job to Azure Machine Learning.

+Azure Machine Learning uses estimator classes to encapsulate job configuration information. This lets you specify how to configure a script execution.

+### Define a worker configuration

+The WorkerConfiguration object tells Azure Machine Learning how to initialize the worker cluster that runs the entry script.

+```python

+# Pip packages we will use for both head and worker

+pip_packages=["ray[rllib]==0.8.3"] # Latest version of Ray has fixes for isses related to object transfers

+# Specify the Ray worker configuration

+worker_conf = WorkerConfiguration(

+

+ # Azure ML compute cluster to run Ray workers

+ compute_target=worker_compute_target,

+

+ # Number of worker nodes

+ node_count=4,

+

+ # GPU

+ use_gpu=False,

+

+ # PIP packages to use

+ pip_packages=pip_packages

+)

+```

+### Define script parameters

+The entry script `pong_rllib.py` accepts a list of parameters that defines how to execute the training job. Passing these parameters through the estimator as a layer of encapsulation makes it easy to change script parameters and run configurations independently of each other.

+Specifying the correct `num_workers` makes the most out of your parallelization efforts. Set the number of workers to the same as the number of available CPUs. For this example, you can use the following calculation:

+The head node is a [Standard_NC6](../../virtual-machines/nc-series.md) with 6 vCPUs. The worker cluster is 4 [Standard_D2_V2 VMs](../../cloud-services/cloud-services-sizes-specs.md#dv2-series) with 2 CPUs each, for a total of 8 CPUs. However, you must subtract 1 CPU from the worker count since 1 must be dedicated to the head node role.

+6 CPUs + 8 CPUs - 1 head CPU = 13 simultaneous workers. Azure Machine Learning uses head and worker clusters to distinguish compute resources. However, Ray does not distinguish between head and workers, and all CPUs are available as worker threads.

+```python

+training_algorithm = "IMPALA"

+rl_environment = "PongNoFrameskip-v4"

+# Training script parameters

+script_params = {

+

+ # Training algorithm, IMPALA in this case

+ "--run": training_algorithm,

+

+ # Environment, Pong in this case

+ "--env": rl_environment,

+

+ # Add additional single quotes at the both ends of string values as we have spaces in the

+ # string parameters, outermost quotes are not passed to scripts as they are not actually part of string

+ # Number of GPUs

+ # Number of ray workers

+ "--config": '\'{"num_gpus": 1, "num_workers": 13}\'',

+

+ # Target episode reward mean to stop the training

+ # Total training time in seconds

+ "--stop": '\'{"episode_reward_mean": 18, "time_total_s": 3600}\'',

+}

+```

+### Define the reinforcement learning estimator

+Use the parameter list and the worker configuration object to construct the estimator.

+```python

+# RL estimator

+rl_estimator = ReinforcementLearningEstimator(

+

+ # Location of source files

+ source_directory='files',

+

+ # Python script file

+ entry_script="pong_rllib.py",

+

+ # Parameters to pass to the script file

+ # Defined above.

+ script_params=script_params,

+

+ # The Azure ML compute target set up for Ray head nodes

+ compute_target=head_compute_target,

+

+ # Pip packages

+ pip_packages=pip_packages,

+

+ # GPU usage

+ use_gpu=True,

+

+ # RL framework. Currently must be Ray.

+ rl_framework=Ray(),

+

+ # Ray worker configuration defined above.

+ worker_configuration=worker_conf,

+

+ # How long to wait for whole cluster to start

+ cluster_coordination_timeout_seconds=3600,

+

+ # Maximum time for the whole Ray job to run

+ # This will cut off the job after an hour

+ max_run_duration_seconds=3600,

+

+ # Allow the docker container Ray runs in to make full use

+ # of the shared memory available from the host OS.

+ shm_size=24*1024*1024*1024

+)

+```

+### Entry script

+The [entry script](https://aka.ms/azure-rl-pong-script) `pong_rllib.py` trains a neural network using the [OpenAI Gym environment](https://github.com/openai/gym/) `PongNoFrameSkip-v4`. OpenAI Gyms are standardized interfaces to test reinforcement learning algorithms on classic Atari games.

+This example uses a training algorithm known as [IMPALA](https://arxiv.org/abs/1802.01561) (Importance Weighted Actor-Learner Architecture). IMPALA parallelizes each individual learning actor to scale across many compute nodes without sacrificing speed or stability.

+[Ray Tune](https://ray.readthedocs.io/en/latest/tune.html) orchestrates the IMPALA worker tasks.

+```python

+import ray

+import ray.tune as tune

+from ray.rllib import train

+import os

+import sys

+from azureml.core import Run

+from utils import callbacks

+DEFAULT_RAY_ADDRESS = 'localhost:6379'

+if __name__ == "__main__":

+ # Parse arguments

+ train_parser = train.create_parser()

+ args = train_parser.parse_args()

+ print("Algorithm config:", args.config)

+ if args.ray_address is None:

+ args.ray_address = DEFAULT_RAY_ADDRESS

+ ray.init(address=args.ray_address)

+ tune.run(run_or_experiment=args.run,

+ config={

+ "env": args.env,

+ "num_gpus": args.config["num_gpus"],

+ "num_workers": args.config["num_workers"],

+ "callbacks": {"on_train_result": callbacks.on_train_result},

+ "sample_batch_size": 50,

+ "train_batch_size": 1000,

+ "num_sgd_iter": 2,

+ "num_data_loader_buffers": 2,

+ "model": {

+ "dim": 42

+ },

+ },

+ stop=args.stop,

+ local_dir='./logs')

+```

+### Logging callback function

+The entry script uses a utility function to define a [custom RLlib callback function](https://docs.ray.io/en/latest/rllib-training.html#callbacks-and-custom-metrics) to log metrics to your Azure Machine Learning workspace. Learn how to view these metrics in the [Monitor and view results](#monitor-and-view-results) section.

+```python

+'''RLlib callbacks module:

+ Common callback methods to be passed to RLlib trainer.

+'''

+from azureml.core import Run

+def on_train_result(info):

+ '''Callback on train result to record metrics returned by trainer.

+ '''

+ run = Run.get_context()

+ run.log(

+ name='episode_reward_mean',

+ value=info["result"]["episode_reward_mean"])

+ run.log(

+ name='episodes_total',

+ value=info["result"]["episodes_total"])

+```

+## Submit a job

+[Run](/python/api/azureml-core/azureml.core.run%28class%29) handles the run history of in-progress or complete jobs.

+```python

+run = exp.submit(config=rl_estimator)

+```

+> [!NOTE]

+> The run may take up to 30 to 45 minutes to complete.

+## Monitor and view results

+Use the Azure Machine Learning Jupyter widget to see the status of your jobs in real time. The widget shows two child jobs: one for head and one for workers.

+```python

+from azureml.widgets import RunDetails

+RunDetails(run).show()

+run.wait_for_completion()

+```

+1. Wait for the widget to load.

+1. Select the head job in the list of jobs.

+Select **Click here to see the job in Azure Machine Learning studio** for additional job information in the studio. You can access this information while the job is in progress or after it completes.

+

+The **episode_reward_mean** plot shows the mean number of points scored per training epoch. You can see that the training agent initially performed poorly, losing its matches without scoring a single point (shown by a reward_mean of -21). Within 100 iterations, the training agent learned to beat the computer opponent by an average of 18 points.

+If you browse logs of the child job, you can see the evaluation results recorded in driver_log.txt file. You may need to wait several minutes before these metrics become available on the Job page.

+In short work, you have learned to configure multiple compute resources to train a reinforcement learning agent to play Pong very well against a computer opponent.

+## Next steps

+In this article, you learned how to train a reinforcement learning agent using an IMPALA learning agent. To see additional examples, go to the [Azure Machine Learning Reinforcement Learning GitHub repository](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/reinforcement-learning/README.md).

+| Azure Application <br>(Managed application) | Monthly | Yes | Usage-based and flat rate |

+Virtual Machine and Azure Application offers are a good fit if you want customers to deploy, manage, and run your packaged app or service (as a VM Image and/or other Azure services in the ARM template) in their own Azure cloud infrastructure.

+[1] Attend [Microsoft Office Hours](https://go.microsoft.com/fwlink/?linkid=2185526) or [support](./support.md).

+[2] VM offer images can be included in the Azure App offer to increase pricing and deployment flexibility.

+[3] This can include your own VM offers.

+[4] Customer pays additional infrastructure costs since Azure services are deployed on the customer tenant for VM and Azure App offers.

+> [!VIDEO ae2b72e2-6591-407f-8740-50cc4860e8ee]

+# Monitor Azure Database for MySQL Flexible Server

+Azure Database for MySQL Flexible Server provides monitoring of servers through Azure Monitor. Monitoring data about your servers helps you troubleshoot and optimize for your workload.

+In this article, you'll learn about the various metrics available and Server logs for your flexible server that give insight into the behavior of your server.

+## Metrics

+Metrics are numerical values that describe some aspect of the resources of your server at a particular time. Monitoring your server's resources helps you troubleshoot and optimize your workload by allowing you to monitor what matters the most to you. Monitoring the right metrics helps you keep the performance, reliability, and availability of your server and applications.

+Azure Database for MySQL Flexible Server provides various metrics to understand how your workload is performing and based on this data, you can understand the impact on your server and application.

+## List of metrics

+## Server logs

+In Azure Database for MySQL Server ΓÇô Flexible Server, users can configure and download server logs to assist with troubleshooting efforts. With this feature enabled, a flexible server starts capturing events of the selected log type and writes them to a file. You can then use the Azure portal and Azure CLI to download the files to work with them.

+The server logs feature is disabled by default. For information about how to enable server logs, see [How to enable and download server logs for Azure Database for MySQL - Flexible Server](./how-to-server-logs-portal.md)

+To perform a historical analysis of your data, in the Azure portal, on the Diagnostics settings pane for your server, add a diagnostic setting to send the logs to Log Analytics workspace, Azure Storage, or event hubs. For more information, see [Set up diagnostics](./tutorial-query-performance-insights.md#set-up-diagnostics).

+**Server logs retention**

+When logging is enabled for an Azure Database for MySQL flexible server, logs are available up to seven days from their creation.

+If the total size of the available logs exceeds 7 GB, then the oldest files are deleted until space is available.

+The 7-GB storage limit for server logs is available free of cost and can't be extended.

+Logs are rotated every 24 hours or 7 GB, whichever comes first.

+- How to enable and download server logs for Azure Database for MySQL - Flexible Server from [Azure portal](./how-to-server-logs-portal.md) or [Azure CLI](./how-to-server-logs-cli.md)

+ Title: 'Monitoring - List and Download Server logs using Azure CLI'

+description: This article describes how to download and list server logs using Azure CLI.

+# List and Download Server logs using Azure CLI

+[[!INCLUDE[applies-to-mysql-flexible-server](../includes/applies-to-mysql-flexible-server.md)]

+This article shows you how to list and download server flexible server using Azure CLI.

+## Prerequisites

+This article requires that you're running the Azure CLI version 2.39.0 or later locally. To see the version installed, run the `az --version` command. If you need to install or upgrade, see [Install Azure CLI](/cli/azure/install-azure-cli).

+You'll need to sign-in to your account using the [az login](/cli/azure/reference-index#az-login) command. Note the **id** property, which refers to **Subscription ID** for your Azure account.

+```azurecli-interactive

+az login

+```

+Select the specific subscription under your account using [az account set](/cli/azure/account) command. Make a note of the **id** value from the **az login** output to use as the value for **subscription** argument in the command. If you have multiple subscriptions, choose the appropriate subscription in which the resource should be billed. To get all your subscription, use [az account list](/cli/azure/account#az-account-list).

+```azurecli

+az account set --subscription <subscription id>

+```

+## List Server logs using Azure CLI

+Once you're configured the prerequisites and connected to your required subscription.

+You can list the server logs from your flexible server by below command.

+```azurecli

+az mysql flexible-server server-logs list --resource-group <myresourcegroup> --server-name <serverlogdemo> --out <table>

+```

+Here are the details for the above command

+LastModifiedTime | Name | ResourceGroup | SizeInKb | TypePropertiesType | Url

+||||||

+2022-08-01T11:09:48+00:00 | mysql-slow-serverlogdemo-2022073111.log | myresourcegroup | 10947 | slowlog | https://00000000000.file.core.windows.net/0000000serverlog/slowlogs/mysql-slow-serverlogdemo-2022073111.log?

+2022-08-02T11:10:00+00:00 | mysql-slow-serverlogdemo-2022080111.log | myresourcegroup | 10927 | slowlog | https://00000000000.file.core.windows.net/0000000serverlog/slowlogs/mysql-slow-serverlogdemo-2022080111.log?

+2022-08-03T11:10:12+00:00 | mysql-slow-serverlogdemo-2022080211.log | myresourcegroup | 10936 | slowlog | https://00000000000.file.core.windows.net/0000000serverlog/slowlogs/mysql-slow-serverlogdemo-2022080211.log?

+2022-08-03T11:12:00+00:00 | mysql-slow-serverlogdemo-2022080311.log | myresourcegroup | 8920 | slowlog | https://00000000000.file.core.windows.net/0000000serverlog/slowlogs/mysql-slow-serverlogdemo-2022080311.log?

+Above list shows LastModifiedTime, Name, ResourceGroup, SizeInKb and Download Url of the Server Logs available.

+Default LastModifiedTime is set to 72 hours, for listing files older than 72 hours, use flag `--file-last-written <Time:HH>`

+```azurecli

+az mysql flexible-server server-logs list --resource-group <myresourcegroup> --server-name <serverlogdemo> --out table --file-last-written <144>

+```

+## Downloading Server logs using Azure CLI

+Below command will download the mentioned server logs to your current directory.

+```azurecli

+az mysql flexible-server server-logs download --resource-group <myresourcegroup> --server-name <serverlogdemo> --name <mysql-slow-serverlogdemo-2022073111.log>

+```

+## Next Steps

+- To enable and disable Server logs from portal, you can refer to the [article].(./how-to-server-logs-portal.md)

+- Learn more about [Configure slow logs using Azure CLI](./tutorial-query-performance-insights.md#configure-slow-query-logs-by-using-the-azure-cli)

+ Title: 'How to enable and download server logs for Azure Database for MySQL - Flexible Server'

+description: This article describes how to download and list server logs using Azure portal.

+# Enable, list and download server logs for Azure Database for MySQL - Flexible Server

+You can use server logs to help monitor and troubleshoot an instance of Azure Database for MySQL - Flexible Server, and to gain detailed insights into the activities that have run on your servers.

+By default, the server logs feature in Azure Database for MySQL - Flexible Server is disabled. However, after you enable the feature, a flexible server starts capturing events of the selected log type and writes them to a file. You can then use the Azure portal or the Azure CLI to download the files to assist with your troubleshooting efforts.

+This article explains how to enable the server logs feature in Azure Database for MySQL - Flexible Server and download server log files. It also provides information about how to disable the feature.

+In this tutorial, youΓÇÖll learn how to:

+- Enable the server logs feature.

+- Disable the server logs feature.

+- Download server log files.

+## Prerequisites

+To complete this tutorial, you need an existing Azure Database for MySQL flexible server. If you need to create a new server, see [Create an Azure Database for MySQL flexible server](./quickstart-create-server-portal.md).

+## Enable Server logs

+To enable the server logs feature, perform the following steps.

+1. In the [Azure portal](https://portal.azure.com), select your MySQL flexible server.

+2. On the left pane, under **Monitoring**, select **Server logs**.

+ :::image type="content" source="./media/how-to-server-logs-portal/1-how-to-serverlog.png" alt-text="Screenshot showing Azure Database for My SQL - Server Logs.":::

+3. To enable server logs, under **Server logs**, select **Enable**.

+ :::image type="content" source="./media/how-to-server-logs-portal/2-how-to-serverlog.png" alt-text="Screenshot showing Enable Server Logs.":::

+>[!Note]

+> You can also enable server logs in the Azure portal, on the [Server parameters](./how-to-configure-server-parameters-portal.md) pane for your server, by setting the value of the log_output parameter to FILE.

+> For more information on the log_output parameter, in the MySQL documentation, see topic Server System Variables ([version 5.7](https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_log_output) or [version 8.0](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_log_output)).

+4. To enable the slow_query_log log, under **Select logs to enable**, select **slow_query_log**.

+ :::image type="content" source="./media/how-to-server-logs-portal/3-how-to-serverlog.png" alt-text="Screenshot showing select slow log - Server Logs.":::

+To configure slow_logs on your Azure Database for MySQL flexible server, see [Query Performance Insight.](./tutorial-query-performance-insights.md)

+## Download Server logs

+To download server logs, perform the following steps.

+> [!Note]

+> After enabling logs, the log files will be available to download after few minutes.

+1. Under **Name**, select the log file you want to download, and then, under **Action**, select **Download**.

+ :::image type="content" source="./media/how-to-server-logs-portal/4-how-to-serverlog.png" alt-text="Screenshot showing Server Logs - Download.":::

+ For HA enabled Azure database for MySQL Flexible server, server logs for standby server can be identified by another four-letter identifier after the hostname of the server as shown below.

+ :::image type="content" source="./media/how-to-server-logs-portal/5-how-to-serverlog.png" alt-text="Screenshot showing server Logs - HA logs.":::

+2. To download multiple log files at one time, under **Name**, select the files you want to download, and then above **Name**, select **Download**.

+ :::image type="content" source="./media/how-to-server-logs-portal/6-how-to-serverlog.png" alt-text="Screenshot showing server Logs - Download all.":::

+## Disable Server Logs

+1. From your Azure portal, select Server logs from Monitoring server pane.

+2. For disabling Server logs to file, Uncheck Enable. (The setting will disable logging for all the log_types available)

+ :::image type="content" source="./media/how-to-server-logs-portal/7-how-to-serverlog.png" alt-text="Screenshot showing server Logs - Disable.":::

+3. Select Save

+ :::image type="content" source="./media/how-to-server-logs-portal/8-how-to-serverlog.png" alt-text="Screenshot showing server Logs - Save.":::

+## Next steps

+- Learn more about [How to enable slow query logs](./tutorial-query-performance-insights.md#configure-slow-query-logs-by-using-the-azure-portal)

+- List and download [Server logs using Azure CLI](./how-to-server-logs-cli.md)

+| <b> Logical/user errors | Recovery from user errors, such as accidentally dropped tables or incorrectly updated data, involves performing a [point-in-time recovery](concepts-backup.md) (PITR), by restoring and recovering the data until the time just before the error had occurred.<br> <br> If you want to restore only a subset of databases or specific tables rather than all databases in the database server, you can restore the database server in a new instance, export the table(s) via [mysqldump](concepts-migrate-dump-restore.md), and then use [restore](concepts-migrate-dump-restore.md#restore-your-mysql-database-using-command-line) to restore those tables into your database. |

+- **Migrating from on-premises environment or Virtual machine** - Azure Database for MySQL doesn't support restore of physical backups, which makes logical backup and restore as the ONLY approach.

+- **Moving your backup storage from locally redundant to geo-redundant storage** - Azure Database for MySQL allows configuring locally redundant or geo-redundant storage for backup is only allowed during server create. Once the server is provisioned, you can't change the backup storage redundancy option. In order to move your backup storage from locally redundant storage to geo-redundant storage, dump and restore is the ONLY option.

+- **Migrating from alternative storage engines to InnoDB** - Azure Database for MySQL supports only InnoDB Storage engine, and therefore doesn't support alternative storage engines. If your tables are configured with other storage engines, convert them into the InnoDB engine format before migration to Azure Database for MySQL.

+- max_allowed_packet ΓÇô set to 1073741824 (that is, 1 GB) to prevent any overflow issue due to long rows.

+- Scale up Storage tier ΓÇô The IOPs for Azure Database for MySQL server increases progressively with the increase in storage tier. For faster loads, you may want to increase the storage tier to increase the IOPs provisioned. Do remember the storage can only be scaled up, not down.

+- [pass] The password for your database (note there's no space between -p and the password)

+For example, to back up a database named 'testdb' on your MySQL server with the username 'testuser' and with no password to a file testdb_backup.sql, use the following command. The command backs up the `testdb` database into a file called `testdb_backup.sql`, which contains all the SQL statements needed to re-create the database. Make sure that the username 'testuser' has at least the SELECT privilege for dumped tables, SHOW VIEW for dumped views, TRIGGER for dumped triggers, and LOCK TABLES if the `--single-transaction` option isn't used.

+To back up more than one database at once, use the `--database` switch and list the database names separated by spaces.

+### Restore your MySQL database using command-line

+Once you've created the target database, you can use the mysql command to restore the data into the specific newly created database from the dump file.

+Here's an example for how to use this **mysql** for **Single Server** :

+Here's an example for how to use this **mysql** for **Flexible Server** :

+>[!Note]

+>You can also use [MySQL Workbench client utility to restore MySQL database](./concepts-migrate-import-export.md#import-and-export-data-by-using-mysql-workbench).

+2. Select your database. Select the database name in the list on the left.

+3. Select the **Export** link. A new page appears to view the dump of database.

+4. In the Export area, select the **Select All** link to choose the tables in your database.

+5. In the SQL options area, select the appropriate options.

+6. Select the **Save as file** option and the corresponding compression option and then select the **Go** button. A dialog box should appear prompting you to save the file locally.

+2. In the phpMyAdmin setup page, select **Add** to add your Azure Database for MySQL server. Provide the connection details and log in information.

+3. Create an appropriately named database and select it on the left of the screen. To rewrite the existing database, select the database name, select all the check boxes beside the table names, and select **Drop** to delete the existing tables.

+4. Select the **SQL** link to show the page where you can type in SQL commands, or upload your SQL file.

+6. Select the **Go** button to export the backup, execute the SQL commands, and re-create your database.

+- If you're looking to migrate large databases with database sizes more than 1 TBs, you may want to consider using community tools like **mydumper/myloader** which supports parallel export and import. Learn [how to migrate large MySQL databases](https://techcommunity.microsoft.com/t5/azure-database-for-mysql/best-practices-for-migrating-large-databases-to-azure-database/ba-p/1362699).

+ * Certificate revocation list isn't accessible.

+ |Whether Network Address and Port Translation (NAPT) should be enabled for this data network. NAPT allows you to translate a large pool of private IP addresses for UEs to a small number of public IP addresses. The translation is performed at the point where traffic enters the data network, maximizing the utility of a limited supply of public IP addresses.</br></br>If you want to use [UE-to-UE traffic](private-5g-core-overview.md#ue-to-ue-traffic) in this data network, keep NAPT disabled. |**NAPT**|

+ :::image type="content" source="media/packet-core-dashboards/packet-core-http-stats-dashboard.png" alt-text="Screenshot of the H T T P stats dashboard. Panels related to H T T P statistics for the Session Management Function are shown." lightbox="media/packet-core-dashboards/packet-core-http-stats-dashboard.png":::

+ - The **4G Interfaces dashboard** displays request and response statistics recorded by each of the packet core instance's 4G interfaces. Note that this dashboard is only available for packet core instances supporting 4G devices.

+ :::image type="content" source="media/packet-core-dashboards/packet-core-4g-interfaces-dashboard.png" alt-text="Screenshot of the 4G Interfaces dashboard. Panels related to activity on the packet core instance's 4G interfaces are shown." lightbox="media/packet-core-dashboards/packet-core-4g-interfaces-dashboard.png":::

+### UE-to-UE traffic

+Azure Private 5G Core supports traffic flow from UE to UE through the user plane, allowing machine-to-machine (M2M) communication between 5G devices for a range of applications including robot control.

+An external router is responsible for hairpinning traffic from UE to UE over the N6 interface. This means that traffic leaving the UPF destined to a UE IP address will be routed back to the UPFΓÇÖs N6 IP address.

+### Multi-Operator Core Network (MOCN)

+Multi-operator Core Network (MOCN) aims to maximize resource usage by sharing a RAN between multiple core networks. Azure Private 5G Core supports MOCN, allowing multiple public land mobile networks (PLMNs) to be shared by a gNodeB (for 5G deployments) or eNodeB (for 4G deployments).

+In the context of private mobile networks, a single RAN can connect to both a private and a standard macro network, with traffic automatically routed to the appropriate core network based on the PLMN ID.

+| Azure Synapse | Microsoft.Synapse/privateLinkHubs | web |

+ Title: Available metadata for Power BI in the Microsoft Purview governance portal

+description: This reference article provides a list of metadata that is available for a Power BI tenant in the Microsoft Purview governance portal.

+# Available metadata

+This article has a list of the metadata that is available for a Power BI tenant in the Microsoft Purview governance portal.

+## Power BI

+| Metadata | Population method | Source of truth | Asset type | Editable | Upstream metadata |

+|--|-|-|--|-||

+| Classification | Manual | Microsoft Purview | All types | Yes | N/A |

+| Sensitivity Labels | Automatic | Microsoft Purview | All types | No | |

+| Glossary terms | Manual | Microsoft Purview | All types | Yes | N/A |

+| Collection | Automatic | Microsoft Purview | All types | | N/A |

+| Hierarchy | Automatic | Microsoft Purview | All types | No | N/A |

+| qualifiedName | Automatic | Microsoft Purview | All types | No | |

+| Asset Description | Automatic/Manual* | Microsoft Purview | All types | Yes | N/A |

+| Contacts - Expert | Manual | Microsoft Purview | All types | Yes | N/A |

+| Contacts - Owner | Manual | Microsoft Purview | All types | Yes | N/A |

+| name | Automatic | Power BI | Power BI Dashboard | Yes | dashboard.DisplayName |

+| isReadOnly | Automatic | Power BI | Power BI Dashboard | No | dashboard.IsReadOnly |

+| EmbedUrl | Automatic | Power BI | Power BI Dashboard | No | dashboard.EmbedUrl |

+| tileNames | Automatic | Power BI | Power BI Dashboard | No | TileTitles |

+| Lineage | Automatic | Power BI | Power BI Dashboard | No | N/A |

+| name | Automatic | Power BI | Power BI dataflow | Yes | dataflow.Name |

+| configured by | Automatic | Power BI | Power BI dataflow | No | dataflow.ConfiguredBy |

+| description | Automatic | Power BI | Power BI dataflow | Yes | dataflow.Description |

+| ModelUrl | Automatic | Power BI | Power BI dataflow | No | dataflow.ModelUrl |

+| ModifiedBy | Automatic | Power BI | Power BI dataflow | No | dataflow.ModifiedBy |

+| ModifiedDateTime | Automatic | Power BI | Power BI dataflow | No | dataflow.ModifiedDateTime |

+| Endorsement | Automatic | Power BI | Power BI dataflow | No | dataflow.EndorsementDetails |

+| name | Automatic | Power BI | Power BI Dataset | Yes | dataset.Name |

+| IsRefreshable | Automatic | Power BI | Power BI Dataset | No | dataset.IsRefreshable |

+| configuredBy | Automatic | Power BI | Power BI Dataset | No | dataset.ConfiguredBy |

+| contentProviderType | Automatic | Power BI | Power BI Dataset | No | dataset.ContentProviderType |

+| createdDate | Automatic | Power BI | Power BI Dataset | No | dataset.CreatedDateTime |

+| targetStorageMode | Automatic | Power BI | Power BI Dataset | No | dataset.TargetStorageMode |

+| Schema | Automatic/Manual | Power BI | Power BI Dataset | | |

+| Lineage | Automatic | Microsoft Purview | Power BI Dataset | No | |

+| description | Automatic | Power BI | Power BI Dataset | Yes | dataset.Description |

+| Endorsement | Automatic | Power BI | Power BI Dataset | No | dataset.EndorsementDetails |

+| name | Automatic | Power BI | Power BI Report | Yes | report.Name |

+| description | Automatic | Power BI | Power BI Report | Yes | report.Description |

+| createdDateTime | Automatic | Power BI | Power BI Report | No | report.CreatedDateTime |

+| WebUrl | Automatic | Power BI | Power BI Report | No | report.WebUrl |

+| EmbedUrl | Automatic | Power BI | Power BI Report | No | report.EmbedUrl |

+| PBIDatasetId | Automatic | Power BI | Power BI Report | No | report.DatasetId; |

+| modifiedBy | Automatic | Power BI | Power BI Report | No | report.ModifiedBy |

+| modifiedDateTime | Automatic | Power BI | Power BI Report | No | report.ModifiedDateTime |

+| reportType | Automatic | Power BI | Power BI Report | No | report.ReportType |

+| Endorsement | Automatic | Power BI | Power BI Report | No | report.EndorsementDetails |

+| Lineage | Automatic | Microsoft Purview | Power BI Report | No | N/A |

+| name | Automatic | Power BI | Power BI Workspace | Yes | workspace.Name |

+| Description | Automatic | Power BI | Power BI Workspace | Yes | workspace.Description |

+| state | Automatic | Power BI | Power BI Workspace | No | workspace.State |

+| type | Automatic | Power BI | Power BI Workspace | No | ResourceType.Workspace |

+| IsReadOnly | Automatic | Power BI | Power BI Workspace | No | workspace.IsReadOnly |

+| IsOnDedicatedCapacity | Automatic | Power BI | Power BI Workspace | No | workspace.IsOnDedicatedCapacity |

+## Next steps

+- [Register and scan a Power BI tenant](register-scan-power-bi-tenant.md)

+- [Register and scan Power BI across tenants](register-scan-power-bi-tenant-cross-tenant.md)

+- [Register and scan Power BI troubleshooting](register-scan-power-bi-tenant-troubleshoot.md)

+As an example, if you named your classification **contoso.hr.employee\_ID**, the friendly name is stored

+description: This article provides best practices for deploying Microsoft Purview (formerly Azure Purview) in your data estate. The Microsoft Purview Data Map and governance portal enable any user to register, discover, understand, and consume data sources.

+This article is a guide to successfully deploying Microsoft Purview (formerly Azure Purview) into production in your data estate. It's intended to help you strategize and phase your deployment from research to hardening your production environment, and is best used in tandem with our [deployment checklist](tutorial-azure-purview-checklist.md).

+If you're looking for a strictly technical deployment guide, use the [deployment checklist](tutorial-azure-purview-checklist.md).

+If you're creating a plan to deploy Microsoft Purview and want to consider best practices as you develop your deployment strategy, then follow the article below. This guide outlines tasks can be completed in phases over the course of a month or more to develop your deployment process for Microsoft Purview. Even organizations who have already deployed Microsoft Purview can use this guide to ensure they're getting the most out of their investment.

+A well-planned deployment of your data governance platform, can give the following benefits:

+- Maximized return on investment

+This guide provides insight on a full deployment lifecycle, from initial planning to a mature environment by following these stages:

+| Stage | Description |

+|-|-|

+|[Identify objectives and goals](#identify-objectives-and-goals)|Consider what your entire organization wants and needs from data governance.|

+|[Gathering questions](#gathering-questions)|What questions might you and your team have as you get started, and where can you look to begin addressing them?|

+|[Deployment models](#deployment-models)|Customize your Microsoft Purview deployment to your data estate.|

+|[Create a process to move to production](#create-a-process-to-move-to-production)|Create a phased deployment strategy tailored to your organization.|

+|[Platform hardening](#platform-hardening)|Continue to grow your deployment to maturity.|

+Many of Microsoft Purview's applications and features have their own individual best practices pages as well. They're referenced often throughout this deployment guide, but you can find all of them in the table of contents under **Concepts** and then **Best practices and guidelines**.

+Some of the common data governance objectives that you might want to identify in the early phases to create a comprehensive data governance experience include:

+### Identify key scenarios

+Microsoft Purview governance services can be used to centrally manage data governance across an organizationΓÇÖs data estate spanning cloud and on-premises environments. To have a successful implementation, you must identify key scenarios that are critical to the business. These scenarios can cross business unit boundaries or affect multiple user personas either upstream or downstream.

+These scenarios can be written up in various ways, but you should include at least these five dimensions:

+1. Persona ΓÇô Who are the users?

+2. Source system ΓÇô What are the data sources such as Azure Data Lake Storage Gen2 or Azure SQL Database?

+3. Impact Area ΓÇô What is the category of this scenario?

+4. Detail scenarios ΓÇô How the users use Microsoft Purview to solve problems?

+5. Expected outcome ΓÇô What is the success criteria?

+The scenarios must be specific, actionable, and executable with measurable results. Some example scenarios that you can use:

+|Scenario|Detail|Persona|

+||||

+|Catalog business-critical assets|I need to have information about each data sets to have a good understanding of what it is. This scenario includes both business and technical metadata data about the data set in the catalog. The data sources include Azure Data Lake Storage Gen2, Azure Synapse DW, and/or Power BI. This scenario also includes on-premises resources such as SQL Server.|Business Analyst, Data Scientist, Data Engineer|

+|Discover business-critical assets|I need to have a search engine that can search through all metadata in the catalog. I should be able to search using technical term, business term with either simple or complex search using wildcard.|Business Analyst, Data Scientist, Data Engineer, Data Admin|

+|Track data to understand its origin and troubleshoot data issues|I need to have data lineage to track data in reports, predictions, or models back to its original source. I also need to understand the changes made to the data, and where the data has resided throughout the data life cycle. This scenario needs to support prioritized data pipelines Azure Data Factory and Databricks.|Data Engineer, Data Scientist|

+|Enrich metadata on critical data assets|I need to enrich the data set in the catalog with technical metadata that is generated automatically. Classification and labeling are some examples.|Data Engineer, Domain/Business Owner|

+|Govern data assets with friendly user experience|I need to have a Business glossary for business-specific metadata. The business users can use Microsoft Purview for self-service scenarios to annotate their data and enable the data to be discovered easily via search.|Domain/Business Owner, Business Analyst, Data Scientist, Data Engineer|

+### Integration points with Microsoft Purview

+ItΓÇÖs likely that a mature organization already has an existing data catalog. The key question is whether to continue to use the existing technology and sync with the Microsoft Purview Data Map and Data Catalog or not. To handle syncing with existing products in an organization, [Microsoft Purview provides Atlas REST APIs](tutorial-using-rest-apis.md). Atlas APIs provide a powerful and flexible mechanism handling both push and pull scenarios. Information can be published to Microsoft Purview using Atlas APIs for bootstrapping or to push latest updates from another system into Microsoft Purview. The information available in Microsoft Purview can also be read using Atlas APIs and then synced back to existing products.

+For other integration scenarios such as ticketing, custom user interface, and orchestration you can use Atlas APIs and Kafka endpoints. In general, there are four integration points with Microsoft Purview:

+* **Data Asset** ΓÇô This enables Microsoft Purview to scan a storeΓÇÖs assets in order to enumerate what those assets are and collect any readily available metadata about them. So for SQL this could be a list of DBs, tables, stored procedures, views and config data about them kept in places like `sys.tables`. For something like Azure Data Factory (ADF) this could be enumerating all the pipelines and getting data on when they were created, last run, current state.

+* **Lineage** ΓÇô This enables Microsoft Purview to collect information from an analysis/data mutation system on how data is moving around. For something like Spark this could be gathering information from the execution of a notebook to see what data the notebook ingested, how it transformed it and where it outputted it. For something like SQL, it could be analyzing query logs to reverse engineer what mutation operations were executed and what they did. We support both push and pull based lineage depending on the needs.

+* **Classification** ΓÇô This enables Microsoft Purview to take physical samples from data sources and run them through our classification system. The classification system figures out the semantics of a piece of data. For example, we may know that a file is a Parquet file and has three columns and the third one is a string. But the classifiers we run on the samples will tell us that the string is a name, address, or phone number. Lighting up this integration point means that we've defined how Microsoft Purview can open up objects like notebooks, pipelines, parquet files, tables, and containers.

+* **Embedded Experience** ΓÇô Products that have a ΓÇ£studioΓÇ¥ like experience (such as ADF, Synapse, SQL Studio, PBI, and Dynamics) usually want to enable users to discover data they want to interact with and also find places to output data. Microsoft PurviewΓÇÖs catalog can help to accelerate these experiences by providing an embedding experience. This experience can occur at the API or the UX level at the partnerΓÇÖs option. By embedding a call to Microsoft Purview, the organization can take advantage of Microsoft PurviewΓÇÖs map of the data estate to find data assets, see lineage, check schemas, look at ratings, contacts etc.

+## Gathering questions

+Once your organization agrees on the high-level objectives and goals, there will be many questions from multiple groups. ItΓÇÖs crucial to gather these questions in order to craft a plan to address all of the concerns. Make sure to [include relevant groups](#include-the-right-stakeholders) as you gather these questions. You can use our documentation to start answering them.

+Some example questions that you may run into during the initial phase:

+- What are the main data sources and data systems in our organization?

+- [What data sources are supported?](microsoft-purview-connector-overview.md)

+- For data sources that aren't supported yet by Microsoft Purview, what are my options?

+- [How should we budget for Microsoft Purview?](concept-guidelines-pricing.md)

+- [How many Microsoft Purview instances do we need?](concept-best-practices-accounts.md)

+- [Who will use Microsoft Purview, and what roles will they have?](catalog-permissions.md)

+- [Who can scan new data sources?](catalog-permissions.md)

+- [Who can modify content inside of Microsoft Purview?](catalog-permissions.md)

+- What processes can I use to improve the data quality in Microsoft Purview?

+- How to bootstrap the platform with existing critical assets, [glossary terms](concept-best-practices-glossary.md), and contacts?

+- How to integrate with existing systems?

+- [How can we secure Microsoft Purview?](concept-best-practices-security.md)

+- How can we gather feedback and build a sustainable process?

+- [What can we do in a disaster situation?](concept-best-practices-migration.md)

+- [We're already using Azure Data Catalog, can we migrate to Microsoft Purview?](../data-catalog/data-catalog-migration-to-azure-purview.md)

+Even if you might not have the answer to most of these questions right away, gathering questions can help your organization to frame this project and ensure all ΓÇ£must-haveΓÇ¥ requirements can be met.

+### Include the right stakeholders

+To ensure the success of implementing Microsoft Purview for your entire organization, itΓÇÖs important to involve the right stakeholders. Only a few people are involved in the initial phase. However, as the scope expands, you'll require more personas to contribute to the project and provide feedback.

+For more information, see our [accounts architecture best practices guide](concept-best-practices-accounts.md) and our [default account guide](concept-default-purview-account.md).

+## Create a process to move to production

+Some organizations may decide to keep things simple by working with a single production version of Microsoft Purview. They probably donΓÇÖt need to go beyond discovery, search, and browse scenarios. However, most organizations that want to deploy Microsoft Purview across various business units will want to have some form of process and control.

+Below we've provided a potential four phase deployment plan that includes tasks, helpful links, and acceptance criteria for each phase:

+1. [Phase 1: Pilot](#phase-1-pilot)

+1. [Phase 2: Minimum viable product](#phase-2-minimum-viable-product)

+1. [Phase 3: Pre-production](#phase-3-pre-production)

+1. [Phase 4: Production](#phase-4-production)

+### Phase 1: Pilot

+#### Tasks to complete

+|[Navigating the Microsoft Purview governance portal](use-azure-purview-studio.md)|Understand how to use Microsoft Purview from the home page.|One Day|

+|[Configure ADF for lineage](how-to-link-azure-data-factory.md)|Identify key pipelines and data assets. Gather all information required to connect to an internal ADF account.|One Day|

+|Scan a data source such as [Azure Data Lake Storage Gen2](register-scan-adls-gen2.md) or a [SQL server.](tutorial-register-scan-on-premises-sql-server.md)|Add the data source and set up a scan. Ensure the scan successfully detects all assets.|Two Day|

+|[Search](how-to-search-catalog.md) and [browse](how-to-browse-catalog.md)|Allow end users to access Microsoft Purview and perform end-to-end search and browse scenarios.|One Day|

+#### Other helpful links

+- [Create a Microsoft Purview account](create-catalog-portal.md)

+- [Create a collection](quickstart-create-collection.md)

+- [Concept: Permissions and access](catalog-permissions.md)

+- [Microsoft Purview product glossary](reference-azure-purview-glossary.md)

+#### Acceptance criteria

+### Phase 2: Minimum viable product

+#### Tasks to complete

+|[Add sensitive labels and scan](how-to-automatically-label-your-content.md)|This might be optional for some organizations, depending on the usage of Labeling from Microsoft 365.|1-2 Weeks|

+|[Get classification and sensitive insights](concept-insights.md)|For reporting and insight in Microsoft Purview, you can access this functionality to get various reports and provide presentation to management.|One Day|

+#### Other helpful links

+- [Collections architecture best practices](concept-best-practices-collections.md)

+- [Classification best practices](concept-best-practices-classification.md)

+- [Glossary best practices](concept-best-practices-glossary.md)

+- [Labeling best practices](concept-best-practices-sensitivity-labels.md)

+- [Asset lifecycle best practices](concept-best-practices-asset-lifecycle.md)

+#### Acceptance criteria

+### Phase 3: Pre-production

+#### Tasks to complete

+|[Refine your scan using scan rule set](create-a-scan-rule-set.md)|Your organization will have many data sources for pre-production. ItΓÇÖs important to pre-define key criteria for scanning so that classifications and file extension can be applied consistently across the board.|1-2 Days|

+|[Assess region availability for scan for each of your sources by checking source pages](microsoft-purview-connector-overview.md)|Depending on the region of the data sources and organizational requirements on compliance and security, you may want to consider what regions must be available for scanning.|One Day|

+|[Understand firewall concept when scanning](concept-best-practices-security.md#network-security)|This step requires some exploration of how the organization configures its firewall and how Microsoft Purview can authenticate itself to access the data sources for scanning.|One Day|

+|[Understand Private Link concept when scanning](catalog-private-link.md)|If your organization uses Private Link, you must lay out the foundation of network security to include Private Link as a part of the requirements.|One Day|

+|[Use Microsoft Purview REST API for integration scenarios](tutorial-using-rest-apis.md)|If you have requirements to integrate Microsoft Purview with other third party technologies such as orchestration or ticketing system, you may want to explore REST API area.|1-4 Weeks|

+|[Understand Microsoft Purview pricing](concept-guidelines-pricing.md)|This step will provide the organization important financial information to make decision.|1-5 Days|

+#### Other helpful links

+- [Labeling best practices](concept-best-practices-sensitivity-labels.md)

+- [Network architecture best practices](concept-best-practices-network.md)

+#### Acceptance criteria

+### Phase 4: Production

+#### Tasks to complete

+|[Enable Private Link](catalog-private-link.md)|If this is optional when Private Link is used. Otherwise, you can skip this as itΓÇÖs a must-have criterion when Private is enabled.|1-5 Days|

+|[Create automated workflow](concept-workflow.md)|Workflow is important to automate process such as approval, escalation, review and issue management.|2-3 Weeks|

+|Create operation documentation|Data governance isn't a one-time project. It's an ongoing program to fuel data-driven decision making and creating opportunities for business. It's critical to document key procedure and business standards.|One Week|

+#### Other helpful links

+- [Manage workflow runs](how-to-workflow-manage-runs.md)

+- [Workflow requests and approvals](how-to-workflow-manage-requests-approvals.md)

+- [Manage integration runtimes](manage-integration-runtimes.md)

+- [Request access to a data asset](how-to-request-access.md)

+#### Acceptance criteria

+* [Use REST APIs](tutorial-atlas-2-2-apis.md) to export critical metadata and properties for backup and recovery

+* [Use workflow](how-to-workflow-business-terms-approval.md) to automate ticketing and eventing to avoid human errors

+* [Use policies](concept-data-owner-policies.md) to manage access to data assets through the Microsoft Purview governance portal.

+## Lifecycle considerations

+Another important aspect to include in your production process is how classifications and labels can be migrated. Microsoft Purview has over 90 system classifiers. You can apply system or custom classifications on file, table, or column assets. Classifications are like subject tags and are used to mark and identify content of a specific type found within your data estate during scanning. Sensitivity labels are used to identify the categories of classification types within your organizational data, and then group the policies you wish to apply to each category. It makes use of the same sensitive information types as Microsoft 365, allowing you to stretch your existing security policies and protection across your entire content and data estate. It can scan and automatically classify documents. For example, if you have a file named multiple.docx and it has a National ID number in its content, Microsoft Purview will add classification such as EU National Identification Number in the Asset Detail page.

+In the Microsoft Purview Data Map, there are several areas where the Catalog Administrators need to ensure consistency and maintenance best practices over its life cycle:

+* **Data assets** ΓÇô Data sources will need to be rescanned across environments. ItΓÇÖs not recommended to scan only in development and then regenerate them using APIs in Production. The main reason is that the Microsoft Purview scanners do a lot more ΓÇ£wiringΓÇ¥ behind the scenes on the data assets, which could be complex to move them to a different Microsoft Purview instance. ItΓÇÖs much easier to just add the same data source in production and scan the sources again. The general best practice is to have documentation of all scans, connections, and authentication mechanisms being used.

+* **Scan rule sets** ΓÇô This is your collection of rules assigned to specific scan such as file type and classifications to detect. If you donΓÇÖt have that many scan rule sets, itΓÇÖs possible to just re-create them manually again via Production. This will require an internal process and good documentation. However, if your rule sets change on a daily or weekly basis, this could be addressed by exploring the REST API route.

+* **Custom classifications** ΓÇô Your classifications may not also change regularly. During the initial phase of deployment, it may take some time to understand various requirements to come up with custom classifications. However, once settled, this will require little change. So the recommendation here's to manually migrate any custom classifications over or use the REST API.

+* **Glossary** ΓÇô ItΓÇÖs possible to export and import glossary terms via the UX. For automation scenarios, you can also use the REST API.

+* **Resource set pattern policies** ΓÇô This functionality is advanced for any typical organizations to apply. In some cases, your Azure Data Lake Storage has folder naming conventions and specific structure that may cause problems for Microsoft Purview to generate the resource set. Your business unit may also want to change the resource set construction with more customizations to fit the business needs. For this scenario, itΓÇÖs best to keep track of all changes via REST API, and document the changes through external versioning platform.

+* **Role assignment** ΓÇô This is where you control who has access to Microsoft Purview and which permissions they have. Microsoft Purview also has REST API to support export and import of users and roles but this isn't Atlas API-compatible. The recommendation is to assign an Azure Security Group and manage the group membership instead.

+>[!IMPORTANT]

+>Most data sources have additional information and prerequisites to register and scan them in Microsoft Purview. For a list of all available sources, and links to source-specific instructions for registeration and scanning, see our [supported sources article.](microsoft-purview-connector-overview.md#microsoft-purview-data-map-available-data-sources)

+You can view all registered sources on the **Data Map** tab of the Microsoft Purview governance portal.

+There are two view types:

+- [The map view](#map-view)

+- [The list view](#table-view)

+>[!TIP]

+>You can remove sources from a hierarchy by selecting *None* for the parent. Unparented sources are grouped in a dotted box in the map view with no arrows linking them to parents.

+||[HDFS](register-scan-hdfs.md)|[Yes](register-scan-hdfs.md)| [Yes](register-scan-hdfs.md)| No | No| No |

+ Title: Connect to and manage HDFS

+description: This guide describes how to connect to HDFS in Microsoft Purview, and use Microsoft Purview's features to scan and manage your HDFS source.

+# Connect to and manage HDFS in Microsoft Purview

+This article outlines how to register Hadoop Distributed File System (HDFS), and how to authenticate and interact with HDFS in Microsoft Purview. For more information about Microsoft Purview, read the [introductory article](overview.md).

+## Supported capabilities

+|**Metadata Extraction**|**Full Scan**|**Incremental Scan**|**Scoped Scan**|**Classification**|**Access Policy**|**Lineage**|**Data Sharing**|

+|||||||||

+| [Yes](#register)| [Yes](#scan)| [Yes](#scan) | [Yes](#scan) | [Yes](#scan) | No| No | No|

+When scanning HDFS source, Microsoft Purview supports extracting technical metadata including HDFS:

+- Namenode

+- Folder

+- File

+- Resource set

+When setting up scan, you can choose to scan the entire HDFS or selective folders. Learn about the supported file format [here](microsoft-purview-connector-overview.md#file-types-supported-for-scanning).

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- An active [Microsoft Purview account](create-catalog-portal.md).

+- You need Data Source Administrator and Data Reader permissions to register a source and manage it in the Microsoft Purview governance portal. For more information about permissions, see [Access control in Microsoft Purview](catalog-permissions.md).

+- Set up the latest [self-hosted integration runtime](https://www.microsoft.com/download/details.aspx?id=39717). For more information, see [the create and configure a self-hosted integration runtime guide](manage-integration-runtimes.md). The minimal supported Self-hosted Integration Runtime version is 5.20.8235.2.

+ * Ensure Visual C++ Redistributable for Visual Studio 2012 Update 4 is installed on the self-hosted integration runtime machine. If you don't have this update installed, [you can download it here](https://www.microsoft.com/download/details.aspx?id=30679).

+ * Ensure JRE or OpenJDK is installed on the self-hosted integration runtime machine for parsing Parquet and ORC files. Learn more from [here](manage-integration-runtimes.md#java-runtime-environment-installation).

+ * To set up your environment to enable Kerberos authentication, see the [Use Kerberos authentication for the HDFS connector](#use-kerberos-authentication-for-the-hdfs-connector) section.

+## Register

+This section describes how to register HDFS in Microsoft Purview using the [Microsoft Purview governance portal](https://web.purview.azure.com/).

+### Steps to register

+To register a new HDFS source in your data catalog, follow these steps:

+1. Navigate to your Microsoft Purview account in the [Microsoft Purview governance portal](https://web.purview.azure.com/resource/).

+1. Select **Data Map** on the left navigation.

+1. Select **Register**

+1. On Register sources, select **HDFS**. Select **Continue**.

+On the **Register sources (HDFS)** screen, follow these steps:

+1. Enter a **Name** that the data source will be listed within the Catalog.

+1. Enter the **Cluster URL** of the HDFS NameNode in the form of `https://<namenode>:<port>` or `http://<namenode>:<port>`, e.g. `https://namenodeserver.com:50470` or `http://namenodeserver.com:50070`.

+1. Select a collection or create a new one (Optional)

+1. Finish to register the data source.

+ :::image type="content" source="media/register-scan-hdfs/register-sources.png" alt-text="Screenshot of HDFS source registration in Purview." border="true":::

+## Scan

+Follow the steps below to scan HDFS to automatically identify assets. For more information about scanning in general, see our [introduction to scans and ingestion](concept-scans-and-ingestion.md).

+### Authentication for a scan

+The supported authentication type for an HDFS source is **Kerberos authentication**.

+### Create and run scan

+To create and run a new scan, follow these steps:

+1. Make sure a self-hosted integration runtime is set up. If it isn't set up, use the steps mentioned [here](./manage-integration-runtimes.md) to create a self-hosted integration runtime.

+1. Navigate to **Sources**.

+1. Select the registered HDFS source.

+1. Select **+ New scan**.

+1. On "**Scan *source_name***"" page, provide the below details:

+ 1. **Name**: The name of the scan

+ 1. **Connect via integration runtime**: Select the configured self-hosted integration runtime. See setup requirements in [Prerequisites](#prerequisites) section.

+ 1. **Credential**: Select the credential to connect to your data source. Make sure to:

+ * Select **Kerberos Authentication** while creating a credential.

+ * Provide the user name in the format of `<username>@<domain>.com` in the User name input field. Learn more from [Use Kerberos authentication for the HDFS connector](#use-kerberos-authentication-for-the-hdfs-connector).

+ * Store the user password used to connect to HDFS in the secret key.

+ :::image type="content" source="media/register-scan-hdfs/scan.png" alt-text="Screenshot of HDFS scan configurations in Purview." border="true":::

+1. Select **Test connection**.

+1. Select **Continue**.

+1. On "**Scope your scan**" page, select the path(s) that you want to scan.

+1. On "**Select a scan rule set**" page, select the scan rule set you want to use for schema extraction and classification. You can choose between the system default, existing custom rule sets, or create a new rule set inline. Learn more from [Create a scan rule set](create-a-scan-rule-set.md).

+1. On "**Set a scan trigger**" page, choose your **scan trigger**. You can set up a schedule or ran the scan once.

+1. Review your scan and select **Save and Run**.

+## Use Kerberos authentication for the HDFS connector

+There are two options for setting up the on-premises environment to use Kerberos authentication for the HDFS connector. You can choose the one that better fits your situation.

+* Option 1: [Join a self-hosted integration runtime machine in the Kerberos realm](#kerberos-join-realm)

+* Option 2: [Enable mutual trust between the Windows domain and the Kerberos realm](#kerberos-mutual-trust)

+For either option, make sure you turn on webhdfs for Hadoop cluster:

+1. Create the HTTP principal and keytab for webhdfs.

+ > [!IMPORTANT]

+ > The HTTP Kerberos principal must start with "**HTTP/**" according to Kerberos HTTP SPNEGO specification. Learn more from [here](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#HDFS_Configuration_Options).

+ ```bash

+ Kadmin> addprinc -randkey HTTP/<namenode hostname>@<REALM.COM>

+ Kadmin> ktadd -k /etc/security/keytab/spnego.service.keytab HTTP/<namenode hostname>@<REALM.COM>

+ ```

+2. HDFS configuration options: add the following three properties in `hdfs-site.xml`.

+ ```xml

+ <property>

+ <name>dfs.webhdfs.enabled</name>

+ <value>true</value>

+ </property>

+ <property>

+ <name>dfs.web.authentication.kerberos.principal</name>

+ <value>HTTP/_HOST@<REALM.COM></value>

+ </property>

+ <property>

+ <name>dfs.web.authentication.kerberos.keytab</name>

+ <value>/etc/security/keytab/spnego.service.keytab</value>

+ </property>

+ ```

+### <a name="kerberos-join-realm"></a>Option 1: Join a self-hosted integration runtime machine in the Kerberos realm

+#### Requirements

+* The self-hosted integration runtime machine needs to join the Kerberos realm and canΓÇÖt join any Windows domain.

+#### How to configure

+**On the KDC server:**

+Create a principal, and specify the password.

+> [!IMPORTANT]

+> The username should not contain the hostname.

+```bash

+Kadmin> addprinc <username>@<REALM.COM>

+```

+**On the self-hosted integration runtime machine:**

+1. Run the Ksetup utility to configure the Kerberos Key Distribution Center (KDC) server and realm.

+ The machine must be configured as a member of a workgroup, because a Kerberos realm is different from a Windows domain. You can achieve this configuration by setting the Kerberos realm and adding a KDC server by running the following commands. Replace *REALM.COM* with your own realm name.

+ ```cmd

+ C:> Ksetup /setdomain REALM.COM

+ C:> Ksetup /addkdc REALM.COM <your_kdc_server_address>

+ ```

+ After you run these commands, restart the machine.

+2. Verify the configuration with the `Ksetup` command. The output should be like:

+ ```cmd

+ C:> Ksetup

+ default realm = REALM.COM (external)

+ REALM.com:

+ kdc = <your_kdc_server_address>

+ ```

+**In your Purview account:**

+* Configure a credential with Kerberos authentication type with your Kerberos principal name and password to scan the HDFS. For configuration details, check the credential setting part in [Scan section](#scan).

+### <a name="kerberos-mutual-trust"></a>Option 2: Enable mutual trust between the Windows domain and the Kerberos realm

+#### Requirements

+* The self-hosted integration runtime machine must join a Windows domain.

+* You need permission to update the domain controller's settings.

+#### How to configure

+> [!NOTE]

+> Replace REALM.COM and AD.COM in the following tutorial with your own realm name and domain controller.

+**On the KDC server:**

+1. Edit the KDC configuration in the *krb5.conf* file to let KDC trust the Windows domain by referring to the following configuration template. By default, the configuration is located at */etc/krb5.conf*.

+ ```config

+ [logging]

+ default = FILE:/var/log/krb5libs.log

+ kdc = FILE:/var/log/krb5kdc.log

+ admin_server = FILE:/var/log/kadmind.log

+

+ [libdefaults]

+ default_realm = REALM.COM

+ dns_lookup_realm = false

+ dns_lookup_kdc = false

+ ticket_lifetime = 24h

+ renew_lifetime = 7d

+ forwardable = true

+

+ [realms]

+ REALM.COM = {

+ kdc = node.REALM.COM

+ admin_server = node.REALM.COM

+ }

+ AD.COM = {

+ kdc = windc.ad.com

+ admin_server = windc.ad.com

+ }

+

+ [domain_realm]

+ .REALM.COM = REALM.COM

+ REALM.COM = REALM.COM

+ .ad.com = AD.COM

+ ad.com = AD.COM

+

+ [capaths]

+ AD.COM = {

+ REALM.COM = .

+ }

+ ```

+ After you configure the file, restart the KDC service.

+2. Prepare a principal named *krbtgt/REALM.COM\@AD.COM* in the KDC server with the following command:

+ ```cmd

+ Kadmin> addprinc krbtgt/REALM.COM@AD.COM

+ ```

+3. In the *hadoop.security.auth_to_local* HDFS service configuration file, add `RULE:[1:$1@$0](.*\@AD.COM)s/\@.*//`.

+**On the domain controller:**

+1. Run the following `Ksetup` commands to add a realm entry:

+ ```cmd

+ C:> Ksetup /addkdc REALM.COM <your_kdc_server_address>

+ C:> ksetup /addhosttorealmmap HDFS-service-FQDN REALM.COM

+ ```

+2. Establish trust from the Windows domain to the Kerberos realm. [password] is the password for the principal *krbtgt/REALM.COM\@AD.COM*.

+ ```cmd

+ C:> netdom trust REALM.COM /Domain: AD.COM /add /realm /password:[password]

+ ```

+3. Select the encryption algorithm that's used in Kerberos.

+ 1. Select **Server Manager** > **Group Policy Management** > **Domain** > **Group Policy Objects** > **Default or Active Domain Policy**, and then select **Edit**.

+ 1. On the **Group Policy Management Editor** pane, select **Computer Configuration** > **Policies** > **Windows Settings** > **Security Settings** > **Local Policies** > **Security Options**, and then configure **Network security: Configure Encryption types allowed for Kerberos**.

+ 1. Select the encryption algorithm you want to use when you connect to the KDC server. You can select all the options.

+ :::image type="content" source="media/register-scan-hdfs/config-encryption-types-for-kerberos.png" alt-text="Screenshot of the Network security: Configure encryption types allowed for Kerberos pane.":::

+ 1. Use the `Ksetup` command to specify the encryption algorithm to be used on the specified realm.

+ ```cmd

+ C:> ksetup /SetEncTypeAttr REALM.COM DES-CBC-CRC DES-CBC-MD5 RC4-HMAC-MD5 AES128-CTS-HMAC-SHA1-96 AES256-CTS-HMAC-SHA1-96

+ ```

+4. Create the mapping between the domain account and the Kerberos principal, so that you can use the Kerberos principal in the Windows domain.

+ 1. Select **Administrative tools** > **Active Directory Users and Computers**.

+ 1. Configure advanced features by selecting **View** > **Advanced Features**.

+ 1. On the **Advanced Features** pane, right-click the account to which you want to create mappings and, on the **Name Mappings** pane, select the **Kerberos Names** tab.

+ 1. Add a principal from the realm.

+ :::image type="content" source="media/register-scan-hdfs/map-security-identity.png" alt-text="Screenshot of the Security Identity Mapping pane.":::

+**On the self-hosted integration runtime machine:**

+* Run the following `Ksetup` commands to add a realm entry.

+ ```cmd

+ C:> Ksetup /addkdc REALM.COM <your_kdc_server_address>

+ C:> ksetup /addhosttorealmmap HDFS-service-FQDN REALM.COM

+ ```

+**In your Purview account:**

+* Configure a credential with Kerberos authentication type with your Kerberos principal name and password to scan the HDFS. For configuration details, check the credential setting part in [Scan section](#scan).

+## Known limitations

+Currently, HDFS connector doesn't support custom resource set pattern rule for [advanced resource set](concept-resource-sets.md#advanced-resource-sets), the built-in resource set patterns will be applied.

+[Sensitivity label](create-sensitivity-label.md) is not yet supported.

+## Next steps

+Now that you've registered your source, follow the below guides to learn more about Microsoft Purview and your data.

+- [Search Data Catalog](how-to-search-catalog.md)

+- [Data Estate Insights in Microsoft Purview](concept-insights.md)

+For a list of metadata available for Power BI, see our [available metadata documentation](available-metadata.md).

+ 2. Microsoft Purview managed identity has **get** and **list** access to secrets.

+ 2. Microsoft Purview managed identity has **get** and **list** access to secrets.

+ > If you switch the configuration of a scan to include or exclude a personal workspace, you trigger a full scan of the Power BI source.

+For a list of metadata available for Power BI, see our [available metadata documentation](available-metadata.md).

+1. Make sure your [Power BI Metadata model is up to date by enabling metadata scanning.](/power-bi/admin/service-admin-metadata-scanning-setup#enable-tenant-settings-for-metadata-scanning)

+1. Make sure your [Power BI Metadata model is up to date by enabling metadata scanning.](/power-bi/admin/service-admin-metadata-scanning-setup#enable-tenant-settings-for-metadata-scanning)

+1. Make sure your [Power BI Metadata model is up to date by enabling metadata scanning.](/power-bi/admin/service-admin-metadata-scanning-setup#enable-tenant-settings-for-metadata-scanning)

+ > Switching the configuration of a scan to include or exclude a personal workspace will trigger a full scan of Power BI source.

+ > Switching the configuration of a scan to include or exclude a personal workspace will trigger a full scan of Power BI source.

+This article lists prerequisites that help you get started quickly on planning and deployment for your Microsoft Purview (formerly Azure Purview) account.

+If you are creating a plan to deploy Microsoft Purview, and also want to consider best practices as you develop your deployment strategy, then use [our deployment best practices guide](deployment-best-practices.md) to get started.

+If you are looking for a strictly technical deployment guide, this deployment checklist is for you.

+### When the same route is learned over ExpressRoute, VPN or SDWAN, which network is preferred?

+The Microsoft Threat Intelligence Matching Analytics matches the log sources in the following tables with domain, IP and Microsoft Defender Threat Intelligence (MDTI) indicators.

+#### [Domain](#tab/domain)

+| [CEF](connect-common-event-format.md) | Matching is done for all CEF logs that are ingested in the Log Analytics **CommonSecurityLog** table, except when the `DeviceVendor` is `Cisco`. <br><br>To match Microsoft generated threat intelligence with domain indicators in CEF logs, make sure to map the domain in the `RequestURL` field of the CEF log.|

+| [DNS](./data-connectors-reference.md#windows-dns-server-preview) | Matching is done for all DNS logs that are lookup queries from clients to DNS services (`SubType == "LookupQuery"`). DNS queries are only processed for IPv4 (`QueryType="A"`) and IPv6 queries (`QueryType="AAAA"`).<br><br>To match Microsoft generated threat intelligence with domain indicators in DNS logs, no manual mapping of columns is needed. All columns are standard from Windows DNS Server, and the domains will be in the `Name` column by default.|

+| [Syslog](connect-syslog.md) | Matching is only done for Syslog events where the `Facility` is `cron`. <br><br>To match Microsoft generated threat intelligence with domain indicators from Syslog, no manual mapping of columns is needed. The details originate from the `SyslogMessage` field by default and the rule parses the domain directly from it.|

+#### [IPv4](#tab/ipv4)

+| Log source | Description |

+| | |

+|[CEF](connect-common-event-format.md) | Matching is done for all CEF logs that are ingested in the Log Analytics **CommonSecurityLog** table, except when the `DeviceVendor` is `Cisco`. <br><br>To match Microsoft generated threat intelligence with IP indicators in CEF logs, no manual mapping needs to be done. The IP is populated in the `DestinationIP` field by default.|

+| [DNS](./data-connectors-reference.md#windows-dns-server-preview) | Matching is done for all DNS logs that are lookup queries from clients to DNS services (`SubType == "LookupQuery"`). DNS queries are only processed for IPv4 (`QueryType="A"`). <br><br>To match Microsoft generated threat intelligence with IP indicators in DNS logs, no manual mapping of columns is needed. All columns are standard from Windows DNS Server, and the IPs will be in the `IPAddresses` column by default.|

+| [Syslog](connect-syslog.md) | Matching is only done for Syslog events where the `Facility` is `cron`. <br><br>To match Microsoft generated threat intelligence with IP indicators from Syslog, no manual mapping of columns is needed. The details originate from the `SyslogMessage` field by default and the rule parses the IP directly from it.|

+Microsoft Threat Intelligence Matching Analytics only matches IPv4 indicators.

+#### [Microsoft Defender Threat Intelligence (MDTI)](#tab/microsoft-defender-threat-intelligence)

+|[CEF](connect-common-event-format.md) | Matching is done for all CEF logs that are ingested in the Log Analytics **CommonSecurityLog** table, except when the `DeviceVendor` is `Cisco`. <br><br>To match Microsoft generated threat intelligence with MDTI indicators in CEF logs, no manual mapping needs to be done. The URL is populated in the `RequestURL` field by default.|

+## BackupRestoreService Events

+When BackupRestoreService (BRS) is enabled on an SF Cluster, it exposes events for user triggered and periodic operations to let user understand the status of operations. User triggered operations cover actions like creating a backup policy on cluster, triggering backup on a partition or any other valid action in BRS context. BRS periodically emits status of current active policies at cluster level, information about last backup and upcoming scheduled backup and status of periodic backup at different stages at partition level.

+**BackupRestoreService partition events**

+| EventId | Name | Description |Source (Task) | Level |

+| | | | | |

+| 65305 | BRSInfo | Periodic backup triggered | BackupRestoreService | Informational |

+| 65307 | BRSWarning | Incremental backup failed, triggering a full backup | BackupRestoreService | Warning |

+| 65309 | BRSError | Periodic backup failed | BackupRestoreService | Error |

+**BackupRestoreService cluster events**

+| EventId | Name | Description |Source (Task) | Level |

+| | | | | |

+| 65306 | BRSInfo | Backup policy created | BackupRestoreService | Informational |

+| 65308 | BRSWarning | Backup policy deleted | BackupRestoreService | Warning |

+| 65310 | BRSError | AddBackupPolicy failed | BackupRestoreService | Error |

+ Title: How to move from classic to modernized VMware disaster recovery?

+description: This article describes how to move from classic to modernized VMware disaster recovery.

+# How to move from classic to modernized VMware disaster recoveryΓÇ»

+This article provides information about how you can move/migrate your VMware replications from [classic](/azure/site-recovery/vmware-azure-architecture) to [modernized](/azure/site-recovery/vmware-azure-architecture-preview) protection architecture. With this capability to migrate, you can successfully transfer your replicated items from a configuration server to an Azure Site Recovery replication appliance. This migration is guided by a smart replication mechanism which ensures that the complete initial replication is not performed again for non-critical replicated items, and only the differential data is transferred.

+> [!Note]

+> - Movement of physical servers to modernized architecture is not yet supported.  

+> - Movement of machines replicated in a Private Endpoint enabled Recovery Services vault is not supported yet.

+> - Recovery plans will not be migrated and will need to be created again in the modernized Recovery Services vault.

+## PrerequisitesΓÇ»

+- [Prepare the required infrastructure](move-from-classic-to-modernized-vmware-disaster-recovery.md#prepare-the-infrastructure).

+- [Prepare the classic Recovery Services vault](move-from-classic-to-modernized-vmware-disaster-recovery.md#prepare-classic-recovery-services-vault).

+- [Prepare the modernized Recovery Services vault](move-from-classic-to-modernized-vmware-disaster-recovery.md#prepare-modernized-recovery-services-vault).

+## Move replicated itemsΓÇ»

+Follow these steps to move the replicated items from classic architecture to modernized architecture:

+1. Navigate to the classic Recovery Services vault and open **Replicated items**.

+ :::image type="Replicated items" source="media/migrate-tool/replicated-items-inline.png" alt-text="Screenshot showing replicated items." lightbox="media/migrate-tool/replicated-items-expanded.png":::

+2. Select **Upgrade to modernized VMware replication**. The **Pre-requisites** details are displayed. Ensure you read through the prerequisites and then select **Next** to proceed to configure the migration settings.

+ :::image type="Pre-requisites" source="media/migrate-tool/prerequisites-inline.png" alt-text="Screenshot showing prerequisites." lightbox="media/migrate-tool/prerequisites-expanded.png":::

+3. Select the modernized vault you plan to move to, machines from the current vault which will be moved to the modernized vault and an appliance for each of them.

+ :::image type="Migration settings" source="media/migrate-tool/migration-settings-inline.png" alt-text="Screenshot showing migration settings." lightbox="media/migrate-tool/migration-settings-expanded.png":::

+4. Select **Next** to review and make sure to check **Maximum migration time**.

+5. Select **I understand the risk. Proceed to move selected replicated item(s)** check box. ΓÇ»

+ :::image type="review" source="media/migrate-tool/review-inline.png" alt-text="Screenshot showing review." lightbox="media/migrate-tool/review-expanded.png":::

+ΓÇ»

+6. Select **Migrate**.

+7. You can monitor the migration jobs in the **Site Recovery jobs** section of the vault.ΓÇ»

+## Allowed actions during migration and post migrationΓÇ»

+### During the migration of machines  

+During the migration of a replicated item, continuous replication may get broken for some time. Replication continues as soon as the migration is complete. During migration, you will be allowed to do **Failover** operation. The last available recovery point will be present for selection and can be chosen for replication.  

+While the migration is in progress, you can only perform **Failover** operation. Once the migration is complete, data will start replicating using the modernized architecture and the new vault. All the operations will be available for you to perform from the new vault.  

+> [!Note]

+> If the migration fails, then we will automatically rollback the changes and ensure the replication starts again from the classic vault.  

+### Post migration operations from Classic vaultΓÇ»

+**Failover** and **Disable replication** operations will continue to be available from the classic vault even after migration is performed successfully. The classic vault will continue to exist till the retention period of last available recovery point has expired. Once the retention period is up, the vault will be cleaned up automatically. During this time, recovery points from both the vaults can be used for failover. It will depend on your failover needs to select a proper recovery point.ΓÇ»

+Till deletion of classic vault, you will continue to get charged for the retention points. Once the deletion has been done, no charges will be associated to the classic vault.

+After migration, if the failover is performed using the classic vault, then the replicated items present in the modernized vault will be automatically cleaned up. Once done, all the further operations, such as commit, re-protect, failback, will only be possible via the classic vault.  

+## Next steps

+[move from classic to modernized VMware disaster recovery](move-from-classic-to-modernized-vmware-disaster-recovery.md)

+ Title: Move from classic to modernized VMware disaster recovery.

+description: Learn about the architecture, necessary infrastructure, and FAQs about moving your VMware replications from classic to modernized protection architecture.

+# Move from classic to modernized VMware disaster recovery  

+This article provides information about the architecture, necessary infrastructure, and FAQs about moving your VMware replications from [classic](/azure/site-recovery/vmware-azure-architecture) to [modernized](/azure/site-recovery/vmware-azure-architecture-preview) protection architecture. With this capability to migrate, you can successfully transfer your replicated items from a configuration server to an Azure Site Recovery replication appliance. This migration is guided by a smart replication mechanism, which ensures that complete initial replication isn't performed again for non-critical replicated items, and only the differential data is transferred.

+> [!Note]

+> - Movement of physical servers to modernized architecture is not yet supported.  

+> - Movement of machines replicated in a Private Endpoint enabled Recovery Services vault is not supported yet.   

+> - Recovery plans won't be migrated and will need to be created again in the modernized Recovery Services vault.

+## ArchitectureΓÇ»

+The components involved in the migration of replicated items of a VMware machine are summarized in the following table:ΓÇ»

+|Component|Requirement|

+||-|

+|Replicated items in a classic Recovery Services vault|One or more replicated items that are protected using the classic architecture and a healthy configuration server.<br></br>The replicated item should be in a non-critical state and must be replicated from on-premises to Azure with the mobility agent running on version 9.50 or later.|

+|Configuration server used by the replicated items|The configuration server, used by the replicated items, should be in a non-critical state and its components should be upgraded to the latest version (9.50 or later).|ΓÇ»

+|A Recovery Services vault with modernized experience|A Recovery Services vault with modernized experience.|

+|A healthy Azure Site Recovery replication appliance|A non-critical Azure Site Recovery replication appliance, which can discover on-premises machines, with all its components upgraded to the latest version (9.50 or later). The exact required versions are as follows:<br></br>Process server: 9.50<br>Proxy server: 1.35.8419.34591<br>Recovery services agent: 2.0.9249.0<br>Replication service: 1.35.8433.24227|

+## Required infrastructureΓÇ»

+Ensure the following for a successful movement of replicated item:

+- A Recovery Services vault using the modernized experience. ΓÇ»

+ >[!Note]

+ >Any new Recovery Services vault created will have the modernized experience switched on by default. You can [switch to the classic experience](/azure/site-recovery/vmware-azure-common-questions#how-do-i-use-the-classic-experience-in-the-recovery-services-vault-rather-than-the-preview-experience) but once done, you canΓÇÖt switch again. ΓÇ»

+- An [Azure Site Recovery replication appliance](/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview), which has been successfully registered to the vault, and all its components are in a non-critical state.  

+- The version of the appliance must be 9.50 or later. For a detailed version description, check [here](#architecture).

+- The vCenter server or vSphere host’s details, where the existing replicated machines reside, are added to the appliance for the on-premises discovery to be successful.  

+## PrerequisitesΓÇ»

+### Prepare the infrastructureΓÇ»

+Ensure the following before you move from classic architecture to modernized architecture:

+- [Create a Recovery Services vault](/azure/site-recovery/azure-to-azure-tutorial-enable-replication#create-a-recovery-services-vault) and ensure the experience has [not been switched to classic](/azure/site-recovery/vmware-azure-common-questions#how-do-i-use-the-classic-experience-in-the-recovery-services-vault-rather-than-the-preview-experience).

+- [Deploy an Azure Site Recovery replication appliance](/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview).

+- [Add the on-premises machine’s vCenter Server details](/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview) to the appliance, so that it successfully performs discovery.  

+### Prepare classic Recovery Services vault  

+Ensure the following for the replicated items you are planning to move:

+- The Recovery Services vault does not have MSI enabled on it.

+- The replicated item is a VMware machine replicating via a configuration server.

+- Replication is not happening to an unmanaged storage account but rather to managed disk.

+- Replication is happening from on-premises to Azure and the replicated item is not in a failed-over or in failed-back state.

+- The replicated item is not replicating the data from Azure to on-premises.ΓÇ»

+- The initial replication is not under progress and has already been completed.  

+- The replicated item is not in the ΓÇÿresynchronizationΓÇÖ state.ΓÇ»

+- The configuration serverΓÇÖs version is 9.50 or later and its health is in a non-critical state.ΓÇ»

+- The configuration server has a healthy heartbeat.ΓÇ»

+- The mobility service agentΓÇÖs version, installed on the source machine, is 9.50 or later.ΓÇ»

+- The replicated item does not use Private Endpoint.  

+- The replicated itemΓÇÖs health is in a non-critical state, or its recovery points are being created successfully.ΓÇ»

+### Prepare modernized Recovery Services vault  

+For the modernized architecture setup, ensure that:  

+- The Recovery Services vault used for modernized architecture setup is in the same geographical location as the classic vault.  

+- An Azure Site Recovery replication appliance is deployed on your on-premises with version 9.50 or later.ΓÇ»

+- The appliance is successfully registered to the vault.  

+- The appliance and all its components are in a non-critical state and the appliance has a healthy heartbeat.ΓÇ»

+- The vCenter Server version is supported by the modernized architecture.ΓÇ»

+- The vCenter Server details of the source machine are added to the appliance.ΓÇ»

+- The Linux distro version is supported by the modernized architecture.ΓÇ»[Learn more](/azure/site-recovery/vmware-physical-azure-support-matrix#for-linux).

+- The Windows Server version is supported by the modernized architecture.ΓÇ»[Learn more](/azure/site-recovery/vmware-physical-azure-support-matrix#for-windows).

+## Calculate total time to moveΓÇ»

+The total time required to move any replicated item from classic vault to modernized vault depends on the itemΓÇÖs replication status and the disk size.

+| State | Time to migrate to modernized vault |

+|-||

+| Replicated itemΓÇÖs protection status is **healthy** and the **last recovery point was created less than 50 minutes ago**|Migration will be complete in **1-2 hours**|

+| Replicated itemΓÇÖs protection status is **not healthy** or the **last recovery point was created more than 50 minutes ago**|Migration time will vary, and it will **depend on the disk size**|

+If your machines protection status is not healthy, then use the formula below to calculate the exact time for your machines:

+Time to migrate = 1 hour + 45 second/GiB

+| Machine configuration | Time to migrate |

+|--||

+| 1 machine with 2 disks, both of size 256 GiB|~ 4 hours 15 mins<br></br>*[Both the disks will be migrated in parallel]*|

+| 10 machines with 2 disks each, both of size 256 GiB|~ 4 hours 15 mins<br></br>*[All the VMs and their disks will be migrated in parallel]*|

+| 1 machine with 4 disks, all of size 512 GiB|~ 7 hours 30 mins<br></br>*[Both the disks will be migrated in parallel]*|

+| 10 machines with 4 disks each, all of size 512 GiB|~ 7 hours 30 mins<br></br>*[All the VMs and their disks will be migrated in parallel]*|

+The same formula will be used to calculate time for migration and is shown on the portal.

+## How to define required infrastructure

+When migrating machines from classic to modernized architecture, you will need to make sure that the required infrastructure has already been registered in the modernized Recovery Services vault. Refer to the replication applianceΓÇÖs [sizing and capacity details](/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview#sizing-and-capacity) to help define the required infrastructure.

+As a rule, you should set up the same number of replication appliances, as the number of process servers in your classic Recovery Services vault. In the classic vault, if there was one configuration server and four process servers, then you should set up four replication appliances in the modernized Recovery Services vault.

+## Pricing

+Site Recovery license fee will continue to be charged on the classic vault till retention period of all recovery points has expired. Once all recovery points have been cleaned up, the pricing will also stop on the classic vault. Once the retention period of all the recovery points has expired, the replicated item will be automatically removed via a system triggered purge replication operation.

+Site Recovery will start charging license fee on replicated items in the modernized vault, only after the first recovery point has been generated and older vault has been cleaned up. If there are any free trial usage days pending on the classic vault, then the same information will be passed on to the modernized vault. Pricing will start on the modernized vault only after this trial period has passed.

+>[!Note]

+> At one point in time, pricing will only happen using one vault, either the classic or modernized vault.

+## FAQsΓÇ»

+### Why should I migrate my machines to the modernized architecture?

+Ultimately, the classic architecture will be deprecated, so one must ensure that they are using the latest modernized architecture. The table below shows a comparison of the two architectures to enable you to select the correct option for enabling disaster recovery for your machines:ΓÇ»

+|Classic architecture|Modernized architecture [New]|

+||--|

+|Multiple setups required for discovering on-premises data.|**Central discovery** of on-premises data center using discovery service.|

+|Extensive number of steps required for initial onboarding.|**Simplified the onboarding experience** by automating artifact creation and introduced defaults to reduce required inputs.|ΓÇ»

+|Utilizes a manually downloaded file to obtain cloud context.|**Introduced replication key** for obtaining cloud context when setting up the appliance.|

+|Extensive number of steps required for a simple enable replication process.|**Simplified the enable replication experience** by reducing the number of required inputs and redefining each blade.|

+|Configuration server continues to be an on-premises infrastructure with extensive setup for various components.|Enhanced the appliance by converting all components into Azure hosted microservices. This **simplifies appliance scaling, monitoring, and troubleshooting.**|

+|Need for scale-out process server and master target server in Azure for Linux machines is a hindering requirement.|**Removed** the need to maintain separate **process server and master target server**.|

+|Used a static passphrase for authentication, which interfered with customerΓÇÖs business requirements of periodic password rotation.|Introduced **certificate-based authentication**, which is more secure and resolves customerΓÇÖs security concerns.|

+|Upgrading to an updated version should be done manually and is a cumbersome process.|Introduced **automatic upgrades** for both appliance components and Mobility service.|

+|The configuration server does not have high availability and might be at the risk of collapsing.|Implemented **high availability of appliance** to ensure resiliency.|

+|Root credentials should be regularly updated to ensure an error-free upgrade experience.|**Eliminated the requirement to maintain machineΓÇÖs root credentials** for performing automatic upgrades.|

+|Static IP address should be assigned to configuration server to maintain connectivity.|Introduced **FQDN based connectivity** between appliance and on-premises machines.|

+|Only that virtual network, which has Site-to-Site VPN or Express Route enabled, should be used.|Removed the need to maintain a Site-to-Site VPN or Express Route for reverse replication.|

+### What machines should be migrated to the modernized architecture?

+All VMware machines, which are replicated using a configuration server, should be migrated to the modernized architecture. As of now, we have released support for VMware machines.  

+### Where should my modernized Recovery Services vault be created?

+The modernized Recovery Services vault should be present in the same region and tenant as the classic vault. It can be a part of any subscription or resource group.  

+### Will my replication continue while the migration is happening?

+No, the replication will break for some time while the migration is in progress. During this time, the last created recovery point, in the classic Recovery Services vault, will be available for you to failover to. Once the migration is complete, a new recovery point will be generated in the modernized Recovery Services vault.  

+### When will my migration operation be marked as complete?

+Migration operation will only be marked complete once the first recovery point has been successfully created in the modernized Recovery Services vault.ΓÇ»

+### What operations can be performed from my classic Recovery Services vault, after migration is done?ΓÇ»

+You can only perform failover and disable replication from your classic vault after the migration. The failover operation is possible via the classic vault until the recovery points are available in the older vault.

+For example, if the retention period for a replicated item is 72 hours (3 days), the latest recovery point on the classic vault will continue to be available for 72 hours (3 days), after a successful post migration. After the stipulated time, Azure Site Recovery will trigger a purge replication operation on the replicated item and perform the cleanup of all associated storage and billing causing items.

+### What if a disaster strikes my machine while the migration operation is in progress?

+Any replicated item on which migration is being performed will continue to support failover operation via the classic Recovery Services vault, till the last recovery point’s retention period expires. In case you try to execute failover operation, it will take a higher priority than migration operation. The job for migration will be aborted. To ensure that your replicated item is migrated, you will need to trigger the migration operation again, at a later point in time.  

+>[!Note]

+> The Compute and Network properties of replicated items can be updated while the migration is in progress. However, the changes may not get replicated in the modernized Recovery Services vault.

+### How many machines can I migrate in one go from classic to modernized vault?

+You can migrate up to 10 machines via the portal, in one go.  

+### Should I recreate the virtual networks, storage accounts, and replication policy to be used in the new vault?

+No, the same resources, which were being used previously will be defaulted to in the modernized vault also. You can always change those from the Compute and Network blade of your replicated item. You must ensure that the resources continue to have the required access.  

+### How will my replication policies be moved to the modernized vault?

+As a prerequisite, Site Recovery will first create replication policies in the modernized vault, with the same configuration as present in the classic vault. So, if a replicated item is being moved, then the policy associated with it will be first created in the modernized vault and then migration will happen. It is recommended that the configuration of replication policies not be changed in the classic vault after migration has been triggered, as the changed values won't be propagated to the modernized vault. This operation should happen before migration is triggered.

+

+The replication policy created in the modernized vault will have its name changed in the modernized vault. It will be prefixed with resource group name and vault name of the modernized Recovery Services vault. So, if the policy name was ΓÇ£default replication policyΓÇ¥ in the classic vault, then in the modernized vault, this policyΓÇÖs name will be ΓÇ£default replication policy contoso-modern-vault_contoso-rgΓÇ¥, given the vaultΓÇÖs name is contoso-modern-vault and the vaultΓÇÖs resource group is contoso-rg.

+### Can I edit my replication policy during migration or post migration in the classic vault?

+If the replica of a replication policy has already been created in the modernized vault, then any changes to the policy in the classic vault won't be propagated to the modernized vault.

+So, if there are 10 replicated items, which are replicated using a policy and you decide to move 5 of those to the modernized experience, then a copy of the policy will be created before migration starts. Now, before performing migration of the remaining 5 items, if any changes are made in the policy in classic vault, the policy from modernized vault won't be updated. You will need to make those configuration changes in the modernized vault too.

+### How do I migrate replicated items, which are present in a replication group, also known as multi-vm consistency groups?

+All replicated items, which are a part of a replication group will be migrated together. You can either select them all, by selecting the replication group, or skip them all. In case the migration fails for some machines in a replication group and succeeds for others, then a rollback to classic experience will be performed for failed replicated items and migration for failed items can be triggered again.

+## Next steps

+[How to move from classic to modernized VMware disaster recovery](how-to-move-from-classic-to-modernized-vmware-disaster-recovery.md)

+Watch a quick example [video](https://youtu.be/1KUVdtvGqw8) showing an on-click failover for a recovery plan for a two-tier WordPress app.

+>[!NOTE]

+>You can now move your existing replicated items to modernized VMware disaster recovery experience. [Learn more](move-from-classic-to-modernized-vmware-disaster-recovery.md).

+>[!NOTE]

+>You can now move your existing replicated items to modernized VMware disaster recovery experience. [Learn more](move-from-classic-to-modernized-vmware-disaster-recovery.md).

+>[!NOTE]

+>You can now move your existing replicated items to modernized VMware disaster recovery experience. [Learn more](move-from-classic-to-modernized-vmware-disaster-recovery.md).

+> Site Recovery does not move or store customer data out of the target region, in which disaster recovery has been set up for the source machines. Customers may select a Recovery Services Vault from a different region if they so choose. The Recovery Services Vault contains metadata but no actual customer data.

+SUSE Linux Enterprise Server 12 (SP1, SP2, SP3, SP4, SP5) | [9.48](https://support.microsoft.com/en-us/topic/update-rollup-61-for-azure-site-recovery-kb5012960-a1cc029b-03ad-446f-9365-a00b41025d39) | All [stock SUSE 12 SP1,SP2,SP3,SP4,SP5 kernels](https://www.suse.com/support/kb/doc/?id=000019587) are supported. </br> 4.12.14-16.85-azure:5 </br> 4.12.14-16.88-azure:5 </br> 4.12.14-122.106-default:5 </br> 4.12.14-122.110-default:5 </br> 4.12.14-122.113-default:5 </br> 4.12.14-122.116-default:5 </br> 4.12.14-122.12-default:5 </br> 4.12.14-122.121-default:5 |

+SUSE Linux Enterprise Server 12 (SP1, SP2, SP3, SP4, SP5) | [9.47](https://support.microsoft.com/topic/update-rollup-60-for-azure-site-recovery-k5011122-883a93a7-57df-4b26-a1c4-847efb34a9e8) | All [stock SUSE 12 SP1,SP2,SP3,SP4,SP5 kernels](https://www.suse.com/support/kb/doc/?id=000019587) are supported.</br></br> 4.12.14-16.80-azure </br> 4.12.14-122.103-default </br> 4.12.14-122.98-default5 |

+SUSE Linux Enterprise Server 12 (SP1, SP2, SP3, SP4, SP5) | [9.46](https://support.microsoft.com/topic/update-rollup-59-for-azure-site-recovery-kb5008707-66a65377-862b-4a4c-9882-fd74bdc7a81e) | All [stock SUSE 12 SP1,SP2,SP3,SP4,SP5 kernels](https://www.suse.com/support/kb/doc/?id=000019587) are supported.</br></br> 4.4.138-4.7-azure to 4.4.180-4.31-azure,</br>4.12.14-6.3-azure to 4.12.14-6.43-azure </br> 4.12.14-16.7-azure to 4.12.14-16.65-azure </br> 4.12.14-16.68-azure </br> 4.12.14-16.73-azure </br> 4.12.14-16.76-azure </br> 4.12.14-122.88-default </br> 4.12.14-122.91-default |

+SUSE Linux Enterprise Server 12 (SP1, SP2, SP3, SP4, SP5) | [9.45](https://support.microsoft.com/en-us/topic/update-rollup-58-for-azure-site-recovery-kb5007075-37ba21c3-47d9-4ea9-9130-a7d64f517d5d) | All [stock SUSE 12 SP1, SP2, SP3, SP4, SP5 kernels](https://www.suse.com/support/kb/doc/?id=000019587) are supported.</br></br> 4.4.138-4.7-azure to 4.4.180-4.31-azure,</br>4.12.14-6.3-azure to 4.12.14-6.43-azure </br> 4.12.14-16.7-azure to 4.12.14-16.65-azure </br> 4.12.14-16.68-azure </br> 4.12.14-16.76-azure |

+Please note that the above limits are applicable to VMware and Hyper-V scenarios only.

+After enabling Azure AD Kerberos authentication, you'll need to explicitly grant admin consent to the new Azure AD application registered in your Azure AD tenant to complete your configuration. You can configure the API permissions from the [Azure portal](https://portal.azure.com) by following these steps.

+2. Select **App registrations** in the left pane.

+3. Select **All Applications** in the right pane.

+ :::image type="content" source="media/storage-troubleshoot-windows-file-connection-problems/azure-portal-azuread-app-registrations.png" alt-text="Screenshot of the Azure portal. Azure Active Directory is open. App registrations is selected in the left pane. All applications is highlighted in the right pane." lightbox="media/storage-troubleshoot-windows-file-connection-problems/azure-portal-azuread-app-registrations.png":::

+4. Select the application with the name matching **[Storage Account] $storageAccountName.file.core.windows.net**.

+> [!NOTE]

+> Azure Data Explorer from Azure Stream Analytics does not support output to Synapse Data Explorer clusters.

+For the Stream Analytics job to access your Cosmos DB using managed identity, the service principal you created must have special permissions to your Azure Cosmos DB account. In this step, you can assign a role to your stream analytics job's system-assigned managed identity. Azure Cosmos DB has multiple built-in roles that you can assign to the managed identity. For this solution, you can use the following role:

+## July 2022

+Here's what changed in July 2022:

+## Scheduled agent updates now generally available

+Scheduled agent updates on Azure Virtual Desktop are now generally available. This feature gives IT admins control over when the Azure Virtual Desktop agent, side-by-side stack, and Geneva Monitoring agent get updated. For more information, see [our blog post](https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/announcing-general-availability-of-scheduled-agent-updates-on/ba-p/3579236).

+## FSLogix 2201 hotfix 2

+The FSLogix 2201 hotfix 2 update includes fixes to multi-session VHD mounting, Cloud Cache meta tracking files, and registry cleanup operations. This update doesn't include new fatures. Learn more at [WhatΓÇÖs new in FSLogix](/fslogix/whats-new?context=%2Fazure%2Fvirtual-desktop%2Fcontext%2Fcontext#fslogix-2201-hotfix-2-29822850276) and [our blog post](https://techcommunity.microsoft.com/t5/azure-virtual-desktop/announcing-fslogix-2201-hotfix-2-2-9-8228-50276-has-been/m-p/3579409).

+## Japan and Australia metadata service now generally available

+The Azure Virtual Desktop metadata database located in Japan and Australia is now generally available. This update allows customers to store their Azure Virtual Desktop objects and metadata within a database located within that geography. For more information, see [our blog post](https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/announcing-general-availability-of-the-azure-virtual-desktop/ba-p/3570756).

+## Azure Virtual Desktop moving away from Storage Blob image type

+Storage Blob images are created from unmanaged disks, which means they lack the availability, scalability, and frictionless user experience that managed images and Shared Image Gallery images offer. As a result, Azure Virtual Desktop will be deprecating support for Storage Blobs image types by August 22, 2022. For more information, see [our blog post](https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/azure-virtual-desktop-is-moving-away-from-storage-blob-image/ba-p/3568364).

+## Azure Virtual Desktop Custom Configuration changing to PowerShell

+Starting July 21, 2022, Azure Virtual Desktop will replace the Custom Configuration Azure Resource Manager template parameters for creating host pools, adding session hosts to host pools, and the Getting Started feature with a PowerShell script URL parameter stored in a publicly accessible location. This replacement includes the parameters' respective Azure Resource Manager templates. For more information, see [our blog post](https://techcommunity.microsoft.com/t5/azure-virtual-desktop/azure-virtual-desktop-custom-configuration-breaking-change/m-p/3568069).

+Deploying Intune user configuration policies from Microsoft Endpoint Manager admin center to Windows 11 Enterprise multi-session virtual machines (VMs) on Azure Virtual Desktop is now in public preview. In this preview, you can configure the following features:

+## DCsv3

+### DCsv3-Type1

+The DCsv3-Type1 is a Dedicated Host SKU utilizing the 3rd Generation Intel® Xeon Scalable Processor 8370C. It offers 48 physical cores, 48 vCPUs, and 384 GiB of RAM. The DCsv3-Type1 runs [DCsv3-series](dcv3-series.md#dcsv3-series) VMs.

+The following packing configuration outlines the max packing of uniform VMs you can put onto a DCsv3-Type1 host.

+| Physical cores | Available vCPUs | Available RAM | VM Size | # VMs |

+|--||-||-|

+| 48 | 48 | 384 GiB | DC1s v3 | 32 |

+| | | | DC2s v3 | 24 |

+| | | | DC4s v3 | 12 |

+| | | | DC8s v3 | 6 |

+| | | | DC16s v3 | 3 |

+| | | | DC24s v3 | 2 |

+## DCdsv3

+### DCdsv3-Type1

+The DCdsv3-Type1 is a Dedicated Host SKU utilizing the 3rd Generation Intel® Xeon Scalable Processor 8370C. It offers 48 physical cores, 48 vCPUs, and 384 GiB of RAM. The DCdsv3-Type1 runs [DCdsv3-series](dcv3-series.md#dcdsv3-series) VMs.

+The following packing configuration outlines the max packing of uniform VMs you can put onto a DCdsv3-Type1 host.

+| Physical cores | Available vCPUs | Available RAM | VM Size | # VMs |

+|--||-||-|

+| 48 | 48 | 384 GiB | DC16ds v3 | 2 |

+| | | | DC24ds v3 | 1 |

+| | | | DC32ds v3 | 1 |

+| | | | DC48ds v3 | 1 |

+As of version 2.7+, The azure linux guest agent has a feature to automatically collect some logs and upload them. This feature currently requires systemd, and utilizes a new systemd slice called azure-walinuxagent-logcollector.slice to manage resources while performing the collection. The log collector's goal is facilitate offline analysis, and therefore produces a ZIP file of some diagnostics logs before uploading them to the VM's Host. The ZIP file can then be retreived by Engineering Teams and Support professionals to investigate issues at the behest of the VM owner. More technical information on the files collected by the guest agent can be found in the azurelinuxagent/common/logcollector_manifests.py file in the [agent's GitHub repository](https://github.com/Azure/WALinuxAgent).

+- **Ensure VMSnapshot extension isn't in a failed state**: Follow the steps in [Troubleshooting](/azure/backup/backup-azure-troubleshoot-vm-backup-fails-snapshot-timeout#usererrorvmprovisioningstatefailedthe-vm-is-in-failed-provisioning-state) to verify and ensure the Azure VM snapshot extension is healthy.

+ Follow these steps to [troubleshoot VSS writer issues](/azure/backup/backup-azure-vms-troubleshoot#extensionfailedvsswriterinbadstatesnapshot-operation-failed-because-vss-writers-were-in-a-bad-state).

+- In the Azure portal, go to **Virtual Machines** > **Settings** > **Extensions** and ensure all extensions are in **provisioning succeeded** state. If not, follow these [steps](/azure/backup/backup-azure-troubleshoot-vm-backup-fails-snapshot-timeout#usererrorvmprovisioningstatefailedthe-vm-is-in-failed-provisioning-state) to resolve the issue.

+ :::image type="content" source="./media/restore-point-troubleshooting/delete-lock-inline.png" alt-text="Screenshot of Delete lock in Azure portal." lightbox="./media/restore-point-troubleshooting/delete-lock-expanded.png":::

+- Not available in Government clouds

+Once you have created the disks, [create a new VM](/azure/virtual-machines/scripts/create-vm-from-managed-os-disks) and [attach these restored disks](/azure/virtual-machines/linux/add-disk#attach-an-existing-disk) to the newly created VM.

+[Learn more](/azure/virtual-machines/backup-recovery) about Backup and restore options for virtual machines in Azure.

+Once the disks are created, [create a new VM](/azure/virtual-machines/windows/create-vm-specialized-portal#create-a-vm-from-a-disk.md) and [attach these restored disks](/azure/virtual-machines/windows/attach-managed-disk-portal) to the newly created VM.

+After you create the disks, [create a new VM](/azure/virtual-machines/windows/create-vm-specialized-portal) and [attach these restored disks](/azure/virtual-machines/windows/attach-disk-ps#using-managed-disks) to the newly created VM.

+RHEL | 7-RAW | RAW | Linux Agent | RHEL 7.x family of images. <br> Attached to regular repositories by default (not EUS).

+>

+> RHEL 6.7, 6.8, 6.9, and 6.10 have [Extended Lifecycle Support](redhat-extended-lifecycle-support.md) available.

+[SAP Cloud Appliance Library](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) offers a quick and easy way to create SAP workloads in Azure. With a few clicks you can set up a fully configured demo environment from an Appliance Template or deploy a standardized system for an SAP product based on default or custom SAP software installation stacks.

+This page lists the latest Appliance Templates and below the latest SAP S/4HANA stacks for production-ready deployments.

+For deployment of an appliance template you will need to authenticate with your S-User or P-User. You can create a P-User free of charge via the [SAP Community](https://community.sap.com/).

+[For details on Azure account creation see the SAP learning video and description](https://www.youtube.com/watch?v=iORePziUMBk&list=PLWV533hWWvDmww3OX9YPhjjS1l1n6o-H2&index=18)

+You will also find detailed answers to your questions related to SAP Cloud Appliance Library on Azure [SAP CAL FAQ](https://caldocs.hana.ondemand.com/caldocs/help/Azure_FAQs.pdf)

+The online library is continuously updated with Appliances for demo, proof of concept and exploration of new business cases. For the most recent ones select ΓÇ£Create ApplianceΓÇ¥ here from the list ΓÇô or visit [cal.sap.com](https://cal.sap.com/catalog#/applianceTemplates) for further templates.

+## Deployment of appliances through SAP Cloud Appliance Library

+| Appliance Templates | Link |

+| -- | : |

+| **SAP Focused Run 3.0 FP03 (configured)** July 28 2022 | [Create Appliance](https://cal.sap.com/registration?sguid=517c6359-6b26-458d-b816-ca25c3e5af7d&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|SAP Focused Run is designed specifically for businesses that need high-volume system and application monitoring, alerting, and analytics. It's a powerful solution for service providers, who want to host all their customers in one central, scalable, safe, and automated environment. It also addresses customers with advanced needs regarding system management, user monitoring, integration monitoring, and configuration and security analytics. | [Details](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/517c6359-6b26-458d-b816-ca25c3e5af7d) |

+| **System Conversion for SAP S/4HANA ΓÇô SAP S/4HANA 2021 FPS01 after technical conversion** July 27 2022 | [Create Appliance](https://cal.sap.com/registration?sguid=93895065-7267-4d51-945b-9300836f6a80&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|Third solution after performing a technical system conversion from SAP ERP to SAP S/4HANA before additional configuration. It has been tested and prepared as converted from SAP EHP7 for SAP ERP 6.0 to SAP S/4HANA 2020 FPS01. | [Details](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/93895065-7267-4d51-945b-9300836f6a80) |

+| **SAP Focused Run 3.0 FP03, unconfigured** July 21 2022 | [Create Appliance](https://cal.sap.com/registration?sguid=4c38b6ff-d598-4dbc-8f39-fdcf96ae0beb&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|SAP Focused Run is designed specifically for businesses that need high-volume system and application monitoring, alerting, and analytics. It's a powerful solution for service providers, who want to host all their customers in one central, scalable, safe, and automated environment. It also addresses customers with advanced needs regarding system management, user monitoring, integration monitoring, and configuration and security analytics. | [Details](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/4c38b6ff-d598-4dbc-8f39-fdcf96ae0beb) |

+| **SAP S/4HANA 2021 FPS02, Fully-Activated Appliance** July 19 2022 | [Create Appliance](https://cal.sap.com/registration?sguid=3f4931de-b15b-47f1-b93d-a4267296b8bc&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|This appliance contains SAP S/4HANA 2021 (FPS02) with pre-activated SAP Best Practices for SAP S/4HANA core functions, and further scenarios for Service, Master Data Governance (MDG), Portfolio Mgmt. (PPM), Human Capital Management (HCM), Analytics, Migration Cockpit, and more. User access happens via SAP Fiori, SAP GUI, SAP HANA Studio, Windows remote desktop, or the backend operating system for full administrative access. | [Details]( https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/3f4931de-b15b-47f1-b93d-a4267296b8bc) |

+ | **System Conversion for SAP S/4HANA ΓÇô Source system SAP ERP6.0 before running SUM** July 05 2022 | [Create Appliance](https://cal.sap.com/registration?sguid=b28b67f3-ebab-4b03-bee9-1cd57ddb41b6&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|Second solution for performing a system conversion from SAP ERP to SAP S/4HANA after preparation steps before running Software Update Manager. It has been tested and prepared to be converted from SAP EHP7 for SAP ERP 6.0 to SAP S/4HANA 2021 FPS01 | [Details](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/b28b67f3-ebab-4b03-bee9-1cd57ddb41b6) |

+| **SAP NetWeaver 7.5 SP15 on SAP ASE** January 20 2020 | [Create Appliance](https://cal.sap.com/registration?sguid=69efd5d1-04de-42d8-a279-813b7a54c1f6&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+|SAP NetWeaver 7.5 SP15 on SAP ASE | [Details](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/solutions/69efd5d1-04de-42d8-a279-813b7a54c1f6) |

+## Deployment of S/4HANA system for productive usage through SAP Cloud Appliance Library

+You now can also deploy S4H systems for productive usage through SAP Cloud Appliance Library. Within a few clicks, you can have your SAP system for productive usage up and running. The following links highlight the solutions that you can quickly deploy on Azure. Just select the "Deploy System" under "Products" link.

+You will need to authenticate with your S-User.

+| All products | Link |

+| -- | : |

+| **SAP S/4HANA 2021 FPS01 for Productive Deployments** | [Deploy System](https://cal.sap.com/catalog#/products) |

+|This solution comes as a standard S/4HANA system installation including High Availability capabilities to ensure higher system uptime for productive usage. The system parameters can be customized during initial provisioning according to the requirements for the target system. You will need a valid license for deployment initiation. |

+| **SAP S/4HANA 2021 Initial Shipment Stack for Productive Deployments** | [Deploy System](https://cal.sap.com/catalog#/products) |

+|This solution comes as a standard S/4HANA system installation including High Availability capabilities to ensure higher system uptime for productive usage. The system parameters can be customized during initial provisioning according to the requirements for the target system. |

+_Within a few hours, a healthy SAP S/4 appliance is deployed in Azure._

+If you bought an SAP CAL subscription, SAP fully supports deployments through SAP CAL on Azure. The support queue is BC-VCM-CAL.

+Each NAT gateway can provide up to 50 Gbps of throughput. This data throughput includes data processed both outbound and inbound through a NAT gateway resource. You can split your deployments into multiple subnets and assign each subnet or group of subnets a NAT gateway to scale out.

+| SNAT Connection Count | Number of new SNAT connections over a given interval of time | Sum | Connection State (Attempted, Established, Failed, Closed, Timed Out), Protocol (6 TCP; 17 UDP) |

+| Total SNAT connection count | Total number of active SNAT connections | Sum | Protocol (6 TCP; 17 UDP) |

+The SNAT connection count metric shows you the number of new SNAT connections within a specified time frame.

+1. Search for *myVMMgmt* in the portal search box.

+ Title: 'Create an ExpressRoute association to Azure Virtual WAN - PowerShell'

+description: Learn how to create an ExpressRoute association from your branch site to Azure Virtual WAN using PowerShell.

+# Create an ExpressRoute association to Virtual WAN - PowerShell

+This article helps you use Virtual WAN to connect to your resources in Azure over an ExpressRoute circuit. For more information about Virtual WAN and Virtual WAN resources, see the [Virtual WAN Overview](virtual-wan-about.md).

+## Prerequisites

+Verify that you've met the following criteria before beginning your configuration.

+* You have a virtual network that you want to connect to. Verify that none of the subnets of your on-premises networks overlap with the virtual networks that you want to connect to. To create a virtual network using PowerShell, see the [Quickstart](../virtual-network/quick-create-powershell.md).

+* Your virtual network doesn't have any virtual network gateways. If your virtual network has a gateway (either VPN or ExpressRoute), you must remove all gateways. This configuration requires that virtual networks are connected instead, to the Virtual WAN hub gateway.

+* Obtain an IP address range for your virtual hub region. A virtual hub is a virtual network that is created and used by Virtual WAN. The address range that you specify for the virtual hub can't overlap with any of your existing virtual networks that you connect to. It also can't overlap with your address ranges that you connect to on-premises. If you're unfamiliar with the IP address ranges located in your on-premises network configuration, coordinate with someone who can provide those details for you.

+* The ExpressRoute circuit must be a Premium or Standard circuit in order to connect to the virtual hub gateway.

+* If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+### Azure PowerShell

+## <a name="signin"></a>Sign in

+## Create a virtual WAN

+Before you can create a virtual wan, you have to create a resource group to host the virtual wan or use an existing resource group. Use one of the following examples.

+**New resource group** - This example creates a new resource group named testRG in the West US location.

+1. Create a resource group.

+ ```azurepowershell-interactive

+ New-AzResourceGroup -Location "West US" -Name "testRG"

+ ```

+1. Create the virtual wan.

+ ```azurepowershell-interactive

+ $virtualWan = New-AzVirtualWan -ResourceGroupName testRG -Name myVirtualWAN -Location "West US"

+ ```

+**Existing resource group** - Use the following steps if you want to create the virtual wan in an already existing resource group.

+1. Set the variables for the existing resource group.

+ ```azurepowershell-interactive

+ $resourceGroup = Get-AzResourceGroup -ResourceGroupName "testRG"

+ ```

+1. Create the virtual wan.

+ ```azurepowershell-interactive

+ $virtualWan = New-AzVirtualWan -ResourceGroupName testRG -Name myVirtualWAN -Location "West US"

+ ```

+## Create a virtual hub and a gateway

+A virtual hub is a virtual network that can contain gateways for site-to-site, ExpressRoute, or point-to-site functionality. Use one of the following examples to create an ExpressRoute gateway in a new or existing virtual hub.

+**New virtual hub** - This example creates a default virtual hub named westushub with the specified address prefix and a location for the virtual hub.

+1. Create a virtual hub.

+ ```azurepowershell-interactive

+ $virtualHub = New-AzVirtualHub -VirtualWan $virtualWan -ResourceGroupName "testRG" -Name "westushub" -AddressPrefix "10.0.0.1/24"

+ ```

+1. Create an ExpressRoute gateway. ExpressRoute gateways are provisioned in units of 2 Gbps. 1 scale unit = 2 Gbps with support up to 10 scale units = 20 Gbps. It takes about 30 minutes for a virtual hub and gateway to fully create.

+ ```azurepowershell-interactive

+ $expressroutegatewayinhub = New-AzExpressRouteGateway -ResourceGroupName "testRG" -Name "testergw" -VirtualHubId $virtualHub.Id -MinScaleUnits 2

+ ```

+**Existing virtual hub** - This example creates an ExpressRoute gateway in an existing virtual hub.

+```azurepowershell-interactive

+$expressroutegatewayinhub = New-AzExpressRouteGateway -MaxScaleUnits <UInt32> -MinScaleUnits 2 -Name 'testExpressRoutegw' -ResourceGroupName 'testRG' -Tag @{"tag1"="value1"; "tag2"="value2"} -VirtualHubName "[hub Name]"

+```

+## Create an Express Route circuit

+The next step is to get the private peering ID of the ExpressRoute circuit. You can either create a new circuit, or get the ID from an existing circuit. Use one of the following examples.

+**New circuit** - This example creates a new ExpressRoute circuit and gets its private peering ID.

+ ```azurepowershell-interactive

+ $ExpressRouteCircuit = New-AzExpressRouteCircuit -ResourceGroupName "testRG" -Name "testExpressRouteCircuit" -Location "West Central US" -SkuTier Premium -SkuFamily MeteredData -ServiceProviderName "Equinix" -PeeringLocation "Silicon Valley" -BandwidthInMbps 200

+ Add-AzExpressRouteCircuitPeeringConfig -Name "AzurePrivatePeering" -ExpressRouteCircuit $ExpressRouteCircuit -PeeringType AzurePrivatePeering -PeerASN 100 -PrimaryPeerAddressPrefix "123.0.0.0/30" -SecondaryPeerAddressPrefix "123.0.0.4/30" -VlanId 300

+ $ExpressRouteCircuit = Set-AzExpressRouteCircuit -ExpressRouteCircuit $ExpressRouteCircuit

+ $ExpressRouteCircuitPeeringId = $ExpressRouteCircuit.Peerings[0].Id

+ ```

+**Existing circuit** - This example gets the details and Private Peering ID from an existing ExpressRoute circuit.

+ ```azurepowershell-interactive

+ $ExpressRouteCircuit = Get-AzExpressRouteCircuit -ResourceGroupName ["resource group name"] -Name ["expressroute circuit name"]

+ $ExpressRouteCircuitPeeringId = $ExpressRouteCircuit.Peerings[0].Id

+ ```

+## Connect your circuit to the gateway

+In this section, you connect an ExpressRoute (ER) circuit to your virtual hub's ExpressRoute gateway.

+* ExpressRoute Standard or Premium circuits that are in ExpressRoute Global Reach-supported locations can connect to a Virtual WAN ExpressRoute gateway and enjoy all Virtual WAN transit capabilities (VPN-to-VPN, VPN, and ExpressRoute transit).

+* ExpressRoute Standard and Premium circuits that are in non-Global Reach locations can connect to Azure resources, but won't be able to use Virtual WAN transit capabilities. ExpressRoute Local is also supported with Azure Virtual WAN virtual hubs.

+Use one of the following examples to connect your circuit. Both examples include optional authorization key steps.

+**Connect - example ER gateway** - This example connects the ExpressRoute circuit that you created earlier to the virtual hub's ExpressRoute gateway ($expressroutegatewayinhub).

+1. Run the following example command:

+ ```azurepowershell-interactive

+ $ExpressrouteConnection = New-AzExpressRouteConnection -ResourceGroupName $expressroutegatewayinhub.ResourceGroupName -ExpressRouteGatewayName $expressroutegatewayinhub.Name -Name "testConnection" -ExpressRouteCircuitPeeringId $ExpressRouteCircuitPeeringId -RoutingWeight 20

+ ```

+Optional - Connect by using ExpressRoute circuit's authorization key

+1. Create authorization key for the ExpressRoute circuit. For steps, see [How To Create Authorization](../expressroute/expressroute-howto-linkvnet-arm.md).

+1. Once authorization is created, get the authorization of the ER circuit.

+ ```azurepowershell-interactive

+ $authorizations = Get-AzExpressRouteCircuitAuthorization -ExpressRouteCircuit $ExpressRouteCircuit

+ ```

+1. Get the authorization key for the first key; use the index for other keys (i.e [1]).

+ ```azurepowershell-interactive

+ $authorizationskey = $authorizationskey[0].AuthorizationKey

+ ```

+1. Connect the ExpressRoute circuit to the virtual hub using the authorization key.

+ ```azurepowershell-interactive

+ $ExpressrouteConnection = New-AzExpressRouteConnection -ResourceGroupName $expressroutegatewayinhub.ResourceGroupName -ExpressRouteGatewayName $expressroutegatewayinhub.Name -Name "testConnectionpowershellauthkey" -ExpressRouteCircuitPeeringId $ExpressRouteCircuitPeeringId -RoutingWeight 2 -AuthorizationKey $authprizationskey

+ ```

+**Connect - existing ER gateway** - The steps in this example help you connect to an existing ExpressRoute gateway.

+1. Get the existing virtual hub ExpressRoute gateway details.

+ ```azurepowershell-interactive

+ $expressroutegatewayinhub = Get-AzExpressRouteGateway -ResourceId "[ERgatewayinhubID]"

+ ```

+1. Connect the ExpressRoute circuit to the virtual hub ExpressRoute gateway.

+ ```azurepowershell-interactive

+ $ExpressrouteConnection = New-AzExpressRouteConnection -ResourceGroupName $expressroutegatewayinhub.ResourceGroupName -ExpressRouteGatewayName $expressroutegatewayinhub.Name -Name "testConnection" -ExpressRouteCircuitPeeringId $ExpressRouteCircuitPeeringId -RoutingWeight 20

+ ```

+Optional - Connect by using ExpressRoute circuit's authorization key.

+1. Create authorization key for the ExpressRoute circuit. For steps, see [How To Create Authorization](../expressroute/expressroute-howto-linkvnet-arm.md).

+1. Once authorization is created, get the authorization of the ER circuit.

+ ```azurepowershell-interactive

+ $authorizations = Get-AzExpressRouteCircuitAuthorization -ExpressRouteCircuit $ExpressRouteCircuit

+ ```

+1. Get the authorization key for the first key; use the index for other keys (i.e [1]).

+ ```azurepowershell-interactive

+ $authorizationskey = $authorizationskey[0].AuthorizationKey

+ ```

+1. Connect the ExpressRoute circuit to the virtual hub ExpressRoute gateway.

+ ```azurepowershell-interactive

+ $ExpressrouteConnection = New-AzExpressRouteConnection -ResourceGroupName $expressroutegatewayinhub.ResourceGroupName -ExpressRouteGatewayName $expressroutegatewayinhub.Name -Name "testConnectionpowershellauthkey" -ExpressRouteCircuitPeeringId $ExpressRouteCircuitPeeringId -RoutingWeight 2 -AuthorizationKey $authprizationskey

+ ```

+### Test connectivity

+After the circuit connection is established, the virtual hub connection status will indicate 'this hub', implying the connection is established to the virtual hub ExpressRoute gateway. Wait approximately 5 minutes before you test connectivity from a client behind your ExpressRoute circuit, for example, a VM in the VNet that you created earlier.

+If you have sites connected to a Virtual WAN VPN gateway in the same virtual hub as the ExpressRoute gateway, you can have bidirectional connectivity between VPN and ExpressRoute end points. Dynamic routing (BGP) is supported. The ASN of the gateways in the virtual hub is fixed and can't be edited at this time.

+### To change gateway size

+In the following example, an ExpressRoute gateway is modified to 3 scale units.

+```azurepowershell-interactive

+Set-AzExpressRouteGateway -ResourceGroupName "testRG" -Name "testergw" -MinScaleUnits 3

+```

+## Next Steps

+Next, to learn more about Virtual WAN, see the [Virtual WAN FAQ](virtual-wan-faq.md).

+> [!NOTE]

+> If you want to inspect traffic destined to private endpoints using Azure Firewall in a secured virtual hub, see [Secure traffic destined to private endpoints in Azure Virtual WAN](../firewall-manager/private-link-inspection-secure-virtual-hub.md).

+You need to add /32 prefix for each private endpoint in the **Private traffic prefixes** under Security configuration of your Azure Firewall manager for them to be inspected via Azure Firewall in secured virtual hub. If these /32 prefixes are not configured, traffic destined to private endpoints will bypass Azure Firewall.

+Before you can create a virtual wan, you have to create a resource group to host the virtual wan or use an existing resource group. Use one of the following examples.

+**New resource group** - This example creates a new resource group named **testRG** in the **West US** location.

+**Existing resource group** - Use the following steps if you want to create the virtual wan in an already existing resource group.

+ Title: 'Tutorial: Create an ExpressRoute association to Azure Virtual WAN'

+# Tutorial: Create an ExpressRoute association to Virtual WAN - Azure portal

+Verify that you've met the following criteria before beginning your configuration:

+* Your virtual network doesn't have any virtual network gateways. If your virtual network has a gateway (either VPN or ExpressRoute), you must remove all gateways. This configuration requires that virtual networks are connected instead, to the Virtual WAN hub gateway.

+* Obtain an IP address range for your hub region. The hub is a virtual network that is created and used by Virtual WAN. The address range that you specify for the hub can't overlap with any of your existing virtual networks that you connect to. It also can't overlap with your address ranges that you connect to on-premises. If you're unfamiliar with the IP address ranges located in your on-premises network configuration, coordinate with someone who can provide those details for you.

+A virtual hub is a virtual network that is created and used by Virtual WAN. It can contain various gateways, such as VPN and ExpressRoute. In this section, you'll create an ExpressRoute gateway for your virtual hub. You can either create the gateway when you [create a new virtual hub](#newhub), or you can create the gateway in an [existing hub](#existinghub) by editing it.

+Once you've created an ExpressRoute gateway, you can view gateway details. Navigate to the hub, select **ExpressRoute**, and view the gateway.

+ * **Virtual network** - Select the virtual network you want to connect to this hub. The virtual network can't have an already existing virtual network gateway (neither VPN, nor ExpressRoute).

+Once the gateway is created, you can connect an [ExpressRoute circuit](../expressroute/expressroute-howto-circuit-portal-resource-manager.md) to it.

+* ExpressRoute Standard or Premium circuits that are in ExpressRoute Global Reach-supported locations can connect to a Virtual WAN ExpressRoute gateway and enjoy all Virtual WAN transit capabilities (VPN-to-VPN, VPN, and ExpressRoute transit).

+* ExpressRoute Standard and Premium circuits that are in non-Global Reach locations can connect to Azure resources, but won't be able to use Virtual WAN transit capabilities. ExpressRoute Local is also supported with Azure Virtual WAN virtual hubs.

+In the portal, go to the **Virtual hub -> Connectivity -> ExpressRoute** page. If you have access in your subscription to an ExpressRoute circuit, you'll see the circuit you want to use in the list of circuits. If you donΓÇÖt see any circuits, but have been provided with an authorization key and peer circuit URI, you can redeem and connect a circuit. See [To connect by redeeming an authorization key](#authkey).

+If you have sites connected to a Virtual WAN VPN gateway in the same hub as the ExpressRoute gateway, you can have bidirectional connectivity between VPN and ExpressRoute end points. Dynamic routing (BGP) is supported. The ASN of the gateways in the hub is fixed and can't be edited at this time.

+If you would like the Azure virtual hub to advertise the default route 0.0.0.0/0 to your ExpressRoute end points, you'll need to enable 'Propagate default route'.