+ npm start

+ "SignedOutCallbackPath": "/signout-oidc

+GET https://graph.microsoft.com/v1.0/organization/organization-id?$select=directorySizeQuota

+"extension_831374b3bd5041bfaa54263ec9e050fc_loyaltyNumber": "212342"

+* Use [scoping filters](define-conditional-rules-for-provisioning-user-accounts.md) in the provisioning app to define users that each app processes.

+* When configuring each provisioning app, select the parent AD domain from the dropdown of available AD domains. Selecting the parent domain ensures forest-wide lookup while generating unique values for attributes like *userPrincipalName*, *samAccountName* and *mail*.

+* Use [scoping filters](define-conditional-rules-for-provisioning-user-accounts.md) in the provisioning app to define users that each app processes.

+* When configuring the provisioning app, select the parent AD domain from the dropdown of available AD domains. Selecting the parent domain ensures forest-wide lookup while generating unique values for attributes like *userPrincipalName*, *samAccountName* and *mail*.

+By default, the provisioning connector app maps the HR user profile status to the user account status. The status is used to determine whether to enable or disable the user account.

+Attributes like CN, samAccountName, and the UPN have unique constraints. You may need to generate unique attribute values when you initiate the Joiners process.

+A deployment consists of stages ranging from the initial pilot to enabling user provisioning. At each stage, ensure that you're testing for expected results. Also, audit the provisioning cycles.

+|HR rehires an employee into a new role.|Behavior depends on how the cloud HR app is configured to generate employee IDs. If the old employee ID is used for a rehired employee, the connector enables the existing Active Directory account for the user. If the rehired employee gets a new employee ID, the connector creates a new Active Directory account for the user.|

+Azure AD can provide more insights into your organization's user provisioning usage and operational health through audit logs and reports.

+> To complete this task, you must have *Microsoft Entra Permissions Management Administrator* permissions. You can't enable Permissions Management as a user from another tenant who has signed in via B2B or via Azure Lighthouse.

+|  | [Ascent Solutions Microsoft Entra Permissions Management Rapid Risk Assessment](https://www.meetascent.com/resources/microsoft-entra-permissions-rapid-risk-assessment)

+|  | [Synergy Advisors Identity Optimization](https://synergyadvisors.biz/solutions-item/identity-optimization/)

+|  | [BDO Digital Managing Permissions Across Multicloud](https://www.bdodigital.com/services/security-compliance/cybersecurity/entra-permissions-management)

+## Prerequisite

+To view information on the Azure AD Insights tab, you must have Permissions Management Administrator role permissions.

+Connect-MgGraph -Scopes "Policy.ReadWrite.ApplicationConfiguration","Policy.Read.All","Application.ReadWrite.All"

+Learn about [authentication session management capabilities](../conditional-access/howto-conditional-access-session-lifetime.md) in Azure AD Conditional Access.

+ public MyIdentityLogger()

+

+

+- OS Type and Version

+1. On the **Members** page, select **Bulk operations** and choose, **Download members** to download a CSV file listing the group members.

+> Recommendations for your admins:

+> - Ensure all your admins sign in after enabling security defaults so that they can register for authentication methods.

+> - Have separate accounts for administration and standard productivity tasks to significantly reduce the number of times your admins are prompted for MFA.

+> [!WARNING]

+> Be aware of the SAP SAML assertion limits and impact of the length of SAP Cloud Foundry role collection names and amount of collections proxied by groups in SAP Cloud Identity Service. See SAP note [2732890](https://launchpad.support.sap.com/?sap-support-cross-site-visitor-id=b73c7292f9a46d52#/notes/2732890) for more information. Exceeded limits result in authorization issues.

+description: Microsoft Entra Identity Governance allows you to balance your organization's need for security and employee productivity with the right processes and visibility. You can define policies for how users should obtain access to your business critical applications integrated with Microsoft Entra Identity Governance.

+The following table illustrates how concepts in organizational role definitions you might be familiar with in other products correspond to capabilities in entitlement management.

+- Investigate further using Microsoft Defender for Identity

+## Can I use the same SAML certificate across different apps?

+When it's the first time configuring SSO on an enterprise app, we do provide a default SAML certificate that is used across Azure AD. However, if you need to use the same certificate across multiple apps that aren't the default Azure AD one, then you need to use an external Certificate Authority and upload the PFX file. The reason is that Azure AD doesn't provide access to private keys from internally issued certificates.

+ |externalId|String||✓

+> Identifier of this application is a fixed string value so only one instance can be configured in one tenant. If your company has more than one organization at Sauce Labs to be integrated with SAML SSO within a single Azure tenant, please refer to the following [documentation](https://docs.saucelabs.com/basics/sso/setting-up-sso-special-cases/#single-identity-provider-and-multiple-organizations-at-sauce-labs).

+To configure single sign-on on Sauce Labs side, please refer this [documentation](https://docs.saucelabs.com/basics/sso/setting-up-sso/#integrating-with-sauce-labs-service-provider) to set up SAML SSO connection properly on both sides. For any help or queries, please contact [Sauce Labs support team](mailto:support@saucelabs.com).

+ Title: Cluster access control with AKS-managed Azure Active Directory integration

+description: Learn how to access clusters when integrating Azure AD in your Azure Kubernetes Service (AKS) clusters.

+# Cluster access control with AKS-managed Azure Active Directory integration

+When you integrate Azure AD with your AKS cluster, you can use [Conditional Access][aad-conditional-access] or Privileged Identity Management (PIM) for just-in-time requests to control access to your cluster. This article shows you how to enable Conditional Access and PIM on your AKS clusters.

+> [!NOTE]

+> Azure AD Conditional Access and Privileged Identity Management are Azure AD Premium capabilities requiring a Premium P2 SKU. For more on Azure AD SKUs, see the [pricing guide][aad-pricing].

+## Before you begin

+* See [AKS-managed Azure Active Directory integration](./managed-azure-ad.md) for an overview and setup instructions.

+## Use Conditional Access with Azure AD and AKS

+1. In the Azure portal, go to the **Azure Active Directory** page and select **Enterprise applications**.

+2. Select **Conditional Access** > **Policies** > **New policy**.

+ :::image type="content" source="./media/managed-aad/conditional-access-new-policy.png" alt-text="Screenshot of adding a Conditional Access policy." lightbox="./media/managed-aad/conditional-access-new-policy.png":::

+3. Enter a name for the policy, such as *aks-policy*.

+4. Under **Assignments**, select **Users and groups**. Choose the users and groups you want to apply the policy to. In this example, choose the same Azure AD group that has administrator access to your cluster.

+ :::image type="content" source="./media/managed-aad/conditional-access-users-groups.png" alt-text="Screenshot of selecting users or groups to apply the Conditional Access policy." lightbox="./media/managed-aad/conditional-access-users-groups.png":::

+5. Under **Cloud apps or actions** > **Include**, select **Select apps**. Search for **Azure Kubernetes Service** and select **Azure Kubernetes Service AAD Server**.

+ :::image type="content" source="./media/managed-aad/conditional-access-apps.png" alt-text="Screenshot of selecting Azure Kubernetes Service AD Server for applying the Conditional Access policy." lightbox="./media/managed-aad/conditional-access-apps.png":::

+6. Under **Access controls** > **Grant**, select **Grant access**, **Require device to be marked as compliant**, and **Require all the selected controls**.

+ :::image type="content" source="./media/managed-aad/conditional-access-grant-compliant.png" alt-text="Screenshot of selecting to only allow compliant devices for the Conditional Access policy." lightbox="./media/managed-aad/conditional-access-grant-compliant.png" :::

+7. Confirm your settings, set **Enable policy** to **On**, and then select **Create**.

+ :::image type="content" source="./media/managed-aad/conditional-access-enable-policy.png" alt-text="Screenshot of enabling the Conditional Access policy." lightbox="./media/managed-aad/conditional-access-enable-policy.png":::

+### Verify your Conditional Access policy has been successfully listed

+1. Get the user credentials to access the cluster using the [`az aks get-credentials`][az-aks-get-credentials] command.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group myResourceGroup --name myManagedCluster

+ ```

+2. Follow the instructions to sign in.

+3. View the nodes in the cluster using the `kubectl get nodes` command.

+ ```azurecli-interactive

+ kubectl get nodes

+ ```

+4. In the Azure portal, navigate to **Azure Active Directory** and select **Enterprise applications** > **Activity** > **Sign-ins**.

+5. Under the **Conditional Access** column you should see a status of *Success*. Select the event and then select the **Conditional Access** tab. Your Conditional Access policy will be listed.

+ :::image type="content" source="./media/managed-aad/conditional-access-sign-in-activity.png" alt-text="Screenshot that shows failed sign-in entry due to Conditional Access policy." lightbox="./media/managed-aad/conditional-access-sign-in-activity.png":::

+## Configure just-in-time cluster access with Azure AD and AKS

+1. In the Azure portal, go to **Azure Active Directory** and select **Properties**.

+2. Note the value listed under **Tenant ID**. It will be referenced in a later step as `<tenant-id>`.

+ :::image type="content" source="./media/managed-aad/jit-get-tenant-id.png" alt-text="Screenshot of the Azure portal screen for Azure Active Directory with the tenant's ID highlighted." lightbox="./media/managed-aad/jit-get-tenant-id.png":::

+3. Select **Groups** > **New group**.

+ :::image type="content" source="./media/managed-aad/jit-create-new-group.png" alt-text="Screenshot of the Azure portal Active Directory groups screen with the New Group option highlighted." lightbox="./media/managed-aad/jit-create-new-group.png":::

+4. Verify the group type **Security** is selected and specify a group name, such as *myJITGroup*. Under the option **Azure AD roles can be assigned to this group (Preview)**, select **Yes** and then select **Create**.

+ :::image type="content" source="./media/managed-aad/jit-new-group-created.png" alt-text="Screenshot of the new group creation screen in the Azure portal." lightbox="./media/managed-aad/jit-new-group-created.png":::

+5. On the **Groups** page, select the group you just created and note the Object ID. It will be referenced in a later step as `<object-id>`.

+ :::image type="content" source="./media/managed-aad/jit-get-object-id.png" alt-text="Screenshot of the Azure portal screen for the just-created group with the Object ID highlighted." lightbox="./media/managed-aad/jit-get-object-id.png":::

+6. Create the AKS cluster with AKS-managed Azure AD integration using the [`az aks create`][az-aks-create] command with the `--aad-admin-group-objects-ids` and `--aad-tenant-id parameters` and include the values noted in the steps earlier.

+ ```azurecli-interactive

+ az aks create -g myResourceGroup -n myManagedCluster --enable-aad --aad-admin-group-object-ids <object-id> --aad-tenant-id <tenant-id>

+ ```

+7. In the Azure portal, select **Activity** > **Privileged Access (Preview)** > **Enable Privileged Access**.

+ :::image type="content" source="./media/managed-aad/jit-enabling-priv-access.png" alt-text="Screenshot of the Privileged access (Preview) page in the Azure portal with Enable privileged access highlighted." lightbox="./media/managed-aad/jit-enabling-priv-access.png":::

+8. To grant access, select **Add assignments**.

+ :::image type="content" source="./media/managed-aad/jit-add-active-assignment.png" alt-text="Screenshot of the Privileged access (Preview) screen in the Azure portal after enabling. The option to Add assignments is highlighted." lightbox="./media/managed-aad/jit-add-active-assignment.png":::

+9. From the **Select role** drop-down list, select the users and groups you want to grant cluster access. These assignments can be modified at any time by a group administrator. Then select **Next**.

+ :::image type="content" source="./media/managed-aad/jit-adding-assignment.png" alt-text="Screenshot of the Add assignments Membership screen in the Azure portal with a sample user selected to be added as a member. The Next option is highlighted." lightbox="./media/managed-aad/jit-adding-assignment.png":::

+10. Under **Assignment type**, select **Active** and then specify the desired duration. Provide a justification and then select **Assign**.

+ :::image type="content" source="./media/managed-aad/jit-set-active-assignment-details.png" alt-text="Screenshot of the Add assignments Setting screen in the Azure portal. An assignment type of Active is selected and a sample justification has been given. The Assign option is highlighted." lightbox="./media/managed-aad/jit-set-active-assignment-details.png":::

+For more information about assignment types, see [Assign eligibility for a privileged access group (preview) in Privileged Identity Management][aad-assignments].

+### Verify just-in-time access is working by accessing the cluster

+1. Get the user credentials to access the cluster using the [`az aks get-credentials`][az-aks-get-credentials] command.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group myResourceGroup --name myManagedCluster

+ ```

+2. Follow the steps to sign in.

+3. Use the `kubectl get nodes` command to view the nodes in the cluster.

+ ```azurecli-interactive

+ kubectl get nodes

+ ```

+4. Note the authentication requirement and follow the steps to authenticate. If successful, you should see an output similar to the following example output:

+ ```output

+ To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code AAAAAAAAA to authenticate.

+ NAME STATUS ROLES AGE VERSION

+ aks-nodepool1-61156405-vmss000000 Ready agent 6m36s v1.18.14

+ aks-nodepool1-61156405-vmss000001 Ready agent 6m42s v1.18.14

+ aks-nodepool1-61156405-vmss000002 Ready agent 6m33s v1.18.14

+ ```

+### Apply just-in-time access at the namespace level

+1. Integrate your AKS cluster with [Azure RBAC](manage-azure-rbac.md).

+2. Associate the group you want to integrate with just-in-time access with a namespace in the cluster using the [`az role assignment create`][az-role-assignment-create] command.

+ ```azurecli-interactive

+ az role assignment create --role "Azure Kubernetes Service RBAC Reader" --assignee <AAD-ENTITY-ID> --scope $AKS_ID/namespaces/<namespace-name>

+ ```

+3. Associate the group you configured at the namespace level with PIM to complete the configuration.

+## Troubleshooting

+If `kubectl get nodes` returns an error similar to the following:

+```output

+Error from server (Forbidden): nodes is forbidden: User "aaaa11111-11aa-aa11-a1a1-111111aaaaa" cannot list resource "nodes" in API group "" at the cluster scope

+```

+Make sure the admin of the security group has given your account an *Active* assignment.

+## Next steps

+* Use [kubelogin](https://github.com/Azure/kubelogin) to access features for Azure authentication that aren't available in kubectl.

+<!-- LINKS - External -->

+[aad-pricing]: https://azure.microsoft.com/pricing/details/active-directory/

+<!-- LINKS - Internal -->

+[aad-conditional-access]: ../active-directory/conditional-access/overview.md

+[az-aks-get-credentials]: /cli/azure/aks#az_aks_get_credentials

+[az-role-assignment-create]: /cli/azure/role/assignment#az_role_assignment_create

+[aad-assignments]: ../active-directory/privileged-identity-management/groups-assign-member-owner.md#assign-an-owner-or-member-of-a-group

+[az-aks-create]: /cli/azure/aks#az_aks_create

+Troubleshooting Azure Kubernetes Service (AKS) cluster issues plays an important role in maintaining your cluster, especially if your cluster is running mission-critical workloads. AKS Diagnostics (preview) is an intelligent, self-diagnostic experience with the following features:

+* Requires no extra configuration or billing costs.

+1. Choose a category that best describes the issue of your cluster, like _Cluster Node Issues_, using the keywords in the homepage tile or typing a keyword that best describes your issue in the search bar.

+After selecting a category, you can view a diagnostic report specific to your cluster. Diagnostic reports intelligently call out any issues in your cluster with status icons. You can drill down on each topic by clicking **More Info** to see a detailed description of:

+* Logging data

+Automatically completed upgrades are functionally the same as manual upgrades. The timing of upgrades is determined by the [selected auto-upgrade channel][planned-maintenance]. When making changes to auto-upgrade, allow 24 hours for the changes to take effect.

+> AKS has a new improved [AKS-managed Azure AD][managed-aad] experience that doesn't require you to manage server or client applications. If you want to migrate follow the instructions [here][managed-aad-migrate].

+[managed-aad]: managed-azure-ad.md

+[managed-aad-migrate]: managed-azure-ad.md#upgrade-a-legacy-azure-ad-cluster-to-aks-managed-azure-ad-integration

+[azure-ad-aks-cli]: managed-azure-ad.md

+[enable-azure-ad-integration-existing-cluster]: managed-azure-ad.md#use-an-existing-cluster

+Learn how to integrate AKS with Azure AD with our [AKS-managed Azure AD integration how-to guide](managed-azure-ad.md).

+[aks-aad]: managed-azure-ad.md

+[aks-aad]: ./managed-azure-ad.md

+To illustrate the application of a cluster with outbound type using a user-defined route, a cluster can be configured on a virtual network with an Azure Firewall on its own subnet. See this example on the [restrict egress traffic with Azure firewall example](limit-egress-traffic.md).

+[aad]: managed-azure-ad.md

+[aks-firewall-requirements]: outbound-rules-control-egress.md#azure-global-required-network-rules

+[aks-firewall-requirements]: outbound-rules-control-egress.md#azure-global-required-network-rules

+> The Kubernetes resource view in the Azure portal is only supported by [managed-AAD enabled clusters](managed-azure-ad.md) or non-AAD enabled clusters. If you're using a managed-AAD enabled cluster, your AAD user or identity needs to have the respective roles/role bindings to access the Kubernetes API and the permission to pull the [user `kubeconfig`](control-kubeconfig-access.md).

+[aks-managed-aad]: managed-azure-ad.md

+[cli-aad-upgrade]: managed-azure-ad.md#upgrade-a-legacy-azure-ad-cluster-to-aks-managed-azure-ad-integration

+ Title: Control egress traffic using Azure Firewall in Azure Kubernetes Service (AKS)

+description: Learn how to control egress traffic using Azure Firewall in Azure Kubernetes Service (AKS)

+#Customer intent: As a cluster operator, I want to restrict egress traffic for nodes to only access defined ports and addresses and improve cluster security.

+# Control egress traffic using Azure Firewall in Azure Kubernetes Service (AKS)

+This article provides a walkthrough of how to use the [Outbound network and FQDN rules for AKS clusters][outbound-fqdn-rules] to control egress traffic using Azure Firewall in AKS. To simplify this configuration, Azure Firewall provides an Azure Kubernetes Service (`AzureKubernetesService`) FQDN that restricts outbound traffic from the AKS cluster. This article also provides an example of how to configure public inbound traffic via the firewall.

+> The FQDN tag contains all the FQDNs listed in [Outbound network and FQDN rules for AKS clusters][outbound-fqdn-rules] and is automatically updated.

+>

+> For production scenarios, we recommend having a *minimum of 20 frontend IPs* on the Azure Firewall to avoid SNAT port exhaustion issues.

+The following information provides an example architecture of the deployment:

+* **Public ingress is forced to flow through firewall filters**

+ * AKS agent nodes are isolated in a dedicated subnet

+ * [Azure Firewall](../firewall/overview.md) is deployed in its own subnet

+ * A DNAT rule translates the firewall public IP into the load balancer frontend IP

+* **Outbound requests start from agent nodes to the Azure Firewall internal IP using a [user-defined route (UDR)](egress-outboundtype.md)**

+ * Requests from AKS agent nodes follow a UDR that has been placed on the subnet the AKS cluster was deployed into

+ * Access to the AKS control plane can be protected by [API server authorized IP ranges](./api-server-authorized-ip-ranges.md), including the firewall public frontend IP address

+* **Internal traffic**

+ * You can use an [internal load balancer](internal-lb.md) for internal traffic, which you could isolate on its own subnet, instead of or alongside a [public load balancer](load-balancer-standard.md)

+## Set configuration using environment variables

+## Create a virtual network with multiple subnets

+Provision a virtual network with two separate subnets: one for the cluster and one for the firewall. Optionally, you can create one for internal service ingress.

+1. Create a resource group using the [`az group create`][az-group-create] command.

+ ```azurecli

+ az group create --name $RG --location $LOC

+ ```

+2. Create a virtual network with two subnets to host the AKS cluster and the Azure Firewall using the [`az network vnet create`][az-network-vnet-create] and [`az network vnet subnet create`][az-network-vnet-subnet-create] commands.

+ ```azurecli

+ # Dedicated virtual network with AKS subnet

+ az network vnet create \

+ --resource-group $RG \

+ --name $VNET_NAME \

+ --location $LOC \

+ --address-prefixes 10.42.0.0/16 \

+ --subnet-name $AKSSUBNET_NAME \

+ --subnet-prefix 10.42.1.0/24

+ # Dedicated subnet for Azure Firewall (Firewall name can't be changed)

+ az network vnet subnet create \

+ --resource-group $RG \

+ --vnet-name $VNET_NAME \

+ --name $FWSUBNET_NAME \

+ --address-prefix 10.42.2.0/24

+ ```

+## Create and set up an Azure Firewall with a UDR

+You need to configure Azure Firewall inbound and outbound rules. The main purpose of the firewall is to enable organizations to configure granular ingress and egress traffic rules into and out of the AKS cluster.

+>

+> If your cluster or application creates a large number of outbound connections directed to the same or a small subset of destinations, you might require more firewall frontend IPs to avoid maxing out the ports per frontend IP.

+>

+> For more information on how to create an Azure Firewall with multiple IPs, see [Create an Azure Firewall with multiple public IP addresses using Bicep](../firewall/quick-create-multiple-ip-bicep.md).

+

+1. Create a standard SKU public IP resource using the [`az network public-ip create`][az-network-public-ip-create] command. This resource will be used as the Azure Firewall frontend address.

+ ```azurecli

+ az network public-ip create -g $RG -n $FWPUBLICIP_NAME -l $LOC --sku "Standard"

+ ```

+2. Register the [Azure Firewall preview CLI extension](https://github.com/Azure/azure-cli-extensions/tree/main/src/azure-firewall) to create an Azure Firewall using the [`az extension add`][az-extension-add] command.

+ ```azurecli

+ az extension add --name azure-firewall

+ ```

+3. Create an Azure Firewall and enable DNS proxy using the [`az network firewall create`][az-network-firewall-create] command and setting the `--enable-dns-proxy` to `true`.

+ ```azurecli

+ az network firewall create -g $RG -n $FWNAME -l $LOC --enable-dns-proxy true

+ ```

+ Setting up the public IP address to the Azure Firewall may take a few minutes. Once it's ready, the IP address created earlier can be assigned to the firewall front end.

+ > [!NOTE]

+ >

+ > To leverage FQDN on network rules, we need DNS proxy enabled. When DNS proxy is enabled, the firewall listens on port 53 and forwards DNS requests to the DNS server specified above. This allows the firewall to translate the FQDN automatically.

+4. Create an Azure Firewall IP configuration using the [`az network firewall ip-config create`][az-network-firewall-ip-config-create] command.

+ ```azurecli

+ az network firewall ip-config create -g $RG -f $FWNAME -n $FWIPCONFIG_NAME --public-ip-address $FWPUBLICIP_NAME --vnet-name $VNET_NAME

+ ```

+5. Once the previous command succeeds, save the firewall frontend IP address for configuration later.

+ ```azurecli

+ FWPUBLIC_IP=$(az network public-ip show -g $RG -n $FWPUBLICIP_NAME --query "ipAddress" -o tsv)

+ FWPRIVATE_IP=$(az network firewall show -g $RG -n $FWNAME --query "ipConfigurations[0].privateIpAddress" -o tsv)

+ ```

+ > [!NOTE]

+ >

+ > If you use secure access to the AKS API server with [authorized IP address ranges](./api-server-authorized-ip-ranges.md), you need to add the firewall public IP into the authorized IP range.

+Azure automatically routes traffic between Azure subnets, virtual networks, and on-premises networks. If you want to change any of Azure's default routing, you can create a route table.

+1. Create an empty route table to be associated with a given subnet using the [`az network route-table create`][az-network-route-table-create] command. The route table will define the next hop as the Azure Firewall created above. Each subnet can have zero or one route table associated to it.

+ ```azurecli

+ az network route-table create -g $RG -l $LOC --name $FWROUTE_TABLE_NAME

+ ```

+2. Create routes in the route table for the subnets using the [`az network route-table route create`][az-network-route-table-route-create] command.

+ ```azurecli

+ az network route-table route create -g $RG --name $FWROUTE_NAME --route-table-name $FWROUTE_TABLE_NAME --address-prefix 0.0.0.0/0 --next-hop-type VirtualAppliance --next-hop-ip-address $FWPRIVATE_IP

+ az network route-table route create -g $RG --name $FWROUTE_NAME_INTERNET --route-table-name $FWROUTE_TABLE_NAME --address-prefix $FWPUBLIC_IP/32 --next-hop-type Internet

+ ```

+For information on how to override Azure's default system routes or add additional routes to a subnet's route table, see the [virtual network route table documentation](../virtual-network/virtual-networks-udr-overview.md#user-defined).

+### Add firewall rules

+>

+> For applications outside of the kube-system or gatekeeper-system namespaces that need to talk to the API server, an additional network rule to allow TCP communication to port 443 for the API server IP in addition to adding application rule for fqdn-tag `AzureKubernetesService` is required.

+This section covers three network rules and an application rule you can use to configure on your firewall. You may need to adapt these rules based on your deployment.

+* The first network rule allows access to port 9000 via TCP.

+* The second network rule allows access to port 1194 and 123 via UDP. If you're deploying to Azure China 21Vianet, see the [Azure China 21Vianet required network rules](./outbound-rules-control-egress.md#azure-china-21vianet-required-network-rules). Both these rules will only allow traffic destined to the Azure Region CIDR in this article, which is East US.

+* The third network rule opens port 123 to `ntp.ubuntu.com` FQDN via UDP. Adding an FQDN as a network rule is one of the specific features of Azure Firewall, so you'll need to adapt it when using your own options.

+* The application rule covers all needed FQDNs accessible through TCP port 443 and port 80.

+1. Create the network rules using the [`az network firewall network-rule create`][az-network-firewall-network-rule-create] command.

+ ```azurecli

+ az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'apiudp' --protocols 'UDP' --source-addresses '*' --destination-addresses "AzureCloud.$LOC" --destination-ports 1194 --action allow --priority 100

+ az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'apitcp' --protocols 'TCP' --source-addresses '*' --destination-addresses "AzureCloud.$LOC" --destination-ports 9000

+ az network firewall network-rule create -g $RG -f $FWNAME --collection-name 'aksfwnr' -n 'time' --protocols 'UDP' --source-addresses '*' --destination-fqdns 'ntp.ubuntu.com' --destination-ports 123

+ ```

+2. Create the application rule using the [`az network firewall application-rule create`][az-network-firewall-application-rule-create] command.

+ ```azurecli

+ az network firewall application-rule create -g $RG -f $FWNAME --collection-name 'aksfwar' -n 'fqdn' --source-addresses '*' --protocols 'http=80' 'https=443' --fqdn-tags "AzureKubernetesService" --action allow --priority 100

+ ```

+To learn more about Azure Firewall, see the [Azure Firewall documentation](../firewall/overview.md).

+To associate the cluster with the firewall, the dedicated subnet for the cluster's subnet must reference the route table created above. Use the [`az network vnet subnet update`][az-network-vnet-subnet-update] command to associate the route table to AKS.

+## Deploy an AKS cluster with a UPR outbound type to the existing network

+Now, you can deploy an AKS cluster into the existing virtual network. You will use the [`userDefinedRouting` outbound type](egress-outboundtype.md), which ensures that any outbound traffic is forced through the firewall and no other egress paths will exist. The [`loadBalancer` outbound type](egress-outboundtype.md#outbound-type-of-loadbalancer) can also be used.

+The target subnet to be deployed into is defined with the environment variable, `$SUBNETID`. Set the value for the subnet ID using the following command:

+> You can add additional features to the cluster deployment, such as [**private clusters**](private-clusters.md).

+> You can add the AKS feature for [**API server authorized IP ranges**](api-server-authorized-ip-ranges.md) to limit API server access to only the firewall's public endpoint. The authorized IP ranges feature is denoted in the diagram as optional. When enabling the authorized IP range feature to limit API server access, your developer tools must use a jumpbox from the firewall's virtual network, or you must add all developer endpoints to the authorized IP range.

+### Create an AKS cluster with system-assigned identities

+> AKS will create a system-assigned kubelet identity in the node resource group if you don't [specify your own kubelet managed identity][Use a pre-created kubelet managed identity].

+>

+> For user-defined routing, system-assigned identity only supports the CNI network plugin.

+Create an AKS cluster using a system-assigned managed identity with the CNI network plugin using the [`az aks create`][az-aks-create] command.

+### Create user-assigned identities

+If you don't have user-assigned identities, follow the steps in this section. If you already have user-assigned identities, skip to [Create an AKS cluster with user-assigned identities](#create-an-aks-cluster-with-user-assigned-identities).

+1. Create a control plane managed identity using the [`az identity create`][az-identity-create] command.

+ ```azurecli-interactive

+ az identity create --name myIdentity --resource-group myResourceGroup

+ ```

+ The output should resemble the following example output:

+ ```output

+ {

+ "clientId": "<client-id>",

+ "clientSecretUrl": "<clientSecretUrl>",

+ "id": "/subscriptions/<subscriptionid>/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myIdentity",

+ "location": "westus2",

+ "name": "myIdentity",

+ "principalId": "<principal-id>",

+ "resourceGroup": "myResourceGroup",

+ "tags": {},

+ "tenantId": "<tenant-id>",

+ "type": "Microsoft.ManagedIdentity/userAssignedIdentities"

+ }

+ ```

+2. Create a kubelet managed identity using the [`az identity create`][az-identity-create] command.

+ ```azurecli

+ az identity create --name myKubeletIdentity --resource-group myResourceGroup

+ ```

+ The output should resemble the following example output:

+ ```output

+ {

+ "clientId": "<client-id>",

+ "clientSecretUrl": "<clientSecretUrl>",

+ "id": "/subscriptions/<subscriptionid>/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myKubeletIdentity",

+ "location": "westus2",

+ "name": "myKubeletIdentity",

+ "principalId": "<principal-id>",

+ "resourceGroup": "myResourceGroup",

+ "tags": {},

+ "tenantId": "<tenant-id>",

+ "type": "Microsoft.ManagedIdentity/userAssignedIdentities"

+ }

+ ```

+ > [!NOTE]

+ > If you create your own VNet and route table where the resources are outside of the worker node resource group, the CLI will add the role assignment automatically. If you're using an ARM template or other client, you need to use the Principal ID of the cluster managed identity to perform a [role assignment][add role to identity].

+### Create an AKS cluster with user-assigned identities

+Create an AKS cluster with your existing identities in the subnet using the [`az aks create`][az-aks-create] command and providing your control plane identity resource ID kubelet managed identity.

+## Enable developer access to the API server

+If you used authorized IP ranges for your cluster in the previous step, you need to add your developer tooling IP addresses to the AKS cluster list of approved IP ranges so you access the API server from there. You can also configure a jumpbox with the needed tooling inside a separate subnet in the firewall's virtual network.

+1. Retrieve your IP address using the following command:

+ ```bash

+ CURRENT_IP=$(dig @resolver1.opendns.com ANY myip.opendns.com +short)

+ ```

+2. Add the IP address to the approved ranges using the [`az aks update`][az-aks-update] command.

+ ```azurecli

+ az aks update -g $RG -n $AKSNAME --api-server-authorized-ip-ranges $CURRENT_IP/32

+ ```

+3. Configure `kubectl` to connect to your AKS cluster using the [`az aks get-credentials`][az-aks-get-credentials] command.

+ ```azurecli

+ az aks get-credentials -g $RG -n $AKSNAME

+ ```

+## Deploy a public service

+You can now start exposing services and deploying applications to this cluster. In this example, we'll expose a public service, but you also might want to expose an internal service using an [internal load balancer](internal-lb.md).

+1. Copy the following YAML and save it as a file named `example.yaml`.

+ ```yaml

+ # voting-storage-deployment.yaml

+ apiVersion: apps/v1

+ kind: Deployment

+ metadata:

+ name: voting-storage

+ spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-storage

+ template:

+ metadata:

+ labels:

+ app: voting-storage

+ spec:

+ containers:

+ - name: voting-storage

+ image: mcr.microsoft.com/aks/samples/voting/storage:2.0

+ args: ["--ignore-db-dir=lost+found"]

+ resources:

+ requests:

+ cpu: 100m

+ memory: 128Mi

+ limits:

+ cpu: 250m

+ memory: 256Mi

+ ports:

+ - containerPort: 3306

+ name: mysql

+ volumeMounts:

+ - name: mysql-persistent-storage

+ mountPath: /var/lib/mysql

+ env:

+ - name: MYSQL_ROOT_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_ROOT_PASSWORD

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+ volumes:

+ - name: mysql-persistent-storage

+ persistentVolumeClaim:

+ claimName: mysql-pv-claim

+

+ # voting-storage-secret.yaml

+ apiVersion: v1

+ kind: Secret

+ metadata:

+ name: voting-storage-secret

+ type: Opaque

+ data:

+ MYSQL_USER: ZGJ1c2Vy

+ MYSQL_PASSWORD: UGFzc3dvcmQxMg==

+ MYSQL_DATABASE: YXp1cmV2b3Rl

+ MYSQL_ROOT_PASSWORD: UGFzc3dvcmQxMg==

+

+ # voting-storage-pv-claim.yaml

+ apiVersion: v1

+ kind: PersistentVolumeClaim

+ metadata:

+ name: mysql-pv-claim

+ spec:

+ accessModes:

+ - ReadWriteOnce

+ resources:

+ requests:

+ storage: 1Gi

+

+ # voting-storage-service.yaml

+ apiVersion: v1

+ kind: Service

+ name: voting-storage

+ labels:

+ ports:

+ - port: 3306

+ name: mysql

+ selector:

+ app: voting-storage

+

+ # voting-app-deployment.yaml

+ apiVersion: apps/v1

+ kind: Deployment

+ name: voting-app

+ spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-app

+ template:

+ metadata:

+ labels:

+ app: voting-app

+ spec:

+ containers:

+ - name: voting-app

+ image: mcr.microsoft.com/aks/samples/voting/app:2.0

+ imagePullPolicy: Always

+ ports:

+ - containerPort: 8080

+ name: http

+ env:

+ - name: MYSQL_HOST

+ value: "voting-storage"

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+ - name: ANALYTICS_HOST

+ value: "voting-analytics"

+

+ # voting-app-service.yaml

+ apiVersion: v1

+ kind: Service

+ metadata:

+ name: voting-app

+ labels:

+ type: LoadBalancer

+ ports:

+ - port: 80

+ targetPort: 8080

+ name: http

+ selector:

+ app: voting-app

+

+ # voting-analytics-deployment.yaml

+ apiVersion: apps/v1

+ kind: Deployment

+ name: voting-analytics

+ spec:

+ replicas: 1

+ selector:

+ matchLabels:

+ app: voting-analytics

+ version: "2.0"

+ template:

+ metadata:

+ labels:

+ app: voting-analytics

+ version: "2.0"

+ spec:

+ containers:

+ - name: voting-analytics

+ image: mcr.microsoft.com/aks/samples/voting/analytics:2.0

+ imagePullPolicy: Always

+ ports:

+ - containerPort: 8080

+ name: http

+ env:

+ - name: MYSQL_HOST

+ value: "voting-storage"

+ - name: MYSQL_USER

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_USER

+ - name: MYSQL_PASSWORD

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_PASSWORD

+ - name: MYSQL_DATABASE

+ valueFrom:

+ secretKeyRef:

+ name: voting-storage-secret

+ key: MYSQL_DATABASE

+

+ # voting-analytics-service.yaml

+ apiVersion: v1

+ kind: Service

+ metadata:

+ name: voting-analytics

+ labels:

+ ports:

+ - port: 8080

+ name: http

+ selector:

+ app: voting-analytics

+ ```

+2. Deploy the service using the `kubectl apply` command.

+ ```bash

+ kubectl apply -f example.yaml

+ ```

+## Add a DNAT rule to Azure Firewall

+>

+> When you use Azure Firewall to restrict egress traffic and create a UDR to force all egress traffic, make sure you create an appropriate DNAT rule in Azure Firewall to correctly allow ingress traffic. Using Azure Firewall with a UDR breaks the ingress setup due to asymmetric routing. The issue occurs if the AKS subnet has a default route that goes to the firewall's private IP address, but you're using a public load balancer - ingress or Kubernetes service of type `loadBalancer`. In this case, the incoming load balancer traffic is received via its public IP address, but the return path goes through the firewall's private IP address. Because the firewall is stateful, it drops the returning packet because the firewall isn't aware of an established session. To learn how to integrate Azure Firewall with your ingress or service load balancer, see [Integrate Azure Firewall with Azure Standard Load Balancer](../firewall/integrate-lb.md).

+To configure inbound connectivity, you need to write a DNAT rule to the Azure Firewall. To test connectivity to your cluster, a rule is defined for the firewall frontend public IP address to route to the internal IP exposed by the internal service. The destination address can be customized. The translated address must be the IP address of the internal load balancer. The translated port must be the exposed port for your Kubernetes service. You also need to specify the internal IP address assigned to the load balancer created by the Kubernetes service.

+1. Get the internal IP address assigned to the load balancer using the `kubectl get services` command.

+ ```bash

+ kubectl get services

+ ```

+ The IP address will be listed in the `EXTERNAL-IP` column, as shown in the following example output:

+ ```bash

+ NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

+ kubernetes ClusterIP 10.41.0.1 <none> 443/TCP 10h

+ voting-analytics ClusterIP 10.41.88.129 <none> 8080/TCP 9m

+ voting-app LoadBalancer 10.41.185.82 20.39.18.6 80:32718/TCP 9m

+ voting-storage ClusterIP 10.41.221.201 <none> 3306/TCP 9m

+ ```

+2. Get the service IP using the `kubectl get svc voting-app` command.

+ ```bash

+ SERVICE_IP=$(kubectl get svc voting-app -o jsonpath='{.status.loadBalancer.ingress[*].ip}')

+ ```

+3. Add the NAT rule using the [`az network firewall nat-rule create`][az-network-firewall-nat-rule-create] command.

+ ```azurecli

+ az network firewall nat-rule create --collection-name exampleset --destination-addresses $FWPUBLIC_IP --destination-ports 80 --firewall-name $FWNAME --name inboundrule --protocols Any --resource-group $RG --source-addresses '*' --translated-port 80 --action Dnat --priority 100 --translated-address $SERVICE_IP

+ ```

+## Validate connectivity

+You should see the AKS voting app. In this example, the firewall public IP was `52.253.228.132`.

+To clean up Azure resources, delete the AKS resource group using the [`az group delete`][az-group-delete] command.

+In this article, you learned how to secure your outbound traffic using Azure Firewall. If needed, you can generalize the steps above to forward the traffic to your preferred egress solution following the [Outbound Type `userDefinedRoute` documentation](egress-outboundtype.md).

+[az-group-create]: /cli/azure/group#az_group_create

+[outbound-fqdn-rules]: ./outbound-rules-control-egress.md

+[az-network-vnet-create]: /cli/azure/network/vnet#az_network_vnet_create

+[az-network-vnet-subnet-create]: /cli/azure/network/vnet/subnet#az_network_vnet_subnet_create

+[az-network-vnet-subnet-update]: /cli/azure/network/vnet/subnet#az_network_vnet_subnet_update

+[az-network-public-ip-create]: /cli/azure/network/public-ip#az_network_public_ip_create

+[az-extension-add]: /cli/azure/extension#az_extension_add

+[az-network-firewall-create]: /cli/azure/network/firewall#az_network_firewall_create

+[az-network-firewall-ip-config-create]: /cli/azure/network/firewall/ip-config#az_network_firewall_ip_config_create

+[az-network-route-table-create]: /cli/azure/network/route-table#az_network_route_table_create

+[az-network-route-table-route-create]: /cli/azure/network/route-table/route#az_network_route_table_route_create

+[az-network-firewall-network-rule-create]: /cli/azure/network/firewall/network-rule#az_network_firewall_network_rule_create

+[az-network-firewall-application-rule-create]: /cli/azure/network/firewall/application-rule#az_network_firewall_application_rule_create

+[az-aks-create]: /cli/azure/aks#az_aks_create

+[az-aks-update]: /cli/azure/aks#az_aks_update

+[az-network-firewall-nat-rule-create]: /cli/azure/network/firewall/nat-rule#az-network-firewall-nat-rule-create

+[az-group-delete]: /cli/azure/group#az_group_delete

+The following example terminates an operation on a node pool on a specified cluster.

+The following example terminates an operation on a specified cluster.

+The following example terminates a process for a specified cluster.

+When you leverage [integrated authentication between Azure Active Directory (Azure AD) and AKS](managed-azure-ad.md), you can use Azure AD users, groups, or service principals as subjects in [Kubernetes role-based access control (Kubernetes RBAC)][kubernetes-rbac]. This feature frees you from having to separately manage user identities and credentials for Kubernetes. However, you still have to set up and manage Azure RBAC and Kubernetes RBAC separately.

+* You need managed Azure AD integration enabled on your cluster before you can add Azure RBAC for Kubernetes authorization. If you need to enable managed Azure AD integration, see [Use Azure AD in AKS](managed-azure-ad.md).

+[managed-aad]: ./managed-azure-ad.md

+ Title: Manage local accounts with AKS-managed Azure Active Directory integration

+description: Learn how to managed local accounts when integrating Azure AD in your Azure Kubernetes Service (AKS) clusters.

+# Manage local accounts with AKS-managed Azure Active Directory integration

+When you deploy an AKS cluster, local accounts are enabled by default. Even when you enable RBAC or Azure AD integration, `--admin` access still exists as a non-auditable backdoor option. This article shows you how to disable local accounts on an existing cluster, create a new cluster with local accounts disabled, and re-enable local accounts on existing clusters.

+## Before you begin

+* See [AKS-managed Azure Active Directory integration](./managed-azure-ad.md) for an overview and setup instructions.

+## Disable local accounts

+You can disable local accounts using the parameter `disable-local-accounts`. The `properties.disableLocalAccounts` field has been added to the managed cluster API to indicate whether the feature is enabled or not on the cluster.

+> [!NOTE]

+>

+> * On clusters with Azure AD integration enabled, users assigned to an Azure AD administrators group specified by `aad-admin-group-object-ids` can still gain access using non-administrator credentials. On clusters without Azure AD integration enabled and `properties.disableLocalAccounts` set to `true`, any attempt to authenticate with user or admin credentials will fail.

+>

+> * After disabling local user accounts on an existing AKS cluster where users might have authenticated with local accounts, the administrator must [rotate the cluster certificates](certificate-rotation.md) to revoke certificates they might have had access to. If this is a new cluster, no action is required.

+### Create a new cluster without local accounts

+1. Create a new AKS cluster without any local accounts using the [`az aks create`][az-aks-create] command with the `disable-local-accounts` flag.

+ ```azurecli-interactive

+ az aks create -g <resource-group> -n <cluster-name> --enable-aad --aad-admin-group-object-ids <aad-group-id> --disable-local-accounts

+ ```

+2. In the output, confirm local accounts are disabled by checking that the field `properties.disableLocalAccounts` is set to `true`.

+ ```output

+ "properties": {

+ ...

+ "disableLocalAccounts": true,

+ ...

+ }

+ ```

+3. Run the [`az aks get-credentials`][az-aks-get-credentials] command to ensure the cluster is set to disable local accounts.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin

+ ```

+ Your output should show the following error message indicating the feature is preventing access:

+ ```output

+ Operation failed with status: 'Bad Request'. Details: Getting static credential isn't allowed because this cluster is set to disable local accounts.

+ ```

+### Disable local accounts on an existing cluster

+1. Disable local accounts on an existing Azure AD integration enabled AKS cluster using the [`az aks update`][az-aks-update] command with the `disable-local-accounts` parameter.

+ ```azurecli-interactive

+ az aks update -g <resource-group> -n <cluster-name> --disable-local-accounts

+ ```

+2. In the output, confirm local accounts are disabled by checking that the field `properties.disableLocalAccounts` is set to `true`.

+ ```output

+ "properties": {

+ ...

+ "disableLocalAccounts": true,

+ ...

+ }

+ ```

+3. Run the [`az aks get-credentials`][az-aks-get-credentials] command to ensure the cluster is set to disable local accounts.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin

+ ```

+ Your output should show the following error message indicating the feature is preventing access:

+ ```output

+ Operation failed with status: 'Bad Request'. Details: Getting static credential isn't allowed because this cluster is set to disable local accounts.

+ ```

+### Re-enable local accounts on an existing cluster

+1. Re-enable a disabled local account on an existing cluster using the [`az aks update`][az-aks-update] command with the `enable-local-accounts` parameter.

+ ```azurecli-interactive

+ az aks update -g <resource-group> -n <cluster-name> --enable-local-accounts

+ ```

+2. In the output, confirm local accounts are re-enabled by checking that the field `properties.disableLocalAccounts` is set to `false`.

+ ```output

+ "properties": {

+ ...

+ "disableLocalAccounts": false,

+ ...

+ }

+ ```

+3. Run the [`az aks get-credentials`][az-aks-get-credentials] command to ensure the cluster is set to enable local accounts.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin

+ ```

+ Your output should show the following message indicating you have successfully enabled local accounts on the cluster:

+ ```output

+ Merged "<cluster-name>-admin" as current context in C:\Users\<username>\.kube\config

+ ```

+## Next steps

+* Learn about [Azure RBAC integration for Kubernetes Authorization][azure-rbac-integration].

+<!-- LINKS - Internal -->

+[az-aks-create]: /cli/azure/aks#az_aks_create

+[az-aks-update]: /cli/azure/aks#az_aks_update

+[az-aks-get-credentials]: /cli/azure/aks#az_aks_get_credentials

+[azure-rbac-integration]: manage-azure-rbac.md

+ Title: AKS-managed Azure Active Directory integration

+description: Learn how to configure Azure AD for your Azure Kubernetes Service (AKS) clusters.

+# AKS-managed Azure Active Directory integration

+AKS-managed Azure Active Directory (Azure AD) integration simplifies the Azure AD integration process. Previously, you were required to create a client and server app, and the Azure AD tenant had to grant Directory Read permissions. Now, the AKS resource provider manages the client and server apps for you.

+Cluster administrators can configure Kubernetes role-based access control (Kubernetes RBAC) based on a user's identity or directory group membership. Azure AD authentication is provided to AKS clusters with OpenID Connect. OpenID Connect is an identity layer built on top of the OAuth 2.0 protocol. For more information on OpenID Connect, see the [Open ID connect documentation][open-id-connect].

+Learn more about the Azure AD integration flow in the [Azure AD documentation](concepts-identity.md#azure-ad-integration).

+## Limitations

+* AKS-managed Azure AD integration can't be disabled.

+* Changing an AKS-managed Azure AD integrated cluster to legacy Azure AD isn't supported.

+* Clusters without Kubernetes RBAC enabled aren't supported with AKS-managed Azure AD integration.

+## Before you begin

+* Make sure you have Azure CLI version 2.29.0 or later is installed and configured. Run `az --version` to find the version. If you need to install or upgrade, see [Install Azure CLI](/cli/azure/install-azure-cli).

+* You need `kubectl` with a minimum version of [1.18.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1181) or [`kubelogin`](https://github.com/Azure/kubelogin). The difference between the minor versions of Kubernetes and `kubectl` shouldn't be more than *one* version. You'll experience authentication issues if you don't use the correct version.

+* If you're using [helm](https://github.com/helm/helm), you need a minimum version of helm 3.3.

+* This article requires you have an Azure AD group for your cluster. This group will be registered as an admin group on the cluster to grant admin permissions. If you don't have an existing Azure AD group, you can create one using the [`az ad group create`](/cli/azure/ad/group#az_ad_group_create) command.

+## Enable AKS-managed Azure AD integration on your AKS cluster

+### Create a new cluster

+1. Create an Azure resource group using the [`az group create`][az-group-create] command.

+ ```azurecli-interactive

+ az group create --name myResourceGroup --location centralus

+ ```

+2. Create an AKS cluster and enable administration access for your Azure AD group using the [`az aks create`][az-aks-create] command.

+ ```azurecli-interactive

+ az aks create -g myResourceGroup -n myManagedCluster --enable-aad --aad-admin-group-object-ids <id> [--aad-tenant-id <id>]

+ ```

+ A successful creation of an AKS-managed Azure AD cluster has the following section in the response body:

+ ```output

+ "AADProfile": {

+ "adminGroupObjectIds": [

+ "5d24****-****-****-****-****afa27aed"

+ ],

+ "clientAppId": null,

+ "managed": true,

+ "serverAppId": null,

+ "serverAppSecret": null,

+ "tenantId": "72f9****-****-****-****-****d011db47"

+ }

+ ```

+### Use an existing cluster

+* Enable AKS-managed Azure AD integration on your existing Kubernetes RBAC enabled cluster using the [`az aks update`][az-aks-update] command. Make sure to set your admin group to keep access on your cluster.

+ ```azurecli-interactive

+ az aks update -g MyResourceGroup -n myManagedCluster --enable-aad --aad-admin-group-object-ids <id-1> [--aad-tenant-id <id>]

+ ```

+ A successful activation of an AKS-managed Azure AD cluster has the following section in the response body:

+ ```output

+ "AADProfile": {

+ "adminGroupObjectIds": [

+ "5d24****-****-****-****-****afa27aed"

+ ],

+ "clientAppId": null,

+ "managed": true,

+ "serverAppId": null,

+ "serverAppSecret": null,

+ "tenantId": "72f9****-****-****-****-****d011db47"

+ }

+ ```

+### Upgrade a legacy Azure AD cluster to AKS-managed Azure AD integration

+* If your cluster uses legacy Azure AD integration, you can upgrade to AKS-managed Azure AD integration with no downtime using the [`az aks update`][az-aks-update] command.

+ ```azurecli-interactive

+ az aks update -g myResourceGroup -n myManagedCluster --enable-aad --aad-admin-group-object-ids <id> [--aad-tenant-id <id>]

+ ```

+ A successful migration of an AKS-managed Azure AD cluster has the following section in the response body:

+ ```output

+ "AADProfile": {

+ "adminGroupObjectIds": [

+ "5d24****-****-****-****-****afa27aed"

+ ],

+ "clientAppId": null,

+ "managed": true,

+ "serverAppId": null,

+ "serverAppSecret": null,

+ "tenantId": "72f9****-****-****-****-****d011db47"

+ }

+ ```

+## Access your AKS-managed Azure AD enabled cluster

+1. Get the user credentials to access your cluster using the [`az aks get-credentials`][az-aks-get-credentials] command.

+ ```azurecli-interactive

+ az aks get-credentials --resource-group myResourceGroup --name myManagedCluster

+ ```

+2. Follow the instructions to sign in.

+3. View the nodes in the cluster using the `kubectl get nodes` command.

+ ```azurecli-interactive

+ kubectl get nodes

+ ```

+## Non-interactive sign-in with kubelogin

+There are some non-interactive scenarios, such as continuous integration pipelines, that aren't currently available with `kubectl`. You can use [`kubelogin`](https://github.com/Azure/kubelogin) to connect to the cluster with a non-interactive service principal credential. Starting with Kubernetes version 1.24, the default format of the clusterUser credential for Azure AD clusters is `exec`, which requires [`kubelogin`](https://github.com/Azure/kubelogin) binary in the execution PATH.

+* When getting the clusterUser credential, you can use the `format` query parameter to overwrite the default behavior change. You can set the value to `azure` to use the original kubeconfig format:

+ ```azurecli-interactive

+ az aks get-credentials --format azure

+ ```

+* Azure AD integrated clusters using a Kubernetes version newer than 1.24 automatically use the `kubelogin` format.

+* If your Azure AD integrated clusters use a Kubernetes version older than 1.24, you need to convert the kubeconfig format manually.

+ ```azurecli-interactive

+ export KUBECONFIG=/path/to/kubeconfig

+ kubelogin convert-kubeconfig

+ ```

+## Troubleshoot access issues with AKS-managed Azure AD

+> [!IMPORTANT]

+> The steps described in this section bypass the normal Azure AD group authentication. Use them only in an emergency.

+If you're permanently blocked by not having access to a valid Azure AD group with access to your cluster, you can still get admin credentials to directly access the cluster. You need to have access to the [Azure Kubernetes Service Cluster Admin](../role-based-access-control/built-in-roles.md#azure-kubernetes-service-cluster-admin-role) built-in role.

+## Next steps

+* Learn about [Azure AD integration with Kubernetes RBAC][azure-ad-rbac].

+* Learn more about [AKS and Kubernetes identity concepts][aks-concepts-identity].

+* Use [Azure Resource Manager (ARM) templates][aks-arm-template] to create AKS-managed Azure AD enabled clusters.

+<!-- LINKS - external -->

+[aks-arm-template]: /azure/templates/microsoft.containerservice/managedclusters

+<!-- LINKS - Internal -->

+[aks-concepts-identity]: concepts-identity.md

+[azure-ad-rbac]: azure-ad-rbac.md

+[az-aks-create]: /cli/azure/aks#az_aks_create

+[az-aks-get-credentials]: /cli/azure/aks#az_aks_get_credentials

+[az-group-create]: /cli/azure/group#az_group_create

+[open-id-connect]:../active-directory/develop/v2-protocols-oidc.md

+[az-aks-update]: /cli/azure/aks#az_aks_update

+[azure-ad-integration]: managed-azure-ad.md

+ Title: Outbound network and FQDN rules for Azure Kubernetes Service (AKS) clusters

+description: Learn what ports and addresses are required to control egress traffic in Azure Kubernetes Service (AKS)

+#Customer intent: As an cluster operator, I want to learn the network and FQDNs rules to control egress traffic and improve security for my AKS clusters.

+# Outbound network and FQDN rules for Azure Kubernetes Service (AKS) clusters

+This article provides the necessary details that allow you to secure outbound traffic from your Azure Kubernetes Service (AKS). It contains the cluster requirements for a base AKS deployment and additional requirements for optional addons and features. You can apply this information to any outbound restriction method or appliance.

+To see an example configuration using Azure Firewall, visit [Control egress traffic using Azure Firewall in AKS](limit-egress-traffic.md).

+## Background

+AKS clusters are deployed on a virtual network. This network can either be customized and pre-configured by you or it can be created and managed by AKS. In either case, the cluster has **outbound**, or egress, dependencies on services outside of the virtual network.

+For management and operational purposes, nodes in an AKS cluster need to access certain ports and fully qualified domain names (FQDNs). These endpoints are required for the nodes to communicate with the API server or to download and install core Kubernetes cluster components and node security updates. For example, the cluster needs to pull base system container images from Microsoft Container Registry (MCR).

+The AKS outbound dependencies are almost entirely defined with FQDNs, which don't have static addresses behind them. The lack of static addresses means you can't use network security groups (NSGs) to lock down the outbound traffic from an AKS cluster.

+By default, AKS clusters have unrestricted outbound internet access. This level of network access allows nodes and services you run to access external resources as needed. If you wish to restrict egress traffic, a limited number of ports and addresses must be accessible to maintain healthy cluster maintenance tasks. The simplest solution to securing outbound addresses is using a firewall device that can control outbound traffic based on domain names. Azure Firewall can restrict outbound HTTP and HTTPS traffic based on the FQDN of the destination. You can also configure your preferred firewall and security rules to allow these required ports and addresses.

+> [!IMPORTANT]

+>

+> This document covers only how to lock down the traffic leaving the AKS subnet. AKS has no ingress requirements by default. Blocking **internal subnet traffic** using network security groups (NSGs) and firewalls isn't supported. To control and block the traffic within the cluster, see [Secure traffic between pods using network policies in AKS][use-network-policies].

+## Required outbound network rules and FQDNs for AKS clusters

+The following network and FQDN/application rules are required for an AKS cluster. You can use them if you wish to configure a solution other than Azure Firewall.

+* IP address dependencies are for non-HTTP/S traffic (both TCP and UDP traffic).

+* FQDN HTTP/HTTPS endpoints can be placed in your firewall device.

+* Wildcard HTTP/HTTPS endpoints are dependencies that can vary with your AKS cluster based on a number of qualifiers.

+* AKS uses an admission controller to inject the FQDN as an environment variable to all deployments under kube-system and gatekeeper-system. This ensures all system communication between nodes and API server uses the API server FQDN and not the API server IP.

+* If you have an app or solution that needs to talk to the API server, you must add an **additional** network rule to allow **TCP communication to port 443 of your API server's IP**.

+* On rare occasions, if there's a maintenance operation, your API server IP might change. Planned maintenance operations that can change the API server IP are always communicated in advance.

+### Azure Global required network rules

+| Destination Endpoint | Protocol | Port | Use |

+|-|-|||

+| **`*:1194`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:1194`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:1194`** <br/> *Or* <br/> **`APIServerPublicIP:1194`** `(only known after cluster creation)` | UDP | 1194 | For tunneled secure communication between the nodes and the control plane. This isn't required for [private clusters][private-clusters], or for clusters with the *konnectivity-agent* enabled. |

+| **`*:9000`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:9000`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:9000`** <br/> *Or* <br/> **`APIServerPublicIP:9000`** `(only known after cluster creation)` | TCP | 9000 | For tunneled secure communication between the nodes and the control plane. This isn't required for [private clusters][private-clusters], or for clusters with the *konnectivity-agent* enabled. |

+| **`*:123`** or **`ntp.ubuntu.com:123`** (if using Azure Firewall network rules) | UDP | 123 | Required for Network Time Protocol (NTP) time synchronization on Linux nodes. This isn't required for nodes provisioned after March 2021. |

+| **`CustomDNSIP:53`** `(if using custom DNS servers)` | UDP | 53 | If you're using custom DNS servers, you must ensure they're accessible by the cluster nodes. |

+| **`APIServerPublicIP:443`** `(if running pods/deployments that access the API Server)` | TCP | 443 | Required if running pods/deployments that access the API Server, those pods/deployments would use the API IP. This port isn't required for [private clusters][private-clusters]. |

+### Azure Global required FQDN / application rules

+| Destination FQDN | Port | Use |

+|-|--|-|

+| **`*.hcp.<location>.azmk8s.io`** | **`HTTPS:443`** | Required for Node <-> API server communication. Replace *\<location\>* with the region where your AKS cluster is deployed. This is required for clusters with *konnectivity-agent* enabled. Konnectivity also uses Application-Layer Protocol Negotiation (ALPN) to communicate between agent and server. Blocking or rewriting the ALPN extension will cause a failure. This isn't required for [private clusters][private-clusters]. |

+| **`mcr.microsoft.com`** | **`HTTPS:443`** | Required to access images in Microsoft Container Registry (MCR). This registry contains first-party images/charts (for example, coreDNS, etc.). These images are required for the correct creation and functioning of the cluster, including scale and upgrade operations. |

+| **`*.data.mcr.microsoft.com`** | **`HTTPS:443`** | Required for MCR storage backed by the Azure content delivery network (CDN). |

+| **`management.azure.com`** | **`HTTPS:443`** | Required for Kubernetes operations against the Azure API. |

+| **`login.microsoftonline.com`** | **`HTTPS:443`** | Required for Azure Active Directory authentication. |

+| **`packages.microsoft.com`** | **`HTTPS:443`** | This address is the Microsoft packages repository used for cached *apt-get* operations. Example packages include Moby, PowerShell, and Azure CLI. |

+| **`acs-mirror.azureedge.net`** | **`HTTPS:443`** | This address is for the repository required to download and install required binaries like kubenet and Azure CNI. |

+### Azure China 21Vianet required network rules

+| Destination Endpoint | Protocol | Port | Use |

+|-|-|||

+| **`*:1194`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.Region:1194`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:1194`** <br/> *Or* <br/> **`APIServerPublicIP:1194`** `(only known after cluster creation)` | UDP | 1194 | For tunneled secure communication between the nodes and the control plane. |

+| **`*:9000`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:9000`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:9000`** <br/> *Or* <br/> **`APIServerPublicIP:9000`** `(only known after cluster creation)` | TCP | 9000 | For tunneled secure communication between the nodes and the control plane. |

+| **`*:22`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:22`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:22`** <br/> *Or* <br/> **`APIServerPublicIP:22`** `(only known after cluster creation)` | TCP | 22 | For tunneled secure communication between the nodes and the control plane. |

+| **`*:123`** or **`ntp.ubuntu.com:123`** (if using Azure Firewall network rules) | UDP | 123 | Required for Network Time Protocol (NTP) time synchronization on Linux nodes. |

+| **`CustomDNSIP:53`** `(if using custom DNS servers)` | UDP | 53 | If you're using custom DNS servers, you must ensure they're accessible by the cluster nodes. |

+| **`APIServerPublicIP:443`** `(if running pods/deployments that access the API Server)` | TCP | 443 | Required if running pods/deployments that access the API Server, those pod/deployments would use the API IP. |

+### Azure China 21Vianet required FQDN / application rules

+| Destination FQDN | Port | Use |

+||--|-|

+| **`*.hcp.<location>.cx.prod.service.azk8s.cn`**| **`HTTPS:443`** | Required for Node <-> API server communication. Replace *\<location\>* with the region where your AKS cluster is deployed. |

+| **`*.tun.<location>.cx.prod.service.azk8s.cn`**| **`HTTPS:443`** | Required for Node <-> API server communication. Replace *\<location\>* with the region where your AKS cluster is deployed. |

+| **`mcr.microsoft.com`** | **`HTTPS:443`** | Required to access images in Microsoft Container Registry (MCR). This registry contains first-party images/charts (for example, coreDNS, etc.). These images are required for the correct creation and functioning of the cluster, including scale and upgrade operations. |

+| **`.data.mcr.microsoft.com`** | **`HTTPS:443`** | Required for MCR storage backed by the Azure Content Delivery Network (CDN). |

+| **`management.chinacloudapi.cn`** | **`HTTPS:443`** | Required for Kubernetes operations against the Azure API. |

+| **`login.chinacloudapi.cn`** | **`HTTPS:443`** | Required for Azure Active Directory authentication. |

+| **`packages.microsoft.com`** | **`HTTPS:443`** | This address is the Microsoft packages repository used for cached *apt-get* operations. Example packages include Moby, PowerShell, and Azure CLI. |

+| **`*.azk8s.cn`** | **`HTTPS:443`** | This address is for the repository required to download and install required binaries like kubenet and Azure CNI. |

+### Azure US Government required network rules

+| Destination Endpoint | Protocol | Port | Use |

+|-|-|||

+| **`*:1194`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:1194`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:1194`** <br/> *Or* <br/> **`APIServerPublicIP:1194`** `(only known after cluster creation)` | UDP | 1194 | For tunneled secure communication between the nodes and the control plane. |

+| **`*:9000`** <br/> *Or* <br/> [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureCloud.<Region>:9000`** <br/> *Or* <br/> [Regional CIDRs](../virtual-network/service-tags-overview.md#discover-service-tags-by-using-downloadable-json-files) - **`RegionCIDRs:9000`** <br/> *Or* <br/> **`APIServerPublicIP:9000`** `(only known after cluster creation)` | TCP | 9000 | For tunneled secure communication between the nodes and the control plane. |

+| **`*:123`** or **`ntp.ubuntu.com:123`** (if using Azure Firewall network rules) | UDP | 123 | Required for Network Time Protocol (NTP) time synchronization on Linux nodes. |

+| **`CustomDNSIP:53`** `(if using custom DNS servers)` | UDP | 53 | If you're using custom DNS servers, you must ensure they're accessible by the cluster nodes. |

+| **`APIServerPublicIP:443`** `(if running pods/deployments that access the API Server)` | TCP | 443 | Required if running pods/deployments that access the API Server, those pods/deployments would use the API IP. |

+### Azure US Government required FQDN / application rules

+| Destination FQDN | Port | Use |

+||--|-|

+| **`*.hcp.<location>.cx.aks.containerservice.azure.us`** | **`HTTPS:443`** | Required for Node <-> API server communication. Replace *\<location\>* with the region where your AKS cluster is deployed.|

+| **`mcr.microsoft.com`** | **`HTTPS:443`** | Required to access images in Microsoft Container Registry (MCR). This registry contains first-party images/charts (for example, coreDNS, etc.). These images are required for the correct creation and functioning of the cluster, including scale and upgrade operations. |

+| **`*.data.mcr.microsoft.com`** | **`HTTPS:443`** | Required for MCR storage backed by the Azure content delivery network (CDN). |

+| **`management.usgovcloudapi.net`** | **`HTTPS:443`** | Required for Kubernetes operations against the Azure API. |

+| **`login.microsoftonline.us`** | **`HTTPS:443`** | Required for Azure Active Directory authentication. |

+| **`packages.microsoft.com`** | **`HTTPS:443`** | This address is the Microsoft packages repository used for cached *apt-get* operations. Example packages include Moby, PowerShell, and Azure CLI. |

+| **`acs-mirror.azureedge.net`** | **`HTTPS:443`** | This address is for the repository required to install required binaries like kubenet and Azure CNI. |

+## Optional recommended FQDN / application rules for AKS clusters

+The following FQDN / application rules aren't required, but are recommended for AKS clusters:

+| Destination FQDN | Port | Use |

+|--||-|

+| **`security.ubuntu.com`, `azure.archive.ubuntu.com`, `changelogs.ubuntu.com`** | **`HTTP:80`** | This address lets the Linux cluster nodes download the required security patches and updates. |

+If you choose to block/not allow these FQDNs, the nodes will only receive OS updates when you do a [node image upgrade](node-image-upgrade.md) or [cluster upgrade](upgrade-cluster.md). Keep in mind that node image upgrades also come with updated packages including security fixes.

+## GPU enabled AKS clusters required FQDN / application rules

+| Destination FQDN | Port | Use |

+|--|--|-|

+| **`nvidia.github.io`** | **`HTTPS:443`** | This address is used for correct driver installation and operation on GPU-based nodes. |

+| **`us.download.nvidia.com`** | **`HTTPS:443`** | This address is used for correct driver installation and operation on GPU-based nodes. |

+| **`download.docker.com`** | **`HTTPS:443`** | This address is used for correct driver installation and operation on GPU-based nodes. |

+## Windows Server based node pools required FQDN / application rules

+| Destination FQDN | Port | Use |

+|-|--|-|

+| **`onegetcdn.azureedge.net, go.microsoft.com`** | **`HTTPS:443`** | To install windows-related binaries |

+| **`*.mp.microsoft.com, www.msftconnecttest.com, ctldl.windowsupdate.com`** | **`HTTP:80`** | To install windows-related binaries |

+If you choose to block/not allow these FQDNs, the nodes will only receive OS updates when you do a [node image upgrade](node-image-upgrade.md) or [cluster upgrade](upgrade-cluster.md). Keep in mind that Node Image Upgrades also come with updated packages including security fixes.

+## AKS addons and integrations

+### Microsoft Defender for Containers

+#### Required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`login.microsoftonline.com`** | **`HTTPS:443`** | Required for Active Directory Authentication. |

+| **`*.ods.opinsights.azure.com`** | **`HTTPS:443`** | Required for Microsoft Defender to upload security events to the cloud.|

+| **`*.oms.opinsights.azure.com`** | **`HTTPS:443`** | Required to Authenticate with LogAnalytics workspaces.|

+### CSI Secret Store

+#### Required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`vault.azure.net`** | **`HTTPS:443`** | Required for CSI Secret Store addon pods to talk to Azure KeyVault server.|

+### Azure Monitor for containers

+There are two options to provide access to Azure Monitor for containers:

+- Allow the Azure Monitor [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags).

+- Provide access to the required FQDN/application rules.

+#### Required network rules

+| Destination Endpoint | Protocol | Port | Use |

+|-|-|||

+| [ServiceTag](../virtual-network/service-tags-overview.md#available-service-tags) - **`AzureMonitor:443`** | TCP | 443 | This endpoint is used to send metrics data and logs to Azure Monitor and Log Analytics. |

+#### Required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`dc.services.visualstudio.com`** | **`HTTPS:443`** | This endpoint is used for metrics and monitoring telemetry using Azure Monitor. |

+| **`*.ods.opinsights.azure.com`** | **`HTTPS:443`** | This endpoint is used by Azure Monitor for ingesting log analytics data. |

+| **`*.oms.opinsights.azure.com`** | **`HTTPS:443`** | This endpoint is used by omsagent, which is used to authenticate the log analytics service. |

+| **`*.monitoring.azure.com`** | **`HTTPS:443`** | This endpoint is used to send metrics data to Azure Monitor. |

+### Azure Policy

+#### Required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`data.policy.core.windows.net`** | **`HTTPS:443`** | This address is used to pull the Kubernetes policies and to report cluster compliance status to policy service. |

+| **`store.policy.core.windows.net`** | **`HTTPS:443`** | This address is used to pull the Gatekeeper artifacts of built-in policies. |

+| **`dc.services.visualstudio.com`** | **`HTTPS:443`** | Azure Policy add-on that sends telemetry data to applications insights endpoint. |

+#### Azure China 21Vianet required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`data.policy.azure.cn`** | **`HTTPS:443`** | This address is used to pull the Kubernetes policies and to report cluster compliance status to policy service. |

+| **`store.policy.azure.cn`** | **`HTTPS:443`** | This address is used to pull the Gatekeeper artifacts of built-in policies. |

+#### Azure US Government required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`data.policy.azure.us`** | **`HTTPS:443`** | This address is used to pull the Kubernetes policies and to report cluster compliance status to policy service. |

+| **`store.policy.azure.us`** | **`HTTPS:443`** | This address is used to pull the Gatekeeper artifacts of built-in policies. |

+## Cluster extensions

+### Required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`<region>.dp.kubernetesconfiguration.azure.com`** | **`HTTPS:443`** | This address is used to fetch configuration information from the Cluster Extensions service and report extension status to the service.|

+| **`mcr.microsoft.com, *.data.mcr.microsoft.com`** | **`HTTPS:443`** | This address is required to pull container images for installing cluster extension agents on AKS cluster.|

+#### Azure US Government required FQDN / application rules

+| FQDN | Port | Use |

+|--|--|-|

+| **`<region>.dp.kubernetesconfiguration.azure.us`** | **`HTTPS:443`** | This address is used to fetch configuration information from the Cluster Extensions service and report extension status to the service. |

+| **`mcr.microsoft.com, *.data.mcr.microsoft.com`** | **`HTTPS:443`** | This address is required to pull container images for installing cluster extension agents on AKS cluster.|

+> [!NOTE]

+>

+> For any addons that aren't explicitly stated here, the core requirements cover it.

+## Next steps

+In this article, you learned what ports and addresses to allow if you want to restrict egress traffic for the cluster.

+If you want to restrict how pods communicate between themselves and East-West traffic restrictions within cluster see [Secure traffic between pods using network policies in AKS][use-network-policies].

+<!-- LINKS - internal -->

+[private-clusters]: ./private-clusters.md

+[use-network-policies]: ./use-network-policies.md

+Trusted Access enables you to give explicit consent to your system-assigned MSI of allowed resources to access your AKS clusters using an Azure resource *RoleBinding*. Your Azure resources access AKS clusters through the AKS regional gateway via system-assigned managed identity authentication with the appropriate Kubernetes permissions via an Azure resource *Role*. The Trusted Access feature allows you to access AKS clusters with different configurations, including but not limited to [private clusters](private-clusters.md), [clusters with local accounts disabled](manage-local-accounts-managed-azure-ad.md#disable-local-accounts), [Azure AD clusters](azure-ad-integration-cli.md), and [authorized IP range clusters](api-server-authorized-ip-ranges.md).

+## Kubernetes version upgrades

+When you upgrade a supported AKS cluster, Kubernetes minor versions can't be skipped. You must perform all upgrades sequentially by major version number. For example, upgrades between *1.14.x* -> *1.15.x* or *1.15.x* -> *1.16.x* are allowed, however *1.14.x* -> *1.16.x* isn't allowed.

+Skipping multiple versions can only be done when upgrading from an *unsupported version* back to a *supported version*. For example, an upgrade from an unsupported *1.10.x* -> a supported *1.15.x* can be completed if available. When performing an upgrade from an *unsupported version* that skips two or more minor versions, the upgrade is performed without any guarantee of functionality and is excluded from the service-level agreements and limited warranty. If your version is significantly out of date, we recommend you recreate your cluster.

+> Any upgrade operation, whether performed manually or automatically, will upgrade the node image version if not already on the latest. The latest version is contingent on a full AKS release and can be determined by visiting the [AKS release tracker][release-tracker].

+* Performing upgrade operations requires the `Microsoft.ContainerService/managedClusters/agentPools/write` RBAC role. For more on Azure RBAC roles, see the [Azure resource provider operations]

+> An AKS cluster upgrade triggers a cordon and drain of your nodes. If you have a low compute quota available, the upgrade may fail. For more information, see [increase quotas](../azure-portal/supportability/regional-quota-requests.md).

+Check which Kubernetes releases are available for your cluster using the [`az aks get-upgrades`][az-aks-get-upgrades] command.

+```output

+Check which Kubernetes releases are available for your cluster using [`Get-AzAksUpgradeProfile`][get-azaksupgradeprofile] command.

+Get-AzAksUpgradeProfile -ResourceGroupName myResourceGroup -ClusterName myAKSCluster |

+Select-Object -Property Name, ControlPlaneProfileKubernetesVersion -ExpandProperty ControlPlaneProfileUpgrade |

+Format-Table -Property *

+```output

+Name ControlPlaneProfileKubernetesVersion IsPreview KubernetesVersion

+- --

+default 1.18.10 1.19.1

+default 1.18.10 1.19.3

+Check which Kubernetes releases are available for your cluster using the following steps:

+4. In **Kubernetes version**, select **Upgrade version**.

+The Azure portal highlights all the deprecated APIs between your current version and newer, available versions you intend to migrate to. For more information, see [the Kubernetes API Removal and Deprecation process][k8s-deprecation].

+### Troubleshoot AKS cluster upgrade error messages

+### [Azure CLI](#tab/azure-cli)

+The following example output means the `appservice-kube` extension isn't compatible with your Azure CLI version (a minimum of version 2.34.1 is required):

+```output

+The 'appservice-kube' extension is not compatible with this version of the CLI.

+You have CLI core version 2.0.81 and this extension requires a min of 2.34.1.

+Table output unavailable. Use the --query option to specify an appropriate query. Use --debug for more info.

+```

+If you receive this output, you need to update your Azure CLI version. The `az upgrade` command was added in version 2.11.0 and doesn't work with versions prior to 2.11.0. You can update older versions by reinstalling Azure CLI as described in [Install the Azure CLI](/cli/azure/install-azure-cli). If your Azure CLI version is 2.11.0 or later, you receive a message to run `az upgrade` to upgrade Azure CLI to the latest version.

+If your Azure CLI is updated and you receive the following example output, it means that no upgrades are available:

+```output

+ERROR: Table output unavailable. Use the --query option to specify an appropriate query. Use --debug for more info.

+```

+If no upgrades are available, create a new cluster with a supported version of Kubernetes and migrate your workloads from the existing cluster to the new cluster. It's not supported to upgrade a cluster to a newer Kubernetes version when `az aks get-upgrades` shows that no upgrades are available.

+### [Azure PowerShell](#tab/azure-powershell)

+If no upgrades are available, create a new cluster with a supported version of Kubernetes and migrate your workloads from the existing cluster to the new cluster. It's not supported to upgrade a cluster to a newer Kubernetes version when `Get-AzAksUpgradeProfile` shows that no upgrades are available.

+### [Azure portal](#tab/azure-portal)

+If no upgrades are available, create a new cluster with a supported version of Kubernetes and migrate your workloads from the existing cluster to the new cluster. It's not supported to upgrade a cluster to a newer Kubernetes version when no upgrades are available.

+## Upgrade an AKS cluster

+During the cluster upgrade process, AKS performs the following operations:

+* Add a new buffer node (or as many nodes as configured in [max surge](#customize-node-surge-upgrade)) to the cluster that runs the specified Kubernetes version.

+* [Cordon and drain][kubernetes-drain] one of the old nodes to minimize disruption to running applications. If you're using max surge, it [cordons and drains][kubernetes-drain] as many nodes at the same time as the number of buffer nodes specified.

+* When the old node is fully drained, it's reimaged to receive the new version and becomes the buffer node for the following node to be upgraded.

+* This process repeats until all nodes in the cluster have been upgraded.

+* At the end of the process, the last buffer node is deleted, maintaining the existing agent node count and zone balance.

+> [!IMPORTANT]

+> Ensure that any `PodDisruptionBudgets` (PDBs) allow for at least *one* pod replica to be moved at a time otherwise the drain/evict operation will fail.

+> If the drain operation fails, the upgrade operation will fail by design to ensure that the applications are not disrupted. Please correct what caused the operation to stop (incorrect PDBs, lack of quota, and so on) and re-try the operation.

+### [Azure CLI](#tab/azure-cli)

+1. Upgrade your cluster using the [`az aks upgrade`][az-aks-upgrade] command.

+ ```azurecli-interactive

+ az aks upgrade \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --kubernetes-version KUBERNETES_VERSION

+ ```

+2. Confirm the upgrade was successful using the [`az aks show`][az-aks-show] command.

+ ```azurecli-interactive

+ az aks show --resource-group myResourceGroup --name myAKSCluster --output table

+ ```

+ The following example output shows that the cluster now runs *1.19.1*:

+ ```output

+ Name Location ResourceGroup KubernetesVersion ProvisioningState Fqdn

+ - - - -

+ myAKSCluster eastus myResourceGroup 1.19.1 Succeeded myakscluster-dns-379cbbb9.hcp.eastus.azmk8s.io

+ ```

+### [Azure PowerShell](#tab/azure-powershell)

+1. Upgrade your cluster using the [`Set-AzAksCluster`][set-azakscluster] command.

+ ```azurepowershell-interactive

+ Set-AzAksCluster -ResourceGroupName myResourceGroup -Name myAKSCluster -KubernetesVersion <KUBERNETES_VERSION>

+ ```

+2. Confirm the upgrade was successful using the [`Get-AzAksCluster`][get-azakscluster] command.

+ ```azurepowershell-interactive

+ Get-AzAksCluster -ResourceGroupName myResourceGroup -Name myAKSCluster |

+ Format-Table -Property Name, Location, KubernetesVersion, ProvisioningState, Fqdn

+ ```

+ The following example output shows that the cluster now runs *1.19.1*:

+ ```output

+ Name Location KubernetesVersion ProvisioningState Fqdn

+ - -- -- -- -

+ myAKSCluster eastus 1.19.1 Succeeded myakscluster-dns-379cbbb9.hcp.eastus.azmk8s.io

+ ```

+### [Azure portal](#tab/azure-portal)

+1. Sign in to the [Azure portal](https://portal.azure.com).

+2. Navigate to your AKS cluster.

+3. Under **Settings**, select **Cluster configuration**.

+4. In **Kubernetes version**, select **Upgrade version**.

+5. In **Kubernetes version**, select your desired version and then select **Save**.

+6. Navigate to your AKS cluster **Overview** page, and select the **Kubernetes version** to confirm the upgrade was successful.

+The Azure portal highlights all the deprecated APIs between your current version and newer, available versions you intend to migrate to. For more information, see [the Kubernetes API removal and deprecation process][k8s-deprecation].

+## View the upgrade events

+When you upgrade your cluster, the following Kubernetes events may occur on each node:

+* **Surge**: Creates a surge node.

+* **Drain**: Evicts pods from the node. Each pod has a 30-second timeout to complete the eviction.

+* **Update**: Update of a node succeeds or fails.

+* **Delete**: Deletes a surge node.

+Use `kubectl get events` to show events in the default namespaces while running an upgrade. For example:

+kubectl get events

+The following example output shows some of the above events listed during an upgrade.

+```output

+...

+default 2m1s Normal Drain node/aks-nodepool1-96663640-vmss000001 Draining node: [aks-nodepool1-96663640-vmss000001]

+...

+default 9m22s Normal Surge node/aks-nodepool1-96663640-vmss000002 Created a surge node [aks-nodepool1-96663640-vmss000002 nodepool1] for agentpool %!s(MISSING)

+...

+```

+## Stop cluster upgrades automatically on API breaking changes (Preview)

+To stay within a supported Kubernetes version, you usually have to upgrade your cluster at least once per year and prepare for all possible disruptions. These disruptions include ones caused by API breaking changes, deprecations, and dependencies such as Helm and CSI. It can be difficult to anticipate these disruptions and migrate critical workloads without experiencing any downtime.

+AKS now automatically stops upgrade operations consisting of a minor version change if deprecated APIs are detected. This feature alerts you with an error message if it detects usage of APIs that are deprecated in the targeted version.

+All of the following criteria must be met in order for the stop to occur:

+* The upgrade operation is a Kubernetes minor version change for the cluster control plane.

+* The Kubernetes version you're upgrading to is 1.26 or later

+* If performed via REST, the upgrade operation uses a preview API version of `2023-01-02-preview` or later.

+* If performed via Azure CLI, the `aks-preview` CLI extension 0.5.134 or later must be installed.

+* The last seen usage of deprecated APIs for the targeted version you're upgrading to must occur within 12 hours before the upgrade operation. AKS records usage hourly, so any usage of deprecated APIs within one hour isn't guaranteed to appear in the detection.

+### Mitigating stopped upgrade operations

+If you attempt an upgrade and all of the previous criteria are met, you receive an error message similar to the following example error message:

+```output

+Bad Request({

+ "code": "ValidationError",

+ "message": "Control Plane upgrade is blocked due to recent usage of a Kubernetes API deprecated in the specified version. Please refer to https://kubernetes.io/docs/reference/using-api/deprecation-guide to migrate the usage. To bypass this error, set IgnoreKubernetesDeprecations in upgradeSettings.overrideSettings. Bypassing this error without migrating usage will result in the deprecated Kubernetes API calls failing. Usage details: 1 error occurred:\n\t* usage has been detected on API flowcontrol.apiserver.k8s.io.prioritylevelconfigurations.v1beta1, and was recently seen at: 2023-03-23 20:57:18 +0000 UTC, which will be removed in 1.26\n\n",

+ "subcode": "UpgradeBlockedOnDeprecatedAPIUsage"

+})

+After receiving the error message, you have two options to mitigate the issue. You can either [remove usage of deprecated APIs (recommended)](#remove-usage-of-deprecated-apis-recommended) or [bypass validation to ignore API changes](#bypass-validation-to-ignore-api-changes).

+### Remove usage of deprecated APIs (recommended)

+1. In the Azure portal, navigate to your cluster's overview page, and select **Diagnose and solve problems**.

+2. Navigate to the **Known Issues, Availability and Performance** category, and select **Selected Kubernetes API deprecations**.

+ :::image type="content" source="./media/upgrade-cluster/applens-api-detection-inline.png" lightbox="./media/upgrade-cluster/applens-api-detection-full.png" alt-text="A screenshot of the Azure portal showing the 'Selected Kubernetes API deprecations' section.":::

+3. Wait 12 hours from the time the last deprecated API usage was seen.

+4. Retry your cluster upgrade.

+You can also check past API usage by enabling [Container Insights][container-insights] and exploring kube audit logs.

+### Bypass validation to ignore API changes

+> [!NOTE]

+> This method requires you to use the `aks-preview` Azure CLI extension version 0.5.134 or later. This method isn't recommended, as deprecated APIs in the targeted Kubernetes version may not work long term. We recommend to removing them as soon as possible after the upgrade completes.

+Bypass validation to ignore API breaking changes using the [`az aks update`][az-aks-update] command and setting the `upgrade-settings` property to `IgnoreKubernetesDeprecations` and setting the `upgrade-override-until` property to define the end of the window during which validation is bypassed. If no value is set, it defaults the window to three days from the current time. The date and time you specify must be in the future.

+```azurecli-interactive

+az aks update --name myAKSCluster --resource-group myResourceGroup --upgrade-settings IgnoreKubernetesDeprecations --upgrade-override-until 2023-04-01T13:00:00Z

+```

+> [!NOTE]

+> `Z` is the zone designator for the zero UTC/GMT offset, also known as 'Zulu' time. This example sets the end of the window to `13:00:00` GMT. For more information, see [Combined date and time representations](https://wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).

+## Customize node surge upgrade

+> [!IMPORTANT]

+>

+> Node surges require subscription quota for the requested max surge count for each upgrade operation. For example, a cluster that has five node pools, each with a count of four nodes, has a total of 20 nodes. If each node pool has a max surge value of 50%, additional compute and IP quota of 10 nodes (2 nodes * 5 pools) is required to complete the upgrade.

+>

+> The max surge setting on a node pool is persistent. Subsequent Kubernetes upgrades or node version upgrades will use this setting. You may change the max surge value for your node pools at any time. For production node pools, we recommend a max-surge setting of 33%.

+>

+> If you're using Azure CNI, validate there are available IPs in the subnet to [satisfy IP requirements of Azure CNI](configure-azure-cni.md).

+By default, AKS configures upgrades to surge with one extra node. A default value of one for the max surge settings enables AKS to minimize workload disruption by creating an extra node before the cordon/drain of existing applications to replace an older versioned node. The max surge value can be customized per node pool to enable a trade-off between upgrade speed and upgrade disruption. When you increase the max surge value, the upgrade process completes faster. If you set a large value for max surge, you might experience disruptions during the upgrade process.

+For example, a max surge value of *100%* provides the fastest possible upgrade process (doubling the node count) but also causes all nodes in the node pool to be drained simultaneously. You might want to use a higher value such as this for testing environments. For production node pools, we recommend a `max_surge` setting of *33%*.

+AKS accepts both integer values and a percentage value for max surge. An integer such as *5* indicates five extra nodes to surge. A value of *50%* indicates a surge value of half the current node count in the pool. Max surge percent values can be a minimum of *1%* and a maximum of *100%*. A percent value is rounded up to the nearest node count. If the max surge value is higher than the required number of nodes to be upgraded, the number of nodes to be upgraded is used for the max surge value.

+During an upgrade, the max surge value can be a minimum of *1* and a maximum value equal to the number of nodes in your node pool. You can set larger values, but the maximum number of nodes used for max surge isn't higher than the number of nodes in the pool at the time of upgrade.

+### Set max surge values

+Set max surge values for new or existing node pools using the following commands:

+```azurecli-interactive

+# Set max surge for a new node pool

+az aks nodepool add -n mynodepool -g MyResourceGroup --cluster-name MyManagedCluster --max-surge 33%

+# Update max surge for an existing node pool

+az aks nodepool update -n mynodepool -g MyResourceGroup --cluster-name MyManagedCluster --max-surge 5

+You can set an auto-upgrade channel on your cluster. For more information, see [Auto-upgrading an AKS cluster][aks-auto-upgrade].

+## Special considerations for node pools that span multiple availability zones

+AKS uses best-effort zone balancing in node groups. During an upgrade surge, the zones for the surge nodes in Virtual Machine Scale Sets are unknown ahead of time, which can temporarily cause an unbalanced zone configuration during an upgrade. However, AKS deletes surge nodes once the upgrade completes and preserves the original zone balance. If you want to keep your zones balanced during upgrades, you can increase the surge to a multiple of *three nodes*, and Virtual Machine Scale Sets balances your nodes across availability zones with best-effort zone balancing.

+If you have PVCs backed by Azure LRS Disks, theyΓÇÖll be bound to a particular zone. They may fail to recover immediately if the surge node doesnΓÇÖt match the zone of the PVC. This could cause downtime on your application when the upgrade operation continues to drain nodes but the PVs are bound to a zone. To handle this case and maintain high availability, configure a [Pod Disruption Budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) on your application to allow Kubernetes to respect your availability requirements during the drain operation.

+This article showed you how to upgrade an existing AKS cluster. To learn more about deploying and managing AKS clusters, see the following tutorials:

+[az-aks-update]: /cli/azure/aks#az_aks_update

+ Title: Migrate your Azure Kubernetes Service (AKS) pod to use workload identity

+This article focuses on migrating from a pod-managed identity to Azure Active Directory (Azure AD) workload identity for your Azure Kubernetes Service (AKS) cluster. It also provides guidance depending on the version of the [Azure Identity][azure-identity-supported-versions] client library used by your container-based application.

+The Azure CLI version 2.40.0 or later. Run `az --version` to find the version, and run `az upgrade` to upgrade the version. If you need to install or upgrade, see [Install Azure CLI][install-azure-cli].

+ > [!NOTE]

+ > The migration sidecar is **not supported for production use**. This feature is meant to give you time to migrate your application SDK's to a supported version, and not meant or intended to be a long-term solution.

+> The migration sidecar is **not supported for production use**. This feature is meant to give you time to migrate your application SDK's to a supported version, and not meant or intended to be a long-term solution.

+This article showed you how to set up your pod to authenticate using a workload identity as a migration option. For more information about Azure AD workload identity, see the following [Overview][workload-identity-overview] article.

+[Defender for APIs](/azure/defender-for-cloud/defender-for-apis-introduction), a capability of [Microsoft Defender for Cloud](/azure/defender-for-cloud/defender-for-cloud-introduction), offers full lifecycle protection, detection, and response coverage for APIs that are managed in Azure API Management. The service empowers security practitioners to gain visibility into their business-critical APIs, understand their security posture, prioritize vulnerability fixes, and detect active runtime threats within minutes.

+> You can also onboard to Defender for APIs directly in the [Defender for Cloud interface](/azure/defender-for-cloud/defender-for-apis-deploy), where more API security insights and inventory experiences are available.

+You can remove APIs from protection by Defender for APIs by using Defender for Cloud in the portal. For more information, see [Manage your Defender for APIs deployment](/azure/defender-for-cloud/defender-for-apis-manage).

+* Learn more about [API findings, recommendations, and alerts](/azure/defender-for-cloud/defender-for-apis-posture) in Defender for APIs

+- [Shared Application Gateway](#shared-application-gateway): Install AGIC in an environment, where Application Gateway is

+- [Cloud Shell](https://shell.azure.com/) is the Azure shell environment, which has `az` CLI, `kubectl`, and `helm` installed. These tools are required for the following commands:

+**Backup your Application Gateway's configuration** before installing AGIC:

+The zip file you downloaded contains JSON templates, bash, and PowerShell scripts you could use to restore App

+[Helm](../aks/kubernetes-helm.md) is a package manager for Kubernetes, used to install the `application-gateway-kubernetes-ingress` package.

+important. The resource group required in the following commands is *not* the one referenced on the AKS portal pane. This is

+1. For the role assignment, commands we need to obtain `principalId` for the newly created identity:

+1. Give the identity `Contributor` access to your Application Gateway. For this you need the ID of the Application Gateway, which

+looks something like this: `/subscriptions/A/resourceGroups/B/providers/Microsoft.Network/applicationGateways/C`

+It's also possible to provide AGIC access to ARM via a Kubernetes secret.

+1. Download helm-config.yaml, which configures AGIC:

+ Or copy the following YAML file:

+ # Specify which application gateway the ingress controller must manage

+ # Setting appgw.shared to "true" creates an AzureIngressProhibitedTarget CRD.

+ # Specify which kubernetes namespace the ingress controller must watch

+## Shared Application Gateway

+By default AGIC assumes full ownership of the Application Gateway it's linked to. AGIC version 0.8.0 and later can

+hosted on Virtual Machine Scale Set and an AKS cluster.

+**Backup your Application Gateway's configuration** before enabling this setting:

+The zip file you downloaded contains JSON templates, bash, and PowerShell scripts you could use to restore Application Gateway

+With default settings, AGIC assumes 100% ownership of the Application Gateway it's pointed to. AGIC overwrites all of App

+defining it in the Kubernetes Ingress, AGIC deletes the `prod.contoso.com` config within seconds.

+Under the `appgw:` section, add `shared` key and set it to `true`.

+As a result your AKS has a new instance of `AzureIngressProhibitedTarget` called `prohibit-all-targets`:

+Helm install with `appgw.shared=true` deploys AGIC, but doesn't make any changes to Application Gateway.

+Since Helm with `appgw.shared=true` and the default `prohibit-all-targets` blocks AGIC from applying a config, broaden AGIC permissions:

+changes, AGIC overwrites or deletes them.

+(`manually-configured-staging-environment`) prohibits AGIC from overwriting Application Gateway configuration related to

+> [!Important]

+> - Automation Update management relies on [Log Analytics agent](../../azure-monitor/agents/log-analytics-agent.md) (aka MMA agent), which is on a deprecation path and wonΓÇÖt be supported after **August 31, 2024**. [Update management center (Preview)](../../update-center/overview.md) (UMC) is the v2 version of Automation Update management and the future of Update management in Azure. UMC is a native service in Azure and does not rely on [Log Analytics agent](../../azure-monitor/agents/log-analytics-agent.md) or [Azure Monitor agent](../../azure-monitor/agents/agents-overview.md).

+> - Guidance for migrating from Automation Update management to Update management center will be provided to customers once the latter is Generally Available. For customers using Automation Update management, we recommend continuing to use the Log Analytics agent and **NOT** migrate to Azure Monitoring agent until migration guidance is provided for Update management or else Automation Update management will not work. Also, the Log Analytics agent would not be deprecated before moving all Automation Update management customers to UMC.

+1. With the `kubeconfig` file pointing to the `apiserver` of your Kubernetes cluster, run this command to create a service account. This example creates the service account in the default namespace, but you can substitute any other namespace for `default`.

+ kubectl create serviceaccount demo-user -n default

+1. Create ClusterRoleBinding to grant this [service account the appropriate permissions on the cluster](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#kubectl-create-rolebinding). If you used a different namespace in the first command, substitute it here for `default`.

+1. With the `kubeconfig` file pointing to the `apiserver` of your Kubernetes cluster, run this command to create a service account. This example creates the service account in the default namespace, but you can substitute any other namespace for `default`.

+ kubectl create serviceaccount demo-user -n default

+1. Create ClusterRoleBinding or RoleBinding to grant this [service account the appropriate permissions on the cluster](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#kubectl-create-rolebinding). If you used a different namespace in the first command, substitute it here for `default`.

+For more information on snapshots, see [Debug snapshots on exceptions in .NET apps](../azure-monitor/app/snapshot-debugger.md) and [Troubleshoot problems enabling Application Insights Snapshot Debugger or viewing snapshots](/troubleshoot/azure/azure-monitor/app-insights/snapshot-debugger-troubleshoot).

+This article demonstrates how to:

+* Translate coordinate location into a human understandable cross street using [Search Address Reverse Cross Street], most often needed in tracking applications that receive a GPS feed from a device or asset, and wish to know where the coordinate is located.

+The example in this section uses [Get Search Address] to convert an address into latitude and longitude coordinates. This process is also called *geocoding*. In addition to returning the coordinates, the response also returns detailed address properties such as street, postal code, municipality, and country/region information.

+3. Select the blue **Send** button. The response body contains data for a single location.

+4. Next, search an address that has more than one possible locations. In the **Params** section, change the `query` key to `400 Broad, Seattle`. Select the blue **Send** button.

+6. Select the **Send** button. The response includes results from multiple countries. To geobias results to the relevant area for your users, always add as many location details as possible to the request.

+[Fuzzy Search] supports standard single line and free-form searches. We recommend that you use the Azure Maps Search Fuzzy API when you don't know your user input type for a search request. The query input can be a full or partial address. It can also be a Point of Interest (POI) token, like a name of POI, POI category or name of brand. Furthermore, to improve the relevance of your search results, constrain the query results using a coordinate location and radius, or by defining a bounding box.

+> [!TIP]

+> Most Search queries default to `maxFuzzyLevel=1` to improve performance and reduce unusual results. Adjust fuzziness levels by using the `maxFuzzyLevel` or `minFuzzyLevel` parameters. For more information on `maxFuzzyLevel` and a complete list of all optional parameters, see [Fuzzy Search URI Parameters].

+The example in this section uses `Fuzzy Search` to search the entire world for *pizza*, then searches over the scope of a specific country. Finally, it demonstrates how to use a coordinate location and radius to scope a search over a specific area, and limit the number of returned results.

+> [!IMPORTANT]

+> To geobias results to the relevant area for your users, always add as many location details as possible. For more information, see [Best Practices for Search].

+ > [!NOTE]

+ > The _json_ attribute in the URL path determines the response format. This article uses json for ease of use and readability. To find other supported response formats, see the `format` parameter definition in the [URI Parameter reference] documentation.

+3. Select **Send** and review the response body.

+ The ambiguous query string for "pizza" returned 10 [point of interest result] (POI) in both the "pizza" and "restaurant" categories. Each result includes details such as street address, latitude and longitude values, view port, and entry points for the location. The results are now varied for this query, and aren't tied to any reference location.

+ In the next step, you'll use the `countrySet` parameter to specify only the countries/regions for which your application needs coverage. For a complete list of supported countries/regions, see [Search Coverage].

+4. The default behavior is to search the entire world, potentially returning unnecessary results. Next, search for pizza only in the United States. Add the `countrySet` key to the **Params** section, and set its value to `US`. Setting the `countrySet` key to `US` bounds the results to the United States.

+5. To get an even more targeted search, you can search over the scope of a lat/lon coordinate pair. The following example uses the lat/lon coordinates of the Seattle Space Needle. Since we only want to return results within a 400-meters radius, we add the `radius` parameter. Also, we add the `limit` parameter to limit the results to the five closest pizza places.

+ | Key | Value |

+ |--||

+ | lat | 47.620525 |

+ | lon | -122.349274|

+ | radius | 400 |

+ | limit | 5 |

+6. Select **Send**. The response includes results for pizza restaurants near the Seattle Space Needle.

+> [!IMPORTANT]

+> To geobias results to the relevant area for your users, always add as many location details as possible. For more information, see [Best Practices for Search].

+> [!TIP]

+> If you have a set of coordinate locations to reverse geocode, you can use [Post Search Address Reverse Batch] to send a batch of queries in a single request.

+This example demonstrates making reverse searches using a few of the optional parameters that are available. For the full list of optional parameters, see [Reverse Search Parameters].

+3. Select **Send**, and review the response body. You should see one query result. The response includes key address information about Safeco Field.

+4. Next, add the following key/value pairs to the **Params** section:

+5. Select **Send**, and review the response body.

+6. Next, we add the `entityType` key, and set its value to `Municipality`. The `entityType` key overrides the `returnMatchType` key in the previous step. `returnSpeedLimit` and `returnRoadUse` also need removed since you're requesting information about the municipality. For all possible entity types, see [Entity Types].

+7. Select **Send**. Compare the results to the results returned in step 5. Because the requested entity type is now `municipality`, the response doesn't include street address information. Also, the returned `geometryId` can be used to request boundary polygon through Azure Maps Get [Search Polygon API].

+> [!TIP]

+> For more information on these as well as other parameters, see [Reverse Search Parameters].

+This example demonstrates how to search for a cross street based on the coordinates of an address.

+3. Select **Send**, and review the response body. Notice that the response contains a `crossStreet` value of `South Atlantic Street`.

+> [Azure Maps Search service]

+> [Best practices for Azure Maps Search service]

+[Azure Maps Search service]: /rest/api/maps/search

+[Best practices for Azure Maps Search service]: how-to-use-best-practices-for-search.md

+[Best Practices for Search]: how-to-use-best-practices-for-search.md#geobiased-search-results

+[Entity Types]: /rest/api/maps/search/getsearchaddressreverse#entitytype

+[Fuzzy Search]: /rest/api/maps/search/getsearchfuzzy

+[Get Search Address]: /rest/api/maps/search/getsearchaddress

+[Post Search Address Batch]: /rest/api/maps/search/postsearchaddressbatch

+[Postman]: https://www.postman.com/

+[Reverse Address Search Results]: /rest/api/maps/search/getsearchaddressreverse#searchaddressreverseresult

+[Reverse Address Search]: /rest/api/maps/search/getsearchaddressreverse

+[Route]: /rest/api/maps/route

+[Search Address Reverse Cross Street]: /rest/api/maps/search/getsearchaddressreversecrossstreet

+[Search Address]: /rest/api/maps/search/getsearchaddress

+[Search service]: /rest/api/maps/search

+[subscription key]: quick-demo-map-app.md#get-the-subscription-key-for-your-account

+[URI Parameter reference]: /rest/api/maps/search/getsearchfuzzy#uri-parameters

+[Weather]: /rest/api/maps/weather

+description: This article describes how to configure log alert pushes with webhook action and available customizations.

+[Log alerts](alerts-log.md) support [configuring webhook action groups](./action-groups.md#webhook). In this article, we describe the properties that are available. You can use webhook actions to invoke a single HTTP POST request. The service that's called should support webhooks and know how to use the payload it receives.

+We recommend that you use [common alert schema](../alerts/alerts-common-schema.md) for your webhook integrations. The common alert schema provides the advantage of having a single extensible and unified alert payload across all the alert services in Azure Monitor.

+For log alert rules that have a custom JSON payload defined, enabling the common alert schema reverts the payload schema to the one described in [Common alert schema](../alerts/alerts-common-schema.md#alert-context-fields-for-log-alerts). If you want to have a custom JSON payload defined, the webhook can't use the common alert schema.

+Alerts with the common schema enabled have an upper size limit of 256 KB per alert. A bigger alert doesn't include search results. When the search results aren't included, use `LinkToFilteredSearchResultsAPI` or `LinkToSearchResultsAPI` to access query results via the Log Analytics API.

+> The `"Severity"` field value changes if you've [switched to the current scheduledQueryRules API](/previous-versions/azure/azure-monitor/alerts/alerts-log-api-switch) from the [legacy Log Analytics Alert API](./api-alerts.md).

+> A custom JSON-based webhook isn't supported from API version `2021-08-01`.

+The following table lists default webhook action properties and their custom JSON parameter names.

+| `AlertRuleName` |#alertrulename |Name of the alert rule. |

+| `Severity` |#severity |Severity set for the fired log alert. |

+| `AlertThresholdOperator` |#thresholdoperator |Threshold operator for the alert rule. |

+| `AlertThresholdValue` |#thresholdvalue |Threshold value for the alert rule. |

+| `LinkToSearchResults` |#linktosearchresults |Link to the Analytics portal that returns the records from the query that created the alert. |

+| `LinkToSearchResultsAPI` |#linktosearchresultsapi |Link to the Analytics API that returns the records from the query that created the alert. |

+| `LinkToFilteredSearchResultsUI` |#linktofilteredsearchresultsui |Link to the Analytics portal that returns the records from the query filtered by dimensions value combinations that created the alert. |

+| `LinkToFilteredSearchResultsAPI` |#linktofilteredsearchresultsapi |Link to the Analytics API that returns the records from the query filtered by dimensions value combinations that created the alert. |

+| `ResultCount` |#searchresultcount |Number of records in the search results. |

+| `Search Interval End time` |#searchintervalendtimeutc |End time for the query in UTC, with the format mm/dd/yyyy HH:mm:ss AM/PM. |

+| `Search Interval` |#searchinterval |Time window for the alert rule, with the format HH:mm:ss. |

+| `Search Interval StartTime` |#searchintervalstarttimeutc |Start time for the query in UTC, with the format mm/dd/yyyy HH:mm:ss AM/PM.

+| `SearchQuery` |#searchquery |Log search query used by the alert rule. |

+| `SearchResults` |"IncludeSearchResults": true|Records returned by the query as a JSON table, limited to the first 1,000 records. "IncludeSearchResults": true is added in a custom JSON webhook definition as a top-level property. |

+| `Dimensions` |"IncludeDimensions": true|Dimensions value combinations that triggered that alert as a JSON section. "IncludeDimensions": true is added in a custom JSON webhook definition as a top-level property. |

+| `Alert Type`| #alerttype | The type of log alert rule configured as [Metric measurement or Number of results](./alerts-unified-log.md#measure).|

+| `WorkspaceID` |#workspaceid |ID of your Log Analytics workspace. |

+| `Application ID` |#applicationid |ID of your Application Insights app. |

+| `Subscription ID` |#subscriptionid |ID of your Azure subscription used. |

+You can use **Include custom JSON payload for webhook** to get a custom JSON payload by using the parameters. You can also generate more properties.

+For example, you might specify the following custom payload that includes a single parameter called `text`. The service that this webhook calls expects this parameter:

+This example payload resolves to something like the following example when it's sent to the webhook:

+Variables in a custom webhook must be specified within a JSON enclosure. For example, referencing `#searchresultcount` in the webhook example generates output based on the alert results.

+To include search results, add **IncludeSearchResults** as a top-level property in the custom JSON. Search results are included as a JSON structure, so results can't be referenced in custom-defined fields.

+> The **View Webhook** button next to the **Include custom JSON payload for webhook** option displays a preview of what was provided. It doesn't contain actual data but is representative of the JSON schema that will be used.

+For example, to create a custom payload that includes only the alert name and the search results, use this configuration:

+ Title: Smart detection in Application Insights | Microsoft Docs

+description: Application Insights performs automatic deep analysis of your app telemetry and warns you about potential problems.

+>You can migrate smart detection on your Application Insights resource to be based on alerts. The migration creates alert rules for the different smart detection modules. After it's created, you can manage and configure these rules like any other Azure Monitor alert rules. You can also configure action groups for these rules to enable multiple methods of taking actions or triggering notification on new detections.

+> For more information, see [Smart detection alerts migration](./alerts-smart-detections-migration.md).

+Smart detection automatically warns you of potential performance problems and failure anomalies in your web application. It performs proactive analysis of the telemetry that your app sends to [Application Insights](../app/app-insights-overview.md). If there's a sudden rise in failure rates or abnormal patterns in client or server performance, you get an alert. This feature needs no configuration. It operates if your application sends enough telemetry.

+You can access the detections issued by smart detection from the emails you receive and from the smart detection pane.

+ 

+ Select **See the analysis of this issue** to see more information in the portal.

+* **The smart detection pane** in Application Insights. Under the **Investigate** menu, select **Smart Detection** to see a list of recent detections.

+ 

+Smart detection detects and notifies you about various issues:

+* [Smart detection - Failure Anomalies](./proactive-failure-diagnostics.md): Notifies if the failure rate goes outside the expected envelope. We use machine learning to set the expected rate of failed requests for your app, correlating with load and other factors.

+* [Smart detection - Performance Anomalies](./smart-detection-performance.md): Notifies if response time of an operation or dependency duration is slowing down compared to the historical baseline. It also notifies if we identify an anomalous pattern in response time or page load time.

+* **General degradations and issues**: [Trace degradation](./proactive-trace-severity.md), [Memory leak](./proactive-potential-memory-leak.md), [Abnormal rise in Exception volume](./proactive-exception-volume.md), and [Security anti-patterns](./proactive-application-security-detection-pack.md).

+The help links in each notification take you to the relevant articles.

+You can configure email notifications for a specific smart detection rule. On the smart detection **Settings** pane, select the rule to open the **Edit rule** pane.

+Alternatively, you can change the configuration by using Azure Resource Manager templates. For more information, see [Manage Application Insights smart detection rules by using Azure Resource Manager templates](./proactive-arm-config.md).

+* [Analytics: Powerful query language](../logs/log-analytics-tutorial.md)

+Smart detection is automatic, but if you want to set up more alerts, see:

+In this article, you learn how to capture logs with Application Insights in .NET apps by using the [`Microsoft.Extensions.Logging.ApplicationInsights`][nuget-ai] provider package. If you use this provider, you can query and analyze your logs by using the Application Insights tools.

+With the NuGet package installed, and the provider being registered with dependency injection, the app is ready to log. With constructor injection, either <xref:Microsoft.Extensions.Logging.ILogger> or the generic-type alternative <xref:Microsoft.Extensions.Logging.ILogger%601> is required. When these implementations are resolved, `ApplicationInsightsLoggerProvider` provides them. Logged messages or exceptions are sent to Application Insights.

+For more information, see [Logging in ASP.NET Core](/aspnet/core/fundamentals/logging) and [What Application Insights telemetry type is produced from ILogger logs? Where can I see ILogger logs in Application Insights?](#what-application-insights-telemetry-type-is-produced-from-ilogger-logs-where-can-i-see-ilogger-logs-in-application-insights).

+For more information, see [What Application Insights telemetry type is produced from ILogger logs? Where can I see ILogger logs in Application Insights?](#what-application-insights-telemetry-type-is-produced-from-ilogger-logs-where-can-i-see-ilogger-logs-in-application-insights).

+### What Application Insights telemetry type is produced from ILogger logs? Where can I see ILogger logs in Application Insights?

+`ApplicationInsightsLoggerProvider` captures `ILogger` logs and creates `TraceTelemetry` from them. If an `Exception` object is passed to the `Log` method on `ILogger`, `ExceptionTelemetry` is created instead of `TraceTelemetry`.

+These telemetry items can be found in the same places as any other `TraceTelemetry` or `ExceptionTelemetry` items for Application Insights, including the Azure portal, analytics, or the Visual Studio local debugger.

+If you prefer to always send `TraceTelemetry`, use this snippet:

+```csharp

+builder.AddApplicationInsights(

+ options => options.TrackExceptionsAsExceptionTelemetry = false);

+```

+To use the Impact analysis workbook, in your Application Insights resources go to **Usage** > **More** and select **User Impact Analysis Workbook**. Or on the **Workbooks** tab, select **Public Templates**. Then under **Usage**, select **User Impact Analysis**.

+* **Sessions tool**: How many sessions of user activity have included certain pages and features of your app? A session is reset after half an hour of user inactivity, or after 24 hours of continuous use.

+ A custom event represents one occurrence of something happening in your app. It's often a user interaction like a button selection or the completion of a task. You insert code in your app to [generate custom events](./api-custom-events-metrics.md#trackevent) or use the [Click Analytics](javascript-feature-extensions.md#feature-extensions-for-the-application-insights-javascript-sdk-click-analytics) extension.

+Clicking **View More Insights** displays the following information:

+- Application Performance: Sessions, Events, and a Performance evaluation related to users' perception of responsiveness.

+- Properties: Charts containing up to six user properties such as browser version, country or region, and operating system.

+- Meet Your Users: View timelines of user activity.

+> The search job feature is currently not supported for workspaces with [customer-managed keys](customer-managed-keys.md).

+To test your move scenario without actually moving the resources, use the [Invoke-AzResourceAction](/powershell/module/az.resources/invoke-azresourceaction) command. Use this command only when you need to predetermine the results.

+$sourceName = "sourceRG"

+$destinationName = "destinationRG"

+$resourcesToMove = @("app1", "app2")

+$sourceResourceGroup = Get-AzResourceGroup -Name $sourceName

+$destinationResourceGroup = Get-AzResourceGroup -Name $destinationName

+$resources = Get-AzResource -ResourceGroupName $sourceName | Where-Object { $_.Name -in $resourcesToMove }

+-ResourceId $sourceResourceGroup.ResourceId `

+-Parameters @{ resources= $resources.ResourceId;targetResourceGroup = $destinationResourceGroup.ResourceId }

+$sourceName = "sourceRG"

+$destinationName = "destinationRG"

+$resourcesToMove = @("app1", "app2")

+$resources = Get-AzResource -ResourceGroupName $sourceName | Where-Object { $_.Name -in $resourcesToMove }

+Move-AzResource -DestinationResourceGroupName $destinationName -ResourceId $resources.ResourceId

+## Use Python

+### Validate

+To test your move scenario without actually moving the resources, use the [ResourceManagementClient.resources.begin_validate_move_resources](/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2022_09_01.operations.resourcesoperations#azure-mgmt-resource-resources-v2022-09-01-operations-resourcesoperations-begin-validate-move-resources) method. Use this method only when you need to predetermine the results.

+```python

+import os

+from azure.identity import AzureCliCredential

+from azure.mgmt.resource import ResourceManagementClient

+credential = AzureCliCredential()

+subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]

+resource_client = ResourceManagementClient(credential, subscription_id)

+source_name = "sourceRG"

+destination_name = "destinationRG"

+resources_to_move = ["app1", "app2"]

+destination_resource_group = resource_client.resource_groups.get(destination_name)

+resources = [

+ resource for resource in resource_client.resources.list_by_resource_group(source_name)

+ if resource.name in resources_to_move

+]

+resource_ids = [resource.id for resource in resources]

+validate_move_resources_result = resource_client.resources.begin_validate_move_resources(

+ source_name,

+ {

+ "resources": resource_ids,

+ "target_resource_group": destination_resource_group.id

+ }

+).result()

+print("Validate move resources result: {}".format(validate_move_resources_result))

+```

+If validation passes, you see no output.

+If validation fails, you see an error message describing why the resources can't be moved.

+### Move

+To move existing resources to another resource group or subscription, use the [ResourceManagementClient.resources.begin_move_resources](/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2022_09_01.operations.resourcesoperations#azure-mgmt-resource-resources-v2022-09-01-operations-resourcesoperations-begin-move-resources) method. The following example shows how to move several resources to a new resource group.

+```python

+import os

+from azure.identity import AzureCliCredential

+from azure.mgmt.resource import ResourceManagementClient

+credential = AzureCliCredential()

+subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]

+resource_client = ResourceManagementClient(credential, subscription_id)

+source_name = "sourceRG"

+destination_name = "destinationRG"

+resources_to_move = ["app1", "app2"]

+destination_resource_group = resource_client.resource_groups.get(destination_name)

+resources = [

+ resource for resource in resource_client.resources.list_by_resource_group(source_name)

+ if resource.name in resources_to_move

+]

+resource_ids = [resource.id for resource in resources]

+resource_client.resources.begin_move_resources(

+ source_name,

+ {

+ "resources": resource_ids,

+ "target_resource_group": destination_resource_group.id

+ }

+)

+```

+- **Agentless backup:** Azure Backup Server doesn't require an agent to be installed on the vCenter Server or ESXi server to back up the VM. Instead, provide the IP address or fully qualified domain name (FQDN) and the sign-in credentials used to authenticate the VMware vCenter Server with Azure Backup Server.

+- **Folder-level auto protection:** vCenter Server lets you organize your VMs into Virtual Machine folders. Azure Backup Server detects these folders. You can use it to protect VMs at the folder level, including all subfolders. During the protection of folders, Azure Backup Server protects the VMs in that folder and protects VMs added later. Azure Backup Server detects new VMs daily, protecting them automatically. As you organize your VMs in recursive folders, Azure Backup Server automatically detects and protects the new VMs deployed in the recursive folders.

+- **Application Consistent Backups:** If the *VMware Tools* isn't installed, a crash consistent backup will be executed. When the *VMware Tools* is installed with Microsoft Windows virtual machines, all applications that support VSS freeze and thaw operations will support application consistent backups. When the *VMware Tools* is installed with Linux virtual machines, application consistent snapshots are supported by calling the pre and post scripts.

+- If you're using *Azure Backup Server V3*, then you must install [Update Rollup 2](https://support.microsoft.com/topic/update-rollup-2-for-microsoft-azure-backup-server-v3-350de164-0ae4-459a-8acf-7777dbb7fd73). New installations from the Azure portal now use *Azure Backup Server V4* that supports vSphere, version *6.5* to *8.0*.

+- You can't back up user snapshots before the first Azure Backup Server backup. After Azure Backup Server finishes the first backup, then you can back up user snapshots.

+> * Is a System Center Operations Manager management server?

+> * Is a node of a cluster?

+The VM must have .NET Framework 4.5 or higher installed.

+ :::image type="content" source="../backup/media/backup-azure-microsoft-azure-backup/downloadcenter.png" alt-text="Screenshot showing the Microsoft Azure Backup files to download.":::

+ :::image type="content" source="../backup/media/backup-azure-microsoft-azure-backup/extract/03.png" alt-text="Screenshot showing the Microsoft Azure Backup files ready to extract.":::

+>- You can also locate the setup.exe file from the folder where you extracted the software package.

+>- To use your own SQL Server instance, ensure that you're using the supported SQL Server versions - SQL Server 2022 and 2019.

+1. On the setup window under **Install**, select **Microsoft Azure Backup** to open the setup wizard and accept any licensing terms from the list that appears.

+ **Configure reporting services with SQL Server 2019 or 2022**

+ If you use your instance of SQL Server, you must configure SQL Server Reporting Services (SSRS) manually. After configuring SSRS, make sure to set the **IsInitialized** property of SSRS to **True**. When set to **True**, Azure Backup Server assumes that SSRS is already configured and skips the SSRS configuration.

+Azure Backup Server only accepts storage volumes. When you add a volume, Azure Backup Server formats the volume to Resilient File System (ReFS), which Modern Backup Storage requires.

+## Upgrade to Azure Backup Server V4 from Azure Backup Server V3

+If you're already using Azure Backup Server V3 to back up AVS VMs, you can [upgrade to Azure Backup Server V4](../backup/backup-azure-microsoft-azure-backup.md#upgrade-mabs) to get access to the latest features and bug fixes.

+ Title: How to generate client access URL for Azure Web PubSub clients

+description: How to generate client access URL for Azure Web PubSub clients.

+# How to generate client access URL for the clients

+A client, be it a browser 💻, a mobile app 📱, or an IoT device 💡, uses a **Client Access URL** to connect and authenticate with your resource. This URL follows a pattern of `wss://<service_name>.webpubsub.azure.com/client/hubs/<hub_name>?access_token=<token>`. This article shows you several ways to get the Client Access URL.

+- For quick start, copy one from the Azure portal

+- For development, generate the value using [Web PubSub server SDK](./reference-server-sdk-js.md)

+- If you're using Azure AD, you can also invoke the [Generate Client Token REST API](/rest/api/webpubsub/dataplane/web-pub-sub/generate-client-token)

+## Copy from the Azure portal

+In the Keys tab in Azure portal, there's a Client URL Generator tool to quickly generate a Client Access URL for you, as shown in the following diagram. Values input here aren't stored.

+## Generate from service SDK

+The same Client Access URL can be generated by using the Web PubSub server SDK.

+# [JavaScript](#tab/javascript)

+1. Follow [Getting started with server SDK](./reference-server-sdk-js.md#getting-started) to create a `WebPubSubServiceClient` object `service`

+2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:

+ * Configure user ID

+ ```js

+ let token = await serviceClient.getClientAccessToken({ userId: "user1" });

+ ```

+ * Configure the lifetime of the token

+ ```js

+ let token = await serviceClient.getClientAccessToken({ expirationTimeInMinutes: 5 });

+ ```

+ * Configure a role that can join group `group1` directly when it connects using this Client Access URL

+ ```js

+ let token = await serviceClient.getClientAccessToken({ roles: ["webpubsub.joinLeaveGroup.group1"] });

+ ```

+ * Configure a role that the client can send messages to group `group1` directly when it connects using this Client Access URL

+ ```js

+ let token = await serviceClient.getClientAccessToken({ roles: ["webpubsub.sendToGroup.group1"] });

+ ```

+ * Configure a group `group1` that the client joins once it connects using this Client Access URL

+ ```js

+ let token = await serviceClient.getClientAccessToken({ groups: ["group1"] });

+ ```

+# [C#](#tab/csharp)

+1. Follow [Getting started with server SDK](./reference-server-sdk-csharp.md#getting-started) to create a `WebPubSubServiceClient` object `service`

+2. Generate Client Access URL by calling `WebPubSubServiceClient.GetClientAccessUri`:

+ * Configure user ID

+ ```csharp

+ var url = service.GetClientAccessUri(userId: "user1");

+ ```

+ * Configure the lifetime of the token

+ ```csharp

+ var url = service.GetClientAccessUri(expiresAfter: TimeSpan.FromMinutes(5));

+ ```

+ * Configure a role that can join group `group1` directly when it connects using this Client Access URL

+ ```csharp

+ var url = service.GetClientAccessUri(roles: new string[] { "webpubsub.joinLeaveGroup.group1" });

+ ```

+ * Configure a role that the client can send messages to group `group1` directly when it connects using this Client Access URL

+ ```csharp

+ var url = service.GetClientAccessUri(roles: new string[] { "webpubsub.sendToGroup.group1" });

+ ```

+ * Configure a group `group1` that the client joins once it connects using this Client Access URL

+ ```csharp

+ var url = service.GetClientAccessUri(groups: new string[] { "group1" });

+ ```

+# [Python](#tab/python)

+1. Follow [Getting started with server SDK](./reference-server-sdk-python.md#install-the-package) to create a `WebPubSubServiceClient` object `service`

+2. Generate Client Access URL by calling `WebPubSubServiceClient.get_client_access_token`:

+ * Configure user ID

+ ```python

+ token = service.get_client_access_token(user_id="user1")

+ ```

+ * Configure the lifetime of the token

+ ```python

+ token = service.get_client_access_token(minutes_to_expire=5)

+ ```

+ * Configure a role that can join group `group1` directly when it connects using this Client Access URL

+ ```python

+ token = service.get_client_access_token(roles=["webpubsub.joinLeaveGroup.group1"])

+ ```

+ * Configure a role that the client can send messages to group `group1` directly when it connects using this Client Access URL

+ ```python

+ token = service.get_client_access_token(roles=["webpubsub.sendToGroup.group1"])

+ ```

+ * Configure a group `group1` that the client joins once it connects using this Client Access URL

+ ```python

+ token = service.get_client_access_token(groups=["group1"])

+ ```

+# [Java](#tab/java)

+1. Follow [Getting started with server SDK](./reference-server-sdk-java.md#getting-started) to create a `WebPubSubServiceClient` object `service`

+2. Generate Client Access URL by calling `WebPubSubServiceClient.getClientAccessToken`:

+ * Configure user ID

+ ```java

+ GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();

+ option.setUserId(id);

+ WebPubSubClientAccessToken token = service.getClientAccessToken(option);

+ ```

+ * Configure the lifetime of the token

+ ```java

+ GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();

+ option.setExpiresAfter(Duration.ofDays(1));

+ WebPubSubClientAccessToken token = service.getClientAccessToken(option);

+ ```

+ * Configure a role that can join group `group1` directly when it connects using this Client Access URL

+ ```java

+ GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();

+ option.addRole("webpubsub.joinLeaveGroup.group1");

+ WebPubSubClientAccessToken token = service.getClientAccessToken(option);

+ ```

+ * Configure a role that the client can send messages to group `group1` directly when it connects using this Client Access URL

+ ```java

+ GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();

+ option.addRole("webpubsub.sendToGroup.group1");

+ WebPubSubClientAccessToken token = service.getClientAccessToken(option);

+ ```

+ * Configure a group `group1` that the client joins once it connects using this Client Access URL

+ ```java

+ GetClientAccessTokenOptions option = new GetClientAccessTokenOptions();

+ option.setGroups(Arrays.asList("group1")),

+ WebPubSubClientAccessToken token = service.getClientAccessToken(option);

+ ```

+In real-world code, we usually have a server side to host the logic generating the Client Access URL. When a client request comes in, the server side can use the general authentication/authorization workflow to validate the client request. Only valid client requests can get the Client Access URL back.

+## Invoke the Generate Client Token REST API

+You can enable Azure AD in your service and use the Azure AD token to invoke [Generate Client Token rest API](/rest/api/webpubsub/dataplane/web-pub-sub/generate-client-token) to get the token for the client to use.

+1. Follow [Authorize from application](./howto-authorize-from-application.md) to enable Azure AD.

+2. Follow [Get Azure AD token](./howto-authorize-from-application.md#use-postman-to-get-the-azure-ad-token) to get the Azure AD token with Postman.

+3. Use the Azure AD token to invoke `:generateToken` with Postman:

+ 1. For the URI, enter `https://{Endpoint}/api/hubs/{hub}/:generateToken?api-version=2022-11-01`

+ 2. On the **Auth** tab, select **Bearer Token** and paste the Azure AD token fetched in the previous step

+ 3. Select **Send** and you see the Client Access Token in the response:

+ ```json

+ {

+ "token": "ABCDEFG.ABC.ABC"

+ }

+ ```

+4. The Client Access URI is in the format of `wss://<endpoint>/client/hubs/<hub_name>?access_token=<token>`

+This code example creates a Web PubSub client that connects to the Web PubSub service instance. A client uses a Client Access URL to connect and authenticate with the service. It's best practice to not hard code the Client Access URL in your code. In the production world, we usually set up an app server to return this URL on demand. [Generate Client Access URL](./howto-generate-client-access-url.md) describes the practice in detail.

+A client can have a few ways to obtain the Client Access URL. For this quick start, you can copy and paste one from Azure portal shown in the following diagram. It's best practice to not hard code the Client Access URL in your code. In the production world, we usually set up an app server to return this URL on demand. [Generate Client Access URL](./howto-generate-client-access-url.md) describes the practice in detail.

+A client, be it a browser 💻, a mobile app 📱, or an IoT device 💡, uses a **Client Access URL** to connect and authenticate with your resource. This URL follows a pattern of `wss://<service_name>.webpubsub.azure.com/client/hubs/<hub_name>?access_token=<token>`. A client can have a few ways to obtain the Client Access URL. For this quick start, you can copy and paste one from Azure portal shown in the following diagram. It's best practice to not hard code the Client Access URL in your code. In the production world, we usually set up an app server to return this URL on demand. [Generate Client Access URL](./howto-generate-client-access-url.md) describes the practice in detail.

+This URL follows a pattern of `wss://<service_name>.webpubsub.azure.com/client/hubs/<hub_name>?access_token=<token>`. A client can have a few ways to obtain the Client Access URL. For this quick start, you can copy and paste one from Azure portal shown in the following diagram. It's best practice to not hard code the Client Access URL in your code. In the production world, we usually set up an app server to return this URL on demand. [Generate Client Access URL](./howto-generate-client-access-url.md) describes the practice in detail.

+**Cause**: Specific FQDN/application rules are required to use cluster extensions in the AKS clusters. [Learn more](../aks/outbound-rules-control-egress.md#cluster-extensions).

+This article describes the prerequisites for Azure Kubernetes Service (AKS) backup.

+Azure Backup now allows you to back up AKS clusters (cluster resources and persistent volumes attached to the cluster) using a backup extension, which must be installed in the cluster. Backup vault communicates with the cluster via this Backup Extension to perform backup and restore operations. Based on the least privileged security model, a Backup vault must have *Trusted Access* enabled to communicate with the AKS cluster.

+- Install Backup Extension on the AKS clusters following the [required FQDN/application rules](../aks/outbound-rules-control-egress.md).

+You can restore individual files from a protected Hyper-V VM recovery point, both disk and online. This feature is only available for Windows Server VMs. Restoring individual files is similar to restoring the entire VM, except you browse into the VMDK and find the file(s) you want, before starting the recovery process.

+> With MABS v4 and later, you can restore an individual file from a Hyper-V VM from both disk and online recovery points. The VM should be a Windows Server VM.

+>

+> Additionally, for item-level recovery from an online recovery point, ensure that the Hyper-V role is installed on the MABS Server, automatic mounting of volumes is enabled, and the VM VHD doesn't contain a dynamic disk. The item-level recovery for online recovery points works by mounting the VM recovery point using *iSCSI* for browsing, and only one VM can be mounted at a given time.

+1. To find the files you want to recover, in the **Path** pane, double-click the item in the Recoverable item column to open it.

+ If you use an online recovery point, wait until the recovery point is mounted. Once the mount is complete, select the *VM*, *VHD disk*, and the *volume* you want to restore until the files and folders are listed.

+ Select the file, files, or folders you want to recover. To select multiple items, press the **Ctrl** key while selecting each item. Use the **Path** pane to search the list of files or folders appearing in the **Recoverable Item** column.**Search list below** doesn't search into subfolders. To search through subfolders, double-click the folder. Use the Up button to move from a child folder into the parent folder. You can select multiple items (files and folders), but they must be in the same parent folder. You can't recover items from multiple folders in the same recovery job.

+>[!Tip]

+>You can perform item-level restore of online recovery points for Hyper-V VMs running Windows also from *Add external DPM Server* to recover VM files and folders quickly.

+>[!Note]

+>By default, *eight parallel recoveries* are supported. You can increase the number of parallel restore jobs by adding the following registry key:

+>**Key Path**: `HKLM\Software\Microsoft\Microsoft Data Protection Manager\Configuration\ MaxParallelRecoveryJobs`

+>- **32 Bit DWORD**: HyperV

+>- **Data**: `<number>`

+| MABS v4 | VMware server 8.0, 7.0, 6.7, or 6.5 (Licensed version) |

+- MABS v4 doesn't support the *DataSets* feature for VMware 8.0.

+## VMware vSphere 6.7, 7.0, and 8.0

+To back up vSphere 6.7, 7.0, and 8.0, follow these steps:

+> Applies To: MABS v4.

+This article explains how to prepare your environment to back up workloads using Microsoft Azure Backup Server (MABS). With Azure Backup Server, you can protect application workloads, such as Hyper-V VMs, VMware VMs, Azure Stack HCI VMs, Microsoft SQL Server, SharePoint Server, Microsoft Exchange, and Windows clients from a single console.

+> To learn more about backing up VMware servers with Azure Backup Server, see the article, [Use Azure Backup Server to back up a VMware server](backup-azure-backup-server-vmware.md). To learn about security capabilities, refer to [Azure Backup security features documentation](backup-azure-security-feature.md).

+When choosing a server for running Azure Backup Server, it's recommended you start with a gallery image of Windows Server 2022 Datacenter or Windows Server 2019 Datacenter. The article, [Create your first Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md?toc=/azure/virtual-machines/windows/toc.json), provides a tutorial for getting started with the recommended virtual machine in Azure, even if you've never used Azure before. The recommended minimum requirements for the server virtual machine (VM) should be: Standard_A4_v2 with four cores and 8-GB RAM.

+| Windows Server 2022 |64 bit |Standard, Datacenter, Essentials |

+| Windows Server 2019 |64 bit |Standard, Datacenter, Essentials |

+ >If you wish to use your own SQL server, the supported SQL Server versions are SQL Server 2022 and 2019. All SQL Server versions should be Standard or Enterprise 64-bit.

+ >If you run into errors in vault registration, ensure that you're using the latest version of the MARS agent, instead of the version packaged with MABS server. You can download the latest version [from here](https://aka.ms/azurebackup_agent) and replace the *MARSAgentInstaller.exe* file in *MARSAgent* folder in the extracted path before installation and registration on new servers.

+4. Install Windows Server on a new machine and give it the same machine name as the original Azure Backup server.

+6. Install Azure Backup Server V4 or later (move MABS Storage pool disks from old server and import).

+ * `www.msftconnecttest.com`

+ * `*.blob.core.windows.net`

+ * `*.queue.core.windows.net`

+ * `*.blob.storage.azure.net`

+### Upgrade from MABS V3 to V4

+> MABS V3 isn't a prerequisite for installing MABS V4. However, you can upgrade to MABS V4 only from MABS V3 (RTM, Update Rollup 1 and Update Rollup 2).

+1. To upgrade from MABS V3 to MABS V4, upgrade your OS to Windows Server 2022 or Windows Server 2019 if needed.

+2. Upgrade your server. The steps are similar to [installation](#install-and-upgrade-azure-backup-server). However, for SQL settings, you'll get an option to upgrade your SQL instance to SQL 2022, or to use your own instance of SQL server.

+## Increase maximum parallel online backups

+You can increase the number of maximum parallel online backup jobs from the default 8 to a configurable number using the following registry keys (if your underlying hardware and network bandwidth can support it).

+The example below increases the limit to 12 jobs.

+- `[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows Azure Backup\DbgSettings\OnlineBackup]`

+ "MaxParallelBackupJobs"=dword:0000000C

+- `[HKEY_LOCAL_MACHINE\Software\Microsoft\Microsoft Data Protection Manager\Configuration\DPMTaskController\MaxRunningTasksThreshold]`

+ "6e7c76f4-a832-4418-a772-8e58fd7466cb"=dword:0000000C

+>Private endpoints are supported with only DPM server 2022, MABS v4, and later.

+### Immutability support for DPM and MABS

+This feature is supported with MARS agent version *2.0.9250.0* and higher from DPM 2022 UR1 and MABS v4.

+| Operation on Immutable vault | Result with DPM 2022 UR1, MABS v4, and latest MARS agent | Result with older DPM/MABS and or MARS agent |

+- MABS v3 UR2, MABS v4, or later supports SQL Server Failover Cluster Instance (FCI) using Cluster Shared Volume (CSV).

+ *Total data size* is the size of the data you want to back up, and disk space to be provisioned on DPM is the space that MABS recommends for the protection group. DPM chooses the ideal backup volume based on the settings. However, you can edit the backup volume choices in the disk allocation details. For the workloads, select the preferred storage in the dropdown menu. The edits change the values for *Total Storage* and *Free Storage* in the **Available Disk Storage** pane. *Underprovisioned space* is the amount of storage that DPM suggests you add to the volume for continuous smooth backups.

+

+> To use Modern Backup Storage, you must run Backup Server V2 or later on Windows Server 2016 or later.

+> To achieve enhanced backup performance we recommend deploying MABS v4 with tiered storage on Windows Server 2022. To configure tiered storage, see [Set up MBS with Tiered Storage](/system-center/dpm/add-storage#set-up-mbs-with-tiered-storage).

+Using Backup Server with volumes as disk storage can help you maintain control over storage. A volume can be a single disk. However, if you want to extend storage in the future, create a volume out of a disk created by using storage spaces. This can help if you want to expand the volume for backup storage. This section offers best practices for creating a volume with this setup.

+## Migrate legacy storage to Modern Backup Storage for MABS v2

+

+>For more information on tbe supported workloads, see the [Azure Backup Server protection matrix](backup-mabs-protection-matrix.md).

+.NET Framework 4.5 or higher must be installed on the virtual machine.

+When choosing a server for Azure Backup Server, start with a Windows Server 2022 Datacenter or Windows Server 2019 Datacenter gallery image. The article, [Create your first Windows virtual machine in the Azure portal](../virtual-machines/windows/quick-create-portal.md?toc=/azure/virtual-machines/windows/toc.json), provides a tutorial for getting started with the recommended virtual machine. The recommended minimum requirements for the server virtual machine (VM) should be: A2 Standard with two cores and 3.5-GB RAM.

+12. Switch to your Azure subscription and your Recovery Services vault. In the **Prepare Infrastructure** menu, select **Download** to download vault credentials. If the **Download** button in step 2 isn't active, select **Already downloaded or using the latest Azure Backup Server installation** to activate the button. The vault credentials download to the location where your downloads are stored. Be aware of this location because you'll need it for the next step.

+- **URLs**

+ - `www.msftncsi.com`

+ - `*.Microsoft.com`

+ - `*.WindowsAzure.com`

+ - `*.microsoftonline.com`

+ - `*.windows.net`

+ - `www.msftconnecttest.com`

+ - `*.blob.core.windows.net`

+ - `*.queue.core.windows.net`

+ - `*.blob.storage.azure.net`

+- **IP addresses**

+ - `20.190.128.0/18`

+ - `40.126.0.0/18`

+- While a subscription is *Deprovisioned*, it loses functionality. If you restore the subscription to *Active*, it revives the backup/restore functionality. If the backup data on the local disk was retained with a sufficiently large retention period, that backup data can be retrieved. However, backup data in Azure is irretrievably lost once the subscription enters the *Deprovisioned* state.

+If Microsoft Azure Backup server fails with errors during the setup phase (backup or restore), see the [error codes document](https://support.microsoft.com/kb/3041338).

+ Title: MABS (Azure Backup Server) V4 protection matrix

+description: This article provides a support matrix listing all workloads, data types, and installations that Azure Backup Server v4 protects.

+# MABS (Azure Backup Server) V4 (and later) protection matrix

+Use the following matrix for MABS v4 (and later):

+>Support for the 32-bit protection agent isn't supported with MABS v4 (and later). See [32-Bit protection agent deprecation](backup-mabs-whats-new-mabs.md#32-bit-protection-agent-deprecation).

+| Client computers (64-bit) | Windows 11, Windows 10 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | V4 | Volume, share, folder, files, deduped volumes <br><br> Protected volumes must be NTFS. FAT and FAT32 aren't supported. <br><br> Volumes must be at least 1 GB. Azure Backup Server uses Volume Shadow Copy Service (VSS) to take the data snapshot and the snapshot only works if the volume is at least 1 GB. |

+| Servers (64-bit) | Windows Server 2022, 2019, 2016, 2012, 2012 R2 <br /><br />(Including Windows Server Core edition) | Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure Stack | V4 | Volume, share, folder, file <br><br> Deduped volumes (NTFS only) <br><br> System state and bare metal (Not supported when workload is running as Azure virtual machine). <br><br> To protect Windows Server 2012 and 2012 R2, install [Visual C++ 2015](https://www.microsoft.com/download/details.aspx?id=48145) on the protected server. |

+| SQL Server | SQL Server 2022, 2019, 2017, 2016 and [supported SPs](https://support.microsoft.com/lifecycle/search?alpha=SQL%20Server%202016), 2014 and supported [SPs](https://support.microsoft.com/lifecycle/search?alpha=SQL%20Server%202014) | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Azure Stack | V4 | All deployment scenarios: database <br><br> SQL database, stored on the Cluster Shared Volume and ReFS volumes. <br><br> MABS doesn't support SQL Server databases hosted on Scale-Out File Servers (SOFS). <br><br> MABS can't protect SQL server Distributed Availability Group (DAG) or Availability Group (AG), where the role name on the failover cluster is different than the named AG on SQL. |

+| Exchange | Exchange 2019, 2016 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure Stack <br><br> Azure virtual machine (when workload is running as Azure virtual machine) | V4 | Protect (all deployment scenarios): Standalone Exchange server, database under a database availability group (DAG) <br><br> Recover (all deployment scenarios): Mailbox, mailbox databases under a DAG <br><br> Backup of Exchange over ReFS is supported with MABS v3 UR1 |

+| SharePoint | SharePoint 2019, 2016 with latest SPs | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Azure Stack | V4 | Protect (all deployment scenarios): Farm, frontend web server content <br><br> Recover (all deployment scenarios): Farm, database, web application, file, or list item, SharePoint search, frontend web server <br><br> Protecting a SharePoint farm that's using the SQL Server 2012 Always On feature for the content databases isn't supported. |

+| Hyper-V host - MABS protection agent on Hyper-V host server, cluster, or VM | Windows Server 2022, 2019, 2016, 2012 R2, 2012 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | V4 | Protect: Virtual machines, cluster shared volumes (CSVs) <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives |

+| Azure Stack HCI | V1, 20H2, 21H2, and 22H2 | Physical server <br><br> Hyper-V / Azure Stack HCI virtual machine <br><br> VMware virtual machine | V4 | Protect: Virtual machines, cluster shared volumes (CSVs) <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives |

+| VMware VMs | VMware server 6.5, 6.7, 7.0, 8.0 (Licensed Version) | Hyper-V virtual machine <br><br> VMware virtual machine | V4 | Protect: VMware VMs on cluster-shared volumes (CSVs), NFS, and SAN storage <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives <br><br> VMware vApps aren't supported. <br><br> vSphere 8.0 DataSets feature isn't supported for backup. |

+| Linux | Linux running as [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) or [VMware](backup-azure-backup-server-vmware.md) or [Stack](backup-mabs-install-azure-stack.md) guest | Physical server, On-premises Hyper-V VM, Stack VM or VMware VM running Windows Server. | V4 | Hyper-V must be running on Windows Server 2016, Windows Server 2019, or Windows Server 2022. Protect: Entire virtual machine <br><br> Recover: Entire virtual machine <br><br> Only file-consistent snapshots are supported. <br><br> For a complete list of supported Linux distributions and versions, see the article, [Linux on distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md). |

+|Servers (64-bit) | Windows Server 2008 R2 SP1, Windows Server 2008 SP2 (You need to install [Windows Management Framework](https://www.microsoft.com/download/details.aspx?id=54616)), Windows Server 2012, Windows Server 2012 R2. | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | Volume, share, folder, file, system state/bare metal |

+>MABS V4 supports the protection of Hyper-V virtual machines and SQL Server Failover Cluster Instance (FCI) on Cluster Shared Volumes (CSVs). Protection of other workloads hosted on CSVs isn't supported.

+## MABS V4 known issues and workarounds

+If you're protecting Windows Server 2012 and 2012 R2, you need to install Visual C++ redistributable 2015 manually on the protected server. You can download [Visual C++ Redistributable for Visual Studio 2015 from Official Microsoft Download Center](https://www.microsoft.com/en-in/download/details.aspx?id=48145).

+## MABS V3 known issues and workarounds

+### Backup and recovery fails for clustered workloads

+>[!Note]

+>This issue is fixed in MABS V4.

+### Upgrade to MABS V3 fails in Russian locale

+### After installing UR1, the MABS reports aren't updated with new RDL files

+You need to confirm the supported scenarios before you back up a SharePoint farm to Azure from the [support matrix](backup-mabs-protection-matrix.md).

+| SharePoint |SharePoint 2019, SharePoint 2016 with latest SPs |SharePoint deployed as an Azure Stack virtual machine <br> -- <br> SQL Always On | Protect SharePoint Farm recovery options: Recovery farm, database, and file or list item from disk recovery points. Farm and database recovery from Azure recovery points. |

+ If there's more than one WFE server, select the one on which you installed *ConfigureSharePoint.exe*.

+# What's new in Microsoft Azure Backup Server (MABS)?

+Microsoft Azure Backup Server gives you enhanced backup capabilities to protect VMs, files and folders, workloads, and more.

+## What's new in MABS V4 RTM

+Microsoft Azure Backup Server version 4 (MABS V4) includes critical bug fixes and the support for Windows Server 2022, SQL 2022, Azure Stack HCI 22H2, and other features and enhancements. To view the list of bugs fixed and the installation instructions for MABS V4, see [KB article 5024199](https://support.microsoft.com/help/5024199/).

+The following table lists the included features in MABS V4:

+| Supported feature | Description |

+| | |

+| Windows Server 2022 support | You can install MABS V4 on and protect Windows Server 2022. To use MABS V4 with *WS2022*, you can either upgrade your operation system (OS) to *WS2022* before installing/upgrading to MABS V4, or you can upgrade your OS after installing/upgrading V4 on *WS2019*. <br><br> MABS V4 is a full release, and can be installed directly on Windows Server 2022, Windows Server 2019, or can be upgraded from MABS V3. Learn more [about the installation prerequisites](backup-azure-microsoft-azure-backup.md#software-package) before you upgrade to or install Backup Server V4. |

+| SQL Server 2022 support | You can install MABS V4 with SQL 2022 as the MABS database. You can upgrade the SQL Server from SQL 2017 to SQL 2022, or install it fresh. You can also back up SQL 2022 workload with MABS V4. |

+| Private Endpoint Support | With MABS V4, you can use private endpoints to send your online backups to Azure Backup Recovery Services vault. [Learn more](backup-azure-private-endpoints-concept.md). |

+| Azure Stack HCI 22H2 support | MABS V4 now supports protection of workloads running in Azure Stack HCI V1 till 22H2. [Learn more](back-up-azure-stack-hyperconverged-infrastructure-virtual-machines.md). |

+| VMware 8.0 support | MABS V4 can now back up VMware VMs running on VMware 8.0. MABS V4 supports VMware, version 6.5 to 8.0. [Learn more](backup-azure-backup-server-vmware.md). <br><br> Note that MABS V4 doesn't support the DataSets feature added in vSphere 8.0. |

+| Item-level recovery from online recovery points for Hyper-V and Stack HCI VMs running Windows Server | With MABS V4, you can perform item-level recovery of files and folders from your online recovery point for VMs running Windows Server on Hyper-V or Stack HCI without downloading the entire recovery point. <br><br> Go to the *Recovery* pane, select a *VM online recovery point* and double-click the *recoverable item* to browse and recover its contents at a file/folder level. <br><br> [Learn more](back-up-hyper-v-virtual-machines-mabs.md). |

+| Parallel Restore of VMware and Hyper-V VMs | MABS V4 supports parallel restore of [VMware](restore-azure-backup-server-vmware.md) and [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) virtual machines. With earlier versions of MABS, restore of VMware VM and Hyper-V virtual machine was restricted to only one restore job at a time. With MABS V4, by default you can restore *eight* VMs in parallel and this number can be increased using a registry key. |

+| Parallel online backup jobs - limit enhancement | MABS V4 supports increasing the maximum parallel online backup jobs from *eight* to a configurable limit based on your hardware and network limitations through a registry key for faster online backups. [Learn more](backup-azure-microsoft-azure-backup.md). |

+| Faster Item Level Recoveries | MABS V4 moves away from File Catalog for online backup of file/folder workloads. File Catalog was necessary to restore individual files and folders from online recovery points, but increased backup time by uploading file metadata. <br><br> MABS V4 uses an *iSCSI mount* to provide faster individual file restores and reduces backup time, because file metadata doesn't need to be uploaded. |

+## WhatΓÇÖs new in MABS v3 UR2?

+With MABS v3 UR2, you can back up Virtual Machines on Azure Stack HCI. [Learn more](./back-up-azure-stack-hyperconverged-infrastructure-virtual-machines.md).

+## What's new in MABS V3 UR1?

+## What's new in MABS V3 RTM?

+You can download MABS from the [Microsoft Download Center](https://go.microsoft.com/fwLink/?LinkId=626082). It can be run on-premises or on an Azure VM.

+**MABS on an Azure VM** | MABS v4 and later: Windows 2022 Datacenter, Windows 2019 Datacenter <br><br> MABS v3, UR1 and UR2: Windows 2019 Datacenter, Windows 2016 Datacenter <br/><br/> We recommend that you start with an image from the marketplace.<br/><br/> Minimum Standard_A4_v2 with four cores and 8-GB RAM.

+**DPM on an Azure VM** | System Center 2012 R2 with Update 3 or later<br/><br/> Windows operating system as [required by System Center](/system-center/dpm/prepare-environment-for-dpm#dpm-server).<br/><br/> We recommend that you start with an image from the marketplace.<br/><br/> Minimum Standard_A4_v2 with four cores and 8-GB RAM.

+**MABS on-premises** | MABS v4 and later: Windows Server 2022 or Windows Server 2019 <br><br> MABS v3, UR1 and UR2: Windows Server 2019 and Windows Server 2016

+**MABS upgrade** | You can directly install MABS v4, or upgrade to MABS v4 from MABS v3 UR1 and UR2. [Learn more](backup-azure-microsoft-azure-backup.md#upgrade-mabs).

+**MABS on Azure Stack VM** | At least size A2. We recommend you start with a Windows Server 2019 or Windows Server 2022 image from Azure Marketplace.<br/><br/> Don't install anything else on the MABS VM.

+**.NET Framework on MABS** | The MABS VM needs .NET Framework 4.5 or later installed on it.

+**Supported backup** | These operating systems are supported for VMs that you want to back up: <br/><br/> Windows Server 2022, Windows Server 2019, Windows Server 20016, Windows Server 2012, Windows Server 2012 R2

+**SQL Server support for Azure Stack VMs** | Back up SQL Server 2022, SQL Server 2019, SQL Server 2017, SQL Server 2016 (SPs), and SQL Server 2014 (SPs).<br/><br/> Back up and recover a database.

+**SharePoint support for Azure Stack VMs** | SharePoint 2019, SharePoint 2016 with latest SPs.<br/><br/> Back up and recover a farm, database, front end, and web server.

+## Networking and access support

+|Domain | The DPM/MABS server should be in a Windows Server 2022, Windows Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012 domain. |

+Deduplication support for MABS depends on operating system support.

+### For NTFS volumes with MABS v4

+| Operating system of protected server | Operating system of MABS server | MABS version | Dedupe support |

+| | | | |

+| Windows Server 2022 | Windows Server 2022 | MABS v4 | Y |

+| Windows Server 2019 | Windows Server 2022 | MABS v4 | Y |

+| Windows Server 2016 | Windows Server 2022 | MABS v4 | Y* |

+| Windows Server 2022 | Windows Server 2019 | MABS v4 | N |

+| Windows Server 2019 | Windows Server 2019 | MABS v4 | Y |

+| Windows Server 2016 | Windows Server 2019 | MABS v4 | Y* |

+**Deduped NTFS volumes in Windows Server 2016 Protected Servers are non-deduplicated during restore.*

+### For NTFS volumes with MABS v3

+| Operating system of protected server | Operating system of MABS server | MABS version | Dedupe support |

+- We've identified a few issues with backups of deduplicated ReFS volumes. We're working on fixing these, and will update this section as soon as we have a fix available. Until then, we're removing the support for backup of deduplicated ReFS volumes from MABS v3 and v4.

+- MABS v3 UR1, MABS v4, and later continues to support protection and recovery of normal ReFS volumes.

+* **Access control**: Vaults (Recovery Services and Backup vaults) provide the management capabilities and are accessible via the Azure portal, Backup Center, Vault dashboards, SDK, CLI, and even REST APIs. It's also an Azure role-based access control (Azure RBAC) boundary, providing you with the option to restrict access to backups only to authorized Backup Admins.

+* Azure Backup provides you with the flexibility to *stop protecting and manage your backups*:

+* When you enable private endpoints for the vault, they're only used for backup and restore of SQL and SAP HANA workloads in an Azure VM, MARS agent, DPM/MABS backups. You can use the vault for the backup of other workloads as well (they wonΓÇÖt require private endpoints though). In addition to the backup of SQL and SAP HANA workloads, backup using the MARS agent and DPM/MABS Server, private endpoints are also used to perform file recovery in the case of Azure VM backup. [Learn more here](private-endpoints-overview.md#recommended-and-supported-scenarios).

+- Whenever new infrastructure is provisioned and new VMs are created, as a backup admin, you need to ensure their protection. You can easily configure backups for one or two VMs. But it becomes complex when you need to configure hundreds or even thousands of VMs at scale. To simplify the process of configuring backups, Azure Backup provides you with a set of built-in Azure Policies to govern your backup estate.

+- **Audit-only Policy**: Azure Backup also provides you with an Audit-only policy that identifies the VMs with no backup configuration.

+* **Reduce the backup storage cost with Selectively backup disks**: Exclude disk (preview feature) provides an efficient and cost-effective choice to selectively backup critical data. For example, you can back up only one disk when you don't want to back up all disks attached to a VM. This is also useful when you have multiple backup solutions. For example, to back up your databases or data with a workload backup solution (SQL Server database in Azure VM backup), use Azure VM level backup for selected disks.

+ * You can send data to an Azure Storage account if you want to retain your log data longer than 90 days for audit, static analysis, or back up. If you only need to retain your events for 90 days or less, you don't need to set up archives to a storage account, since Activity Log events are kept in the Azure platform for 90 days. [Learn more](../azure-monitor/essentials/activity-log.md#send-to-azure-storage).

+ Title: MABS (Azure Backup Server) V3 UR1 protection matrix

+description: This article provides a support matrix listing all workloads, data types, and installations that Azure Backup Server protects.

+# MABS (Azure Backup Server) V3 UR1 (and later) protection matrix

+This article lists the various servers and workloads that you can protect with Azure Backup Server. The following matrix lists what can be protected with Azure Backup Server.

+Use the following matrix for MABS v3 UR1 (and later):

+* Workloads ΓÇô The workload type of technology.

+* Version ΓÇô Supported MABS version for the workloads.

+* MABS installation ΓÇô The computer/location where you wish to install MABS.

+* Protection and recovery ΓÇô List the detailed information about the workloads such as supported storage container or supported deployment.

+>[!NOTE]

+>Support for the 32-bit protection agent is deprecated with MABS v3 UR1 (and later). See [32-Bit protection agent deprecation](backup-mabs-whats-new-mabs.md#32-bit-protection-agent-deprecation).

+## Protection support matrix

+The following sections details the protection support matrix for MABS:

+* [Applications Backup](#applications-backup)

+* [VM Backup](#vm-backup)

+* [Linux](#linux)

+## Applications backup

+| **Workload** | **Version** | **Azure Backup Server installation** | **Azure Backup Server** | **Protection and recovery** |

+| -- | | | | |

+| Client computers (64-bit) | Windows 11, Windows 10 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | V3 UR1 and V3 UR2 | Volume, share, folder, files, deduped volumes <br><br> Protected volumes must be NTFS. FAT and FAT32 aren't supported. <br><br> Volumes must be at least 1 GB. Azure Backup Server uses Volume Shadow Copy Service (VSS) to take the data snapshot and the snapshot only works if the volume is at least 1 GB. |

+| Servers (64-bit) | Windows Server 2022, 2019, 2016, 2012 R2, 2012 <br /><br />(Including Windows Server Core edition) | Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure Stack | V3 UR1 and V3 UR2 | Volume, share, folder, file <br><br> Deduped volumes (NTFS only) <br><br>When protecting a WS 2016 NTFS deduped volume with MABS v3 running on Windows Server 2019, the recoveries may be affected. We have a fix for doing recoveries in a non-deduped way that will be part of later versions of MABS. Contact MABS support if you need this fix on MABS v3 UR1.<br><br> When protecting a WS 2019 NTFS deduped volume with MABS v3 on Windows Server 2016, the backups and restores will be non-deduped. This means that the backups will consume more space on the MABS server than the original NTFS deduped volume. <br><br> System state and bare metal (Not supported when workload is running as Azure virtual machine) |

+| SQL Server | SQL Server 2019, 2017, 2016 and [supported SPs](https://support.microsoft.com/lifecycle/search?alpha=SQL%20Server%202016), 2014 and supported [SPs](https://support.microsoft.com/lifecycle/search?alpha=SQL%20Server%202014) | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Azure Stack | V3 UR1 and V3 UR2 | All deployment scenarios: database <br><br> MABS v3 UR2 and later supports the backup of SQL database, stored on the Cluster Shared Volume. <br><br> MABS v3 UR1 supports the backup of SQL databases over ReFS volumes <br><br> MABS doesn't support SQL Server databases hosted on Windows Server 2012 Scale-Out File Servers (SOFS). <br><br> MABS can't protect SQL server Distributed Availability Group (DAG) or Availability Group (AG), where the role name on the failover cluster is different than the named AG on SQL. |

+| Exchange | Exchange 2019, 2016 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure Stack <br><br> Azure virtual machine (when workload is running as Azure virtual machine) | V3 UR1 and V3 UR2 | Protect (all deployment scenarios): Standalone Exchange server, database under a database availability group (DAG) <br><br> Recover (all deployment scenarios): Mailbox, mailbox databases under a DAG <br><br> Backup of Exchange over ReFS is supported with MABS v3 UR1 |

+| SharePoint | SharePoint 2019, 2016 with latest SPs | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine <br><br> Azure virtual machine (when workload is running as Azure virtual machine) <br><br> Azure Stack | V3 UR1 and V3 UR2 | Protect (all deployment scenarios): Farm, frontend web server content <br><br> Recover (all deployment scenarios): Farm, database, web application, file, or list item, SharePoint search, frontend web server <br><br> Protecting a SharePoint farm that's using the SQL Server 2012 Always On feature for the content databases isn't supported. |

+## VM Backup

+| **Workload** | **Version** | **Azure Backup Server installation** | **Supported Azure Backup Server** | **Protection and recovery** |

+| | - | | - | |

+| Hyper-V host - MABS protection agent on Hyper-V host server, cluster, or VM | Windows Server 2022, 2019, 2016, 2012 R2, 2012 | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | V3 UR1 and V3 UR2 | Protect: Virtual machines, cluster shared volumes (CSVs) <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives |

+| Azure Stack HCI | V1, 20H2, and 21H2 | Physical server <br><br> Hyper-V / Azure Stack HCI virtual machine <br><br> VMware virtual machine | V3 UR2 and later | Protect: Virtual machines, cluster shared volumes (CSVs) <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives |

+| VMware VMs | VMware server 5.5, 6.0, or 6.5, 6.7 (Licensed Version) | Hyper-V virtual machine <br><br> VMware virtual machine | V3 UR1 | Protect: VMware VMs on cluster-shared volumes (CSVs), NFS, and SAN storage <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives <br><br> VMware vApps aren't supported. |

+| VMware VMs | VMware server 7.0, 6.7, 6.5 or 6.0 (Licensed Version) | Hyper-V virtual machine <br><br> VMware virtual machine | V3 UR2 and later | Protect: VMware VMs on cluster-shared volumes (CSVs), NFS, and SAN storage <br><br> Recover: Virtual machine, Item-level recovery of files and folders available only for Windows, volumes, virtual hard drives <br><br> VMware vApps aren't supported. |

+>[!NOTE]

+> MABS doesn't support backup of virtual machines with pass-through disks or those that use a remote VHD. We recommend that in these scenarios you use guest-level backup using MABS, and install an agent on the virtual machine to back up the data.

+## Linux

+| **Workload** | **Version** | **Azure Backup Server installation** | **Supported Azure Backup Server** | **Protection and recovery** |

+| | -- | | - | |

+| Linux | Linux running as [Hyper-V](back-up-hyper-v-virtual-machines-mabs.md) or [VMware](backup-azure-backup-server-vmware.md) guest | Physical server, On-premises Hyper-V VM, Windows VM in VMware | V3 UR1 and V3 UR2 | Hyper-V must be running on Windows Server 2012 R2, Windows Server 2016, or Windows Server 2019. Protect: Entire virtual machine <br><br> Recover: Entire virtual machine <br><br> Only file-consistent snapshots are supported. <br><br> For a complete list of supported Linux distributions and versions, see the article, [Linux on distributions endorsed by Azure](../virtual-machines/linux/endorsed-distros.md). |

+## Azure ExpressRoute support

+You can back up your data over Azure ExpressRoute with public peering (available for old circuits) and Microsoft peering. Backup over private peering isn't supported.

+With public peering: Ensure access to the following domains/addresses:

+* URLs

+ * `www.msftncsi.com`

+ * `*.Microsoft.com`

+ * `*.WindowsAzure.com`

+ * `*.microsoftonline.com`

+ * `*.windows.net`

+ * `www.msftconnecttest.com`

+* IP addresses

+ * 20.190.128.0/18

+ * 40.126.0.0/18

+With Microsoft peering, select the following services/regions and relevant community values:

+* Azure Active Directory (12076:5060)

+* Microsoft Azure Region (according to the location of your Recovery Services vault)

+* Azure Storage (according to the location of your Recovery Services vault)

+For more information, see the [ExpressRoute routing requirements](../expressroute/expressroute-routing.md).

+>[!NOTE]

+>Public Peering is deprecated for new circuits.

+## Operating systems and applications at end of support

+Support for the following operating systems and applications in MABS are deprecated. We recommended you to upgrade them to continue protecting your data.

+If the existing commitments prevent upgrading Windows Server or SQL Server, migrate them to Azure and [use Azure Backup to protect the servers](./index.yml). For more information, see [migration of Windows Server, apps and workloads](https://azure.microsoft.com/migration/windows-server/).

+For on-premises or hosted environments that you can't upgrade or migrate to Azure, activate Extended Security Updates for the machines for protection and support. Note that only limited editions are eligible for Extended Security Updates. For more information, see [Frequently asked questions](https://www.microsoft.com/windows-server/extended-security-updates).

+|Workload |Version |Azure Backup Server installation |Azure Backup Server |Protection and recovery |

+||--||--|--|

+|Servers (64-bit) | Windows Server 2008 R2 SP1, Windows Server 2008 SP2 (You need to install [Windows Management Framework](https://www.microsoft.com/download/details.aspx?id=54616)) | Physical server <br><br> Hyper-V virtual machine <br><br> VMware virtual machine | Volume, share, folder, file, system state/bare metal |

+## Cluster support

+Azure Backup Server can protect data in the following clustered applications:

+* File servers

+* SQL Server

+* Hyper-V - If you protect a Hyper-V cluster using scaled-out MABS protection agent, you can't add secondary protection for the protected Hyper-V workloads.

+* Exchange Server - Azure Backup Server can protect non-shared disk clusters for supported Exchange Server versions (cluster-continuous replication), and can also protect Exchange Server configured for local continuous replication.

+* SQL Server - Azure Backup Server doesn't support backing up SQL Server databases hosted on cluster-shared volumes (CSVs).

+>[!NOTE]

+>- MABS V3 UR1 supports the protection of Hyper-V virtual machines on Cluster Shared Volumes (CSVs). Protection of other workloads hosted on CSVs isn't supported.

+>- MABS v3 UR2 additionally supports SQL Server Failover Cluster Instance (FCI) using Cluster Shared Volumes (CSVs).

+Azure Backup Server can protect cluster workloads that are located in the same domain as the MABS server, and in a child or trusted domain. If you want to protect data sources in untrusted domains or workgroups, use NTLM or certificate authentication for a single server, or certificate authentication only for a cluster.

+## Data protection issues

+* MABS can't back up VMs using shared drives (which are potentially attached to other VMs) as the Hyper-V VSS writer can't back up volumes that are backed up by shared VHDs.

+* When you protect a shared folder, the path to the shared folder includes the logical path on the volume. If you move the shared folder, protection will fail. If you must move a protected shared folder, remove it from its protection group and then add it to protection after the move. Also, if you change the path of a protected data source on a volume that uses the Encrypting File System (EFS) and the new file path exceeds 5120 characters, data protection will fail.

+* You can't change the domain of a protected computer and continue protection without disruption. Also, you can't change the domain of a protected computer and associate the existing replicas and recovery points with the computer when it's reprotected. If you must change the domain of a protected computer, then first remove the data sources on the computer from protection. Then protect the data source on the computer after it has a new domain.

+* You can't change the name of a protected computer and continue protection without disruption. Also, you can't change the name of a protected computer and associate the existing replicas and recovery points with the computer when it's reprotected. If you must change the name of a protected computer, then first remove the data sources on the computer from protection. Then protect the data source on the computer after it has a new name.

+* MABS automatically identifies the time zone of a protected computer during installation of the protection agent. If a protected computer is moved to a different time zone after protection is configured, ensure that you change the computer time in Control Panel. Then update the time zone in the MABS database.

+* MABS can protect workloads in the same domain as the MABS server, or in child and trusted domains. You can also protect the following workloads in workgroups and untrusted domains using NTLM or certificate authentication:

+ * SQL Server

+ * File Server

+ * Hyper-V

+ These workloads can be running on a single server or in a cluster configuration. To protect a workload that isn't in a trusted domain, see [Prepare computers in workgroups and untrusted domains](/system-center/dpm/back-up-machines-in-workgroups-and-untrusted-domains?view=sc-dpm-2019&preserve-view=true#supported-scenarios) for exact details of what's supported and what authentication is required.

+## Unsupported data types

+MABS doesn't support protecting the following data types:

+* Hard links

+* Reparse points, including DFS links and junction points

+* Mount point metadata - A protection group can contain data with mount points. In this case DPM protects the mounted volume that is the target of the mount point, but it doesn't protect the mount point metadata. When you recover data containing mount points, you'll need to manually recreate your mount point hierarchy.

+* Data in mounted volumes within mounted volumes

+* Recycle Bin

+* Paging files

+* System Volume Information folder. To protect system information for a computer, you'll need to select the computer's system state as the protect group member.

+* Non-NTFS volumes

+* Files containing hard links or symbolic links from Windows Vista.

+* Data on file shares hosting UPDs (User Profile Disks)

+* Files with any of the following combinations of attributes:

+ * Encryption and reparse

+ * Encryption and Single Instance Storage (SIS)

+ * Encryption and case-sensitivity

+ * Encryption and sparse

+ * Case-sensitivity and SIS

+ * Compression and SIS

+## Next steps

+* [Support matrix for backup with Microsoft Azure Backup Server or System Center DPM](backup-support-matrix-mabs-dpm.md)

+|VMware VMs|[VMware vSphere Licensed version 6.7, 7.0](backup-azure-backup-server-vmware.md#vmware-vsphere-67-70-and-80) |Physical server, <br/>On-premises Hyper-V VM, <br/> Windows VM in VMware|V3|VMware VMs on cluster-shared volumes (CSVs), NFS, and SAN storage<br /> Item-level recovery of files and folders is available only for Windows VMs, VMware vApps are not supported.|

+>Private endpoints are supported with only DPM server 2022, MABS v4, and later.

+2. On the **Browse** pane, browse or filter to find the VM you want to recover. Once you select a VM or folder, the Recovery points for pane displays the available recovery points.

+2. On the **Browse** pane, browse or filter to find the VM you want to recover. Once you select a VM or folder, the **Recovery points for pane** displays the available recovery points.

+## VMware parallel restore in MABS v4 (and later)

+MABS v4 supports restoring more than one VMware VMs protected from the same vCenter in parallel. By default, eight parallel recoveries are supported. You can increase the number of parallel restore jobs by adding the following registry key.

+>[!Note]

+>Before you increase the number of parallel recoveries, you need to consider the VMware performance. Considering the number of resources in use and additional usage required on VMware vSphere Server, you need to determine the number of recoveries to run in parallel.

+>

+>**Key Path**: `HKLM\ Software\Microsoft\Microsoft Data Protection Manager\Configuration\ MaxParallelRecoveryJobs`

+>- **32 Bit DWORD**: VMware

+>- **Data**: `<number>`. The value should be the number (decimal) of virtual machines that you select for parallel recovery.

+- April 2023

+ - [Microsoft Azure Backup Server v4 is now generally available](#microsoft-azure-backup-server-v4-is-now-generally-available)

+## Microsoft Azure Backup Server v4 is now generally available

+Azure Backup now provides Microsoft Azure Backup Server (MABS) v4, the latest edition of on-premises backup solution.

+- It can *protect* and *run on* Windows Server 2022, Azure Stack HCI 22H2, vSphere 8.0, and SQL Server 2022.

+- It contains stability improvements and bug fixes on *MABS v3 UR2*.

+For more information see [What's new in MABS](backup-mabs-whats-new-mabs.md).

+| OpenAI resources per region per Azure subscription | 3 |

+ContainerAppConsoleLogs_CL

+| where ContainerAppName_s =~ "my-container-app"

+The application logs for all the apps hosted in your Kubernetes cluster are logged to the Log Analytics workspace in the custom log table named `ContainerAppConsoleLogs_CL`.

+ Title: Scale Dapr applications with KEDA scalers using Bicep

+description: Learn how to use KEDA scalers to scale an Azure Container App and its Dapr sidecar.

+# Scale Dapr applications with KEDA scalers

+[Azure Container Apps automatically scales HTTP traffic to zero.](./scale-app.md) However, to scale non-HTTP traffic (like [Dapr](https://docs.dapr.io/) pub/sub and bindings), you can use [KEDA scalers](https://keda.sh/) to scale your application and its Dapr sidecar up and down, based on the number of pending inbound events and messages.

+This guide demonstrates how to configure the scale rules of a Dapr pub/sub application with a KEDA messaging scaler. For context, refer to the corresponding sample pub/sub applications:

+- [Microservice communication using pub/sub in **C#**](https://github.com/Azure-Samples/pubsub-dapr-csharp-servicebus)

+- [Microservice communication using pub/sub in **JavaScript**](https://github.com/Azure-Samples/pubsub-dapr-nodejs-servicebus)

+- [Microservice communication using pub/sub in **Python**](https://github.com/Azure-Samples/pubsub-dapr-python-servicebus)

+In the above samples, the application uses the following elements:

+1. The `checkout` publisher is an application that is meant to run indefinitely and never scale down to zero, despite never receiving any incoming HTTP traffic.

+1. The Dapr Azure Service Bus pub/sub component.

+1. An `order-processor` subscriber container app picks up messages received via the `orders` topic and processed as they arrive.

+1. The scale rule for Azure Service Bus, which is responsible for scaling up the `order-processor` service and its Dapr sidecar when messages start to arrive to the `orders` topic.

+Let's take a look at how to apply the scaling rules in a Dapr application.

+## Publisher container app

+The `checkout` publisher is a headless service that runs indefinitely and never scales down to zero.

+By default, [the Container Apps runtime assigns an HTTP-based scale rule to applications](./scale-app.md), which drives scaling based on the number of incoming HTTP requests. In the following example, `minReplicas` is set to `1`. This configuration ensures the container app doesn't follow the default behavior of scaling to zero with no incoming HTTP traffic.

+```bicep

+resource checkout 'Microsoft.App/containerApps@2022-03-01' = {

+ name: 'ca-checkout-${resourceToken}'

+ location: location

+ identity: {

+ type: 'SystemAssigned'

+ }

+ properties: {

+ //...

+ template: {

+ //...

+ // Scale the minReplicas to 1

+ scale: {

+ minReplicas: 1

+ maxReplicas: 1

+ }

+ }

+ }

+}

+```

+## Subscriber container app

+The following `order-processor` subscriber app includes a custom scale rule that monitors a resource of type `azure-servicebus`. With this rule, the app (and its sidecar) scales up and down as needed based on the number of pending messages in the Bus.

+```bicep

+resource orders 'Microsoft.App/containerApps@2022-03-01' = {

+ name: 'ca-orders-${resourceToken}'

+ location: location

+ tags: union(tags, {

+ 'azd-service-name': 'orders'

+ })

+ identity: {

+ type: 'SystemAssigned'

+ }

+ properties: {

+ managedEnvironmentId: containerAppsEnvironment.id

+ configuration: {

+ //...

+ // Enable Dapr on the container app

+ dapr: {

+ enabled: true

+ appId: 'orders'

+ appProtocol: 'http'

+ appPort: 5001

+ }

+ //...

+ }

+ template: {

+ //...

+ // Set the scale property on the order-processor resource

+ scale: {

+ minReplicas: 0

+ maxReplicas: 10

+ rules: [

+ {

+ name: 'topic-based-scaling'

+ custom: {

+ type: 'azure-servicebus'

+ metadata: {

+ topicName: 'orders'

+ subscriptionName: 'membership-orders'

+ messageCount: '30'

+ }

+ auth: [

+ {

+ secretRef: 'sb-root-connectionstring'

+ triggerParameter: 'connection'

+ }

+ ]

+ }

+ }

+ ]

+ }

+ }

+ }

+}

+```

+## How the scaler works

+Notice the `messageCount` property on the scaler's configuration in the subscriber app:

+```bicep

+{

+ //...

+ properties: {

+ //...

+ template: {

+ //...

+ scale: {

+ //...

+ rules: [

+ //...

+ custom: {

+ //...

+ metadata: {

+ //...

+ messageCount: '30'

+ }

+ }

+ ]

+ }

+ }

+ }

+}

+```

+This property tells the scaler how many messages each instance of the application can process at the same time. In this example, the value is set to `30`, indicating that there should be one instance of the application created for each group of 30 messages waiting in the topic.

+For example, if 150 messages are waiting, KEDA scales the app out to five instances. The `maxReplicas` property is set to `10`, meaning even with a large number of messages in the topic, the scaler never creates more than `10` instances of this application. This setting ensures you don't scale up too much and accrue too much cost.

+## Next steps

+[Learn more about using Dapr components with Azure Container Apps.](./dapr-overview.md)

+- [Scale your Dapr applications using KEDA scalers][dapr-keda]

+[dapr-keda]: ./dapr-keda-scaling.md

+- [Scale your Dapr applications using KEDA scalers](./dapr-keda-scaling.md)

+- Learn more about [Azure Developer CLI](/azure/developer/azure-developer-cli/overview) and [making your applications compatible with `azd`](/azure/developer/azure-developer-cli/make-azd-compatible).

+- [Scale your Dapr applications using KEDA scalers](./dapr-keda-scaling.md)

+# Attestation in Confidential containers on Azure Container Instances

+## Full attestation

+Expanding upon this concept of attestation. Full attestation captures all the components that are part of the Trusted Execution Environment that is remotely verifiable. To achieve full attestation, in Confidential Containers, we have introduced the notion of a cce policy, which defines a set of rules, which is enforced in the utility VM. The security policy is encoded in the attestation report as an SHA-256 digest stored in the HostData attribute, as provided to the AMD SEV-SNP hardware by the host operating system during the VM boot-up. This means that the security policy enforced by the utility VM is immutable throughout the lifetime of the utility VM.

+The exhaustive list of attributes that are part of the SEV-SNP attestation can be found [here](https://www.amd.com/system/files/TechDocs/56860.pdf).

+| Claim | Sample value | Description |

+|::|:-:|:--:|

+| x-ms-attestation-type | sevsnpvm | String value that describes the attestation type. For example, in this scenario sevsnp hardware |

+| x-ms-compliance-status | azure-compliant-uvm | Compliance status of the utility VM that runs the container group. |

+| x-ms-sevsnpvm-hostdata | 670fff86714a650a49b58fadc1e90fedae0eb32dd51e34931c1e7a1839c08f6f | Hash of the cce policy that was generated using tooling during deployment. |

+| x-ms-sevsnpvm-is-debuggable | false | Flag to indicate whether the underlying hardware is running in debug mode |

+* [Required outbound network rules and FQDNs for AKS clusters](../aks/outbound-rules-control-egress.md#required-outbound-network-rules-and-fqdns-for-aks-clusters)

+# What is Azure Cosmos DB for MongoDB vCore? (Preview)

+ Title: EA billing administration for partners in the Azure portal

+description: This article explains the common tasks that a partner administrator accomplishes in the Azure portal to manage indirect enterprise agreements.

+# EA billing administration for partners in the Azure portal

+This article explains the common tasks that a partner administrator accomplishes in the Azure portal https://portal.azure.com to manage indirect EAs. An indirect EA is one where a customer signs an agreement with a Microsoft partner. The partner administrator manages their indirect EAs on behalf of their customers.

+## Access the Azure portal

+The partner organization is referred to as the **billing account** in the Azure portal. Partner administrators can sign in to the Azure portal to view and manage their partner organization. The partner organization contains their customer's enrollments. However, the partner doesn't have an enrollment of their own. A customer's enrollment is shown in the Azure portal as a **billing profile**.

+A partner administrator user can have access to multiple partner organizations (billing account scopes). All the information and activity in the Azure portal are in the context of a billing account _scope_. It's important that the partner administrator first selects a billing scope and then does administrative tasks in the context of the selected scope.

+### Select a billing scope

+1. Sign in to the [Azure portal](https://portal.azure.com/#blade/Microsoft_Azure_GTM/ModernBillingMenuBlade/AllBillingScopes).

+1. Search for **Cost Management + Billing** and select it.

+ :::image type="content" source="./media/ea-billing-administration-partners/search-cost-management-billing.png" alt-text="Screenshot showing search for Cost Management + Billing." lightbox="./media/ea-billing-administration-partners/search-cost-management-billing.png" :::

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+ :::image type="content" source="./media/ea-billing-administration-partners/billing-scopes.png" alt-text="Screenshot showing select a billing scope." lightbox="./media/ea-billing-administration-partners/billing-scopes.png" :::

+## Manage a partner organization

+Partner administrator users can view and manage the partner organization. After a partner administrator selects a partner organization billing scope from Cost management + Billing, they see the Partner management overview page where they can view the following information:

+- Partner organization details such as name, ID, and authentication setting

+- List of active and extended enrollments and the option to download details

+- List of enrollments expiring in the next 180 days, so that partner admin can act to renew

+- List of enrollments with other status

+The partner administrator uses the left navigation menu items to perform the following tasks:

+- **Access control (IAM)** ΓÇô To add, edit, and delete partner administrator users.

+- **Billing profiles** ΓÇô To view a list of enrollments.

+- **Billing scopes** ΓÇô To view a list of all billing scopes that they have access to.

+- **New support request** ΓÇô To create new support request.

+## Manage partner administrators

+Every partner administrator in the Azure portal can add or remove other partner administrators. Partner administrators are associated with partner organizations billing account. They aren't associated _directly_ with the enrollments.

+Partners can view all the details of the billing account and billing profiles for indirect enrollments. The partner administrator can perform the following write operations.

+- Update the billing account authentication type

+- Add, edit, and delete another partner administrator user

+- Set the markup of the billing profile for indirect enrollments

+- Update the PO number of the billing profile for indirect enrollments

+- Generate API the key of billing profile for indirect enrollments

+A partner administrator with read-only access can view all billing account and billing profiles details. However, they can't perform any write operations.

+### Add a partner administrator

+You can add a new partner administrator with the following steps:

+1. In the Azure portal, sign in as a partner administrator.

+1. Search for **Cost Management + Billing** and select it.

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+1. In the left navigation menu, select **Access control (IAM)**.

+ :::image type="content" source="./media/ea-billing-administration-partners/access-control.png" alt-text="Screenshot showing select Access Control (IAM)." lightbox="./media/ea-billing-administration-partners/access-control.png" :::

+1. At the top of the page, select **Add Partner Admin**.

+ :::image type="content" source="./media/ea-billing-administration-partners/add-partner-admin.png" alt-text="Screenshot showing select Add Partner Admin." lightbox="./media/ea-billing-administration-partners/add-partner-admin.png" :::

+1. In the Add Role Assignment window, enter the email address of the user to whom you want to give access.

+1. Select the authentication type.

+1. Select **Provide read-only access** if you want to provide read-only (reader) access.

+1. Enter a notification contact if you want to inform someone about the role assignment.

+1. Select the notification frequency.

+1. Select **Add**.

+ :::image type="content" source="./media/ea-billing-administration-partners/add-role-assignment.png" alt-text="Screenshot showing Add role assignment window." lightbox="./media/ea-billing-administration-partners/add-role-assignment.png" :::

+### Edit a partner administrator

+You can edit a partner administrator user role using the following steps.

+1. In the Azure portal, sign in as a partner administrator.

+1. Search for **Cost Management + Billing** and select it.

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+1. In the left navigation menu, select **Access control (IAM)**.

+1. In the list of administrators, in the row for the user that you want to edit, select the ellipsis (**…**) symbol, and then select **Edit**.

+ :::image type="content" source="./media/ea-billing-administration-partners/edit-role-assignment.png" alt-text="Screenshot showing Edit partner admin." lightbox="./media/ea-billing-administration-partners/edit-role-assignment.png" :::

+1. In the Edit role assignment window, select **Provide read-only access**.

+1. Select the **Notification frequency** option and choose the frequency.

+1. **Apply** the changes.

+### Remove a partner administrator

+To revoke access of a partner administrator/reader, you delete the user from the billing account. After access is revoked, the user can't view or manage the billing account.

+1. In the Azure portal, sign in as a partner administrator.

+1. Search for **Cost Management + Billing** and select it.

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+1. In the left navigation menu, select **Access control (IAM)**.

+1. In the list of administrators, in the row for the user that you want to delete, select the ellipsis (**…**) symbol, and then select **Delete**.

+1. In the Delete role assignment window, select **Yes, I want to delete this partner administrator** to confirm that you want to delete the partner administrator.

+1. At the bottom of the window, select **Delete**.

+## Manage partner notifications

+Partner administrators can manage the frequency that they receive usage notifications for their enrollments. They automatically receive weekly notifications of their unbilled balance. They can change the notification frequency to daily, weekly, monthly, or disable them completely.

+If a user doesn't receive a notification, verify that the user's notification settings are correct with the following steps.

+1. In the Azure portal, sign in as a partner administrator.

+1. Search for **Cost Management + Billing** and select it.

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+1. In the left navigation menu, select **Access control (IAM)**.

+1. In the list of administrators, in the row for the user that you want to edit, select the ellipsis (**…**) symbol, and then select **Edit**.

+1. In the Edit role assignment window, in the **Notification frequency** list, select a frequency.

+1. **Apply** the changes.

+## View and manage enrollments

+Partner administrators can view a list of their customer enrollments (billing profiles) in the Azure portal. Each customer's EA enrollment is represented as a billing profile to the partner.

+### View the enrollment list

+1. In the Azure portal, sign in as a partner administrator.

+1. Search for **Cost Management + Billing** and select it.

+1. In the left navigation menu, select **Billing scopes** and then select the billing account that you want to work with.

+1. In the left navigation menu, select **Billing profiles**.

+ :::image type="content" source="./media/ea-billing-administration-partners/billing-profiles.png" alt-text="Screenshot showing the Billing profiles enrollment list." lightbox="./media/ea-billing-administration-partners/billing-profiles.png" :::

+By default, all active enrollments are shown. You can change the status filter to view the entire list of enrollments associated with the partner organization. Then you can select an enrollment to manage.

+## Next steps

+- To view usage and charges for a specific enrollment, see the [View your usage summary details and download reports for EA enrollments](direct-ea-azure-usage-charges-invoices.md) article.

+The start date of a new Azure Prepayment (previously called monetary commitment) is defined by the date that the regional operations center processed it. Since Azure Prepayment orders via the Azure portal are processed in the UTC time zone, you may experience some delay if your Azure Prepayment purchase order was processed in a different region. The coverage start date on the purchase order shows the start of the Azure Prepayment. The coverage start date is when the Azure Prepayment appears in the Azure portal.

+**Pending** - The enrollment administrator needs to sign in to the Azure portal. After the administrator signs in, the enrollment switches to **Active** status.

+**Active** - The enrollment is accessible and usable. You can create departments, accounts, and subscriptions in the [Azure portal](https://portal.azure.com). The enrollment remains active until the enterprise agreement end date.

+In the Azure portal, Partner Price Markup helps to enable better cost reporting for customers. The Azure portal shows usage and prices configured by partners for their customers.

+Markup allows partner administrators to add a percentage markup to their indirect enterprise agreements. Percentage markup applies to all Microsoft first party service information in the Azure portal such as: meter rates, Azure Prepayment, and orders. After the markup is published by the partner, the customer sees Azure costs in the Azure portal. For example, usage summary, price lists, and downloaded usage reports.

+The LSP provides a single percentage number in the Azure portal.  All commercial information on the portal will be uplifted by the percentage provided by the LSP. Example:

+**You can add price markup on Azure portal with the following steps:**

+

+**In the Azure portal,**

+- Sign in as a partner administrator.

+- Search for Cost Management + Billing and select it.

+- In the left navigation menu, select Billing scopes and then select the billing account that you want to work with.

+- In the left navigation menu, select Billing Profile and then select the billing profile that you want to work with.

+- In the left navigation menu, select Markup.

+- To add markup, click on ΓÇ£set markupΓÇ¥.

+- Enter the markup percentage and select Preview.

+- Review the credit and usage charges before and after markup update.

+- Accept the disclaimer and click on Publish to published the markup.

+- End customer should be able to view credits and charges details.

+**You can add price markup on Azure Enterprise portal with the following steps:**

+**To check markup status of an enrollment on Azure portal, follow the below steps:**

+- In the Azure portal, sign in as a partner administrator.

+- Search for Cost Management + Billing and select it.

+- Select Billing scopes and then select the billing account that you want to work with.

+- In the left navigation menu, select Billing scopes and then select the billing account that you want to work with.

+- In the left navigation menu, select Billing Profile

+- You can view the markup status of an enrollment

+Partners can use the markup feature (on Azure EA portal or Azure portal) after a Change of Channel Partner is processed; no need to wait for the next anniversary term.

+EA customer can view price sheet in Azure portal. See [view price sheet in Azure portal](ea-pricing.md#download-pricing-for-an-enterprise-agreement).

+|Purchase reservations|✔|✘⁶|✔|✘|✘|✘|✘|

+- ⁶ The Enterprise Administrator (read only) role doesn't allow reservation purchases. However, if the EA Admin (read only) is also a subscription owner or subscription reservation purchaser, they can purchase a reservation.

+|View usage and cost details|✔|✔|✔|✔⁷|✔⁷|✔⁸|✔|

+- ⁷ Requires that the Enterprise Administrator enable **DA view charges** policy in the Enterprise portal. The Department Administrator can then see cost details for the department.

+- ⁸ Requires that the Enterprise Administrator enable **AO view charges** policy in the Enterprise portal. The Account Owner can then see cost details for the account.

+**Amortized cost** - Shows a reservation purchase split as an amortized cost over the duration of the reservation term. With the same example above, cost analysis shows a different amount for each month depending on the number of days in the month. If you group costs by VM in this example, you'd see cost attributed to each VM that received the reservation benefit. However, _unused reservation_ costs are not attributed to the subscription used to buy the reservation because the unused portion isn't attributable to any specific resource or subscription.

+- Read [Charge back Azure Reservation costs](charge-back-usage.md) to learn more about charge back processes.

+| Microsoft Sentinel data connector and workbook | Yes | Yes |

+| microsoft-defender-publisher-ds-* | kube-system | [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) | Publish the collected data to Microsoft Defender for Containers backend service where the data will be processed for and analyzed. | N/A | memory: 200Mi  <br> <br> cpu: 60m | Https 443 <br> <br> Learn more about the [outbound access prerequisites](../aks/outbound-rules-control-egress.md#microsoft-defender-for-containers) |

+Yes, you can use the REST API to return to the Defender for Storage (classic) plan.

+If you want to switch back to the Defender for Storage (classic) plan, you need to do two things. First, disable the new Defender for Storage plan that is enabled now. Second, check if there are any policies that can re-enable the new plan and turn them off too. **The two Azure built-in policies enabling the new plan are Configure Microsoft Defender for Storage to be enabled** and **Configure basic Microsoft Defender for Storage to be enabled (Activity Monitoring only).**

+**To enable agentless vulnerability assessment on Azure**:

+> [Security Policy Enhancements in Defender for Cloud](episode-twenty-nine.md)

+ Title: Security policy enhancements in Defender for Cloud | Defender for Cloud in the field

+description: Learn about security policy enhancements and dashboard in Defender for Cloud

+# Security policy enhancements in Defender for Cloud

+**Episode description**: In this episode of Defender for Cloud in the field, Tuval Rozner joins Yuri Diogenes to talk about the new security policy enhancements. Tuval covers the new security policy dashboard within Defender for Cloud, how to filter, and create exemptions from a single place without having to make changes in the Azure Policy dashboard. Tuval also demonstrates how to use the new dashboard and customize policies.

+<br>

+<br>

+<iframe src="https://aka.ms/docs/player?id=1145810e-fc14-4d73-8d63-ea861aefb30b" width="1080" height="530" allowFullScreen="true" frameBorder="0"></iframe>

+- [01:21](/shows/mdc-in-the-field/security-policy#time=01m21s) - The rationale behind changing the security policy assignment experience

+- [02:20](/shows/mdc-in-the-field/security-policy#time=02m20s) - What's new in the security policy assignment in Defender for Cloud?

+- [04:20](/shows/mdc-in-the-field/security-policy#time=04m20s) - Demonstration

+- [12:02](/shows/mdc-in-the-field/security-policy#time=12m02s) - What's next?

+## Recommended resources

+ - Learn more about [managing security policies](tutorial-security-policy.md)

+ - Subscribe to [Microsoft Security on YouTube](https://www.youtube.com/playlist?list=PL3ZTgFEc7LysiX4PfHhdJPR7S8mGO14YS)

+ - Join our [Tech Community](https://aka.ms/SecurityTechCommunity)

+ - For more about [Microsoft Security](https://msft.it/6002T9HQY)

+- Follow us on social media:

+ - [LinkedIn](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbFk5TXZuQld2NlpBRV9BQlJqMktYSm95WWhCZ3xBQ3Jtc0tsQU13MkNPWGNFZzVuem5zc05wcnp0VGxybHprVTkwS2todWw0b0VCWUl4a2ZKYVktNGM1TVFHTXpmajVLcjRKX0cwVFNJaDlzTld4MnhyenBuUGRCVmdoYzRZTjFmYXRTVlhpZGc4MHhoa3N6ZDhFMA&q=https%3A%2F%2Fwww.linkedin.com%2Fshowcase%2Fmicrosoft-security%2F)

+ - [Twitter](https://twitter.com/msftsecurity)

+- Join our [Tech Community](https://aka.ms/SecurityTechCommunity)

+- Learn more about [Microsoft Security](https://msft.it/6002T9HQY)

+## Next steps

+> [!div class="nextstepaction"]

+> [New AWS Connector in Microsoft Defender for Cloud](episode-one.md)

+- Add the [Required FQDN/application rules for Azure policy](../aks/outbound-rules-control-egress.md#azure-policy).

+ :::image type="content" source="media/quickstart-onboard-aws/add-aws-account-plans-selection.png" alt-text="The select plans tab is where you choose which Defender for Cloud capabilities to enable for this AWS account." lightbox="media/quickstart-onboard-aws/add-aws-account-plans-selection.png":::

+- [Two recommendations related to missing Operating System (OS) updates were released to GA](#two-recommendations-related-to-missing-operating-system-os-updates-were-released-to-ga)

+### Two recommendations related to missing Operating System (OS) updates were released to GA

+The recommendations `System updates should be installed on your machines (powered by Update management center)` and `Machines should be configured to periodically check for missing system updates` have been released for General Availability.

+To use the new recommendation, you need to:

+- Connect your non-Azure machines to Arc.

+- [Enable the periodic assessment property](../update-center/assessment-options.md#periodic-assessment). You can use the [Fix button](implement-security-recommendations.md).

+ in the new recommendation, `Machines should be configured to periodically check for missing system updates` to fix the recommendation.

+After completing these steps, you can remove the old recommendation `System updates should be installed on your machines`, by disabling it from Defender for Cloud's built-in initiative in Azure policy.

+The two versions of the recommendations:

+- [`System updates should be installed on your machines`](https://ms.portal.azure.com/#view/Microsoft_Azure_Security/SystemUpdatesRecommendationDetailsWithRulesBlade/assessmentKey/4ab6e3c5-74dd-8b35-9ab9-f61b30875b27/subscriptionIds~/%5B%220cd6095b-b140-41ec-ad1d-32f2f7493386%22%2C%220ee78edb-a0ad-456c-a0a2-901bf542c102%22%2C%2284ca48fe-c942-42e5-b492-d56681d058fa%22%2C%22b2a328a7-ffff-4c09-b643-a4758cf170bc%22%2C%22eef8b6d5-94da-4b36-9327-a662f2674efb%22%2C%228d5565a3-dec1-4ee2-86d6-8aabb315eec4%22%2C%22e0fd569c-e34a-4249-8c24-e8d723c7f054%22%2C%22dad45786-32e5-4ef3-b90e-8e0838fbadb6%22%2C%22a5f9f0d3-a937-4de5-8cf3-387fce51e80c%22%2C%220368444d-756e-4ca6-9ecd-e964248c227a%22%2C%22e686ef8c-d35d-4e9b-92f8-caaaa7948c0a%22%2C%222145a411-d149-4010-84d4-40fe8a55db44%22%2C%22212f9889-769e-45ae-ab43-6da33674bd26%22%2C%22615f5f56-4ba9-45cf-b644-0c09d7d325c8%22%2C%22487bb485-b5b0-471e-9c0d-10717612f869%22%2C%22cb9eb375-570a-4e75-b83a-77dd942bee9f%22%2C%224bbecc02-f2c3-402a-8e01-1dfb1ffef499%22%2C%22432a7068-99ae-4975-ad38-d96b71172cdf%22%2C%22c0620f27-ac38-468c-a26b-264009fe7c41%22%2C%22a1920ebd-59b7-4f19-af9f-5e80599e88e4%22%2C%22b43a6159-1bea-4fa2-9407-e875fdc0ff55%22%2C%22d07c0080-170c-4c24-861d-9c817742986a%22%2C%22ae71ef11-a03f-4b4f-a0e6-ef144727c711%22%2C%2255a24be0-d9c3-4ecd-86b6-566c7aac2512%22%2C%227afc2d66-d5b4-4e84-970b-a782e3e4cc46%22%2C%2252a442a2-31e9-42f9-8e3e-4b27dbf82673%22%2C%228c4b5b03-3b24-4ed0-91f5-a703cd91b412%22%2C%22e01de573-132a-42ac-9ee2-f9dea9dd2717%22%2C%22b5c0b80f-5932-4d47-ae25-cd617dac90ce%22%2C%22e4e06275-58d1-4081-8f1b-be12462eb701%22%2C%229b4236fe-df75-4289-bf00-40628ed41fd9%22%2C%2221d8f407-c4c4-452e-87a4-e609bfb86248%22%2C%227d411d23-59e5-4e2e-8566-4f59de4544f2%22%2C%22b74d5345-100f-408a-a7ca-47abb52ba60d%22%2C%22f30787b9-82a8-4e74-bb0f-f12d64ecc496%22%2C%22482e1993-01d4-4b16-bff4-1866929176a1%22%2C%2226596251-f2f3-4e31-8a1b-f0754e32ad73%22%2C%224628298e-882d-4f12-abf4-a9f9654960bb%22%2C%224115b323-4aac-47f4-bb13-22af265ed58b%22%2C%22911e3904-5112-4232-a7ee-0d1811363c28%22%2C%22cd0fa82d-b6b6-4361-b002-050c32f71353%22%2C%22dd4c2dac-db51-4cd0-b734-684c6cc360c1%22%2C%22d2c9544f-4329-4642-b73d-020e7fef844f%22%2C%22bac420ed-c6fc-4a05-8ac1-8c0c52da1d6e%22%2C%2250ff7bc0-cd15-49d5-abb2-e975184c2f65%22%2C%223cd95ff9-ac62-4b5c-8240-0cd046687ea0%22%2C%2213723929-6644-4060-a50a-cc38ebc5e8b1%22%2C%2209fa8e83-d677-474f-8f73-2a954a0b0ea4%22%2C%22ca38bc19-cf50-48e2-bbe6-8c35b40212d8%22%2C%22bf163a87-8506-4eb3-8d14-c2dc95908830%22%2C%221278a874-89fc-418c-b6b9-ac763b000415%22%2C%223b2fda06-3ef6-454a-9dd5-994a548243e9%22%2C%226560575d-fa06-4e7d-95fb-f962e74efd7a%22%2C%22c3547baf-332f-4d8f-96bd-0659b39c7a59%22%2C%222f96ae42-240b-4228-bafa-26d8b7b03bf3%22%2C%2229de2cfc-f00a-43bb-bdc8-3108795bd282%22%2C%22a1ffc958-d2c7-493e-9f1e-125a0477f536%22%2C%2254b875cc-a81a-4914-8bfd-1a36bc7ddf4d%22%2C%22407ff5d7-0113-4c5c-8534-f5cfb09298f5%22%2C%22365a62ee-6166-4d37-a936-03585106dd50%22%2C%226d17b59e-06c4-4203-89d2-de793ebf5452%22%2C%229372b318-ed3a-4504-95a6-941201300f78%22%2C%223c1bb38c-82e3-4f8d-a115-a7110ba70d05%22%2C%22c6dcd830-359f-44d0-b4d4-c1ba95e86f48%22%2C%2209e8ad18-7bdb-43b8-80c4-43ee53460e0b%22%2C%22dcbdac96-1896-478d-89fc-c95ed43f4596%22%2C%22d23422cf-c0f2-4edc-a306-6e32b181a341%22%2C%228c2c7b23-848d-40fe-b817-690d79ad9dfd%22%2C%221163fbbe-27e7-4b0f-8466-195fe5417043%22%2C%223905431d-c062-4c17-8fd9-c51f89f334c4%22%2C%227ea26ded-0260-4e78-9336-285d4d9e33d2%22%2C%225ccdbd03-f1b1-4b59-a609-300685e17ce3%22%2C%22bcdc6eb0-74cd-40b6-b3a9-584b33cea7b6%22%2C%22d557e825-27b1-4819-8af5-dc2429af91c9%22%2C%222bb50811-92b6-43a1-9d80-745962d9c759%22%2C%22409111bf-3097-421c-ad68-a44e716edf58%22%2C%2249e3f635-484a-43d1-b953-b29e1871ba88%22%2C%22b77ec8a9-04ed-48d2-a87a-e5887b978ba6%22%2C%22075423e9-7d33-4166-8bdf-3920b04e3735%22%2C%22ef143bbb-6a7e-4a3f-b64f-2f23330e0116%22%2C%2224afc59a-f969-4f83-95c9-3b70f52d833d%22%2C%22a8783cc5-1171-4c34-924f-6f71a20b21ec%22%2C%220079a9bb-e218-496a-9880-d27ad6192f52%22%2C%226f53185c-ea09-4fc3-9075-318dec805303%22%2C%22588845a8-a4a7-4ab1-83a1-1388452e8c0c%22%2C%22b68b2f37-1d37-4c2f-80f6-c23de402792e%22%2C%22eec2de82-6ab2-4a84-ae5f-57e9a10bf661%22%2C%22227531a4-d775-435b-a878-963ed8d0d18f%22%2C%228cff5d56-95fb-4a74-ab9d-079edb45313e%22%2C%22e72e5254-f265-4e95-9bd2-9ee8e7329051%22%2C%228ae1955e-f748-4273-a507-10159ba940f9%22%2C%22f6869ac6-2a40-404f-acd3-d07461be771a%22%2C%2285b3dbca-5974-4067-9669-67a141095a76%22%2C%228168a4f2-74d6-4663-9951-8e3a454937b7%22%2C%229ec1d932-0f3f-486c-acc6-e7d78b358f9b%22%2C%2279f57c16-00fe-48da-87d4-5192e86cd047%22%2C%22bac044cf-49e1-4843-8dda-1ce9662606c8%22%2C%22009d0e9f-a42a-470e-b315-82496a88cf0f%22%2C%2268f3658f-0090-4277-a500-f02227aaee97%22%5D/showSecurityCenterCommandBar~/false/assessmentOwners~/null)

+- [`System updates should be installed on your machines (powered by Update management center)`](https://ms.portal.azure.com/#view/Microsoft_Azure_Security/SystemUpdatesV2RecommendationDetailsBlade/assessmentKey/e1145ab1-eb4f-43d8-911b-36ddf771d13f/subscriptionIds~/%5B%220cd6095b-b140-41ec-ad1d-32f2f7493386%22%2C%220ee78edb-a0ad-456c-a0a2-901bf542c102%22%2C%2284ca48fe-c942-42e5-b492-d56681d058fa%22%2C%22b2a328a7-ffff-4c09-b643-a4758cf170bc%22%2C%22eef8b6d5-94da-4b36-9327-a662f2674efb%22%2C%228d5565a3-dec1-4ee2-86d6-8aabb315eec4%22%2C%22e0fd569c-e34a-4249-8c24-e8d723c7f054%22%2C%22dad45786-32e5-4ef3-b90e-8e0838fbadb6%22%2C%22a5f9f0d3-a937-4de5-8cf3-387fce51e80c%22%2C%220368444d-756e-4ca6-9ecd-e964248c227a%22%2C%22e686ef8c-d35d-4e9b-92f8-caaaa7948c0a%22%2C%222145a411-d149-4010-84d4-40fe8a55db44%22%2C%22212f9889-769e-45ae-ab43-6da33674bd26%22%2C%22615f5f56-4ba9-45cf-b644-0c09d7d325c8%22%2C%22487bb485-b5b0-471e-9c0d-10717612f869%22%2C%22cb9eb375-570a-4e75-b83a-77dd942bee9f%22%2C%224bbecc02-f2c3-402a-8e01-1dfb1ffef499%22%2C%22432a7068-99ae-4975-ad38-d96b71172cdf%22%2C%22c0620f27-ac38-468c-a26b-264009fe7c41%22%2C%22a1920ebd-59b7-4f19-af9f-5e80599e88e4%22%2C%22b43a6159-1bea-4fa2-9407-e875fdc0ff55%22%2C%22d07c0080-170c-4c24-861d-9c817742986a%22%2C%22ae71ef11-a03f-4b4f-a0e6-ef144727c711%22%2C%2255a24be0-d9c3-4ecd-86b6-566c7aac2512%22%2C%227afc2d66-d5b4-4e84-970b-a782e3e4cc46%22%2C%2252a442a2-31e9-42f9-8e3e-4b27dbf82673%22%2C%228c4b5b03-3b24-4ed0-91f5-a703cd91b412%22%2C%22e01de573-132a-42ac-9ee2-f9dea9dd2717%22%2C%22b5c0b80f-5932-4d47-ae25-cd617dac90ce%22%2C%22e4e06275-58d1-4081-8f1b-be12462eb701%22%2C%229b4236fe-df75-4289-bf00-40628ed41fd9%22%2C%2221d8f407-c4c4-452e-87a4-e609bfb86248%22%2C%227d411d23-59e5-4e2e-8566-4f59de4544f2%22%2C%22b74d5345-100f-408a-a7ca-47abb52ba60d%22%2C%22f30787b9-82a8-4e74-bb0f-f12d64ecc496%22%2C%22482e1993-01d4-4b16-bff4-1866929176a1%22%2C%2226596251-f2f3-4e31-8a1b-f0754e32ad73%22%2C%224628298e-882d-4f12-abf4-a9f9654960bb%22%2C%224115b323-4aac-47f4-bb13-22af265ed58b%22%2C%22911e3904-5112-4232-a7ee-0d1811363c28%22%2C%22cd0fa82d-b6b6-4361-b002-050c32f71353%22%2C%22dd4c2dac-db51-4cd0-b734-684c6cc360c1%22%2C%22d2c9544f-4329-4642-b73d-020e7fef844f%22%2C%22bac420ed-c6fc-4a05-8ac1-8c0c52da1d6e%22%2C%2250ff7bc0-cd15-49d5-abb2-e975184c2f65%22%2C%223cd95ff9-ac62-4b5c-8240-0cd046687ea0%22%2C%2213723929-6644-4060-a50a-cc38ebc5e8b1%22%2C%2209fa8e83-d677-474f-8f73-2a954a0b0ea4%22%2C%22ca38bc19-cf50-48e2-bbe6-8c35b40212d8%22%2C%22bf163a87-8506-4eb3-8d14-c2dc95908830%22%2C%221278a874-89fc-418c-b6b9-ac763b000415%22%2C%223b2fda06-3ef6-454a-9dd5-994a548243e9%22%2C%226560575d-fa06-4e7d-95fb-f962e74efd7a%22%2C%22c3547baf-332f-4d8f-96bd-0659b39c7a59%22%2C%222f96ae42-240b-4228-bafa-26d8b7b03bf3%22%2C%2229de2cfc-f00a-43bb-bdc8-3108795bd282%22%2C%22a1ffc958-d2c7-493e-9f1e-125a0477f536%22%2C%2254b875cc-a81a-4914-8bfd-1a36bc7ddf4d%22%2C%22407ff5d7-0113-4c5c-8534-f5cfb09298f5%22%2C%22365a62ee-6166-4d37-a936-03585106dd50%22%2C%226d17b59e-06c4-4203-89d2-de793ebf5452%22%2C%229372b318-ed3a-4504-95a6-941201300f78%22%2C%223c1bb38c-82e3-4f8d-a115-a7110ba70d05%22%2C%22c6dcd830-359f-44d0-b4d4-c1ba95e86f48%22%2C%2209e8ad18-7bdb-43b8-80c4-43ee53460e0b%22%2C%22dcbdac96-1896-478d-89fc-c95ed43f4596%22%2C%22d23422cf-c0f2-4edc-a306-6e32b181a341%22%2C%228c2c7b23-848d-40fe-b817-690d79ad9dfd%22%2C%221163fbbe-27e7-4b0f-8466-195fe5417043%22%2C%223905431d-c062-4c17-8fd9-c51f89f334c4%22%2C%227ea26ded-0260-4e78-9336-285d4d9e33d2%22%2C%225ccdbd03-f1b1-4b59-a609-300685e17ce3%22%2C%22bcdc6eb0-74cd-40b6-b3a9-584b33cea7b6%22%2C%22d557e825-27b1-4819-8af5-dc2429af91c9%22%2C%222bb50811-92b6-43a1-9d80-745962d9c759%22%2C%22409111bf-3097-421c-ad68-a44e716edf58%22%2C%2249e3f635-484a-43d1-b953-b29e1871ba88%22%2C%22b77ec8a9-04ed-48d2-a87a-e5887b978ba6%22%2C%22075423e9-7d33-4166-8bdf-3920b04e3735%22%2C%22ef143bbb-6a7e-4a3f-b64f-2f23330e0116%22%2C%2224afc59a-f969-4f83-95c9-3b70f52d833d%22%2C%22a8783cc5-1171-4c34-924f-6f71a20b21ec%22%2C%220079a9bb-e218-496a-9880-d27ad6192f52%22%2C%226f53185c-ea09-4fc3-9075-318dec805303%22%2C%22588845a8-a4a7-4ab1-83a1-1388452e8c0c%22%2C%22b68b2f37-1d37-4c2f-80f6-c23de402792e%22%2C%22eec2de82-6ab2-4a84-ae5f-57e9a10bf661%22%2C%22227531a4-d775-435b-a878-963ed8d0d18f%22%2C%228cff5d56-95fb-4a74-ab9d-079edb45313e%22%2C%22e72e5254-f265-4e95-9bd2-9ee8e7329051%22%2C%228ae1955e-f748-4273-a507-10159ba940f9%22%2C%22f6869ac6-2a40-404f-acd3-d07461be771a%22%2C%2285b3dbca-5974-4067-9669-67a141095a76%22%2C%228168a4f2-74d6-4663-9951-8e3a454937b7%22%2C%229ec1d932-0f3f-486c-acc6-e7d78b358f9b%22%2C%2279f57c16-00fe-48da-87d4-5192e86cd047%22%2C%22bac044cf-49e1-4843-8dda-1ce9662606c8%22%2C%22009d0e9f-a42a-470e-b315-82496a88cf0f%22%2C%2268f3658f-0090-4277-a500-f02227aaee97%22%5D/showSecurityCenterCommandBar~/false/assessmentOwners~/null)

+

+will both be available until the [Log Analytics agent is deprecated on August 31, 2024](https://azure.microsoft.com/updates/were-retiring-the-log-analytics-agent-in-azure-monitor-on-31-august-2024/), which is when the older version (`System updates should be installed on your machines`) of the recommendation will be deprecated as well. Both recommendations return the same results and are available under the same control `Apply system updates`.

+The new recommendation `System updates should be installed on your machines (powered by Update management center)`, has a remediation flow available through the Fix button, which can be used to remediate any results through the Update Management Center (Preview). This remediation process is still in Preview.

+The new recommendation `System updates should be installed on your machines (powered by Update management center)`, isn't expected to affect your Secure Score, as it will have the same results as the old recommendation `System updates should be installed on your machines`.

+The prerequisite recommendation ([Enable the periodic assessment property](../update-center/assessment-options.md#periodic-assessment)) will have a negative effect on your Secure Score. You can be remediated the effect with the available [Fix button](implement-security-recommendations.md).

+**Feature/Plan** | **Azure** | **Azure Government** | **Azure China**<br/><br/>**21Vianet**

+ | | |

+**FOUNDATIONAL CSPM FEATURES** | | |

+**DEFENDER FOR CLOUD PLANS** | | |

+**[Agentless discovery for Kubernetes](concept-agentless-containers.md)** | Public preview | NA | NA

+**[Agentless vulnerability assessments for container images.](concept-agentless-containers.md)**<br/><br/> Including registry scanning (up to 20 unique images per billable resources) | Public preview | NA | NA

+**[Defender for Azure SQL database servers](defender-for-sql-introduction.md)**<br/><br/> Partial GA in Vianet21<br/> - A subset of alerts/vulnerability assessments is available.<br/>- Behavioral threat protection isn't available.| GA | GA | GA

+**[Defender for Containers](defender-for-containers-introduction.md)**<br/><br/>Support for Arc-enabled Kubernetes clusters (and therefore AWS EKS too) is in public preview and not available on Azure Government.<br/>Run-time visibility of vulnerabilities in container images is also a preview feature. | GA | GA | GA

+[Defender extension for Azure Arc-enabled Kubernetes clusters/servers/data services](defender-for-kubernetes-azure-arc.md). Requires Defender for Containers/Defender for Kubernetes. | Public preview | NA | NA

+**[Defender for Kubernetes](./defender-for-kubernetes-introduction.md)**<br/><br/> Defender for Kubernetes is deprecated and replaced by Defender for Containers. Support for Azure Arc-enabled clusters is in public preview and not available in government clouds. [Learn more](defender-for-kubernetes-introduction.md). | GA | GA | GA

+**DEFENDER FOR SERVERS FEATURES** | | |

+[Docker host hardening](./harden-docker-hosts.md) | GA | GA | GA

+[Defender for Endpoint integration](./integration-defender-for-endpoint.md) | GA | GA | NA

+**[Defender for Storage](./defender-for-storage-introduction.md)**<br/><br/> Some threat protection alerts for Defender for Storage are in public preview. | GA | GA (activity monitoring) | NA

+**[Kubernetes workload protection](kubernetes-workload-protections.md)** | GA | GA | GA

+This article describes Azure services and client operating systems that are supported by Microsoft Defender for Cloud. For Azure cloud support, review [this article](support-matrix-cloud-environment.md)

+Legacy PCI DSS v3.2.1 and legacy SOC TSP are set to be fully deprecated and replaced by [SOC 2 Type 2](/azure/compliance/offerings/offering-soc-2) initiative and [PCI DSS v4](/azure/compliance/offerings/offering-pci-dss) initiative.

+The existing recommendation "Container registry images should have vulnerability findings resolved" will be replaced by a new recommendation powered by MDVM:

+The recommendation "Running container images should have vulnerability findings resolved" (assessment key 41503391-efa5-47ee-9282-4eff6131462c) will be temporarily removed and will be replaced soon by a new recommendation powered by MDVM.

+ The current container recommendations in Defender for Containers will be renamed as follows:

+1. Select **Generation** and set it to **Generation 1**, and then select **Next**.

+For information about transferring data disks for claimable lab VMs, see [Transfer the data disk](devtest-lab-add-claimable-vm.md#transfer-the-data-disk).

+1. On the VM Overview page, your VM shows **Opted-in** status for auto-start.

+

+ :::image type="content" source="media/devtest-lab-auto-startup-vm/vm-overview-auto-start.png" alt-text="Screenshot showing vm with opted-in status for auto-start checked." lightbox="media/devtest-lab-auto-startup-vm/vm-overview-auto-start.png":::

+ You can also see the auto-start status for the VM on the lab Overview page.

+ :::image type="content" source="media/devtest-lab-auto-startup-vm/lab-overview-auto-start-status.png" alt-text="Screenshot showing the lab overview page, with VM auto-start set to Yes." lightbox="media/devtest-lab-auto-startup-vm/lab-overview-auto-start-status.png":::

+- For Azure PowerShell, the [Az PowerShell module](/powershell/azure/new-azureps-module-az) installed on your workstation. Make sure you have the latest version. If necessary, run `Update-Module -Name Az` to update the module.

+The following script provides [Azure CLI](/cli/azure/get-started-with-azure-cli) commands for starting or stopping a lab VM. The variables in this script are for a Windows environment, like a command prompt. Bash or other environments have slight variations.

+ set SUBSCRIPTIONID=<Subscription ID>

+ REM az account set --subscription %SUBSCRIPTIONID%

+> [!NOTE]

+> Terraform currently does not support private endpoint creation for Azure Data Manager for Energy.

+ |**Target sub-resource**| **Azure Data Manager for Energy** (for Azure Data Manager for Energy Preview) by default|

+To learn more about using Customer Lockbox as an interface to review and approve or reject access requests.

+* Active flows

+* [Link a VNet to an ExpressRoute circuit](expressroute-howto-linkvnet-arm.md)

+## **Configuration overviews**

+## Configuring Log Analytics permissions

+2. Select **Access control (IAM)** from the left-hand navigation pane. For more information on access control, see [identity documentation](/azure/cloud-adoption-framework/decision-guides/identity/).

+3. On this page, select **+Add** to create a new role assignment.

+4. From the **Role** tab, select **Contributor**. Click **Next**.

+5. Open the **Members** tab. Click **+ Select members** to open a configuration pane. Search for **ΓÇ£EASM APIΓÇ¥** and click on the value in the members list. Once done, click **Select**, then **Review + assign.**

+6. Once the role assignment has been created, select **Agents** from the **Settings** section of the left-hand navigation menu.

+7. Expand the **Log Analytics agent instructions** section to view your Workspace ID and Primary key. These values will be used to set up your data connection. Save the values in the following format: *WorkspaceId=XXX;ApiKey=YYY*

+

+Please note that use of this data connection is subject to the pricing structure of Log Analytics. See [Azure monitor pricing](https://azure.microsoft.com/pricing/details/monitor/) for more information.

+

+

+## Configuring Data Explorer permissions

+## Add a data connection

+## Edit or delete a data connection

+1. From this page, users can elect to reconnect, edit or delete their data connection.

+ - **Reconnect**: this option attempts to validate the data connection without any changes to the configuration. This option is best for those who have validated the authentication credentials used for the data connection.

+ - **Edit**: this option allows users to change the configuration for the data connection.

+ - **Delete**: this option deletes the data connection.

+

+|Office365<br><br>For example: Office365.Skype.Optimize|Several Office 365 tags are available to allow outbound access by Office 365 product and category. For more information, see [Use Azure Firewall to protect Office 365](protect-office-365.md).|

+ Title: Use Azure Firewall to protect Office 365

+description: Learn how to use Azure Firewall to protect Office 365

+# Use Azure Firewall to protect Office 365

+You can use the Azure Firewall built-in Service Tags and FQDN tags to allow outbound communication to [Office 365 endpoints and IP addresses](/microsoft-365/enterprise/urls-and-ip-address-ranges).

+## Tags creation

+For each Office 365 product and category, Azure Firewall automatically retrieves the required endpoints and IP addresses, and creates tags accordingly:

+- Tag name: all names begin with **Office365** and are followed by:

+ - Product: Exchange / Skype / SharePoint / Common

+ - Category: Optimize / Allow / Default

+ - Required / Not required (optional)

+- Tag type:

+ - **FQDN tag** represents only the required FQDNs for the specific product and category that communicate over HTTP/HTTPS (ports 80/443) and can be used in Application Rules to secure traffic to these FQDNs and protocols.

+ - **Service tag** represents only the required IPv4 addresses and ranges for the specific product and category and can be used in Network Rules to secure traffic to these IP addresses and to any required port.

+You should accept a tag being available for a specific combination of product, category and required / not required in the following cases:

+- For a Service Tag ΓÇô this specific combination exists and has required IPv4 addresses listed.

+- For an FQDN Rule ΓÇô this specific combination exists and has required FQDNs listed which communicate to ports 80/443.

+Tags are updated automatically with any modifications to the required IPv4 addresses and FQDNs. New tags might be created automatically in the future as well if new combinations of product and category are added.

+Network rule collection:

+Application rule collection:

+## Rules configuration

+These built-in tags provide granularity to allow and protect the outbound traffic to Office 365 based on your preferences and usage. You can allow outbound traffic only to specific products and categories for a specific source. You can also use [Azure Firewall PremiumΓÇÖs TLS Inspection and IDPS](premium-features.md) to monitor some of the traffic. For example, traffic to endpoints in the Default category that can be treated as normal Internet outbound traffic. For more information about Office 365 endpoint categories, see [New Office 365 endpoint categories](/microsoft-365/enterprise/microsoft-365-network-connectivity-principles#new-office-365-endpoint-categories).

+When you create the rules, ensure you define the required TCP ports (for network rules) and protocols (for application rules) as required by Office 365. If a specific combination of product, category and required/not required have both a Service Tag and an FQDN tag, you should create representative rules for both tags to fully cover the required communication.

+## Limitations

+If a specific combination of product, category and required/not required has only FQDNs required, but uses TCP ports that aren't 80/443, an FQDN tag isn't be created for this combination. Application Rules can only cover HTTP, HTTPS or MSSQL. To allow communication to these FQDNs, create your own network rules with these FQDNs and ports.

+For more information, see [Use FQDN filtering in network rules](fqdn-filtering-network-rules.md).

+## Next steps

+- Learn more about Office 365 network connectivity: [Microsoft 365 network connectivity overview](/microsoft-365/enterprise/microsoft-365-networking-overview)

+- Tags for the required IP addresses of Office365 services, split by Office365 product and category. You must define the TCP/UDP ports in your rules. For more information, see [Use Azure Firewall to protect Office 365](protect-office-365.md).

+| Alma | AlmaLinux | 9 |

+| Microsoft | CBL-Mariner | 1 - 2 |

+| Microsoft | Windows Server | 2012 - 2022 |

+| Oracle | Oracle-Linux | 7.x - 8.x |

+| OpenLogic | CentOS | 7.3 - 8.x |

+| Red Hat | Red Hat Enterprise Linux\* | 7.4 - 9.x |

+| Rocky | Rocky Linux | 9 |

+- [Required outbound network rules and fully qualified domain names (FQDNs) for AKS clusters](../../../aks/outbound-rules-control-egress.md#required-outbound-network-rules-and-fqdns-for-aks-clusters)

+ Title: Use MirrorMaker 2 to migrate Kafka clusters between different Azure HDInsight versions - Azure HDInsight

+description: Learn how to use MirrorMaker 2 to migrate Kafka clusters between different Azure HDInsight versions

+# Use MirrorMaker 2 to migrate Kafka clusters between different Azure HDInsight versions

+> [!NOTE]

+> 1. We can use mirroring cluster as a fault tolerance.

+> 2. This is valid only is primary cluster HDI Kafka 2.4.1, 3.2.0 and secondary cluster is HDI Kafka 3.2.0 versions.

+> 3. Secondary cluster would work seamlessly if your primary cluster went down.

+> 4. Consumer group offsets will be automatically translated to secondary cluster.

+> 5. Just point your primary cluster consumers to secondary cluster with same consumer group and your consumer group will start consuming from the offset where it left in primary cluster.

+> 6. The only difference would be that the topic name in backup cluster will change from TOPIC_NAME to primary-cluster-name.TOPIC_NAME.

+ | Cluster name |HDInsight version| Resource group | Virtual network | Storage account |

+ | primary-kafka-cluster | 5.0|kafka-primary-rg | kafka-primary-vnet | kafkaprimarystorage |

+ | secondary-kafka-cluster |5.1|kafka-secondary-rg | kafka-secondary-vnet | kafkasecondarystorage |

+1. After you making the changes, the `/etc/hosts` file for `SECONDARYCLUSTER` looks like the given image.

+1. You can use regular expressions to specify the topics and their configurations that you want to replicate. By setting the `replication.factor` parameter to 3, you can ensure that all topics created by the MirrorMaker script hsd a replication factor of 3.

+1. For automated consumer offset sync, we need to enable replication and control the sync duration too. Following property syncs offset every 30 second. For active-active scenario, we need to do it both ways.

+ ```

+ groups=.*

+ emit.checkpoints.enabled = true

+ source->destination.sync.group.offsets.enabled = true

+ source->destination.sync.group.offsets.interval.ms=30000

+ destination->source.sync.group.offsets.enabled = true

+ destination->source.sync.group.offsets.interval.ms=30000

+ ```

+1. If we donΓÇÖt want to replicate internal topics across clusters, then use following property

+

+ ```

+ topics.blacklist="*.internal,__.*"

+ ```

+

+ emit.checkpoints.enabled = true

+ primary-kafka-cluster->secondary-kafka-cluster.sync.group.offsets.enabled=true

+ primary-kafka-cluster->secondary-kafka-cluster.sync.group.offsets.interval.ms=30000

+ secondary-kafka-cluster->primary-kafka-cluster.sync.group.offsets.enabled = true

+ secondary-kafka-cluster->primary-kafka-cluster.sync.group.offsets.interval.ms=30000

+ export KAFKAZKHOSTS='zk0-primar:2181'

+

+

+ # For Kafka 2.4

+ bash /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --zookeeper $KAFKAZKHOSTS --topic $TOPICNAME

+ # For Kafka 3.2

+ bash /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --boostrap-server $KAFKABROKERS --topic $TOPICNAME

+1. Now start the consumer in PRIMARYCLUSTER with a consumer group

+ ```

+ //Start Consumer

+

+ # For Kafka 2.4

+ bash /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper $KAFKAZKHOSTS --topic $TOPICNAME -ΓÇôgroup my-group ΓÇô-from- beginning

+

+ # For Kafka 3.2

+ bash /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --boostrap-server $KAFKABROKERS --topic $TOPICNAME -ΓÇôgroup my-group ΓÇô-from-beginning

+1. Now stop the consumer in PRIMARYCONSUMER and start consumer in SECONDARYCLUSTER with same consumer group

+ ```

+ export clusterName='secondary-kafka-cluster'

+ export TOPICNAME='primary-kafka-cluster.TestMirrorMakerTopic'

+ export KAFKABROKERS='wn0-second:9092'

+ export KAFKAZKHOSTS='zk0-second:2181'

+

+ # List all the topics whether they're replicated or not

+ You can notice that in secondary cluster consumer group my-group cant't consume any messages because, already consumed by primary cluster consumer group. Now produce more messages in primary-cluster and try to consumer then in secondary-cluster. You are able to consume from `SECONDARYCLUSTER`.

+This article provides an introductory overview of the MedTech service. The MedTech service is a Platform as a Service (PaaS) within the Azure Health Data Services. The MedTech service enables you to ingest device data, transform it into a unified FHIR format, and store it in an enterprise-scale, secure, and compliant cloud environment. 

+1. **Ingest** - The MedTech service asynchronously reads the device message from the event hub at high speed.

+5. **Persist** - After the transformation is done, the newly transformed data is sent to the FHIR service and persisted as FHIR Observations.

+For each IoT hub, you can expect the following:

+* **One to two weeks before migration**: The subscription owners of each IoT hub receive an email notification informing them of their migration date. This notification doesn't apply to hubs that are manually migrated.

+* **Day of the migration**: The IoT hub switches its TLS certificate to the DigiCert Global Root G2, which results in no downtime for the IoT hub. IoT Hub doesn't force device reconnections.

+* **Following the migration**: The subscription owners receive a notification confirming that the IoT hub was migrated. Devices attempt to reconnect based on their individual retry logic, at which point they request and receive the new server certificate from IoT Hub and reconnect only if they trust the Digicert Global Root G2.

+You can migrate your application from the Baltimore CyberTrust Root to the DigiCert Global G2 Root on your own schedule. We recommend the following process:

+2. **In addition** to the Baltimore Root, ensure the DigiCert Global G2 Root is added to your trusted root store.

+3. Make sure you arenΓÇÖt pinning any intermediate or leaf certificates and are using the public roots to perform TLS server validation.

+ 1. Select **DigiCert Global G2 Root** to migrate to the new certificate root.

+ 2. Click **Save** to initiate the migration.

+ 3. If needed, you can migrate back to the Baltimore root by selecting **Baltimore CyberTrust Root** and saving the changes. This option is available until 15 May 2023 and will then be disabled as Microsoft will start initiating the migration.

+The [Azure Digital Twins](../digital-twins/overview.md) service lets you build and maintain models that are live, up-to-date representations of the real world. You can query, analyze, and generate visualizations from these models to extract business insights. An example model might be a representation of a building that includes information about the rooms, the devices in the rooms, and the relationships between the rooms and devices. The real-world data that populates these models is typically collected from IoT devices and sent through an IoT hub.

+If a backend endpoint's health probe fails, established TCP connections to this backend endpoint continue. However, if a backend pool only contains a single endpoint, then existing flows will terminate.

+If all probes for all instances in a backend pool fail, no new flows will be sent to the backend pool. Standard Load Balancer will permit established TCP flows to continue given that a backend pool has more than one backend endpoint. Basic Load Balancer will terminate all existing TCP flows to the backend pool.

+* For AKS cluster without Azure Arc connection, configure the [AKS extension network requirements](../aks/outbound-rules-control-egress.md#cluster-extensions).

+- [Disabling local accounts](../aks/manage-local-accounts-managed-azure-ad.md#disable-local-accounts) for AKS is **not supported** by Azure Machine Learning. When the AKS Cluster is deployed, local accounts are enabled by default.

+**Description**: Recommended environment for Deep Learning with PyTorch on Azure containing the Azure Machine Learning SDK with the latest compatible versions of Ubuntu, Python, PyTorch, CUDA\RocM, and NebulaML combined with optimizers like ORT Training, +DeepSpeed+MSCCL+ORT MoE, and checkpointing using NebulaML and more.

+## 2023-04-10

+### Azure Machine Learning SDK for Python v1.50.0

+ + **azureml-contrib-automl-dnn-forecasting**

+ + Added support for forecasting at given quantiles for TCN models.

+ + **azureml-responsibleai**

+ + updated common environment and azureml-responsibleai package to raiwidgets and responsibleai 0.26.0

+ + **azureml-train-automl-runtime**

+ + Fix MLTable handling for model test scenario

+ + **azureml-training-tabular**

+ + Added quantiles as parameter in the forecast_quantile method.

+ + Starting with v1.49.0 and above, the following AutoML algorithms won't be supported.

+ + Runs will not fail anymore because of "Failed to calculate TCN metrics" error. The warning message that says "Forecast Metric calculation resulted in error, reporting back worst scores" will still be logged. Instead we raise exception when we face inf/nan validation loss for more than two times consecutively with a message "Invalid Model, TCN training didn't converge.". The customers need be aware of the fact that loaded models may return nan/inf values as predictions while inferencing after this change.

+ + Fixed bug where generated code doesn't do train/test splits correctly.

+ + There's a corner case where samples are reduced to 1 after the cross validation split but sample_size still points to the count before the split and hence batch_size ends up being more than sample count in some cases. In this fix we initialize sample_size after the split

+ + Automatic cross-validation parameter configuration is now available for AutoML forecasting tasks. Users can now specify "auto" for n_cross_validations and cv_step_size or leave them empty, and AutoML will provide those configurations base on your data. However, currently this feature isn't supported when TCN is enabled.

+ + Don't print run detail anymore if `pipeline_run.wait_for_completion` with `show_output=False`

+ + New optional fields added to Kubernetes.attach_configuration(identity_type=None, identity_ids=None) which allow attaching KubernetesCompute with either SystemAssigned or UserAssigned identity. New identity fields are included when calling print(compute_target) or compute_target.serialize(): identity_type, identity_id, principal_id, and tenant_id/client_id.

+ + We're removing the dependency azureml-model-management-sdk==1.0.1b6.post1 from azureml-defaults.

+ + Fixed a bug where docker settings in Environment object on ScriptRunConfig aren't respected.

+ + Fixed an issue where Naive models would be recommended in AutoMLStep runs and fail with lag or rolling window features. These models won't be recommended when target lags or target rolling window size are set.

+ + Previously, it was possibly for users to create provisioning configurations for ComputeTarget's that didn't satisfy the password strength requirements for the `admin_user_password` field (that is, that they must contain at least 3 of the following: One lowercase letter, one uppercase letter, one digit, and one special character from the following set: ``\`~!@#$%^&*()=+_[]{}|;:./'",<>?``). If the user created a configuration with a weak password and ran a job using that configuration, the job would fail at runtime. Now, the call to `AmlCompute.provisioning_configuration` throws a `ComputeTargetException` with an accompanying error message explaining the password strength requirements.

+ + Additionally, it was also possible in some cases to specify a configuration with a negative number of maximum nodes. It's no longer possible to do this. Now, `AmlCompute.provisioning_configuration` throws a `ComputeTargetException` if the `max_nodes` argument is a negative integer.

+ + Show error message 'Environment name expected str, {} found' when provided environment name isn't a string.

+ + Fixed a bug where classical forecasting models (for example, AutoArima) could receive training data wherein rows with imputed target values weren't present. This violated the data contract of these models. * Fixed various bugs with lag-by-occurrence behavior in the time-series lagging operator. Previously, the lag-by-occurrence operation didn't mark all imputed rows correctly and so wouldn't always generate the correct occurrence lag values. Also fixed some compatibility issues between the lag operator and the rolling window operator with lag-by-occurrence behavior. This previously resulted in the rolling window operator dropping some rows from the training data that it should otherwise use.

+ + Customers shouldn't see changes to existing run data visualization using the widget, and now will have support if they optionally use conditional hyperparameters.

+ + framework_version added in OptimizationConfig. It's used when model is registered with framework MULTI.

+ + framework_version added in OptimizationConfig. It's used when model is registered with framework MULTI.

+ + Users can't rename an experiment while reactivating it.

+ + Resolved a bug in mlflow.projects.run against azureml backend where Finalizing state wasn't handled properly.

+ + Bug fix when environment object isn't passed to ScriptRunConfig constructor.

+ + Grid Profiling removed from the SDK and isn't longer supported.

+ + Grid Profiling removed from the SDK and isn't longer supported.

+ + Fixed issue where AutoML fails with ImportError: can't import name `RollingOriginValidator`.

+ + A new method `run.get_detailed_status()` now shows the detailed explanation of current run status. It's currently only showing explanation for `Queued` status.

+ + While enabling private link on an existing workspace, please note that if there are compute targets associated with the workspace, those targets won't work if they are not behind the same virtual network as the workspace private endpoint.

+ + Warning messages are printed if no files were downloaded from the datastore in a run.

+ + A new ForecastingParameters class is used instead of forecasting parameters in a dict format

+ + Environment.docker.base_dockerfile accepts filepath. If able to resolve a file, the content is read into base_dockerfile environment property

+ + When calling `keep_columns` or `drop_columns` that results in a time series, label, or image column being dropped, the corresponding capabilities are dropped for the dataset as well.

+ + When calling `keep_columns` or `drop_columns` that results in a time series column being dropped, the corresponding capabilities are dropped for the dataset as well.

+ + When calling `keep_columns` or `drop_columns` that results in a time series, label, or image column being dropped, the corresponding capabilities are dropped for the dataset as well.

+ + When calling `keep_columns` or `drop_columns` that results in a time series column being dropped, the corresponding capabilities are dropped for the dataset as well.

+ + Announcing two new editions (also referred to as a SKU interchangeably) in Azure Machine Learning. With this release, you can now create either a Basic or Enterprise Azure Machine Learning workspace. All existing workspaces are defaulted to the Basic edition, and you can go to the Azure portal or to the studio to upgrade the workspace anytime. You can create either a Basic or Enterprise workspace from the Azure portal. Read [our documentation](./how-to-manage-workspace.md) to learn more. From the SDK, the edition of your workspace can be determined using the "sku" property of your workspace object.

+ + Added a guardrail for forecasting tasks, to check whether a specified max_horizon causes a memory issue on the given machine or not. If it will, a guardrail message is displayed.

+ + New methods `create_from_model()` and `create_from_dataset()` to the [`DataDriftDetector`](/python/api/azureml-datadrift/azureml.datadrift.datadriftdetector%28class%29) class. The `create()` method is deprecated.

+ + Added a parallelize argument to `append_columns`. If True, data is loaded into memory but execution will run in parallel; if False, execution is streaming but single-threaded.

+ + In forecasting tasks, the `target_lags` parameter now accepts a single integer value or a list of integers. If the integer was provided, only one lag is created. If a list is provided, the unique values of lags will be taken. target_lags=[1, 2, 2, 4] will create lags of one, two and four periods.

+ + `Datastore.register_azure_blob_container` now optionally takes a `blob_cache_timeout` value (in seconds) which configures blobfuse's mount parameters to enable cache expiration for this datastore. The default is no timeout, such as when a blob is read, it stays in the local cache until the job is finished. Most jobs prefer this setting, but some jobs need to read more data from a large dataset than will fit on their nodes. For these jobs, tuning this parameter helps them succeed. Take care when tuning this parameter: setting the value too low can result in poor performance, as the data used in an epoch may expire before being used again. All reads are done from blob storage/network rather than the local cache, which negatively impacts training times.

+ 2. Look for the appliance Storage Account in the Resource Group. The Storage Account has a name that resembles *migrategwsa\*\*\*\*\*\*\*\*\*\**. This is the value of parameter [account] in the above command.

+ 3. Search for your storage account in the Azure portal. Ensure that the subscription you use to search is the same subscription (target subscription) in which the storage account is created. Go to Containers in the Blob Service section. Select **+Container** and create a Container. Retain Public Access Level to the default selected value.

+ 4. Go to **Settings** > **Shared Access Signature** and select **Container** in **Allowed Resource Type**.

+

+ 5. Select Generate SAS and connection string and copy the SAS token. If you're using PowerShell, ensure you enclose the URL with single quotation marks (**' '**).

+ 5. Execute the above command in Command Prompt by replacing account, container, SAS with the values obtained in steps b, c, and e respectively.

+ > [!Note]

+ > This is applicable only for the projects that are set up with public endpoint.<br/> A Service bus refers to the ServiceBusNamespace type resource in the resource group for a Migrate project. The name of the Service Bus is of the formatΓÇ»*migratelsa(keyvaultsuffix)*. The Migrate key vault suffix is available in the gateway.json file on the appliance. <br/>

+ > For example, if the gateway.json contains: <br/>

+ > *"AzureKeyVaultArmId": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroupName>/providers/Microsoft.KeyVault/vaults/migratekv1329610309"*,<br/> the service bus namespace resource will be *migratelsa1329610309*.

+ This test checks if the Azure Migrate appliance can communicate to the Azure Migrate Cloud Service backend. The appliance communicates to the service backend through Service Bus and Event Hubs message queues. To validate connectivity from the appliance to the Service Bus, [download](https://go.microsoft.com/fwlink/?linkid=2139104) the Service Bus Explorer, try to connect to the appliance Service Bus and perform the send message/receive message operations. If there's no issue, this should be successful.

+ This test will check whether the Azure Migrate appliance can communicate to the Azure Migrate Cloud Service backend. The appliance communicates to the service backend through Service Bus and Event Hubs message queues. To validate connectivity from the appliance to the Service Bus, [download](https://go.microsoft.com/fwlink/?linkid=2139104) the Service Bus Explorer, try to connect to the appliance Service Bus and perform the send message/receive message operations. If there's no issue, this should be successful.

+The Azure Migrate: App Containerization tool currently supports:

+- Containerizing ASP.NET apps and deploying them on Windows containers on Azure App Service. [Learn more](./tutorial-app-containerization-aspnet-app-service.md).

+- Containerizing Java Web apps on Apache Tomcat (on Linux servers) and deploying them on Linux containers on AKS. [Learn more](./tutorial-app-containerization-java-kubernetes.md).

+- Containerizing Java Web apps on Apache Tomcat (on Linux servers) and deploying them on Linux containers on App Service. [Learn more](./tutorial-app-containerization-java-app-service.md).

+The Azure Migrate: App Containerization tool helps you to:

+- **Improved infrastructure utilization** - With containers, multiple applications can share resources and be hosted on the same infrastructure. This can help you consolidate infrastructure and improve utilization.

+- **Simplified management** - By hosting your applications on a modern managed platform like AKS and App Service, you can simplify your management practices. You can achieve this by retiring or reducing the infrastructure maintenance and management processes that you'd traditionally perform with owned infrastructure.

+- **Application portability** - With increased adoption and standardization of container specification formats and platforms, application portability is no longer a concern.

+- **Adopt modern management with DevOps** - Helps you adopt and standardize on modern practices for management and security and transition to DevOps.

+**Identify a machine to install the tool** | A Windows machine to install and run the Azure Migrate: App Containerization tool. The Windows machine could be a server (Windows Server 2016 or later) or client (Windows 10) operating system, meaning that the tool can run on your desktop as well. <br/><br/> The Windows machine running the tool should have network connectivity to the servers/virtual machines hosting the ASP.NET applications to be containerized.<br/><br/> Ensure that 6-GB space is available on the Windows machine running the Azure Migrate: App Containerization tool for storing application artifacts. <br/><br/> The Windows machine should have internet access, directly or via a proxy. <br/> <br/>Install the Microsoft Web Deploy tool on the machine running the App Containerization helper tool and application server if not already installed. You can download the tool from [here](https://aka.ms/webdeploy3.6).

+**Application servers** | Enable PowerShell remoting on the application servers: Sign in to the application server and follow [these](/powershell/module/microsoft.powershell.core/enable-psremoting) instructions to turn on PowerShell remoting. <br/><br/> If the application server is running Windows Server 2008 R2, ensure that PowerShell 5.1 is installed on the application server. Follow the instruction [here](/powershell/scripting/windows-powershell/wmf/setup/install-configure) to download and install PowerShell 5.1 on the application server. <br/><br/> Install the Microsoft Web Deploy tool on the machine running the App Containerization helper tool and application server if not already installed. You can download the tool from [here](https://aka.ms/webdeploy3.6).

+**ASP.NET application** | The tool currently supports:<br/> - ASP.NET applications using Microsoft .NET framework 3.5 or later. <br/>- Application servers running Windows Server 2008 R2 or later (application servers should be running PowerShell version 5.1). <br/>- Applications running on Internet Information Services (IIS) 7.5 or later. <br/><br/> The tool currently doesn't support: <br/>- Applications requiring Windows authentication (The App Containerization tool currently doesn't support gMSA). <br/>- Applications that depend on other Windows services hosted outside IIS.

+Once your subscription is set up, you need an Azure user account with:

+- Owner permissions on the Azure subscription.

+- Permissions to register Azure Active Directory apps.

+ 

+1. Assign the following role. For detailed steps, see [Assigning Azure roles using the Azure portal](../role-based-access-control/role-assignments-portal.md).

+ | **Setting** | **Value** |

+ 

+ 

+ 

+### Complete tool prerequisites

+ 

+- Containerizing ASP.NET web apps and deploying them on Windows containers on App Service. [Learn more](./tutorial-app-containerization-aspnet-app-service.md).

+- Containerizing Java web apps on Apache Tomcat (on Linux servers) and deploying them on Linux containers on AKS. [Learn more](./tutorial-app-containerization-java-kubernetes.md).

+- Containerizing Java web apps on Apache Tomcat (on Linux servers) and deploying them on Linux containers on App Service. [Learn more](./tutorial-app-containerization-java-app-service.md).

+## April 2023

+- **Known issues**

+ [Storage Auto-grow](./concepts-service-tiers-storage.md#storage-auto-grow): When [storage auto-grow feature](./concepts-service-tiers-storage.md#storage-auto-grow) is enabled and pre-provisioned [IOPS](./concepts-service-tiers-storage.md#iops) is increased, it may result in unexpected increase in the storage size of the instance. We are actively working to resolve this issue and will provide updates as soon as they are available.

+>

+> If your cluster uses any part of the 100.64.0.0/16 IP address range, you cannot migrate to OVN-Kubernetes because it uses this IP address range internally.

+|Azure Machine Learning| See [Azure Machine Learning feature availability across Azure in China cloud regions](../machine-learning/reference-machine-learning-cloud-parity.md#azure-china-21vianet). | |

+### Security

+This section outlines variations and considerations when using Security services.

+| Product | Unsupported, limited, and/or modified features | Notes |

+||--||

+| Microsoft Sentinel| For Microsoft Sentinel availability, see [Microsoft Sentinel availability](../sentinel/feature-availability.md). |

+| Azure Container Apps Default Domain | \*.azurecontainerapps.io | No default domain is provided for external environment. The [custom domain](/azure/container-apps/custom-domains-certificates) is required. |

+> [!CAUTION]

+> If you're using the [SharePoint Online indexer (Preview)](search-howto-index-sharepoint-online.md), you should avoid incremental enrichment. Under certain circumstances, the cache becomes invalid, requiring an [indexer reset and run](search-howto-run-reset-indexers.md), should you choose to reload it.

+> [!CAUTION]

+> If you're using the [SharePoint Online indexer (Preview)](search-howto-index-sharepoint-online.md), you should avoid incremental enrichment. Under certain circumstances, the cache becomes invalid, requiring an [indexer reset and run](search-howto-run-reset-indexers.md), should you choose to reload it.

+- [Find ASIM-based domain essential solutions](sentinel-solutions-catalog.md) like the Network Session Essentials and DNS Essentials Solution for Microsoft Sentinel

+- [Using the Advanced Security Information Model (ASIM)](/azure/sentinel/normalization-about-parsers)

+ Title: Cloud feature availability in Microsoft Sentinel

+description: This article describes feature availability in Microsoft Sentinel across different Azure environments.

+# Cloud feature availability in Microsoft Sentinel

+This article describes feature availability in Microsoft Sentinel across different Azure environments.

+## Analytics

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Analytics rules health](monitor-analytics-rule-integrity.md) |Public Preview |&#10060; |

+|[MITRE ATT&CK dashboard](mitre-coverage.md) |Public Preview |&#10060; |

+|[NRT rules](near-real-time-rules.md) |Public Preview |&#x2705; |

+|[Recommendations](detection-tuning.md) |Public Preview |&#10060; |

+|[Scheduled](detect-threats-built-in.md) and [Microsoft rules](create-incidents-from-alerts.md) |GA |&#x2705; |

+## Content and content management

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Content hub](sentinel-solutions.md) and [solutions](sentinel-solutions-catalog.md) |Public preview |&#10060; |

+|[Repositories](ci-cd.md?tabs=github) |Public preview |&#10060; |

+|[Workbooks](monitor-your-data.md) |GA |&#x2705; |

+## Data collection

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Amazon Web Services](connect-aws.md?tabs=ct) |GA |&#10060; |

+|[Amazon Web Services S3 (Preview)](connect-aws.md?tabs=s3) |Public Preview |&#10060; |

+|[Azure Active Directory](connect-azure-active-directory.md) |GA |&#x2705; ^{[1](#logsavailable)} |

+|[Azure Active Directory Identity Protection](connect-services-api-based.md) |GA |&#10060; |

+|[Azure Activity](data-connectors/azure-activity.md) |GA |&#x2705; |

+|[Azure DDoS Protection](connect-services-diagnostic-setting-based.md) |GA |&#10060; |

+|[Azure Firewall](data-connectors/azure-firewall.md) |GA |&#x2705; |

+|[Azure Information Protection (Preview)](data-connectors/azure-information-protection.md) |Deprecated |&#10060; |

+|[Azure Key Vault](data-connectors/azure-key-vault.md) |Public Preview |&#x2705; |

+|[Azure Kubernetes Service (AKS)](data-connectors/azure-kubernetes-service-aks.md) |Public Preview |&#x2705; |

+|[Azure SQL Databases](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/azure-sentinel-sql-solution-query-deep-dive/ba-p/2597961) |GA |&#x2705; |

+|[Azure Web Application Firewall (WAF)](data-connectors/azure-web-application-firewall-waf.md) |GA |&#x2705; |

+|[Cisco ASA](data-connectors/cisco-asa.md) |GA |&#x2705; |

+|[Codeless Connectors Platform](create-codeless-connector.md?tabs=deploy-via-arm-template%2Cconnect-via-the-azure-portal) |Public Preview |&#10060; |

+|[Common Event Format (CEF)](connect-common-event-format.md) |GA |&#x2705; |

+|[Common Event Format (CEF) via AMA (Preview)](connect-cef-ama.md) |Public Preview |&#x2705; |

+|[Data Connectors health](monitor-data-connector-health.md#use-the-sentinelhealth-data-table-public-preview) |Public Preview |&#10060; |

+|[DNS](data-connectors/dns.md) |Public Preview |&#x2705; |

+|[GCP Pub/Sub Audit Logs](connect-google-cloud-platform.md) |Public Preview |&#10060; |

+|[Microsoft 365 Defender](connect-microsoft-365-defender.md?tabs=MDE) |GA |&#10060; |

+|[Microsoft Purview Insider Risk Management (Preview)](sentinel-solutions-catalog.md#domain-solutions) |Public Preview |&#10060; |

+|[Microsoft Defender for Cloud](connect-defender-for-cloud.md) |GA |&#x2705; |

+|[Microsoft Defender for IoT](connect-services-api-based.md) |GA |&#10060; |

+|[Microsoft Power BI (Preview)](data-connectors/microsoft-powerbi.md) |Public Preview |&#10060; |

+|[Microsoft Project (Preview)](data-connectors/microsoft-project.md) |Public Preview |&#10060; |

+|[Microsoft Purview (Preview)](connect-services-diagnostic-setting-based.md) |Public Preview |&#10060; |

+|[Microsoft Purview Information Protection](connect-microsoft-purview.md) |Public Preview |&#10060; |

+|[Office 365](connect-services-api-based.md) |GA |&#x2705; |

+|[Security Events via Legacy Agent](connect-services-windows-based.md#log-analytics-agent-legacy) |GA |&#x2705; |

+|[Syslog](connect-syslog.md) |GA |&#x2705; |

+|[Windows DNS Events via AMA (Preview)](connect-dns-ama.md) |Public Preview |&#10060; |

+|[Windows Firewall](data-connectors/windows-firewall.md) |GA |&#x2705; |

+|[Windows Forwarded Events (Preview)](connect-services-windows-based.md) |Public Preview |&#x2705; |

+|[Windows Security Events via AMA](connect-services-windows-based.md) |GA |&#x2705; |

+^{<a name="logsavailable"></a>1} Supports only sign-in logs and audit logs.

+## Hunting

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Hunting blade](hunting.md) |GA |&#x2705; |

+|[Restore historical data](restore.md) |GA |&#x2705; |

+|[Search large datasets](search-jobs.md) |GA |&#x2705; |

+## Incidents

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Add entities to threat intelligence](add-entity-to-threat-intelligence.md?tabs=incidents) |Public Preview |&#10060; |

+|[Advanced and/or conditions](add-advanced-conditions-to-automation-rules.md) |Public Preview |&#x2705; |

+|[Automation rules](automate-incident-handling-with-automation-rules.md) |Public Preview |&#x2705; |

+|[Automation rules health](monitor-automation-health.md) |Public Preview |&#10060; |

+|[Create incidents manually](create-incident-manually.md) |Public Preview |&#x2705; |

+|[Cross-tenant/Cross-workspace incidents view](multiple-workspace-view.md) |GA |&#x2705; |

+|[Incident advanced search](investigate-cases.md#search-for-incidents) |GA |&#x2705; |

+|[Incident tasks](incident-tasks.md) |Public Preview |&#x2705; |

+|[Microsoft 365 Defender incident integration](microsoft-365-defender-sentinel-integration.md#working-with-microsoft-365-defender-incidents-in-microsoft-sentinel-and-bi-directional-sync) |Public Preview |&#10060; |

+|[Microsoft Teams integrations](collaborate-in-microsoft-teams.md) |Public Preview |&#10060; |

+|[Playbook template gallery](use-playbook-templates.md) |Public Preview |&#10060; |

+|[Run playbooks on entities](respond-threats-during-investigation.md) |Public Preview |&#10060; |

+|[Run playbooks on incidents](automate-responses-with-playbooks.md) |Public Preview |&#x2705; |

+|[SOC incident audit metrics](manage-soc-with-incident-metrics.md) |GA |&#x2705; |

+## Machine Learning

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Anomalous RDP login detection - built-in ML detection](configure-connector-login-detection.md) |Public Preview |&#x2705; |

+|[Anomalous SSH login detection - built-in ML detection](connect-syslog.md#configure-the-syslog-connector-for-anomalous-ssh-login-detection) |Public Preview |&#x2705; |

+|[Bring Your Own ML (BYO-ML)](bring-your-own-ml.md) |Public Preview |&#10060; |

+|[Fusion](fusion.md) - advanced multistage attack detections ^{[1](#partialga)} |GA |&#x2705; |

+|[Fusion detection for ransomware](fusion.md#fusion-for-ransomware) |Public Preview |&#x2705; |

+|[Fusion for emerging threats](fusion.md#fusion-for-emerging-threats) |Public Preview |&#x2705; |

+^{<a name="partialga"></a>1} Partially GA: The ability to disable specific findings from vulnerability scans is in public preview.

+## Normalization

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Advanced Security Information Model (ASIM)](normalization.md) |Public Preview |&#x2705; |

+## Notebooks

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Notebooks](notebooks.md) |GA |&#x2705; |

+|[Notebook integration with Azure Synapse](notebooks-with-synapse.md) |Public Preview |&#x2705; |

+## SAP

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Threat protection for SAP](sap/deployment-overview.md)^[1](#sap) |GA |&#x2705; |

+^{<a name="sap"></a>1} Deploy SAP security content [via GitHub](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/SAP).

+## Threat intelligence support

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[GeoLocation and WhoIs data enrichment](work-with-threat-indicators.md) |Public Preview |&#10060; |

+|[Import TI from flat file](indicators-bulk-file-import.md) |Public Preview |&#x2705; |

+|[Threat intelligence matching analytics](use-matching-analytics-to-detect-threats.md) |Public Preview |&#10060; |

+|[Threat Intelligence Platform data connector](understand-threat-intelligence.md) |Public Preview |&#x2705; |

+|[Threat Intelligence Research blade](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/what-s-new-threat-intelligence-menu-item-in-public-preview/ba-p/1646597) |GA |&#x2705; |

+|[Threat Intelligence - TAXII data connector](understand-threat-intelligence.md) |GA |&#x2705; |

+|[Threat Intelligence workbook](/azure/architecture/example-scenario/data/sentinel-threat-intelligence) |GA |&#x2705; |

+|[URL detonation](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/using-the-new-built-in-url-detonation-in-azure-sentinel/ba-p/996229) |Public Preview |&#10060; |

+## UEBA

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Active Directory sync via MDI](enable-entity-behavior-analytics.md#how-to-enable-user-and-entity-behavior-analytics) |Public preview |&#10060; |

+|[Azure resource entity pages](entity-pages.md) |Public Preview |&#10060; |

+|[Entity insights](identify-threats-with-entity-behavior-analytics.md) |GA |&#x2705; |

+|[Entity pages](entity-pages.md) |GA |&#x2705; |

+|[Identity info table data ingestion](investigate-with-ueba.md) |GA |&#x2705; |

+|[IoT device entity page](/azure/defender-for-iot/organizations/iot-advanced-threat-monitoring#investigate-further-with-iot-device-entities) |Public Preview |&#10060; |

+|[Peer/Blast radius enrichments](identify-threats-with-entity-behavior-analytics.md#what-is-user-and-entity-behavior-analytics-ueba) |Public preview |&#10060; |

+|[SOC-ML anomalies](soc-ml-anomalies.md#what-are-customizable-anomalies) |GA |&#10060; |

+|[UEBA anomalies](soc-ml-anomalies.md#ueba-anomalies) |GA |&#10060; |

+|[UEBA enrichments\insights](investigate-with-ueba.md) |GA |&#x2705; |

+## Watchlists

+|Feature |Azure commercial |Azure China 21Vianet |

+||||

+|[Large watchlists from Azure Storage](watchlists.md) |Public Preview |&#10060; |

+|[Watchlists](watchlists.md) |GA |&#x2705; |

+|[Watchlist templates](watchlist-schemas.md) |Public Preview |&#10060; |

+## Next steps

+In this article, you learned about available features in Microsoft Sentinel.

+- [Learn about Microsoft Sentinel](overview.md)

+- [Plan your Microsoft Sentinel architecture](design-your-workspace-architecture.md)

+|**[DNS Essentials Solution](https://azuremarketplace.microsoft.com/marketplace/apps/azuresentinel.azure-sentinel-solution-dns-domain?tab=Overview)**|[Analytics rules, hunting queries, playbooks, workbook](domain-based-essential-solutions.md)|Security - Network | Microsoft|

+Say, there are three messages in the queue and two consumers.

+1. Consumer 1 picks up message 1.

+1. Consumer 2 picks up message 2.

+1. Consumer 2 finishes processing message 2 and picks up message 3, while Consumer 1 isn't done with processing message 1 yet.

+1. Consumer 2 finishes processing message 3, but consumer 1 is still not done with processing message 1 yet.

+1. Finally, consumer 1 completes processing message 1.

+

+So, the messages are processed in this order: message 2, message 3, and message 1. If you need message 1, 2, and 3 to be processed in order, you need to use sessions.

+If messages just need to be retrieved in order, you don't need to use sessions. If messages need to be processed in order, use sessions. The same session ID should be set on messages that belong together, which could be message 1, 4, and 8 in a set, and 2, 3, and 6 in another set.

+- **Integrated API** support provided by managed Azure Functions, with the option to link an existing function app, web app, container app, or API Management instance using a standard account. If you need your API in a region that doesn't support [managed functions](apis-functions.md), you can [bring your own functions](functions-bring-your-own.md) to your app.

+This article shows how to download a blob using the [Azure Storage client library for Java](/java/api/overview/azure/storage-blob-readme). You can download blob data to various destinations, including a local file path, stream, or text string. You can also open a blob stream and read from it.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operation:

+ - [Get Blob](/rest/api/storageservices/get-blob#authorization)

+- The package **azure-storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and Java](storage-blob-java-get-started.md#set-up-your-project).

+## Download a blob

+You can use any of the following methods to download a blob:

+The following example downloads a blob to a local file path:

+The following example downloads a blob to an `OutputStream` object:

+The following example assumes that the blob is a text file, and downloads the blob to a `String` object:

+This article shows how to download a blob using the [Azure Storage client library for JavaScript](https://www.npmjs.com/package/@azure/storage-blob). You can download blob data to various destinations, including a local file path, stream, or text string.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operation:

+ - [Get Blob](/rest/api/storageservices/get-blob#authorization)

+- The package **@azure/storage-blob** installed to your project directory. To learn more about setting up your project, see [Get started with Azure Blob Storage and JavaScript](storage-blob-javascript-get-started.md).

+## Download a blob

+You can use any of the following methods to download a blob:

+This article shows how to download a blob using the [Azure Storage client library for Python](/python/api/overview/azure/storage). You can download blob data to various destinations, including a local file path, stream, or text string. You can also open a blob stream and read from it.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operation:

+ - [Get Blob](/rest/api/storageservices/get-blob#authorization)

+- The package **azure-storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and Java](storage-blob-java-get-started.md#set-up-your-project).

+## Download a blob

+You can use the following method to download a blob:

+This article shows how to download a blob using the [Azure Storage client library for JavaScript](https://www.npmjs.com/package/@azure/storage-blob). You can download blob data to various destinations, including a local file path, stream, or text string.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operation:

+ - [Get Blob](/rest/api/storageservices/get-blob#authorization)

+- The package **@azure/storage-blob** installed to your project directory. To learn more about setting up your project, see [Get started with Azure Blob Storage and TypeScript](storage-blob-typescript-get-started.md).

+## Download a blob

+You can use any of the following methods to download a blob:

+This article shows how to download a blob using the [Azure Storage client library for .NET](/dotnet/api/overview/azure/storage). You can download blob data to various destinations, including a local file path, stream, or text string. You can also open a blob stream and read from it.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operation:

+ - [Get Blob](/rest/api/storageservices/get-blob#authorization)

+- The package **Azure.Storage.Blobs** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and .NET](storage-blob-dotnet-get-started.md#set-up-your-project).

+## Download a blob

+You can use any of the following methods to download a blob:

+You can also open a stream to read from a blob. The stream only downloads the blob as the stream is read from. You can use either of the following methods:

+The following example downloads a blob to a local file path. If the specified directory doesn't exist, the code throws a [DirectoryNotFoundException](/dotnet/api/system.io.directorynotfoundexception). If the file already exists at `localFilePath`, it's overwritten by default during subsequent downloads.

+The following example downloads a blob by creating a [Stream](/dotnet/api/system.io.stream) object and then downloads to that stream. If the specified directory doesn't exist, the code throws a [DirectoryNotFoundException](/dotnet/api/system.io.directorynotfoundexception).

+The following example assumes that the blob is a text file, and downloads the blob to a string:

+The following example downloads a blob by reading from a stream:

+### Code samples

+- [View code samples from this article (GitHub)](https://github.com/Azure-Samples/AzureStorageSnippets/blob/master/blobs/howto/dotnet/BlobDevGuideBlobs/DownloadBlob.cs)

+### See also

+- [Performance tuning for uploads and downloads](storage-blobs-tune-upload-download.md).

+This article shows how to upload a block blob using the [Azure Storage client library for Java](/java/api/overview/azure/storage-blob-readme). You can upload data to a block blob from a file path, a stream, a binary object, or a text string. You can also upload blobs with index tags.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operations:

+ - [Put Blob](/rest/api/storageservices/put-blob#authorization)

+ - [Put Block](/rest/api/storageservices/put-block#authorization)

+- The package **azure-storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and Java](storage-blob-java-get-started.md#set-up-your-project).

+## Upload data to a block blob

+To upload a block blob from a stream or a binary object, use the following method:

+To upload a block blob from a file path, use the following method:

+## Upload a block blob from a local file path

+The following example uploads a file to a block blob using a `BlobClient` object:

+The following example uploads a block blob by creating a `ByteArrayInputStream` object, then uploading that stream object:

+## Upload a block blob from a BinaryData object

+The following example uploads `BinaryData` to a block blob using a `BlobClient` object:

+- [Put Block](/rest/api/storageservices/put-block) (REST API)

+This article shows how to upload a blob using the [Azure Storage client library for JavaScript](https://www.npmjs.com/package/@azure/storage-blob). You can upload data to a block blob from a file path, a stream, a buffer, or a text string. You can also upload blobs with index tags.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operations:

+ - [Put Blob](/rest/api/storageservices/put-blob#authorization)

+ - [Put Block](/rest/api/storageservices/put-block#authorization)

+- The package **@azure/storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and JavaScript](storage-blob-javascript-get-started.md#set-up-your-project).

+## Upload data to a block blob

+You can use any of the following methods to upload data to a block blob:

+- [upload](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-upload) (non-parallel uploading method)

+- [uploadData](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploaddata)

+- [uploadFile](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploadfile) (only available in Node.js runtime)

+- [uploadStream](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploadstream) (only available in Node.js runtime)

+Each of these methods can be called using a [BlockBlobClient](/javascript/api/@azure/storage-blob/blockblobclient) object.

+## Upload a block blob from a file path

+## Upload a block blob from a stream

+## Upload a block blob from a buffer

+## Upload a block blob from a string

+- [Put Block](/rest/api/storageservices/put-block) (REST API)

+This article shows how to upload a blob using the [Azure Storage client library for Python](/python/api/overview/azure/storage). You can upload data to a block blob from a file path, a stream, a binary object, or a text string. You can also upload blobs with index tags.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operations:

+ - [Put Blob](/rest/api/storageservices/put-blob#authorization)

+ - [Put Block](/rest/api/storageservices/put-block#authorization)

+- The package **azure-storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and Python](storage-blob-python-get-started.md#set-up-your-project).

+## Upload data to a block blob

+To upload a blob using a stream or a binary object, use the following method:

+- [BlobClient.upload_blob](/python/api/azure-storage-blob/azure.storage.blob.blobclient#azure-storage-blob-blobclient-upload-blob)

+## Upload a block blob from a local file path

+The following example uploads a file to a block blob using a `BlobClient` object:

+## Upload binary data to a block blob

+The following example uploads binary data to a block blob using a `BlobClient` object:

+- [Put Block](/rest/api/storageservices/put-block) (REST API)

+This article shows how to upload a blob using the [Azure Storage client library for JavaScript](https://www.npmjs.com/package/@azure/storage-blob). You can upload data to a block blob from a file path, a stream, a buffer, or a text string. You can also upload blobs with index tags.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operations:

+ - [Put Blob](/rest/api/storageservices/put-blob#authorization)

+ - [Put Block](/rest/api/storageservices/put-block#authorization)

+- The package **@azure/storage-blob** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and TypeScript](storage-blob-typescript-get-started.md#set-up-your-project).

+## Upload data to a block blob

+You can use any of the following methods to upload data to a block blob:

+- [upload](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-upload) (non-parallel uploading method)

+- [uploadData](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploaddata)

+- [uploadFile](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploadfile) (only available in Node.js runtime)

+- [uploadStream](/javascript/api/@azure/storage-blob/blockblobclient#@azure-storage-blob-blockblobclient-uploadstream) (only available in Node.js runtime)

+Each of these methods can be called using a [BlockBlobClient](/javascript/api/@azure/storage-blob/blockblobclient) object.

+## Upload a block blob from a file path

+## Upload a block blob from a stream

+## Upload a block blob from a buffer

+## Upload a block blob from a string

+- [Put Block](/rest/api/storageservices/put-block) (REST API)

+This article shows how to upload a blob using the [Azure Storage client library for .NET](/dotnet/api/overview/azure/storage). You can upload data to a block blob from a file path, a stream, a binary object, or a text string. You can also open a blob stream and write to it, or upload large blobs in blocks.

+## Prerequisites

+To work with the code examples in this article, make sure you have:

+- An authorized client object to connect to Blob Storage data resources. To learn more, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).

+- Permissions to perform an upload operation. To learn more, see the authorization guidance for the following REST API operations:

+ - [Put Blob](/rest/api/storageservices/put-blob#authorization)

+ - [Put Block](/rest/api/storageservices/put-block#authorization)

+- The package **Azure.Storage.Blobs** installed to your project directory. To learn more about setting up your project, see [Get Started with Azure Storage and .NET](storage-blob-dotnet-get-started.md#set-up-your-project).

+## Upload data to a block blob

+You can use either of the following methods to upload data to a block blob:

+To open a stream in Blob Storage and write to that stream, use either of the following methods:

+## Upload a block blob from a local file path

+The following example uploads a block blob from a local file path:

+## Upload a block blob from a stream

+The following example uploads a block blob by creating a [Stream](/dotnet/api/system.io.stream) object and uploading the stream.

+## Upload a block blob from a BinaryData object

+The following example uploads a block blob from a [BinaryData](/dotnet/api/system.binarydata) object.

+## Upload a block blob from a string

+The following example uploads a block blob from a string:

+The following example uploads a block blob with index tags:

+## Upload to a stream in Blob Storage

+You can open a stream in Blob Storage and write to it. The following example creates a zip file in Blob Storage and writes files to it. Instead of building a zip file in local memory, only one file at a time is in memory.

+## Upload a block blob by staging blocks and committing

+You can have greater control over how to divide uploads into blocks by manually staging individual blocks of data. When all of the blocks that make up a blob are staged, you can commit them to Blob Storage. You can use this approach to enhance performance by uploading blocks in parallel.

+- [Put Block](/rest/api/storageservices/put-block) (REST API)

+### Code samples

+- [View code samples from this article (GitHub)](https://github.com/Azure-Samples/AzureStorageSnippets/blob/master/blobs/howto/dotnet/BlobDevGuideBlobs/UploadBlob.cs)

+- [Performance tuning for uploads and downloads](storage-blobs-tune-upload-download.md).

+When an application transfers data using the Azure Storage client library for .NET, there are several factors that can affect speed, memory usage, and even the success or failure of the request. To maximize performance and reliability for data transfers, it's important to be proactive in configuring client library transfer options based on the environment your app runs in.

+[InitialTransferSize](/dotnet/api/azure.storage.storagetransferoptions.initialtransfersize) is the size of the first range request in bytes. An HTTP range request is a partial request, with the size defined by `InitialTransferSize` in this case. Blobs smaller than this size are transferred in a single request. Blobs larger than this size continue to be transferred in chunks of size `MaximumTransferSize`.

+It's important to note that the value you specify for `MaximumTransferSize` *does not* limit the value that you define for `InitialTransferSize`. `InitialTransferSize` defines a separate size limitation for an initial request to perform the entire operation at once, with no subtransfers. It's often the case that you want `InitialTransferSize` to be *at least* as large as the value you define for `MaximumTransferSize`, if not larger. Depending on the size of the data transfer, this approach can be more performant, as the transfer is completed with a single request and avoids the overhead of multiple requests.

+[MaximumConcurrency](/dotnet/api/azure.storage.storagetransferoptions.maximumconcurrency) is the maximum number of workers that may be used in a parallel transfer. Currently, only asynchronous operations can parallelize transfers. Synchronous operations ignore this value and work in sequence.

+The following code example shows how to define values for a `StorageTransferOptions` instance and pass these configuration options as a parameter to `UploadAsync`. The values provided in this sample aren't intended to be a recommendation. To properly tune these values, you need to consider the specific needs of your app.

+In this example, we set the number of parallel transfer workers to 2, using the `MaximumConcurrency` property. This configuration opens up to two connections simultaneously, allowing the upload to happen in parallel. The initial HTTP range request attempts to upload up to 8 MiB of data, as defined by the `InitialTransferSize` property. Note that `InitialTransferSize` only applies for uploads when [using a seekable stream](#initialtransfersize-on-upload). If the blob size is smaller than 8 MiB, only a single request is necessary to complete the operation. If the blob size is larger than 8 MiB, all subsequent transfer requests have a maximum size of 4 MiB, which we set with the `MaximumTransferSize` property.

+During an upload, the Storage client libraries split a given upload stream into multiple subuploads based on the values defined in the `StorageTransferOptions` instance. Each subupload has its own dedicated call to the REST operation. For a `BlobClient` object or `BlockBlobClient` object, this operation is [Put Block](/rest/api/storageservices/put-block). For a `DataLakeFileClient` object, this operation is [Append Data](/rest/api/storageservices/datalakestoragegen2/path/update). The Storage client library manages these REST operations in parallel (depending on transfer options) to complete the full upload.

+Depending on whether the upload stream is seekable or non-seekable, the client library handles buffering and `InitialTransferSize` differently, as described in the following sections. A seekable stream is a stream that supports querying and modifying the current position within a stream. To learn more about streams in .NET, see the [Stream class](/dotnet/api/system.io.stream#remarks) reference.

+The Storage REST layer doesnΓÇÖt support picking up a REST upload operation where you left off; individual transfers are either completed or lost. To ensure resiliency for non-seekable stream uploads, the Storage client libraries buffer data for each individual REST call before starting the upload. In addition to network speed limitations, this buffering behavior is a reason to consider a smaller value for `MaximumTransferSize`, even when uploading in sequence. Decreasing the value of `MaximumTransferSize` decreases the maximum amount of data that is buffered on each request and each retry of a failed request. If you're experiencing frequent timeouts during data transfers of a certain size, reducing the value of `MaximumTransferSize` reduces the buffering time, and may result in better performance.

+Another scenario where buffering occurs is when you're uploading data with parallel REST calls to maximize network throughput. The client libraries need sources they can read from in parallel, and since streams are sequential, the Storage client libraries buffer the data for each individual REST call before starting the upload. This buffering behavior occurs even if the provided stream is seekable.

+When a seekable stream is provided for upload, the stream length is checked against the value of `InitialTransferSize`. If the stream length is less than this value, the entire stream is uploaded as a single REST call, regardless of other `StorageTransferOptions` values. Otherwise, the upload is done in multiple parts as described earlier. `InitialTransferSize` has no effect on a non-seekable stream and is ignored.

+During a download, the Storage client libraries split a given download request into multiple subdownloads based on the values defined in the `StorageTransferOptions` instance. Each subdownload has its own dedicated call to the REST operation. Depending on transfer options, the client libraries manage these REST operations in parallel to complete the full download.

+During a download, the Storage client libraries make one download range request using `InitialTransferSize` before doing anything else. During this initial download request, the client libraries know the total size of the resource. If the initial request successfully downloaded all of the content, the operation is complete. Otherwise, the client libraries continue to make range requests up to `MaximumTransferSize` until the full download is complete.

+- [REST API](#enable-and-configure-with-rest-api)

+Learn more in the [ARM template reference](/azure/templates/microsoft.security/pricings?pivots=deployment-language-arm-template).

+- [REST API](#rest-api)

+If you want to disable Defender for Storage on the storage account or disable one of the features (On-upload malware scanning or Sensitive data threat detection), select **Settings**, edit the settings, and select **Save**.

+ "type": "Microsoft.Security/DefenderForStorageSettings",

+ "name": "current",

+ },

+ "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('StorageAccountName'))]"

+To modify the monthly threshold for malware scanning in your storage accounts, simply adjust the `capGBPerMonth` parameter to your preferred value. This parameter sets a cap on the maximum data that can be scanned for malware each month, per storage account. If you want to permit unlimited scanning, assign the value `-1`. The default limit is set at 5,000 GB.

+If you want to turn off the **On-upload malware scanning** or **Sensitive data threat detection** features, you can change the `isEnabled` value to `false` under the `malwareScanning` or `sensitiveDataDiscovery` properties sections.

+To disable the entire Defender plan for the storage account, set the `isEnabled` property value to `false` and remove the `malwareScanning` and `sensitiveDataDiscovery` sections from the properties.

+resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' ...

+resource defenderForStorageSettings 'Microsoft.Security/DefenderForStorageSettings@2022-12-01-preview' = {

+ name: 'current'

+ scope: storageAccount

+To modify the monthly threshold for malware scanning in your storage accounts, simply adjust the `capGBPerMonth` parameter to your preferred value. This parameter sets a cap on the maximum data that can be scanned for malware each month, per storage account. If you want to permit unlimited scanning, assign the value `-1`. The default limit is set at 5,000 GB.

+If you want to turn off the **On-upload malware scanning** or **Sensitive data threat detection** features, you can change the `isEnabled` value to `false` under the `malwareScanning` or `sensitiveDataDiscovery` properties sections.

+To disable the entire Defender plan for the storage account, set the `isEnabled` property value to `false` and remove the `malwareScanning` and `sensitiveDataDiscovery` sections from the properties.

+PUT

+https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Storage/storageAccounts/{accountName}/providers/Microsoft.Security/defenderForStorageSettings/current?api-version=2022-12-01-preview

+ "overrideSubscriptionLevelSettings": true

+To modify the monthly threshold for malware scanning in your storage accounts, simply adjust the `capGBPerMonth` parameter to your preferred value. This parameter sets a cap on the maximum data that can be scanned for malware each month, per storage account. If you want to permit unlimited scanning, assign the value `-1`. The default limit is set at 5,000 GB.

+If you want to turn off the **On-upload malware scanning** or **Sensitive data threat detection** features, you can change the `isEnabled` value to `false` under the `malwareScanning` or `sensitiveDataDiscovery` properties sections.

+To disable the entire Defender plan for the storage account, set the `isEnabled` property value to `false` and remove the `malwareScanning` and `sensitiveDataDiscovery` sections from the properties.

+https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Storage/storageAccounts/{accountName}/providers/Microsoft.Security/DefenderForStorageSettings/current/providers/Microsoft.Insights/diagnosticSettings/service?api-version=2021-05-01-preview

+ "workspaceId": "/subscriptions/{subscriptionId}/resourcegroups/{resourceGroup}/providers/microsoft.operationalinsights/workspaces/{workspaceName}",

+ "category": "ScanResults",

+https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Storage/storageAccounts/{accountName}/providers/Microsoft.Security/DefenderForStorageSettings/current?api-version=2022-12-01-preview

+ "scanResultsEventGridTopicResourceId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.EventGrid/topics/{EventGridTopicName}"

+ https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{accountName}/providers/Microsoft.Security/DefenderForStorageSettings/current?api-version=2022-12-01-preview

+You can configure storage accounts to allow access only from specific subnets. The allowed subnets may belong to a VNet in the same subscription or a different subscription, including those belonging to a different Azure Active Directory tenant. With [cross-region service endpoints](#azure-storage-cross-region-service-endpoints), the allowed subnets can also be in different regions from the storage account.

+### Azure Storage cross-region service endpoints

+Cross-region service endpoints for Azure Storage became generally available in April of 2023. They work between virtual networks and storage service instances in any region. With cross-region service endpoints, subnets will no longer use a public IP address to communicate with any storage account, including those in another region. Instead, all the traffic from subnets to storage accounts will use a private IP address as a source IP. As a result, any storage accounts that use IP network rules to permit traffic from those subnets will no longer have an effect.

+Configuring service endpoints between virtual networks and service instances in a [paired region](../../best-practices-availability-paired-regions.md) can be an important part of your disaster recovery plan. Service endpoints allow continuity during a regional failover and access to read-only geo-redundant storage (RA-GRS) instances. Network rules that grant access from a virtual network to a storage account also grant access to any RA-GRS instance.

+> Local and cross-region service endpoints cannot coexist on the same subnet.

+> To replace existing service endpoints with cross-region ones, delete the existing **Microsoft.Storage** endpoints and recreate them as cross-region endpoints (**Microsoft.Storage.Global**).

+You can manage virtual network rules for storage accounts through the Azure portal, PowerShell, or CLIv2.

+> If you want to enable access to your storage account from a virtual network/subnet in another Azure AD tenant, you must use PowerShell or the Azure CLI. The Azure portal does not show subnets in other Azure AD tenants.

+ > Presently, only virtual networks belonging to the same Azure Active Directory tenant are shown for selection during rule creation. To grant access to a subnet in a virtual network belonging to another tenant, please use PowerShell, Azure CLI or REST APIs.

+ Get-AzVirtualNetwork -ResourceGroupName "myresourcegroup" -Name "myvnet" | Set-AzVirtualNetworkSubnetConfig -Name "mysubnet" -AddressPrefix "10.0.0.0/24" -ServiceEndpoint "Microsoft.Storage.Global" | Set-AzVirtualNetwork

+ az network vnet subnet update --resource-group "myresourcegroup" --vnet-name "myvnet" --name "mysubnet" --service-endpoints "Microsoft.Storage.Global"

+| Azure File Sync | Microsoft.StorageSync | Enables you to transform your on-premises file server to a cache for Azure File shares. Allowing for multi-site sync, fast disaster-recovery, and cloud-side backup. [Learn more](../file-sync/file-sync-planning.md) |

+If you create a private endpoint for the Data Lake Storage Gen2 storage resource, then you should also create one for the Blob Storage resource. That's because operations that target the Data Lake Storage Gen2 endpoint might be redirected to the Blob endpoint. Similarly, if you add a private endpoint for Blob Storage only, and not for Data Lake Storage Gen2, some operations (such as Manage ACL, Create Directory, Delete Directory, etc.) will fail since the Gen2 APIs require a DFS private endpoint. By creating a private endpoint for both resources, you ensure that all operations can complete successfully.

+1. Select **+ Add** and for **Service** select **Microsoft.Storage.Global**.

+$virtualNetwork | Set-AzVirtualNetworkSubnetConfig -Name $subnetName -AddressPrefix $subnet.AddressPrefix -ServiceEndpoint "Microsoft.Storage.Global" | Set-AzVirtualNetwork

+az network vnet subnet update --resource-group "myresourcegroup" --vnet-name "myvnet" --name "mysubnet" --service-endpoints "Microsoft.Storage.Global"

+By default, no network access is allowed to any volumes in a volume group. Adding a virtual network to your volume group lets you establish iSCSI connections from clients in the same virtual network and subnet to the volumes in the volume group. For details on accessing your volumes from another region, see [Azure Storage cross-region service endpoints](elastic-san-networking.md#azure-storage-cross-region-service-endpoints).

+1. Select **+ Add** and for **Service** select **Microsoft.Storage.Global**.

+$virtualNetwork | Set-AzVirtualNetworkSubnetConfig -Name $subnetName -AddressPrefix $subnet.AddressPrefix -ServiceEndpoint "Microsoft.Storage.Global" | Set-AzVirtualNetwork

+az network vnet subnet update --resource-group "myresourcegroup" --vnet-name "myvnet" --name "mysubnet" --service-endpoints "Microsoft.Storage.Global"

+By default, no network access is allowed to any volumes in a volume group. Adding a virtual network to your volume group lets you establish iSCSI connections from clients in the same virtual network and subnet to the volumes in the volume group. For details on accessing your volumes from another region, see [Azure Storage cross-region service endpoints](elastic-san-networking.md#azure-storage-cross-region-service-endpoints).

+$virtualNetwork | Set-AzVirtualNetworkSubnetConfig -Name $subnetName -AddressPrefix $subnet.AddressPrefix -ServiceEndpoint "Microsoft.Storage.Global" | Set-AzVirtualNetwork

+az network vnet subnet update --resource-group "myresourcegroup" --vnet-name "myvnet" --name "mysubnet" --service-endpoints "Microsoft.Storage.Global"

+### Available virtual network regions

+Service endpoints for Azure Storage work between virtual networks and service instances in any region.

+Configuring service endpoints between virtual networks and service instances in a [paired region](../../best-practices-availability-paired-regions.md) can be an important part of your disaster recovery plan. Service endpoints allow continuity during a regional failover and access to read-only geo-redundant storage (RA-GRS) instances. Network rules that grant access from a virtual network to a storage account also grant access to any RA-GRS instance.

+When planning for disaster recovery during a regional outage, you should create the VNets in the paired region in advance. Enable service endpoints for Azure Storage, with network rules granting access from these alternative virtual networks. Then apply these rules to your geo-redundant storage accounts.

+#### Azure Storage cross-region service endpoints

+Cross-region service endpoints for Azure became generally available in April of 2023. With cross-region service endpoints, subnets will no longer use a public IP address to communicate with any storage account. Instead, all the traffic from subnets to storage accounts will use a private IP address as a source IP. As a result, any storage accounts that use IP network rules to permit traffic from those subnets will no longer have an effect.

+To use cross-region service endpoints, it might be necessary to delete existing **Microsoft.Storage** endpoints and recreate them as cross-region (**Microsoft.Storage.Global**).

+You can manage virtual network rules for volume groups through the Azure portal, PowerShell, or CLI.

+> If you want to enable access to your storage account from a virtual network/subnet in another Azure AD tenant, you must use PowerShell or the Azure CLI. The Azure portal does not show subnets in other Azure AD tenants.

+ Get-AzVirtualNetwork -ResourceGroupName "myresourcegroup" -Name "myvnet" | Set-AzVirtualNetworkSubnetConfig -Name "mysubnet" -AddressPrefix "10.0.0.0/24" -ServiceEndpoint "Microsoft.Storage.Global" | Set-AzVirtualNetwork

+ az network vnet subnet update --resource-group "myresourcegroup" --vnet-name "myvnet" --name "mysubnet" --service-endpoints "Microsoft.Storage.Global"

+To mount the container called `mycontainer`, `mssparkutils` first needs to check whether you have the permission to access the container. Currently, Azure Synapse Analytics supports three authentication methods for the trigger mount operation: `LinkedService`, `accountKey`, and `sastoken`.

+ {"LinkedService":"mygen2account"}

+ Map("LinkedService" -> "myblobstorageaccount")

+To allow a user to create or drop a server-level credential, admin can GRANT ALTER ANY CREDENTIAL permission to the user:

+To allow a user to create or drop a database scoped credential, admin can GRANT CONTROL permission on the database to the user:

+```sql

+GRANT CONTROL ON DATABASE::[database_name] TO [user_name];

+```

+To use the credential, a user must have `REFERENCES` permission on a specific credential.

+To grant a `REFERENCES` permission ON a server-level credential for a specific_user, execute:

+GRANT REFERENCES ON CREDENTIAL::[server-level_credential] TO [specific_user];

+To grant a `REFERENCES` permission ON a DATABASE SCOPED CREDENTIAL for a specific_user, execute:

+```sql

+GRANT REFERENCES ON DATABASE SCOPED CREDENTIAL::[database-scoped_credential] TO [specific_user];

+```

+## Server-level credential

+Server-level credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-level credential **must** match the base URL of Azure storage (optionally followed by a container name). A credential is added by running [CREATE CREDENTIAL](/sql/t-sql/statements/create-credential-transact-sql?view=azure-sqldw-latest&preserve-view=true). You'll need to provide a CREDENTIAL NAME argument.

+Server-level credentials enable access to Azure storage using the following authentication types:

+Server-level credential isn't required to allow access to publicly available files. Create [data source without credential](develop-tables-external-tables.md?tabs=sql-ondemand#example-for-create-external-data-source) to access publicly available files on Azure storage.

+Database scoped credential isn't required to allow access to publicly available files. Create [data source without credential](develop-tables-external-tables.md?tabs=sql-ondemand#example-for-create-external-data-source) to access publicly available files on Azure storage.

+WITH ( LOCATION = 'https://<storage_account>.dfs.core.windows.net/<public_container>/<path>' )

+WITH ( LOCATION = 'https://<storage_account>.dfs.core.windows.net/<container>/<path>'

+| **Storage shared access signature (SAS) token authentication** | No | Yes, using [DATABASE SCOPED CREDENTIAL](/sql/t-sql/statements/create-database-scoped-credential-transact-sql?view=azure-sqldw-latest&preserve-view=true) with [shared access signature token](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#database-scoped-credential) in [EXTERNAL DATA SOURCE](/sql/t-sql/statements/create-external-data-source-transact-sql?view=azure-sqldw-latest&preserve-view=true) or instance-level [CREDENTIAL](/sql/t-sql/statements/create-credential-transact-sql?view=azure-sqldw-latest&preserve-view=true) with [shared access signature](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential). |

+| **SERVER LEVEL CREDENTIAL** | No | Yes, the [server level credentials](develop-storage-files-storage-access-control.md?tabs=user-identity#server-level-credential) are used by the `OPENROWSET` function that do not uses explicit data source. |

+Make sure you can access your file. If your file is protected with SAS key or custom Azure identity, you will need to set up a [server level credential for sql login](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential).

+The JSON document in the preceding sample query includes an array of objects. The query returns each object as a separate row in the result set. Make sure that you can access this file. If your file is protected with SAS key or custom identity, you would need to set up [server level credential for sql login](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential).

+Make sure that you can access this file. If your file is protected with SAS key or custom Azure identity, you would need to set up [server level credential for sql login](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential).

+Option `firstrow` is used to skip the first row in the CSV file that represents header in this case. Make sure that you can access this file. If your file is protected with SAS key or custom identity, your would need to setup [server level credential for sql login](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential).

+If you have a shared access signature key that you should use to access files, make sure that you created a [server-level](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-level-credential) or [database-scoped](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#database-scoped-credential) credential that contains that credential. The credentials are required if you need to access data by using the workspace [managed identity](develop-storage-files-storage-access-control.md?tabs=managed-identity#database-scoped-credential) and custom [service principal name (SPN)](develop-storage-files-storage-access-control.md?tabs=service-principal#database-scoped-credential).

+> [!Important]

+> - [Automation Update management](../automation/update-management/overview.md) relies on [Log Analytics agent](../azure-monitor/agents/log-analytics-agent.md) (aka MMA agent), which is on a deprecation path and wonΓÇÖt be supported after **August 31, 2024**. Update management center (Preview) is the v2 version of Automation Update management and the future of Update management in Azure. UMC is a native service in Azure and does not rely on [Log Analytics agent](../azure-monitor/agents/log-analytics-agent.md) or [Azure Monitor agent](../azure-monitor/agents/agents-overview.md).

+> - Guidance for migrating from Automation Update management to Update management center will be provided to customers once the latter is Generally Available. For customers using Automation Update management, we recommend continuing to use the Log Analytics agent and **NOT** migrate to Azure Monitoring agent until migration guidance is provided for Update management or else Automation Update management will not work. Also, the Log Analytics agent would not be deprecated before moving all Automation Update management customers to UMC.

+- An Azure virtual machine (VM) with a [supported version of Windows 10 Enterprise multi-session](/lifecycle/products/windows-10-enterprise-and-education).

+ - [Windows 10 Language Pack ISO (version 2004 or later)](https://software-download.microsoft.com/download/pr/19041.1.191206-1406.vb_release_CLIENTLANGPACKDVD_OEM_MULTI.iso)

+ - [Windows 10 FOD Disk 1 ISO (version 2004 or later)](https://software-download.microsoft.com/download/pr/19041.1.191206-1406.vb_release_amd64fre_FOD-PACKAGES_OEM_PT1_amd64fre_MULTI.iso)

+ - [Windows 10 Inbox Apps ISO (version 21H1 or later)](https://software-download.microsoft.com/download/sg/19041.928.210407-2138.vb_release_svc_prod1_amd64fre_InboxApps.iso)

+ - If you use Local Experience Pack (LXP) ISO files to localize your images, you'll also need to download the appropriate LXP ISO for the best language experience. Use the information in [Adding languages in Windows 10: Known issues](/windows-hardware/manufacture/desktop/language-packs-known-issue) to figure out which of the following LXP ISOs is right for you:

+- Use of explicit HTTP proxies defined on the client endpoint device should work, but isn't supported.

+ "storageProfile": {

+ {

+ "diskSizeGB": 1023,

+ "deleteOption": "Delete"

+ }

+ ]

+ },

+ "networkProfile": {

+ {

+ "id": "/subscriptions/.../Microsoft.Network/networkInterfaces/myNIC",

+ "deleteOption": "Delete"

+ }

+ }

+ ]

+ }

+          "properties": {

+            "deleteOption": "Delete"

+ }

+ "id": "/subscriptions/subID/resourceGroups/resourcegroup/providers/Microsoft.Network/networkInterfaces/nic336",

+ "deleteOption": "Delete"

+ }

+ }

+ }

+```output

+ ```bash

+ ```bash

+```bash

+sudo sed -i 's/GSS_USE_PROXY="yes"/GSS_USE_PROXY="no"/g' /etc/sysconfig/nfs

+```bash

+sudo echo 1 > /proc/sys/vm/drop_caches [frees page-cache]

+sudo echo 2 > /proc/sys/vm/drop_caches [frees slab objects e.g. dentries, inodes]

+sudo echo 3 > /proc/sys/vm/drop_caches [cleans page-cache and slab objects]

+```output

+| OS Support for SRIOV RDMA | CentOS/RHEL 7.9+, Ubuntu 18.04+, SLES 15.4, WinServer 2016+ |

+> [!IMPORTANT]

+> This document references a release version of Linux that is nearing or at, End of Life(EOL). Please consider updating to a more current version.

+This template builds an Ubuntu 20.04 LTS image, installs NGINX, then deprovisions the VM.

+Assign the [Network contributor role](../role-based-access-control/built-in-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json#network-contributor) or a [Custom role](../role-based-access-control/custom-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json) with the appropriate [Permissions](#permissions).

+For more information about the association and dissociation of a network security group, see [Associate or dissociate a network security group](virtual-network-network-interface.md#associate-or-dissociate-a-network-security-group).

+To manage network security groups, security rules, and application security groups, your account must be assigned to the [Network contributor](../role-based-access-control/built-in-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json#network-contributor) role. A [Custom role](../role-based-access-control/custom-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json) can also be used that's assigned the appropriate permissions as listed in the following tables:

+Azure automatically routes traffic between Azure subnets, virtual networks, and on-premises networks. If you want to change Azure's default routing, you do so by creating a route table. If you're new to routing in virtual networks, you can learn more about it in [virtual network traffic routing](virtual-networks-udr-overview.md) or by completing a [tutorial](tutorial-create-route-table-portal.md).

+Assign the [Network contributor role](../role-based-access-control/built-in-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json#network-contributor) or a [Custom role](../role-based-access-control/custom-roles.md?toc=%2fazure%2fvirtual-network%2ftoc.json) with the appropriate [Permissions](#permissions).

+You can optionally associate a route table to a subnet. A route table can be associated to zero or more subnets. Route tables aren't associated to virtual networks. You must associate a route table to each subnet you want the route table associated to.

+Azure routes all traffic leaving the subnet based on routes you've created:

+* Within route tables

+* [Default routes](virtual-networks-udr-overview.md#default)

+* Routes propagated from an on-premises network, if the virtual network is connected to an Azure virtual network gateway (ExpressRoute or VPN).

+You can only associate a route table to subnets in virtual networks that exist in the same Azure location and subscription as the route table.

+ :::image type="content" source="./media/manage-route-table/add-route.png" alt-text="Screenshot of add a route page for a route table.":::

+ :::image type="content" source="./media/manage-route-table/add-route.png" alt-text="Screenshot of add a route page for a route table.":::

+- **[Azure Storage cross-region service endpoints](../storage/common/storage-network-security.md?toc=%2fazure%2fvirtual-network%2ftoc.json#azure-storage-cross-region-service-endpoints)** (*Microsoft.Storage.Global*): Generally available in all Azure regions.

+- Endpoints are enabled on subnets configured in Azure virtual networks. Endpoints can't be used for traffic from your on-premises services to Azure services. For more information, see [Secure Azure service access from on-premises](#secure-azure-services-to-virtual-networks)

+- For Azure SQL, a service endpoint applies only to Azure service traffic within a virtual network's region.

+- For Azure SQL Database, virtual networks must be in the same region as the Azure service resource. For all other services, you can secure Azure service resources to virtual networks in any region.

+Each part of this article helps you form a basic building block for enabling BGP in your network connectivity. If you complete all three parts (configure BGP on the gateway, S2S connection, and VNet-to-VNet connection) you build the topology as shown in **Diagram 1**. You can combine parts together to build a more complex, multi-hop, transit network that meets your needs.

+For context, referring to Diagram 1, if BGP were to be disabled between TestVNet2 and TestVNet1, TestVNet2 wouldn't learn the routes for the on-premises network, Site5, and therefore couldn't communicate with Site 5. Once you enable BGP, all three networks will be able to communicate over the S2S IPsec and VNet-to-VNet connections.

+The following steps configure your Azure VPN gateway in active-active modes. The key differences between the active-active and active-standby gateways:

+* You need to install the Azure Resource Manager PowerShell cmdlets if you don't want to use Cloud Shell in your browser. See [Overview of Azure PowerShell](/powershell/azure/) for more information about installing the PowerShell cmdlets.

+For this exercise, we start by declaring our variables. If you use the "Try It" Cloud Shell, you'll automatically connect to your account. If you use PowerShell locally, use the following example to help you connect:

+The following example declares the variables using the values for this exercise. Be sure to replace the values with your own when configuring for production. You can use these variables if you're running through the steps to become familiar with this type of configuration. Modify the variables, and then copy and paste into your PowerShell console.

+Use the following example to create a new resource group:

+The following example creates a virtual network named TestVNet1 and three subnets, one called GatewaySubnet, one called FrontEnd, and one called Backend. When substituting values, it's important that you always name your gateway subnet specifically GatewaySubnet. If you name it something else, your gateway creation fails.

+Request two public IP addresses to be allocated to the gateway you'll create for your VNet. You'll also define the subnet and IP configurations required.

+Create the virtual network gateway for TestVNet1. There are two GatewayIpConfig entries, and the EnableActiveActiveFeature flag is set. Creating a gateway can take a while (45 minutes or more to complete, depending on the selected SKU).

+Once the gateway is created, you need to obtain the BGP Peer IP address on the Azure VPN Gateway. This address is needed to configure the Azure VPN Gateway as a BGP Peer for your on-premises VPN devices.

+The order of the public IP addresses for the gateway instances and the corresponding BGP Peering Addresses are the same. In this example, the gateway VM with public IP of 40.112.190.5 uses 10.12.255.4 as its BGP Peering Address, and the gateway with 138.91.156.129 uses 10.12.255.5. This information is needed when you set up your on premises VPN devices connecting to the active-active gateway. The gateway is shown in the following diagram with all addresses:

+Before proceeding, make sure you have completed [Part 1](#aagateway) of this exercise.

+* If there is only one on-premises VPN device (as shown in the example), the active-active connection can work with or without BGP protocol. This example uses BGP for the cross-premises connection.

+* As a reminder, you must use different BGP ASNs between your on-premises networks and Azure VNet. If they're the same, you need to change your VNet ASN if your on-premises VPN device already uses the ASN to peer with other BGP neighbors.

+Before you continue, make sure you're still connected to Subscription 1. Create the resource group if it isn't yet created.

+The following example lists the parameters that you enter into the BGP configuration section on your on-premises VPN device for this exercise:

+The connection should be established after a few minutes, and the BGP peering session will start once the IPsec connection is established. This example so far has configured only one on-premises VPN device, resulting in the following diagram:

+Similarly, the following example lists the parameters you'll enter into the second VPN device:

+Once the connection (tunnels) are established, you'll have dual redundant VPN devices and tunnels connecting your on-premises network and Azure:

+This section creates an active-active VNet-to-VNet connection with BGP. The following instructions continue from the previous steps. You must complete [Part 1](#aagateway) to create and configure TestVNet1 and the VPN Gateway with BGP.

+It's important to make sure that the IP address space of the new virtual network, TestVNet2, doesn't overlap with any of your VNet ranges.

+In this example, the virtual networks belong to the same subscription. You can set up VNet-to-VNet connections between different subscriptions; refer to [Configure a VNet-to-VNet connection](vpn-gateway-vnet-vnet-rm-ps.md) to learn more details. Make sure you add the "-EnableBgp $True" when creating the connections to enable BGP.

+Request two public IP addresses to be allocated to the gateway you'll create for your VNet. You'll also define the subnet and IP configurations required.

+Create the VPN gateway with the AS number and the "EnableActiveActiveFeature" flag. You must override the default ASN on your Azure VPN gateways. The ASNs for the connected VNets must be different to enable BGP and transit routing.

+Make sure you sign in and connect to Subscription 1.

+In this step, you create the connection from TestVNet1 to TestVNet2, and the connection from TestVNet2 to TestVNet1.

+After completing these steps, the connection will be established in a few minutes, and the BGP peering session will be up once the VNet-to-VNet connection is completed with dual redundancy:

+In this step, you enable active-active mode and update the gateway. In the example, the VPN gateway is currently using a legacy Standard SKU. However, active-active doesn't support the Standard SKU. To resize the legacy SKU to one that is supported (in this case, HighPerformance), you simply specify the supported legacy SKU that you want to use.

+When you're using this in your environment, if you don't need to resize the gateway, you won't need to specify the -GatewaySku. Notice that in this step, you must set the gateway object in PowerShell to trigger the actual update. This update can take 30 to 45 minutes, even if you aren't resizing your gateway.

+Every Azure VPN gateway consists of two instances in an active-standby configuration. For any planned maintenance or unplanned disruption that happens to the active instance, the standby instance would take over (failover) automatically, and resume the S2S VPN or VNet-to-VNet connections. The switch over will cause a brief interruption. For planned maintenance, the connectivity should be restored within 10 to 15 seconds. For unplanned issues, the connection recovery is longer, about 1 to 3 minutes in the worst case. For P2S VPN client connections to the gateway, the P2S connections are disconnected and the users need to reconnect from the client machines.

+You can create an Azure VPN gateway in an active-active configuration, where both instances of the gateway VMs establish S2S VPN tunnels to your on-premises VPN device, as shown the following diagram:

+In this configuration, each Azure gateway instance has a unique public IP address, and each will establish an IPsec/IKE S2S VPN tunnel to your on-premises VPN device specified in your local network gateway and connection. Note that both VPN tunnels are actually part of the same connection. You'll still need to configure your on-premises VPN device to accept or establish two S2S VPN tunnels to those two Azure VPN gateway public IP addresses.

+The most reliable option is to combine the active-active gateways on both your network and Azure, as shown in the following diagram.

+All gateways and tunnels are active from the Azure side, so the traffic is spread among all 4 tunnels simultaneously, although each TCP or UDP flow will again follow the same tunnel or path from the Azure side. Even though by spreading the traffic, you may see slightly better throughput over the IPsec tunnels, the primary goal of this configuration is for high availability. And due to the statistical nature of the spreading, it's difficult to provide the measurement on how different application traffic conditions will affect the aggregate throughput.

+This topology requires two local network gateways and two connections to support the pair of on-premises VPN devices, and BGP is required to allow the two connections to the same on-premises network. These requirements are the same as the [above](#activeactiveonprem).

+## Highly Available VNet-to-VNet

+The same active-active configuration can also apply to Azure VNet-to-VNet connections. You can create active-active VPN gateways for both virtual networks, and connect them together to form the same full mesh connectivity of 4 tunnels between the two VNets, as shown in the following diagram:

+See [Configure active-active gateways](active-active-portal.md) using the [Azure portal](active-active-portal.md) or [PowerShell](vpn-gateway-activeactive-rm-powershell.md).