+> [!IMPORTANT]

+> When content filtering is triggered for a prompt and a `"status": 400` is received as part of the response there may be a charge for this request as the prompt was evaluated by the service. [Charges will also occur](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) when a `"status":200` is received with `"finish_reason": "content_filter"`. In this case the prompt did not have any issues, but the completion generated by the model was detected to violate the content filtering rules which results in the completion being filtered.

+> We don't recommend using preview models in production. We will upgrade all deployments of preview models to a future stable version. Models designated preview do not follow the standard Azure OpenAI model lifecycle.

+| Available to all subscriptions with Azure OpenAI access | | Australia East <br> Canada East <br> France Central <br> Sweden Central <br> Switzerland North | Australia East <br> Canada East <br> East US 2 <br> France Central <br> Norway East <br> South India <br> Sweden Central <br> UK South <br> West US | Sweden Central <br> Switzerland North <br> West US |

+| Available to subscriptions with current access to the model version in the region | East US <br> France Central <br> South Central US <br> UK South | East US <br> East US 2 <br> Japan East <br> UK South | | Australia East |

+ <td>30 K</td>

+> [!NOTE]

+> When implementing **availability zones with the [cluster autoscaler](./cluster-autoscaler-overview.md)**, we recommend using a single node pool for each zone. You can set the `--balance-similar-node-groups` parameter to `True` to maintain a balanced distribution of nodes across zones for your workloads during scale up operations. When this approach isn't implemented, scale down operations can disrupt the balance of nodes across zones.

+description: Learn how to configure Azure CNI Overlay networking in Azure Kubernetes Service (AKS), including deploying an AKS cluster into an existing virtual network and subnets.

+In Overlay networking, only the Kubernetes cluster nodes are assigned IPs from subnets. Pods receive IPs from a private CIDR provided at the time of cluster creation. Each node is assigned a `/24` address space carved out from the same CIDR. Extra nodes created when you scale out a cluster automatically receive `/24` address spaces from the same CIDR. Azure CNI assigns IPs to pods from this `/24` space.

+- **Cluster Nodes**: When setting up your AKS cluster, make sure your VNet subnets have enough room to grow for future scaling. You can assign each node pool to a dedicated subnet. A `/24`subnet can fit up to 251 nodes since the first three IP addresses are reserved for management tasks.

+## Add a new nodepool to a dedicated subnet

+After your have created a cluster with Azure CNI Overlay, you can create another nodepool and assign the nodes to a new subnet of the same VNet.

+This approach can be usefull if you want to control the ingress or egress IPs of the host from/ towards targets in the same VNET or peered VNets.

+```azurecli-interactive

+clusterName="myOverlayCluster"

+resourceGroup="myResourceGroup"

+location="westcentralus"

+nodepoolName="newpool1"

+subscriptionId=$(az account show --query id -o tsv)

+vnetName="yourVnetName"

+subnetName="yourNewSubnetName"

+subnetResourceId="/subscriptions/$subscriptionId/resourceGroups/$resourceGroup/providers/Microsoft.Network/virtualNetworks/$vnetName/subnets/$subnetName"

+az aks nodepool add -g $resourceGroup --cluster-name $clusterName \

+ --name $nodepoolName --node-count 1 \

+ --mode system --vnet-subnet-id $subnetResourceId

+```

+ * **Increase the Cluster Autoscaler scan interval**: If the diagnostic reports show that [Cluster Autoscaler throttling has been detected](/troubleshoot/azure/azure-kubernetes/429-too-many-requests-errors#analyze-and-identify-errors-by-using-aks-diagnose-and-solve-problems), you can [increase the scan interval](./cluster-autoscaler.md#update-the-cluster-autoscaler-settings) to reduce the number of calls to Virtual Machine Scale Sets from the Cluster Autoscaler.

+When considering cluster autoscaling, the decision of when to remove a node involves a tradeoff between optimizing resource utilization and ensuring resource availability. Eliminating underutilized nodes enhances cluster utilization but might result in new workloads having to wait for resources to be provisioned before they can be deployed. It's important to find a balance between these two factors that aligns with your cluster and workload requirements and [configure the cluster autoscaler profile settings accordingly](./cluster-autoscaler.md#update-the-cluster-autoscaler-settings).

+Input/output operations per second (IOPS) refers to the number of read and write operations that a disk can perform in a second. Throughput refers to the amount of data that can be transferred in a given time period.

+For guidance on a designing an enterprise-scale implementation of AKS, see [Plan your AKS design][plan-aks-design].

+[plan-aks-design]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: Cluster autoscaling in Azure Kubernetes Service (AKS) overview

+description: Learn about cluster autoscaling in Azure Kubernetes Service (AKS) using the cluster autoscaler.

+# Cluster autoscaling in Azure Kubernetes Service (AKS) overview

+To keep up with application demands in Azure Kubernetes Service (AKS), you might need to adjust the number of nodes that run your workloads. The cluster autoscaler component watches for pods in your cluster that can't be scheduled because of resource constraints. When the cluster autoscaler detects issues, it scales up the number of nodes in the node pool to meet the application demand. It also regularly checks nodes for a lack of running pods and scales down the number of nodes as needed.

+This article helps you understand how the cluster autoscaler works in AKS. It also provides guidance, best practices, and considerations when configuring the cluster autoscaler for your AKS workloads. If you want to enable, disable, or update the cluster autoscaler for your AKS workloads, see [Use the cluster autoscaler in AKS](./cluster-autoscaler.md).

+## About the cluster autoscaler

+Clusters often need a way to scale automatically to adjust to changing application demands, such as between workdays and evenings or weekends. AKS clusters can scale in the following ways:

+* The **cluster autoscaler** periodically checks for pods that can't be scheduled on nodes because of resource constraints. The cluster then automatically increases the number of nodes. Manual scaling is disabled when you use the cluster autoscaler. For more information, see [How does scale up work?](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-does-scale-up-work).

+* The **[Horizontal Pod Autoscaler][horizontal-pod-autoscaler]** uses the Metrics Server in a Kubernetes cluster to monitor the resource demand of pods. If an application needs more resources, the number of pods is automatically increased to meet the demand.

+* The **[Vertical Pod Autoscaler][vertical-pod-autoscaler]** automatically sets resource requests and limits on containers per workload based on past usage to ensure pods are scheduled onto nodes that have the required CPU and memory resources.

+It's a common practice to enable cluster autoscaler for nodes and either the Vertical Pod Autoscaler or Horizontal Pod Autoscaler for pods. When you enable the cluster autoscaler, it applies the specified scaling rules when the node pool size is lower than the minimum or greater than the maximum. The cluster autoscaler waits to take effect until a new node is needed in the node pool or until a node might be safely deleted from the current node pool. For more information, see [How does scale down work?](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-does-scale-down-work)

+## Best practices and considerations

+* When implementing **availability zones with the cluster autoscaler**, we recommend using a single node pool for each zone. You can set the `--balance-similar-node-groups` parameter to `True` to maintain a balanced distribution of nodes across zones for your workloads during scale up operations. When this approach isn't implemented, scale down operations can disrupt the balance of nodes across zones.

+* For **clusters with more than 400 nodes**, we recommend using Azure CNI or Azure CNI Overlay.

+* To **effectively run workloads concurrently on both Spot and Fixed node pools**, consider using [*priority expanders*](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-are-expanders). This approach allows you to schedule pods based on the priority of the node pool.

+* Exercise caution when **assigning CPU/Memory requests on pods**. The cluster autoscaler scales up based on pending pods rather than CPU/Memory pressure on nodes.

+* For **clusters concurrently hosting both long-running workloads, like web apps, and short/bursty job workloads**, we recommend separating them into distinct node pools with [Affinity Rules](./operator-best-practices-advanced-scheduler.md#node-affinity)/[expanders](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-are-expanders) or using [PriorityClass](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) to help prevent unnecessary node drain or scale down operations.

+* We **don't recommend making direct changes to nodes in autoscaled node pools**. All nodes in the same node group should have uniform capacity, labels, and system pods running on them.

+* Nodes don't scale up if pods have a PriorityClass value below -10. Priority -10 is reserved for [overprovisioning pods](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler). For more information, see [Using the cluster autoscaler with Pod Priority and Preemption](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-does-cluster-autoscaler-work-with-pod-priority-and-preemption).

+* **Don't combine other node autoscaling mechanisms**, such as Virtual Machine Scale Set autoscalers, with the cluster autoscaler.

+* The cluster autoscaler **might be unable to scale down if pods can't move, such as in the following situations**:

+ * A directly created pod not backed by a controller object, such as a Deployment or ReplicaSet.

+ * A pod disruption budget (PDB) that's too restrictive and doesn't allow the number of pods to fall below a certain threshold.

+ * A pod uses node selectors or anti-affinity that can't be honored if scheduled on a different node.

+ For more information, see [What types of pods can prevent the cluster autoscaler from removing a node?](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-types-of-pods-can-prevent-ca-from-removing-a-node).

+## Cluster autoscaler profile

+The [cluster autoscaler profile](./cluster-autoscaler.md#cluster-autoscaler-profile-settings) is a set of parameters that control the behavior of the cluster autoscaler. You can configure the cluster autoscaler profile when you create a cluster or update an existing cluster.

+### Optimizing the cluster autoscaler profile

+You should fine-tune the cluster autoscaler profile settings according to your specific workload scenarios while also considering tradeoffs between performance and cost. This section provides examples that demonstrate those tradeoffs.

+It's important to note that the cluster autoscaler profile settings are cluster-wide and applied to all autoscale-enabled node pools. Any scaling actions that take place in one node pool can affect the autoscaling behavior of other node pools, which can lead to unexpected results. Make sure you apply consistent and synchronized profile configurations across all relevant node pools to ensure you get your desired results.

+#### Example 1: Optimizing for performance

+For clusters that handle substantial and bursty workloads with a primary focus on performance, we recommend increasing the `scan-interval` and decreasing the `scale-down-utilization-threshold`. These settings help batch multiple scaling operations into a single call, optimizing scaling time and the utilization of compute read/write quotas. It also helps mitigate the risk of swift scale down operations on underutilized nodes, enhancing the pod scheduling efficiency.

+For clusters with daemonset pods, we recommend setting `ignore-daemonset-utilization` to `true`, which effectively ignores node utilization by daemonset pods and minimizes unnecessary scale down operations.

+#### Example 2: Optimizing for cost

+If you want a cost-optimized profile, we recommend setting the following parameter configurations:

+* Reduce `scale-down-unneeded-time`, which is the amount of time a node should be unneeded before it's eligible for scale down.

+* Reduce `scale-down-delay-after-add`, which is the amount of time to wait after a node is added before considering it for scale down.

+* Increase `scale-down-utilization-threshold`, which is the utilization threshold for removing nodes.

+* Increase `max-empty-bulk-delete`, which is the maximum number of nodes that can be deleted in a single call.

+## Common issues and mitigation recommendations

+### Not triggering scale up operations

+| Common causes | Mitigation recommendations |

+|--|--|

+| PersistentVolume node affinity conflicts, which can arise when using the cluster autoscaler with multiple availability zones or when a pod's or persistent volume's zone differs from the node's zone. | Use one node pool per availability zone and enabling `--balance-similar-node-groups`. You can also set the [`volumeBindingMode` field to `WaitForFirstConsumer`](./azure-disk-csi.md#create-a-custom-storage-class) in the pod specification to prevent the volume from being bound to a node until a pod using the volume is created. |

+| Taints and Tolerations/Node affinity conflicts | Assess the taints assigned to your nodes and review the tolerations defined in your pods. If necessary, make adjustments to the [taints and tolerations](./operator-best-practices-advanced-scheduler.md#provide-dedicated-nodes-using-taints-and-tolerations) to ensure that your pods can be efficiently scheduled on your nodes. |

+### Scale up operation failures

+| Common causes | Mitigation recommendations |

+|--|--|

+| IP address exhaustion in the subnet | Add another subnet in the same virtual network and add another node pool into the new subnet. |

+| Core quota exhaustion | Approved core quota has been exhausted. [Request a quota increase](../quotas/quickstart-increase-quota-portal.md). The cluster autoscaler enters an [exponential backoff state](#node-pool-in-backoff) within the specific node group when it experiences multiple failed scale up attempts. |

+| Max size of node pool | Increase the max nodes on the node pool or create a new node pool. |

+| Requests/Calls exceeding the rate limit | See [429 Too Many Requests errors](/troubleshoot/azure/azure-kubernetes/429-too-many-requests-errors). |

+### Scale down operation failures

+| Common causes | Mitigation recommendations |

+|--|--|

+| Pod preventing node drain/Unable to evict pod |ΓÇó View [what types of pods can prevent scale down](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-types-of-pods-can-prevent-ca-from-removing-a-node). <br> ΓÇó For pods using local storage, such as hostPath and emptyDir, set the cluster autoscaler profile flag `skip-nodes-with-local-storage` to `false`. <br> ΓÇó In the pod specification, set the `cluster-autoscaler.kubernetes.io/safe-to-evict` annotation to `true`. <br> ΓÇó Check your [PDB](https://kubernetes.io/docs/tasks/run-application/configure-pdb/), as it might be restrictive. |

+| Min size of node pool | Reduce the minimum size of the node pool. |

+| Requests/Calls exceeding the rate limit | See [429 Too Many Requests errors](/troubleshoot/azure/azure-kubernetes/429-too-many-requests-errors). |

+| Write operations locked | Don't make any changes to the [fully managed AKS resource group](./cluster-configuration.md#fully-managed-resource-group-preview) (see [AKS support policies](./support-policies.md)). Remove or reset any [resource locks](../azure-resource-manager/management/lock-resources.md) you previously applied to the resource group. |

+### Other issues

+| Common causes | Mitigation recommendations |

+|--|--|

+| PriorityConfigMapNotMatchedGroup | Make sure that you add all the node groups requiring autoscaling to the [expander configuration file](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/priority/readme.md#configuration). |

+### Node pool in backoff

+Node pool in backoff was introduced in version 0.6.2 and causes the cluster autoscaler to back off from scaling a node pool after a failure.

+Depending on how long the scaling operations have been experiencing failures, it may take up to 30 minutes before making another attempt. You can reset the node pool's backoff state by disabling and then re-enabling autoscaling.

+<!-- LINKS >

+[vertical-pod-autoscaler]: vertical-pod-autoscaler.md

+[horizontal-pod-autoscaler]:concepts-scale.md#horizontal-pod-autoscaler

+description: Learn how to use the cluster autoscaler to automatically scale your Azure Kubernetes Service (AKS) workloads to meet application demands.

+# Use the cluster autoscaler in Azure Kubernetes Service (AKS)

+To keep up with application demands in AKS, you might need to adjust the number of nodes that run your workloads. The cluster autoscaler component watches for pods in your cluster that can't be scheduled because of resource constraints. When the cluster autoscaler detects issues, it scales up the number of nodes in the node pool to meet the application demands. It also regularly checks nodes for a lack of running pods and scales down the number of nodes as needed.

+This article shows you how to enable and manage the cluster autoscaler in AKS, which is based on the [open-source Kubernetes version][kubernetes-cluster-autoscaler].

+## Use the cluster autoscaler on an AKS cluster

+> [!IMPORTANT]

+> The cluster autoscaler is a Kubernetes component. Although the AKS cluster uses a virtual machine scale set for the nodes, don't manually enable or edit settings for scale set autoscaling. Let the Kubernetes cluster autoscaler manage the required scale settings. For more information, see [Can I modify the AKS resources in the node resource group?][aks-faq-node-resource-group]

+> You can manually scale your cluster after disabling the cluster autoscaler using the [`az aks scale`][az-aks-scale] command. If you use the horizontal pod autoscaler, it continues to run with the cluster autoscaler disabled, but pods might end up unable to be scheduled if all node resources are in use.

+### Re-enable the cluster autoscaler on a cluster

+## Use the cluster autoscaler on node pools

+### Use the cluster autoscaler on multiple node pools

+You can use the cluster autoscaler with [multiple node pools][aks-multiple-node-pools] and can enable the cluster autoscaler on each individual node pool and pass unique autoscaling rules to them.

+* Update the settings on an existing node pool using the [`az aks nodepool update`][az-aks-nodepool-update] command.

+ az aks nodepool update \

+ --cluster-name myAKSCluster \

+ --name nodepool1 \

+### Disable the cluster autoscaler on a node pool

+* Disable the cluster autoscaler on a node pool using the [`az aks nodepool update`][az-aks-nodepool-update] command and the `--disable-cluster-autoscaler` parameter.

+ ```azurecli-interactive

+ az aks nodepool update \

+ --resource-group myResourceGroup \

+ --cluster-name myAKSCluster \

+ --name nodepool1 \

+ --disable-cluster-autoscaler

+ ```

+### Re-enable the cluster autoscaler on a node pool

+You can re-enable the cluster autoscaler on a node pool using the [`az aks nodepool update`][az-aks-nodepool-update] command and specifying the `--enable-cluster-autoscaler`, `--min-count`, and `--max-count` parameters.

+> [!NOTE]

+> If you plan on using the cluster autoscaler with node pools that span multiple zones and leverage scheduling features related to zones, such as volume topological scheduling, we recommend you have one node pool per zone and enable `--balance-similar-node-groups` through the autoscaler profile. This ensures the autoscaler can successfully scale up and keep the sizes of the node pools balanced.

+## Update the cluster autoscaler settings

+As your application demands change, you might need to adjust the cluster autoscaler node count to scale efficiently.

+* Change the node count using the [`az aks update`][az-aks-update] command and update the cluster autoscaler using the `--update-cluster-autoscaler` parameter and specifying your updated node `--min-count` and `--max-count`.

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --update-cluster-autoscaler \

+ --min-count 1 \

+ --max-count 5

+> [!NOTE]

+> The cluster autoscaler enforces the minimum count in cases where the actual count drops below the minimum due to external factors, such as during a spot eviction or when changing the minimum count value from the AKS API.

+## Use the cluster autoscaler profile

+You can configure more granular details of the cluster autoscaler by changing the default values in the cluster-wide autoscaler profile. For example, a scale down event happens after nodes are under-utilized after 10 minutes. If you have workloads that run every 15 minutes, you might want to change the autoscaler profile to scale down under-utilized nodes after 15 or 20 minutes. When you enable the cluster autoscaler, a default profile is used unless you specify different settings.

+> [!IMPORTANT]

+> The cluster autoscaler profile affects **all node pools** that use the cluster autoscaler. You can't set an autoscaler profile per node pool. When you set the profile, any existing node pools with the cluster autoscaler enabled immediately start using the profile.

+### Cluster autoscaler profile settings

+The following table lists the available settings for the cluster autoscaler profile:

+|Setting |Description |Default value |

+|--||--|

+| `scan-interval` | How often the cluster is reevaluated for scale up or down. | 10 seconds |

+| `scale-down-delay-after-add` | How long after scale up that scale down evaluation resumes. | 10 minutes |

+| `scale-down-delay-after-delete` | How long after node deletion that scale down evaluation resumes. | `scan-interval` |

+| `scale-down-delay-after-failure` | How long after scale down failure that scale down evaluation resumes. | Three minutes |

+| `scale-down-unneeded-time` | How long a node should be unneeded before it's eligible for scale down. | 10 minutes |

+| `scale-down-unready-time` | How long an unready node should be unneeded before it's eligible for scale down. | 20 minutes |

+| `ignore-daemonsets-utilization` (Preview) | Whether DaemonSet pods will be ignored when calculating resource utilization for scale down. | `false` |

+| `daemonset-eviction-for-empty-nodes` (Preview) | Whether DaemonSet pods will be gracefully terminated from empty nodes. | `false` |

+| `daemonset-eviction-for-occupied-nodes` (Preview) | Whether DaemonSet pods will be gracefully terminated from non-empty nodes. | `true` |

+| `scale-down-utilization-threshold` | Node utilization level, defined as sum of requested resources divided by capacity, in which a node can be considered for scale down. | 0.5 |

+| `max-graceful-termination-sec` | Maximum number of seconds the cluster autoscaler waits for pod termination when trying to scale down a node. | 600 seconds |

+| `balance-similar-node-groups` | Detects similar node pools and balances the number of nodes between them. | `false` |

+| `expander` | Type of node pool [expander](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-are-expanders) uses in scale up. Possible values include `most-pods`, `random`, `least-waste`, and `priority`. | |

+| `skip-nodes-with-local-storage` | If `true`, cluster autoscaler doesn't delete nodes with pods with local storage, for example, EmptyDir or HostPath. | `true` |

+| `skip-nodes-with-system-pods` | If `true`, cluster autoscaler doesn't delete nodes with pods from kube-system (except for DaemonSet or mirror pods). | `true` |

+| `max-empty-bulk-delete` | Maximum number of empty nodes that can be deleted at the same time. | 10 nodes |

+| `new-pod-scale-up-delay` | For scenarios such as burst/batch scale where you don't want CA to act before the Kubernetes scheduler could schedule all the pods, you can tell CA to ignore unscheduled pods before they reach a certain age. | 0 seconds |

+| `max-total-unready-percentage` | Maximum percentage of unready nodes in the cluster. After this percentage is exceeded, CA halts operations. | 45% |

+| `max-node-provision-time` | Maximum time the autoscaler waits for a node to be provisioned. | 15 minutes |

+| `ok-total-unready-count` | Number of allowed unready nodes, irrespective of max-total-unready-percentage. | Three nodes |

+### [Azure portal](#tab/azure-portal)

+* Navigate to *Node pools* from your cluster's overview page in the Azure portal. Select any of the tiles for autoscale events, autoscale warnings, or scale ups not triggered to get more details.

+ :::image type="content" source="./media/cluster-autoscaler/main-blade-tiles-inline.png" alt-text="Screenshot of the Azure portal page for a cluster's node pools. The section displaying autoscaler events, warning, and scale ups not triggered is highlighted." lightbox="./media/cluster-autoscaler/main-blade-tiles.png":::

+ This shows a list of Kubernetes events filtered to `source: cluster-autoscaler` that have occurred within the last hour. You can use this information to troubleshoot and diagnose any issues that might arise while scaling your nodes.

+For more information, see the [Kubernetes/autoscaler GitHub project FAQ][kubernetes-faq].

+ if [[ $PVC_CREATION_TIME >= $STARTTIMESTAMP ]]; then

+ * `startTimeStamp` - Provide a start time **before** PVC creation time in the format **yyyy-mm-ddthh:mm:ssz**

+ Title: 'Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using the Bicep extensibility Kubernetes provider'

+description: Learn how to quickly deploy a Kubernetes cluster using the Bicep extensibility Kubernetes provider and deploy an application in Azure Kubernetes Service (AKS).

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+# Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using the Bicep extensibility Kubernetes provider (preview)

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: 'Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using Bicep'

+description: Learn how to quickly deploy a Kubernetes cluster using a Bicep file and deploy an application in Azure Kubernetes Service (AKS).

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+* To deploy a Bicep file, you need write access on the resources you create and access to all operations on the `Microsoft.Resources/deployments` resource type. For example, to create a virtual machine, you need `Microsoft.Compute/virtualMachines/write` and `Microsoft.Resources/deployments/*` permissions. For a list of roles and permissions, see [Azure built-in roles](../../role-based-access-control/built-in-roles.md).

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in Azure Kubernetes Service (AKS) using Azure CLI.

+#Customer intent: As a developer or cluster operator, I want to deploy an AKS cluster and deploy an application so I can see how to run applications using the managed Kubernetes service in Azure.

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[intro-azure-linux]: ../../azure-linux/intro-azure-linux.md

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in Azure Kubernetes Service (AKS) using the Azure portal.

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run and monitor applications using the managed Kubernetes service in Azure.

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in Azure Kubernetes Service (AKS) using PowerShell.

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: 'Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using an ARM template'

+description: Learn how to quickly deploy a Kubernetes cluster using an Azure Resource Manager template and deploy an application in Azure Kubernetes Service (AKS).

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: 'Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using Terraform'

+description: Learn how to quickly deploy a Kubernetes cluster using Terraform and deploy an application in Azure Kubernetes Service (AKS).

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy an application so that I can see how to run applications using the managed Kubernetes service in Azure.

+# Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using Terraform

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using Azure CLI

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in a Windows Server container in Azure Kubernetes Service (AKS) using Azure CLI.

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy a Windows Server container so that I can see how to run applications running on a Windows Server container using the managed Kubernetes service in Azure.

+# Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using Azure CLI

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+This quickstart assumes a basic understanding of Kubernetes concepts. For more information, see [Kubernetes core concepts for Azure Kubernetes Service (AKS)](../concepts-clusters-workloads.md).

+- This quickstart requires version 2.0.64 or later of the Azure CLI. If you are using Azure Cloud Shell, then the latest version is already installed.

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using the Azure portal

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in a Windows Server container in Azure Kubernetes Service (AKS) using the Azure portal.

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy a Windows Server container so that I can see how to run applications running on a Windows Server container using the managed Kubernetes service in Azure.

+# Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using the Azure portal

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+This quickstart assumes a basic understanding of Kubernetes concepts. For more information, see [Kubernetes core concepts for Azure Kubernetes Service (AKS)](../concepts-clusters-workloads.md).

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+ Title: Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using PowerShell

+description: Learn how to quickly deploy a Kubernetes cluster and deploy an application in a Windows Server container in Azure Kubernetes Service (AKS) using PowerShell.

+#Customer intent: As a developer or cluster operator, I want to quickly deploy an AKS cluster and deploy a Windows Server container so that I can see how to run applications running on a Windows Server container using the managed Kubernetes service in Azure.

+# Deploy a Windows Server container on an Azure Kubernetes Service (AKS) cluster using PowerShell

+> [!NOTE]

+> To get started with quickly provisioning an AKS cluster, this article includes steps to deploy a cluster with default settings for evaluation purposes only. Before deploying a production-ready cluster, we recommend that you familiarize yourself with our [baseline reference architecture][baseline-reference-architecture] to consider how it aligns with your business requirements.

+This quickstart assumes a basic understanding of Kubernetes concepts. For more information, see [Kubernetes core concepts for Azure Kubernetes Service (AKS)](../concepts-clusters-workloads.md).

+[aks-solution-guidance]: /azure/architecture/reference-architectures/containers/aks-start-here?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+[baseline-reference-architecture]: /azure/architecture/reference-architectures/containers/aks/baseline-aks?toc=/azure/aks/toc.json&bc=/azure/aks/breadcrumb/toc.json

+For more information, see [use the cluster autoscaler](cluster-autoscaler.md#use-the-cluster-autoscaler-on-multiple-node-pools).

+ memory: 85Mi

+ memory: 85Mi

+You can also autoscale the nodes in your cluster. For more information, see [Use the cluster autoscaler with node pools](./cluster-autoscaler.md#use-the-cluster-autoscaler-on-node-pools).

+In AKS, you can create node pools that run Linux or Windows Server as the operating system (OS) on the nodes. Windows Server nodes can run native Windows container applications, such as .NET Framework. The Linux OS and Windows OS have different container support and configuration considerations. For more information, see [Windows container considerations in Kubernetes][windows-vs-linux]. To learn more about how various industries are using Windows containers on AKS, see [Windows AKS customer stories](./windows-aks-customer-stories.md).

+Windows Server 2022 is the default OS for Kubernetes version 1.25 and later. Windows Server 2019 will retire after Kubernetes version 1.32 reaches end of service and won't be supported in future releases. For more information, see the [AKS release notes][aks-release-notes].

+ Title: Windows container considerations in Azure Kubernetes Service

+description: See the Windows container considerations with Azure Kubernetes Service (AKS).

+# Windows container considerations with Azure Kubernetes Service

+When you create deployments that use Windows Server containers on Azure Kubernetes Service (AKS), there are a few differences relative to Linux deployments you should keep in mind. For a detailed comparison of the differences between Windows and Linux in upstream Kubernetes, see [Windows containers in Kubernetes](https://kubernetes.io/docs/concepts/windows/intro/).

+> [!TIP]

+> If you want to load-balance requests across multiple services and endpoints, consider configuring a [load-balanced backend pool](backends.md#load-balanced-pool-preview).

+description: Learn about custom backends in Azure API Management

+API Management supports custom backends so you can manage the backend services of your API. Use custom backends for one or more of the following:

+* Authorize the credentials of requests to the backend service

+* Protect your backend from too many requests

+* Route or load-balance requests to multiple backends

+Configure and manage custom backends in the Azure portal, or using Azure APIs or tools.

+## Reference backend using set-backend-service policy

+After creating a backend, you can reference the backend in your APIs. Use the [`set-backend-service`](set-backend-service-policy.md) policy to direct an incoming API request to the custom backend. If you already configured a backend web service for an API, you can use the `set-backend-service` policy to redirect the request to a custom backend instead of the default backend web service configured for that API. For example:

+```xml

+<policies>

+ <inbound>

+ <base />

+ <set-backend-service backend-id="myBackend" />

+ </inbound>

+ [...]

+<policies/>

+```

+You can use conditional logic with the `set-backend-service` policy to change the effective backend based on location, gateway that was called, or other expressions.

+For example, here is a policy to route traffic to another backend based on the gateway that was called:

+```xml

+<policies>

+ <inbound>

+ <base />

+ <choose>

+ <when condition="@(context.Deployment.Gateway.Id == "factory-gateway")">

+ <set-backend-service backend-id="backend-on-prem" />

+ </when>

+ <when condition="@(context.Deployment.Gateway.IsManaged == false)">

+ <set-backend-service backend-id="self-hosted-backend" />

+ </when>

+ <otherwise />

+ </choose>

+ </inbound>

+ [...]

+<policies/>

+```

+Use the API Management [REST API](/rest/api/apimanagement/backend) or a Bicep or ARM template to configure a circuit breaker in a backend. In the following example, the circuit breaker in *myBackend* in the API Management instance *myAPIM* trips when there are three or more `5xx` status codes indicating server errors in a day. The circuit breaker resets after one hour.

+Include a snippet similar to the following in your Bicep template for a backend resource with a circuit breaker:

+ name: 'myAPIM/myBackend'

+ }

+ }

+Include a JSON snippet similar to the following in your ARM template for a backend resource with a circuit breaker:

+ "name": "myAPIM/myBackend",

+## Load-balanced pool (preview)

+Starting in API version 2023-05-01 preview, API Management supports backend *pools*, when you want to implement multiple backends for an API and load-balance requests across those backends. Currently, the backend pool supports round-robin load balancing.

+Use a backend pool for scenarios such as the following:

+* Spread the load to multiple backends, which may have individual backend circuit breakers.

+* Shift the load from one set of backends to another for upgrade (blue-green deployment).

+To create a backend pool, set the `type` property of the backend to `pool` and specify a list of backends that make up the pool.

+> [!NOTE]

+> Currently, you can only include single backends in a backend pool. You can't add a backend of type `pool` to another backend pool.

+### Example

+Use the API Management [REST API](/rest/api/apimanagement/backend) or a Bicep or ARM template to configure a backend pool. In the following example, the backend *myBackendPool* in the API Management instance *myAPIM* is configured with a backend pool. Example backends in the pool are named *backend-1* and *backend-2*.

+#### [Bicep](#tab/bicep)

+Include a snippet similar to the following in your Bicep template for a backend resource with a load-balanced pool:

+```bicep

+resource symbolicname 'Microsoft.ApiManagement/service/backends@2023-05-01-preview' = {

+ name: 'myAPIM/myBackendPool'

+ properties: {

+ description: 'Load balancer for multiple backends'

+ type: 'Pool'

+ protocol: 'http'

+ url: 'http://unused'

+ pool: {

+

+ {

+ id: '/backends/backend-1'

+ }

+ {

+ id: '/backends/backend-2'

+ }

+ ]

+ }

+ }

+}

+```

+#### [ARM](#tab/arm)

+Include a JSON snippet similar to the following in your ARM template for a backend resource with a load-balanced pool:

+```json

+{

+ "type": "Microsoft.ApiManagement/service/backends",

+ "apiVersion": "2023-05-01-preview",

+ "name": "myAPIM/myBackendPool",

+ "properties": {

+ "description": "Load balancer for multiple backends",

+ "type": "Pool",

+ "protocol": "http",

+ "url": "http://unused",

+ "pool": {

+ "services": [

+ {

+ "id": "/backends/backend-1"

+ },

+ {

+ "id": "/backends/backend-2"

+ }

+ ]

+ }

+ }

+}

+```

+## Related content

+zone_pivot_groups: app-service-platform-windows-linux-windows-container

+Use the following button to deploy on **Windows**:

+[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.web%2Fapp-service-docs-windows%2Fazuredeploy.json)

+Use the following button to deploy on **Windows container**:

+[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.web%2Fapp-service-docs-windows-container%2Fazuredeploy.json)

+The template used in this quickstart is from [Azure Quickstart Templates](/samples/azure/azure-quickstart-templates/app-service-docs-windows-container/). It deploys an App Service plan and an App Service app on a Windows container.

+Two Azure resources are defined in the template:

+* [**Microsoft.Web/serverfarms**](/azure/templates/microsoft.web/serverfarms): create an App Service plan.

+* [**Microsoft.Web/sites**](/azure/templates/microsoft.web/sites): create an App Service app.

+This template contains several parameters that are predefined for your convenience. See the table below for parameter defaults and their descriptions:

+| Parameters | Type | Default value | Description |

+||||-|

+| webAppName | string | "webApp-**[`<uniqueString>`](../azure-resource-manager/templates/template-functions-string.md#uniquestring)**" | App name |

+| appServicePlanName | string | "webAppPlan-**[`<uniqueString>`](../azure-resource-manager/templates/template-functions-string.md#uniquestring)**" | App Service Plan name |

+| location | string | "[[resourceGroup().location](../azure-resource-manager/templates/template-functions-resource.md#resourcegroup)]" | App region |

+| skuTier | string | "P1v3" | Instance size ([View available SKUs](configure-custom-container.md?tabs=debian&pivots=container-windows#customize-container-memory)) |

+| appSettings | string | "[{"name": "PORT","value": "8080"}]" | App Service listening port. Needs to be 8080. |

+| kind | string | "windows" | External Git repo (optional) |

+| hyperv | string | "true" | External Git repo (optional) |

+| windowsFxVersion | string | "DOCKER&#124;mcr.microsoft.com/dotnet/samples:aspnetapp" | External Git repo (optional) |

+az deployment group create --resource-group myResourceGroup --parameters webAppName="<app-name>" linuxFxVersion="PYTHON|3.9" \

+Run the code below to deploy a [.NET app](https://mcr.microsoft.com/product/dotnet/samples/tags) on a Windows container.

+```azurecli-interactive

+az group create --name myResourceGroup --location "southcentralus" &&

+az deployment group create --resource-group myResourceGroup \

+--parameters webAppName="<app-name>" \

+--template-uri "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.web/app-service-docs-windows-container/azuredeploy.json"

+To check this, open your App Configuration resource in the Azure portal and go to **Configuration explorer**.

+- * East US

+- East US

+- East US 2

+- Central US

+- North Central US

+- * Canada Central

+- Australia East

+- Japan East

+- East Asia

+- Central India

+### ESU prerequisites

+Ensure that both the licensing package and servicing stack update (SSU) are downloaded for the Azure Arc-enabled server as documented at [KB5031043: Procedure to continue receiving security updates after extended support has ended on October 10, 2023](https://support.microsoft.com/topic/kb5031043-procedure-to-continue-receiving-security-updates-after-extended-support-has-ended-on-october-10-2023-c1a20132-e34c-402d-96ca-1e785ed51d45). Ensure you are following all of the networking prerequisites as recorded at [Prepare to deliver Extended Security Updates for Windows Server 2012](prepare-extended-security-updates.md?tabs=azure-cloud#networking).

+### Error: Trying to check IMDS again (HRESULT 12002)

+If installing the Extended Security Update enabled by Azure Arc fails with errors such as "ESU: Trying to Check IMDS Again LastError=HRESULT_FROM_WIN32(12029)" or "ESU: Trying to Check IMDS Again LastError=HRESULT_FROM_WIN32(12002)", you may need to update the intermediate certificate authorities trusted by your computer using one of the following two methods:

+1. Configure your network firewall and/or proxy server to allow access from the Windows Server 2012 (R2) machines to `https://microsoft.com/pkiops/certs`. This will allow the machine to automatically retrieve updated intermediate certificates as required and is Microsoft's preferred approach.

+1. Download all intermediate CAs from a machine with internet access, copy them to each Windows Server 2012 (R2) machine, and import them to the machine's intermediate certificate authority store:

+ 1. Download the 4 intermediate CA certificates:

+ 1. [Microsoft Azure TLS Issuing CA 01](https://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20TLS%20Issuing%20CA%2001%20-%20xsign.crt)

+ 1. [Microsoft Azure TLS Issuing CA 02](https://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20TLS%20Issuing%20CA%2002%20-%20xsign.crt)

+ 1. [Microsoft Azure TLS Issuing CA 05](https://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20TLS%20Issuing%20CA%2005%20-%20xsign.crt)

+ 1. [Microsoft Azure TLS Issuing CA 06](https://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20TLS%20Issuing%20CA%2006%20-%20xsign.crt)

+ 1. Copy the certificate files to your Windows Server 2012 (R2) machine.

+ 1. Run the following commands in an elevated command prompt or PowerShell session to add the certificates to the "Intermediate Certificate Authorities" store for the local computer. The command should be run from the same directory as the certificate files. The commands are idempotent and won't make any changes if you've already imported the certificate:

+ ```powershell

+ certstore -addstore CA "Microsoft Azure TLS Issuing CA 01 - xsign.crt"

+ certstore -addstore CA "Microsoft Azure TLS Issuing CA 02 - xsign.crt"

+ certstore -addstore CA "Microsoft Azure TLS Issuing CA 05 - xsign.crt"

+ certstore -addstore CA "Microsoft Azure TLS Issuing CA 06 - xsign.crt"

+ ```

+After allowing the servers to reach the PKI URL or manually importing the intermediate certificates, try installing the Extended Security Updates again using Windows Update or your preferred patch management software. You may need to reboot your computer for the changes to take effect.

+### Error: Not eligible (HRESULT 1633)

+|Tier | Basic, Standard, Premium |Enterprise, Enterprise Flash |

+| |::|::|

+|Available | Yes | Yes |

+> For more information, see [How to export if I have firewall enabled on my storage account?](cache-how-to-import-export-data.md#how-to-export-if-i-have-firewall-enabled-on-my-storage-account)

+For **Basic, Standard, and Premium tier** caches, your application should connect to `<cachename>.redis.cache.windows.net` on port `6380`. A private DNS zone, named `*.privatelink.redis.cache.windows.net`, is automatically created in your subscription. The private DNS zone is vital for establishing the TLS connection with the private endpoint. We recommend avoiding the use of `<cachename>.privatelink.redis.cache.windows.net` in configuration or connection string.

+For **Enterprise and Enterprise Flash** tier caches, your application should connect to `<cachename>.<region>.redisenterprise.cache.azure.net` on port `10000`.

+- Private endpoints can't be used with your cache instance if your cache is already a VNet injected cache.

+- On Premium tier caches, you have a limit of one private link for clustered caches. Enterprise and Enterprise Flash tier caches do not have this limitation for clustered caches. For all other caches, your limit is 100 private links.

+- You try to [persist data to storage account](cache-how-to-premium-persistence.md) where firewall rules are applied might prevent you from creating the Private Link.

+- Private links can't be added to caches that are already using [passive geo-replication](cache-how-to-geo-replication.md) in the Premium tier. To add a private link to a geo-replicated cache: 1. Unlink the geo-replication. 2. Add a Private Link. 3. Last, relink the geo-replication. (Enterprise tier caches using [active geo-replication](cache-how-to-active-geo-replication.md) do not have this restriction.)

+ In this article, only double-underscores are used, since they're supported on both operating systems. Most of the settings that support managed identity connections use double-underscores.

+> [!IMPORTANT]

+> Azure Functions proxies is a legacy feature for [versions 1.x through 3.x](functions-versions.md) of the Azure Functions runtime. For more information about legacy support in version 4.x, see [Functions proxies](functions-proxies.md).

+By default, Functions proxies use a shortcut to send API calls from proxies directly to functions in the same function app. This shortcut is used instead of creating a new HTTP request. This setting allows you to disable that shortcut behavior.

+> [!IMPORTANT]

+> Azure Functions proxies is a legacy feature for [versions 1.x through 3.x](functions-versions.md) of the Azure Functions runtime. For more information about legacy support in version 4.x, see [Functions proxies](functions-proxies.md).

+Configures the runtime [hosting environment](/dotnet/api/microsoft.extensions.hosting.environments) of the function app when running in Azure. This value is read during initialization, and only these values are honored by the runtime:

+| Value | Description |

+| | |

+| `Production` | Represents a production environment, with reduced logging and full performance optimizations. This is the default when `AZURE_FUNCTIONS_ENVIRONMENT` either isn't set or is set to an unsupported value. |

+| `Staging` | Represents a staging environment, such as when running in a [staging slot](functions-deployment-slots.md). |

+| `Development` | A development environment supports more verbose logging and other reduced performance optimizations. The Azure Functions Core Tools sets `AZURE_FUNCTIONS_ENVIRONMENT` to `Development` when running on your local computer. This setting can't be overridden in the local.settings.json file. |

+Use this setting instead of `ASPNETCORE_ENVIRONMENT` when you need to change the runtime environment in Azure to something other than `Production`. For more information, see [Environment-based Startup class and methods](/aspnet/core/fundamentals/environments#environments).

+This setting isn't available in version 1.x of the Functions runtime.

+_This setting is deprecated and is only supported when running on version 1.x of the Azure Functions runtime._

+Optional storage account connection string for storing logs and displaying them in the **Monitor** tab in the portal. The storage account must be a general-purpose one that supports blobs, queues, and tables. To learn more, see [Storage account requirements](storage-considerations.md#storage-account-requirements).

+Specifies the connection string for an Azure Storage account that the Functions runtime uses for normal operations. Some uses of this storage account by Functions include key management, timer trigger management, and Event Hubs checkpoints. The storage account must be a general-purpose one that supports blobs, queues, and tables. For more information, see [Storage account requirements](storage-considerations.md#storage-account-requirements).

+Instead of a connection string, you can use an identity based connection for this storage account. For more information, see [Connecting to host storage with an identity](functions-reference.md#connecting-to-host-storage-with-an-identity).

+## AzureWebJobsStorage__accountName

+When using an identity-based storage connection, sets the account name of the storage account instead of using the connection string in `AzureWebJobsStorage`. This syntax is unique to `AzureWebJobsStorage` and can't be used for other identity-based connections.

+|Key|Sample value|

+|||

+|AzureWebJobsStorage__accountName|`<STORAGE_ACCOUNT_NAME>`|

+For sovereign clouds or when using a custom DNS, you must instead use the service-specific `AzureWebJobsStorage__*ServiceUri` settings.

+## AzureWebJobsStorage__blobServiceUri

+When using an identity-based storage connection, sets the data plane URI of the blob service of the storage account.

+|Key|Sample value|

+|||

+|AzureWebJobsStorage__blobServiceUri|`https://<STORAGE_ACCOUNT_NAME>.blob.core.windows.net`|

+Use this setting instead of `AzureWebJobsStorage__accountName` in sovereign clouds or when using a custom DNS. For more information, see [Connecting to host storage with an identity](functions-reference.md#connecting-to-host-storage-with-an-identity).

+## AzureWebJobsStorage__queueServiceUri

+When using an identity-based storage connection, sets the data plane URI of the queue service of the storage account.

+|Key|Sample value|

+|||

+|AzureWebJobsStorage__queueServiceUri|`https://<STORAGE_ACCOUNT_NAME>.queue.core.windows.net`|

+Use this setting instead of `AzureWebJobsStorage__accountName` in sovereign clouds or when using a custom DNS. For more information, see [Connecting to host storage with an identity](functions-reference.md#connecting-to-host-storage-with-an-identity).

+## AzureWebJobsStorage__tableServiceUri

+When using an identity-based storage connection, sets data plane URI of a table service of the storage account.

+|Key|Sample value|

+|||

+|AzureWebJobsStorage__tableServiceUri|`https://<STORAGE_ACCOUNT_NAME>.table.core.windows.net`|

+Use this setting instead of `AzureWebJobsStorage__accountName` in sovereign clouds or when using a custom DNS. For more information, see [Connecting to host storage with an identity](functions-reference.md#connecting-to-host-storage-with-an-identity).

+## DOCKER_REGISTRY_SERVER_PASSWORD

+Indicates the password used to access a private container registry. This setting is only required when deploying your containerized function app from a private container registry. For more information, see [Environment variables and app settings in Azure App Service](../app-service/reference-app-settings.md#custom-containers).

+## DOCKER_REGISTRY_SERVER_URL

+Indicates the URL of a private container registry. This setting is only required when deploying your containerized function app from a private container registry. For more information, see [Environment variables and app settings in Azure App Service](../app-service/reference-app-settings.md#custom-containers).

+## DOCKER_REGISTRY_SERVER_USERNAME

+Indicates the account used to access a private container registry. This setting is only required when deploying your containerized function app from a private container registry. For more information, see [Environment variables and app settings in Azure App Service](../app-service/reference-app-settings.md#custom-containers).

+Indicates whether you're able to edit your function app in the Azure portal. Valid values are `readwrite` and `readonly`.

+The value is set by the runtime based on the language stack and deployment status of your function app. For more information, see [Development limitations in the Azure portal](functions-how-to-use-azure-function-app-settings.md#development-limitations-in-the-azure-portal).

+> This setting is no longer supported. It was originally provided to enable a short-term workaround for apps that targeted the v2.x runtime to be able to instead run on the v3.x runtime while it was still supported. Except for legacy apps that run on version 1.x, all function apps must run on version 4.x of the Functions runtime: `FUNCTIONS_EXTENSION_VERSION=~4`. For more information, see [Azure Functions runtime versions overview](functions-versions.md).

+Specifies the maximum number of language worker processes, with a default value of `1`. The maximum value allowed is `10`. Function invocations are evenly distributed among language worker processes. Language worker processes are spawned every 10 seconds until the count set by `FUNCTIONS_WORKER_PROCESS_COUNT` is reached. Using multiple language worker processes isn't the same as [scaling](functions-scale.md). Consider using this setting when your workload has a mix of CPU-bound and I/O-bound invocations. This setting applies to all language runtimes, except for .NET running in process (`FUNCTIONS_WORKER_RUNTIME=dotnet`).

+The language or language stack of the worker runtime to load in the function app. This corresponds to the language being used in your application (for example, `python`). Starting with version 2.x of the Azure Functions runtime, a given function app can only support a single language.

+| Value | Language/language stack |

+This setting is required for Consumption and Elastic Premium plan apps running on both Windows and Linux. It's not required for Dedicated plan apps, which aren't dynamically scaled by Functions.

+Azure Files doesn't support using managed identity when accessing the file share. For more information, see [Azure Files supported authentication scenarios](../storage/files/storage-files-active-directory-overview.md#supported-authentication-scenarios).

+The name of the file share that Functions uses to store function app code and configuration files. This content is required by event-driven scaling plans. Used with `WEBSITE_CONTENTAZUREFILECONNECTIONSTRING`. Default is a unique string generated by the runtime, which begins with the function app name. For more information, see [Storage account connection setting](storage-considerations.md#storage-account-connection-setting).

+The share is created when your function app is created. Changing or removing this setting can cause your function app to not start. To learn more, see [this troubleshooting article](functions-recover-storage-account.md#storage-account-application-settings-were-deleted).

+The following considerations apply when using an Azure Resource Manager (ARM) template or Bicep file to create a function app during deployment:

+The [WEBSITE_CONTENTAZUREFILECONNECTIONSTRING](#website_contentazurefileconnectionstring) and [WEBSITE_CONTENTSHARE](#website_contentshare) settings have extra validation checks to ensure that the app can be properly started. Creation of application settings fail when the function app can't properly call out to the downstream Storage Account or Key Vault due to networking constraints or other limiting factors. When WEBSITE_SKIP_CONTENTSHARE_VALIDATION is set to `1`, the validation check is skipped; otherwise the value defaults to `0` and the validation takes place.

+## WEBSITES_ENABLE_APP_SERVICE_STORAGE

+Indicates whether the `/home` directory is shared across scaled instances, with a default value of `true`. You should set this to `false` when deploying your function app in a container. d

+The Azure Functions host uses the storage connection set in [`AzureWebJobsStorage`](functions-app-settings.md#azurewebjobsstorage) to enable core behaviors such as coordinating singleton execution of timer triggers and default app key storage. This connection can also be configured to use an identity.

+> In addition, your function app might be reusing `AzureWebJobsStorage` for other storage connections in their triggers, bindings, and/or function code. Make sure that all uses of `AzureWebJobsStorage` are able to use the identity-based connection format before changing this connection from a connection string.

+## Use regular expressions to convert values

+| Standard storage with cool access in Azure NetApp Files | Public preview | Public preview [(in select regions)](cool-access-introduction.md#supported-regions) |

+* US Gov Arizona

+ Title: Azure NetApp Files tools

+description: Learn about the tools available to you to maximize your experience and savings with Azure NetApp Files.

+documentationcenter: ''

+editor: ''

+ms.assetid:

+ na

+# Azure NetApp Files tools

+Azure NetApp Files offers [multiple tools](https://azure.github.io/azure-netapp-files/) to estimate costs, understand features and availability, and monitor your Azure NetApp Files deployment.

+* [**Azure NetApp Files Performance Calculator**](https://aka.ms/anfcalc)

+ The Azure NetApp Files Performance Calculator enables you to easily calculate the performance and estimated cost of a volume based on the size and service level or performance requirements. It also helps you estimate backup and replication costs.

+* [**Azure NetApp Files datastore for Azure VMware Solution TCO Estimator**](https://aka.ms/anfavscalc)

+ This tool assists you with sizing an Azure VMware Solution. In the Estimator, provide the details of your current VMware environment to learn how much you can save by using Azure NetApp Files datastores.

+* [**SAP on Azure NetApp Files Sizing Estimator**](https://aka.ms/anfsapcalc)

+ This comprehensive tool estimates the infrastructure costs of an SAP HANA on Azure NetApp Files landscape. The estimate includes primary storage, backup, and replication costs.

+* [**Azure NetApp Files Standard storage with cool access cost savings estimator**](https://aka.ms/anfcoolaccesscalc)

+ Standard storage with cool access enables you to transparently move infrequently accessed data to less expensive storage. This cost savings estimator helps you understand how much money you can save by enabling Standard storage with cool access.

+* [**Azure NetApp Files Region and Feature Map**](https://aka.ms/anfmap)

+

+ Use this interactive map to understand which regions support Azure NetApp Files and its numerous features.

+* [**Azure NetApp Files on YouTube**](https://www.youtube.com/@azurenetappfiles)

+ Learn about the latest features in Azure NetApp Files and watch detailed how-to videos on the Azure NetApp Files YouTube channel.

+* [**ANFCapacityManager**](https://github.com/ANFTechTeam/ANFCapacityManager)

+ ANFCapacityManager is an Azure logic application that automatically creates metric alert rules in Azure Monitor to notify you when volumes are approaching their capacity. Optionally, it can increase the volumes' sizes automatically to keep your applications online.

+* [**ANFHealthCheck**](https://github.com/seanluce/ANFHealthCheck)

+ ANFHeathCheck is a PowerShell runbook that generates artful HTML reports of your entire Azure NetApp Files landscape. Optionally, it can automatically reduce over-sized volumes and capacity pools to reduce your TCO.

+To correctly use `protobuf.reliable.webpubsub.azure.v1` subprotocol, the client must follow the [How to create reliable clients](./howto-develop-reliable-clients.md) to implement reconnection, publisher and subscriber.

+| Supported capabilities|IR | Managed private endpoint|

+|| --| --|

+|[Copy activity](copy-activity-overview.md) (source/sink)|&#9312; &#9313;|Γ£ô |

+|[Mapping data flow](concepts-data-flow-overview.md) (source/sink)|&#9312; |- |

+|[Lookup activity](control-flow-lookup-activity.md)|&#9312; &#9313;|Γ£ô |

+|[GetMetadata activity](control-flow-get-metadata-activity.md)|&#9312; &#9313;|Γ£ô |

+|[Delete activity](delete-activity.md)|&#9312; &#9313;|Γ£ô |

+1. Browse to the **Manage** tab in your Azure Data Factory or Synapse workspace and select Linked Services, then select New:

+ "name": "LakehouseTableDataset",

+ "properties": {

+ "type": "LakehouseTable",

+ "linkedServiceName": {

+ "referenceName": "<Microsoft Fabric Lakehouse linked service name>",

+ "type": "LinkedServiceReference"

+ },

+ "typeProperties": {

+ },

+ "schema": [< physical schema, optional, retrievable during authoring >]

+ }

+| Merge Schema | Merge schema option allows schema evolution, that is, any columns that are present in the current incoming stream but not in the target Delta table is automatically added to its schema. This option is supported across all update methods. | no | `true` or `false` | mergeSchema: true |

+ validateSchema: false,

+ input(

+ CustomerID as string,

+ NameStyle as string,

+ Title as string,

+ FirstName as string,

+ MiddleName as string,

+ LastName as string,

+ Suffix as string,

+ CompanyName as string,

+ SalesPerson as string,

+ EmailAddress as string,

+ Phone as string,

+ PasswordHash as string,

+ PasswordSalt as string,

+ rowguid as string,

+ ModifiedDate as string

+ ),

+ deletable:false,

+ insertable:true,

+ updateable:false,

+ upsertable:false,

+ optimizedWrite: true,

+ mergeSchema: true,

+ autoCompact: true,

+ skipDuplicateMapInputs: true,

+ skipDuplicateMapOutputs: true) ~> CustomerTable

+| [Microsoft Fabric Lakehouse](connector-microsoft-fabric-lakehouse.md) | √/√ | √/√ | √ | x/x | √/√ | √ | √ | √ | √ | √/√ |

+You can use the Delete Activity in Azure Data Factory to delete files or folders from on-premises storage stores or cloud storage stores. Use this activity to clean up or archive files when they're no longer needed.

+- Make sure you aren't deleting files that are being written at the same time.

+- If you want to delete files or folder from an on-premises system, make sure you're using a self-hosted integration runtime with a version greater than 3.14.

+- [Azure Blob storage](connector-azure-blob-storage.md)

+- [Azure Data Lake Storage Gen1](connector-azure-data-lake-store.md)

+- [Azure Data Lake Storage Gen2](connector-azure-data-lake-storage.md)

+- [Azure Files](connector-azure-file-storage.md)

+- [File System](connector-file-system.md)

+- [FTP](connector-ftp.md)

+- [SFTP](connector-sftp.md)

+- [Microsoft Fabric Lakehouse](connector-microsoft-fabric-lakehouse.md)

+- [Amazon S3](connector-amazon-simple-storage-service.md)

+- [Amazon S3 Compatible Storage](connector-amazon-s3-compatible-storage.md)

+- [Google Cloud Storage](connector-google-cloud-storage.md)

+- [Oracle Cloud Storage](connector-oracle-cloud-storage.md)

+- [HDFS](connector-hdfs.md)

+1. Select the new Delete activity on the canvas if it isn't already selected, and its **Source** tab, to edit its details.

+| enable logging | Indicates whether you need to record the deleted folder or file names. If true, you need to further provide a storage account to save the log file, so that you can track the behaviors of the Delete activity by reading the log file. | No |

+| logStorageSettings | Only applicable when enablelogging = true.<br/><br/>A group of storage properties that can be specified where you want to save the log file containing the folder or file names deleted by the Delete activity. | No |

+| linkedServiceName | Only applicable when enablelogging = true.<br/><br/>The linked service of [Azure Storage](connector-azure-blob-storage.md#linked-service-properties), [Azure Data Lake Storage Gen1](connector-azure-data-lake-store.md#linked-service-properties), or [Azure Data Lake Storage Gen2](connector-azure-data-lake-storage.md#linked-service-properties) to store the log file that contains the folder or file names deleted by the Delete activity. Be aware it must be configured with the same type of Integration Runtime from the one used by delete activity to delete files. | No |

+| path | Only applicable when enablelogging = true.<br/><br/>The path to save the log file in your storage account. If you don't provide a path, the service creates a container for you. | No |

+Now you're using the Delete activity to delete folder or files by the combination of different property value from the dataset and the Delete activity:

+You can create a pipeline to periodically clean up the time partitioned folder or files. For example, the folder structure is similar as: `/mycontainer/2018/12/14/*.csv`. You can use the service system variable from schedule trigger to identify which folder or files should be deleted in each pipeline run.

+You can create a pipeline to clean up the old or expired files by using file attribute filter: "LastModified" in dataset.

+You can move a file by using a Copy activity to copy a file and then a Delete activity to delete a file in a pipeline. When you want to move multiple files, you can use the GetMetadata activity + Filter activity + Foreach activity + Copy activity + Delete activity as in the following sample.

+- Delete activity doesn't support deleting list of folders described by wildcard.

+- When using file attribute filter in delete activity: modifiedDatetimeStart and modifiedDatetimeEnd to select files to be deleted, make sure to set "wildcardFileName": "*" in delete activity as well.

+- [Copy Data tool](copy-data-tool.md)

+| Folder path | The directory of the delta lake | yes | String | folderPath |

+| Folder path | The directory of the delta lake | yes | String | folderPath |

+| [Four new recommendations for Azure Stack HCI resource type](#four-new-recommendations-for-azure-stack-hci-resource-type) | January 11, 2024 | February 2024 |

+## Four new recommendations for Azure Stack HCI resource type

+**Announcement date: January 11, 2024**

+**Estimated date for change: February 2024**

+Azure Stack HCI is set to be a new resource type that can be managed through Microsoft Defender for Cloud. We're adding four recommendations that are specific to the HCI resource type:

+| Recommendation | Description | Severity |

+|-|-|-|

+| Azure Stack HCI servers should meet Secured-core requirements | Ensure that all Azure Stack HCI servers meet the Secured-core requirements. (Related policy: [Guest Configuration extension should be installed on machines - Microsoft Azure](https://ms.portal.azure.com/#view/Microsoft_Azure_Security/GenericRecommendationDetailsBlade/assessmentKey/6c99f570-2ce7-46bc-8175-cde013df43bc)) | Low |

+| Azure Stack HCI servers should have consistently enforced application control policies | At a minimum, apply the Microsoft WDAC base policy in enforced mode on all Azure Stack HCI servers. Applied Windows Defender Application Control (WDAC) policies must be consistent across servers in the same cluster. | High |

+| Azure Stack HCI systems should have encrypted volumes | Use BitLocker to encrypt the OS and data volumes on Azure Stack HCI systems | High |

+| Host and VM networking should be protected on Azure Stack HCI systems | Protect data on the Azure Stack HCI hostΓÇÖs network and on virtual machine network connections. | Low |

+The Defender for Servers built-in vulnerability assessment solution powered by Qualys is on a retirement path which is estimated to complete on **May 1st, 2024**. If you're currently using the vulnerability assessment solution powered by Qualys, you should plan your [transition to the integrated Microsoft Defender vulnerability management solution](how-to-transition-to-built-in.md).

+The Defender for Cloud Containers Vulnerability Assessment powered by Qualys is now on a retirement path completing on **March 1st, 2024**. If you're currently using container vulnerability assessment powered by Qualys, start planning your transition to [Vulnerability assessments for Azure with Microsoft Defender Vulnerability Management](agentless-vulnerability-assessment-azure.md).

+## Azure regions for Dev Box

+Before setting up Dev Box, you need to choose the best regions for your organization. Check [Products available by region](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?products=dev-box) and [Azure geographies](https://azure.microsoft.com/explore/global-infrastructure/geographies/#choose-your-region) to help you decide on the regions you use. If the region you prefer isnΓÇÖt available for Dev Box, choose a region within 500 miles.

+You specify a region for your dev center and projects. Typically, these resources are in the same region as your main office or IT management center.

+The region of the virtual network specified in a network connection determines the region for a dev box. You can create multiple network connections based on the regions where you support developers. You can then use those connections when you're creating dev box pools to ensure that dev box users create dev boxes in a region close to them. Using a region close to the dev box user provides the best experience.

+ Title: Manage a Microsoft Dev Box dev center

+Development teams vary in the way they function and can have different needs. A dev center helps you manage these scenarios by enabling you to group similar sets of projects together and apply similar settings.

+| Action | Permissions required |

+|||

+| _Create or delete a dev center_ | Owner or Contributor permissions on an Azure subscription or a specific resource group. |

+| _Manage a dev center_ | Owner or Contributor role, or specific Write permission to the dev center. |

+| _Attach or remove a network connection_ | Network Contributor permissions on an existing network connection (Owner or Contributor). |

+ :::image type="content" source="./media/how-to-manage-dev-center/search-dev-center.png" alt-text="Screenshot that shows the Azure portal with the search box and the result for dev centers." lightbox="./media/how-to-manage-dev-center/search-dev-center.png":::

+ :::image type="content" source="./media/how-to-manage-dev-center/create-dev-center.png" alt-text="Screenshot that shows the Azure portal with the Create button on the page for dev centers." lightbox="./media/how-to-manage-dev-center/create-dev-center.png":::

+ | Setting | Value |

+ |||

+ | **Subscription** | Select the subscription in which you want to create the dev center. |

+ | **ResourceGroup** | Select an existing resource group, or select **Create new** and then enter a name for the new resource group. |

+ | **Name** | Enter a name for your dev center. |

+ | **Location** | Select the location or region where you want the dev center to be created. |

+ :::image type="content" source="./media/how-to-manage-dev-center/create-dev-center-basics.png" alt-text="Screenshot that shows the Basics tab on the pane for creating a dev center." lightbox="./media/how-to-manage-dev-center/create-dev-center-basics.png":::

+ For a list of the currently supported Azure locations with capacity, see [Frequently asked questions about Microsoft Dev Box](https://aka.ms/devbox_acom).

+ :::image type="content" source="./media/how-to-manage-dev-center/create-dev-center-tags.png" alt-text="Screenshot that shows the Tags tab on the page for creating a dev center." lightbox="./media/how-to-manage-dev-center/create-dev-center-tags.png":::

+ :::image type="content" source="./media/how-to-manage-dev-center/azure-notifications.png" alt-text="Screenshot that shows the Notifications pane in the Azure portal." lightbox="./media/how-to-manage-dev-center/azure-notifications.png":::

+1. When the deployment completes, select **Go to resource**. Confirm that the dev center page appears.

+Attached network connections and their associated virtual networks aren't deleted when you delete a dev center.

+ :::image type="content" source="./media/how-to-manage-dev-center/delete-dev-center.png" alt-text="Screenshot of the Delete button on the page for a dev center." lightbox="./media/how-to-manage-dev-center/delete-dev-center.png":::

+- **Owner**: Grants full access to manage all resources, including the ability to assign roles in Azure role-based access control (RBAC).

+- **Contributor**: Grants full access to manage all resources, but doesn't allow the user to assign roles in Azure RBAC, manage assignments in Azure Blueprints, or share image galleries.

+- **Reader**: Grants the ability to view all resources, but doesn't allow the user to make any changes.

+1. Assign a role by configuring the following settings. For detailed steps, see [Assign Azure roles using the Azure portal](../role-based-access-control/role-assignments-portal.md).

+ |||

+- [Create a dev box definition](quickstart-configure-dev-box-service.md#create-a-dev-box-definition)

+ Title: Request a quota limit increase for Dev Box resources

+This article describes how to submit a support request to increase the number of resources for Microsoft Dev Box in your Azure subscription.

+To ensure that resources are available for customers, Microsoft Dev Box has a limit on the number of each type of resource that can be used in a subscription. This limit is called a _quota_.

+There are different types of quota limits that you might encounter, depending on the resource type. Here are some examples:

+- There are limits on the number of vCPUs available for dev boxes. You might encounter this quota error in the Microsoft **[developer portal](https://aka.ms/devbox-portal)** during dev box creation.

+- There are limits for dev centers, network connections, and dev box definitions. You can find information about these limits through the **Azure portal**.

+When you reach the limit for a resource in your subscription, you can request a limit increase (sometimes called a capacity increase, or a quota increase) to extend the number of resources available. The request process allows the Microsoft Dev Box team to ensure your subscription isn't involved in any cases of fraud or unintentional, sudden large-scale deployments.

+The time it takes to increase your quota varies depending on the virtual machine size, region, and number of resources requested. You don't have to go through the process of requesting extra capacity often. To ensure you have the resources you require when you need them, you should:

+- Make incremental requests for virtual machine cores rather than making large, bulk requests. Break requests for large numbers of cores into smaller requests for extra flexibility in how those requests are fulfilled.

+ For each of your subscriptions, you can check your current usage of each Deployment Environments resource type in each region. Determine your current usage by following the steps in [Determine usage and quota](./how-to-determine-your-quota-usage.md).

+ Dev Box resources can exist in many regions. You can choose to deploy resources in multiple regions located near to your dev box users. For more information about Azure regions, how they relate to global geographies, and which services are available in each region, see [Azure global infrastructure](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/).

+- **Choose the quota type of the additional quota**

+## Initiate a support request

+Azure presents two ways to get you the right help and assist you with submitting a request for support:

+- The **Support + troubleshooting** feature available on the toolbar

+- The **Help + support** page available on the Azure portal menu

+The **Support + troubleshooting** feature uses questions like **How can we help you?** to guide you through the process.

+Both the **Support + troubleshooting** feature and the **Help + support** page help you fill out and submit a classic style support request form.

+To begin the process, choose the tab that offers the input style that's appropriate for your experience, then follow the steps to request a quota limit increase.

+# [**Support + troubleshooting** (questions)](#tab/Questions/)

+1. On the Azure portal home page, select the **Support + Troubleshooting** icon (question mark) on the toolbar.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-question.png" alt-text="Screenshot showing the How can we help question view for the Support plus troubleshooting feature." lightbox="media/how-to-request-quota-increase/help-support-question.png":::

+1. In the **How can we help you?** box, enter **quota limit**, and then select **Go**. The view updates to show the **Current selection** section.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-quota-limit.png" alt-text="Screenshot showing the How can we help question and quota limit answer with the Current selection section." lightbox="media/how-to-request-quota-increase/help-support-quota-limit.png":::

+1. In the **Which service are you having an issue with?** dropdown list, select **Service and subscription limits (quotas)**.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-service-list.png" alt-text="Screenshot showing the open dropdown list for the Which service are you having an issue with field." lightbox="media/how-to-request-quota-increase/help-support-service-list.png":::

+1. Confirm your choice for the **Current selection** and then select **Next**. The view updates to include an option to create a support request for quotas.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-service-list-next.png" alt-text="Screenshot showing the Service and subscription limits (quotas) item selected and the Next button highlighted." lightbox="media/how-to-request-quota-increase/help-support-service-list-next.png":::

+1. In the **Service and subscription limits (quotas)** section, select **Create a support request**.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-result.png" alt-text="Screenshot showing the Service and subscription limits (quotas) section and the Create a support request button highlighted." lightbox="media/how-to-request-quota-increase/help-support-result.png":::

+The **New support request** page opens. Continue to the [following section](#describe-the-requested-quota-increase) to fill out the support request form.

+# [**Help + support**](#tab/AzureADJoin/)

+1. On the Azure portal home page, expand the Azure portal menu, and select **Help + support**.

+ :::image type="content" source="./media/how-to-request-quota-increase/help-plus-support-portal.png" alt-text="Screenshot of the Azure portal menu on the home page and the Help plus support option selected." lightbox="./media/how-to-request-quota-increase/help-plus-support-portal.png":::

+1. On the **Help + support** page, select **Create a support request**.

+ :::image type="content" source="./media/how-to-request-quota-increase/create-support-request.png" alt-text="Screenshot of the Help plus support page and the Create a support request highlighted." lightbox="./media/how-to-request-quota-increase/create-support-request.png":::

+The **New support request** page opens. Continue to the [following section](#describe-the-requested-quota-increase) to fill out the support request form.

+## Describe the requested quota increase

+Follow these steps to describe your requested quota increase and fill out the support form.

+1. On the **New support request** page, on the **1. Problem description** tab, configure the following settings, and then select **Next**.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-request-problem.png" alt-text="Screenshot showing the problem description tab for a new support request with the required fields highlighted." lightbox="media/how-to-request-quota-increase/help-support-request-problem.png":::

+ | Setting | Value |

+ |||

+ | **Issue type** | Select **Service and subscription limits (quotas)**. |

+ | **Subscription** | Select the subscription to which the request applies. |

+ | **Quota type** | Select **Microsoft Dev Box**. |

+ After you select **Next**, the tool skips the **2. Recommended solution** tab and opens the **3. Additional details** tab. This tab contains four sections: **Problem details**, **Advanced diagnostic information**, **Support method**, and **Contact information**.

+1. On the **3. Additional details** tab, in the **Problem details** section, select **Enter details**. The **Quota details** pane opens.

+ :::image type="content" source="media/how-to-request-quota-increase/help-support-request-additional-details.png" alt-text="Screenshot showing the additional details tab for a new support request with the Enter details link highlighted." lightbox="media/how-to-request-quota-increase/help-support-request-enter-details.png":::

+1. In the **Quota details** pane, configure the following settings:

+

+ | Setting | Value |

+ |||

+ | **Region** | Select the **Region** in which you want to increase your quota. |

+ | **Quota type** | When you select a **Region**, Azure updates the view to display your current usage and current limit for all quota types. After the view updates, set the **Quota type** field to the quota that you want to increase. |

+ | **New total limit** | Enter the new total limit that you want to request. |

+ | **Is it a limit decrease?** | Select **Yes** or **No**. |

+ | **Additional information** | Enter any extra information about your request. |

+ :::image type="content" source="media/how-to-request-quota-increase/quota-details.png" alt-text="Screenshot of the Quota details pane showing current usage and current limit for all quota types for a specific region." lightbox="media/how-to-request-quota-increase/quota-details.png":::

+1. Select **Save and Continue**.

+To complete the support request form, configure the remaining settings. When you're ready, review your information and submit the request.

+1. On the **Additional details** tab, in the **Advanced diagnostic information** section, configure the following setting:

+ | Setting | Value |

+ |||

+ | **Allow collection of advanced diagnostic information** | Select **Yes** (Recommended) or **No**. |

+ :::image type="content" source="media/how-to-request-quota-increase/request-advanced-diagnostics-info.png" alt-text="Screenshot showing the Advanced diagnostic information section for a new support request." lightbox="media/how-to-request-quota-increase/request-advanced-diagnostics-info.png":::

+1. In the **Support method** section, configure the following settings:

+ | Setting | Value |

+ |||

+ | **Support plan** | Select your support plan. |

+ | **Severity** | Select the severity of the issue. |

+ | **Preferred contact method** | Select **Email** or **Phone**. |

+ | **Your availability** | Enter your availability. |

+ | **Support language** | Select your language preference. |

+ :::image type="content" source="media/how-to-request-quota-increase/request-support-method.png" alt-text="Screenshot showing the Support method section for a new support request." lightbox="media/how-to-request-quota-increase/request-support-method.png":::

+1. In the **Contact info** section, configure the following settings:

+ | Setting | Value |

+ |||

+ | **First name** | Enter your first name. |

+ | **Last name** | Enter your last name. |

+ | **Email** | Enter your contact email. |

+ | **Additional email for notification** | Enter an email for notifications. |

+ | **Phone** | Enter your contact phone number. |

+ | **Country/region** | Enter your location. |

+ | **Save contact changes for future support requests.** | Select the check box to save changes. |

+ :::image type="content" source="media/how-to-request-quota-increase/request-contact-info.png" alt-text="Screenshot showing the Contact info section for a new support request and the Next button." lightbox="media/how-to-request-quota-increase/request-contact-info.png":::

+1. On the **4. Review + create** tab, review your information. When you're ready to submit the request, select **Create**.

+- Check your quota usage by [determining usage and quota](./how-to-determine-your-quota-usage.md)

+- Check the default quota for each resource type by subscription type with [Microsoft Dev Box limits](../azure-resource-manager/management/azure-subscription-service-limits.md#microsoft-dev-box-limits)

+Depending on the project configuration and your permissions, you have access to different projects and associated dev box configurations. If you have a choice of projects and dev box pools, select the project and dev box pool that best fits your needs. For example, you might choose a project that has a dev box pool located near to you for least latency.

+description: Configure the Azure DNS Private Resolver for a centralized or noncentralized architecture

+Consider the following hub and spoke VNet topology in Azure with a private resolver located in the hub and a ruleset link to the spoke VNet. Both the hub and the spoke use Azure-provided DNS in their VNet settings:

+**DNS resolution in the spoke VNet**: The virtual network link from the ruleset to the spoke VNet enables the spoke VNet to resolve **azure.contoso.com** using the configured forwarding rule. A link from the private zone to the spoke VNet isn't required here. The spoke VNet sends queries for **azure.contoso.com** to the hub's inbound endpoint via Azure-provided DNS because there is a rule matching this domain name in the linked ruleset. Queries for other namespaces can also be forwarded by configuring additional rules. DNS queries that don't match a ruleset rule are not forwarded and are resolved using Azure-provided DNS.

+Consider the following hub and spoke VNet topology with an inbound endpoint provisioned as custom DNS in the spoke VNet. The spoke VNet uses a Custom DNS setting of 10.10.0.4, corresponding to the Hub's private resolver inbound endpoint:

+**DNS resolution in the hub VNet**: The virtual network link from the private zone to the Hub VNet enables resources inside the hub VNet to automatically resolve DNS records in **azure.contoso.com** using Azure-provided DNS ([168.63.129.16](../virtual-network/what-is-ip-address-168-63-129-16.md)). If configured, ruleset rules determine how DNS names are forwarded and resolved. Namespaces that don't match a ruleset rule are resolved without forwarding using Azure-provided DNS.

+description: This article describes various concepts of authentication in Azure Data Manager for Energy.

+# Authentication concepts in Azure Data Manager for Energy

+Authentication confirms the identity of users. The access flows can be user triggered, system triggered, or system API communication. In this article, you learn about service principals and authorization tokens.

+In an Azure Data Manager for Energy instance:

+- No service principals are created.

+- The app ID is used for API access. The same app ID is used to provision an Azure Data Manager for Energy instance.

+- The app ID doesn't have access to infrastructure resources.

+- The app ID also gets added as OWNER to all OSDU groups by default.

+- For service-to-service communication, Azure Data Manager for Energy uses Managed Service Identity.

+In an OSDU instance:

+- Terraform scripts create two service principals:

+ - The first service principal is used for API access. It can also manage infrastructure resources.

+ - The second service principal is used for service-to-service communications.

+## Generate an authorization token

+To generate the authorization token, follow the steps in [Generate auth token](how-to-generate-auth-token.md).

+ Title: Entitlement concepts in Azure Data Manager for Energy

+description: This article describes various concepts of the entitlement service in Azure Data Manager for Energy.

+Access management is a critical function for any service or resource. The entitlement service lets you control who can use your Azure Data Manager for Energy instance, what they can see or change, and which services or data they can use.

+The entitlement service of Azure Data Manager for Energy allows you to create groups and manage memberships of the groups. An entitlement group defines permissions on services or data sources for a specific data partition in your Azure Data Manager for Energy instance. Users added to a specific group obtain the associated permissions. All group identifiers (emails) are of the form `{groupType}.{serviceName|resourceName}.{permission}@{partition}.{domain}`.

+Different groups and associated user entitlements must be set for every *new data partition*, even in the same Azure Data Manager for Energy instance.

+The entitlement service enables three use cases for authorization:

+- **Data groups** are used to enable authorization for data.

+ - The data groups start with the word "data," such as `data.welldb.viewers` and `data.welldb.owners`.

+ - Individual users are added to the data groups, which are added in the ACL of individual data records to enable `viewer` and `owner` access of the data after the data is loaded in the system.

+ - To `upload` the data, you need to have entitlements of various OSDU services, which are used during the ingestion process. The combination of OSDU services depends on the method of ingestion. For example, for manifest ingestion, see [Manifest-based ingestion concepts](concepts-manifest-ingestion.md) to understand the OSDU services that APIs used. The user *doesn't need to be part of the ACL* to upload the data.

+- **Service groups** are used to enable authorization for services.

+ - The service groups start with the word "service," such as `service.storage.user` and `service.storage.admin`.

+ - The service groups are *predefined* when OSDU services are provisioned in each data partition of the Azure Data Manager for Energy instance.

+ - These groups enable `viewer`, `editor`, and `admin` access to call the OSDU APIs corresponding to the OSDU services.

+- **User groups** are used for hierarchical grouping of user and service groups.

+ - The service groups start with the word "users," such as `users.datalake.viewers` and `users.datalake.editors`.

+ - Some user groups are created by default when a data partition is provisioned. For information on these groups and their hierarchy scope, see [Bootstrapped OSDU entitlement groups](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/osdu-entitlement-roles.md).

+ - There's one exception of this group naming rule for the "users" group. It gets created when a new data partition is provisioned and its name follows the pattern of `users@{partition}.{domain}`. It has the list of all the users with any type of access in a specific data partition. Before you add a new user to any entitlement groups, you also need to add the new user to the `users@{partition}.{domain}` group.

+You can add individual users to a `user group`. The `user group` is then added to a `data group`. The data group is added to the ACL of the data record. It enables abstraction for the data groups because individual users don't need to be added one by one to the data group. Instead, you can add users to the `user group`. Then you can use the `user group` repeatedly for multiple `data groups`. The nested structure helps provide scalability to manage memberships in OSDU.

+For each OSDU group, you can add a user as either an OWNER or a MEMBER:

+- If you're an OWNER of an OSDU group, you can add or remove the members of that group or delete the group.

+- If you're a MEMBER of an OSDU group, you can view, edit, or delete the service or data depending on the scope of the OSDU group. For example, if you're a MEMBER of the `service.legal.editor` OSDU group, you can call the APIs to change the legal service.

+> Don't delete the OWNER of a group unless there's another OWNER to manage the users.

+For a full list of Entitlement API endpoints, see [OSDU entitlement service](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/blob/release/0.15/docs/tutorial/Entitlements-Service.md#entitlement-service-api). A few illustrations of how to use Entitlement APIs are available in [Manage users](how-to-manage-users.md).

+> The OSDU documentation refers to v1 endpoints, but the scripts noted in this documentation refer to v2 endpoints, which work and have been successfully validated.

+For the next step, see:

+- [Manage users](how-to-manage-users.md)

+- [Manage legal tags](how-to-manage-legal-tags.md)

+- [Manage ACLs](how-to-manage-acls.md)

+You can also ingest data into your Azure Data Manager for Energy instance:

+ Title: Generate a refresh token for Azure Data Manager for Energy

+description: This article describes how to generate an auth token.

+#Customer intent: As a developer, I want to learn how to generate an auth token.

+# Generate an auth token

+In this article, you learn how to generate the service principal auth token, a user's auth token, and a user's refresh token.

+1. To provision the Azure Data Manager for Energy platform, you must register your app on the [Azure portal app registration page](https://go.microsoft.com/fwlink/?linkid=2083908). You can use either a Microsoft account or a work or school account to register an app. For steps on how to configure, see [Register your app documentation](../active-directory/develop/quickstart-register-app.md#register-an-application).

+1. In the app overview section, if there are no redirect URIs specified, you can select **Add a platform** > **Web**, add `http://localhost:8080`, and select **Save**.

+

+ :::image type="content" source="media/how-to-generate-auth-token/app-registration-uri.png" alt-text="Screenshot that shows adding the URI to the app.":::

+1. Fetch the `redirect-uri` (or reply URL) for your app to receive responses from Microsoft Entra ID.

+You can also find the parameters after the app is registered on the Azure portal.

+### Find tenant-id

+1. Go to the Microsoft Entra account for your organization. You can search for **Microsoft Entra ID** in the Azure portal's search bar.

+1. On the **Overview** tab, under the **Basic information** section, find **Tenant ID**.

+1. Copy the `tenant-ID` value and paste it into an editor to be used later.

+ :::image type="content" source="media/how-to-generate-auth-token/azure-active-directory.png" alt-text="Screenshot that shows searching for Microsoft Entra ID.":::

+ :::image type="content" source="media/how-to-generate-auth-token/tenant-id.png" alt-text="Screenshot that shows finding the tenant ID.":::

+### Find client-id

+A `client-id` is the same value that you use to register your application during the provisioning of your [Azure Data Manager for Energy instance](quickstart-create-microsoft-energy-data-services-instance.md). It's often referred to as `app-id`.

+1. Go to the Azure Data Manager for Energy **Overview** page. On the **Essentials** pane, find **client ID**.

+1. Copy the `client-id` value and paste it into an editor to be used later.

+1. Currently, one Azure Data Manager for Energy instance allows one `app-id` to be associated with one instance.

+ > [!IMPORTANT]

+ > The `client-id` that's passed as a value in the Entitlement API calls needs to be the same one that was used for provisioning your Azure Data Manager for Energy instance.

+ :::image type="content" source="media/how-to-generate-auth-token/client-id-or-app-id.png" alt-text="Screenshot that shows finding the client ID for your registered app.":::

+### Find client-secret

+A `client-secret` is a string value your app can use in place of a certificate to identify itself. It's sometimes referred to as an application password.

+1. Go to **App registrations**.

+1. Under the **Manage** section, select **Certificates & secrets**.

+1. Select **New client secret** to create a client secret for the client ID that you used to create your Azure Data Manager for Energy instance.

+1. Record the secret's **Value** for later use in your client application code.

+

+ The access token of the `app-id` and `client-secret` has the infrastructure administrator access to the instance.

+ > [!CAUTION]

+ > Don't forget to record the secret's value. This secret value is never displayed again after you leave this page for client secret creation.

+ :::image type="content" source="media/how-to-generate-auth-token/client-secret.png" alt-text="Screenshot that shows finding the client secret.":::

+#### Find the URL for your Azure Data Manager for Energy instance

+1. Create an [Azure Data Manager for Energy instance](quickstart-create-microsoft-energy-data-services-instance.md).

+1. Go to your Azure Data Manager for Energy **Overview** page on the Azure portal.

+1. On the **Essentials** pane, copy the URI.

+ :::image type="content" source="media/how-to-generate-auth-token/endpoint-url.png" alt-text="Screenshot that shows finding the URI for the Azure Data Manager for Energy instance.":::

+#### Find data-partition-id

+- **Option 1**: Under the **Advanced** section of your Azure Data Manager for Energy UI, go to the **Data Partitions** menu item.

+ :::image type="content" source="media/how-to-generate-auth-token/data-partition-id.png" alt-text="Screenshot that shows finding the data-partition-id from the Azure Data Manager for Energy instance.":::

+- **Option 2**: On the **Essentials** pane of your Azure Data Manager for Energy **Overview** page, underneath the **Data Partitions** field, select **view**.

+ :::image type="content" source="media/how-to-generate-auth-token/data-partition-id-second-option.png" alt-text="Screenshot that shows finding the data-partition-id from the Azure Data Manager for Energy instance Overview page.":::

+ :::image type="content" source="media/how-to-generate-auth-token/data-partition-id-second-option-step-2.png" alt-text="Screenshot that shows finding the data-partition-id from the Azure Data Manager for Energy instance Overview page with the data partitions.":::

+## Generate the client-id auth token

+Run the following curl command in Azure Cloud Bash after you replace the placeholder values with the corresponding values found earlier in the previous steps. The access token in the response is the `client-id` auth token.

+## Generate the user auth token

+Generating a user's auth token is a two-step process.

+### Get the authorization code

+The first step to get an access token for many OpenID Connect (OIDC) and OAuth 2.0 flows is to redirect the user to the Microsoft identity platform `/authorize` endpoint. Microsoft Entra ID signs the user in and requests their consent for the permissions your app requests. In the authorization code grant flow, after consent is obtained, Microsoft Entra ID returns an authorization code to your app that it can redeem at the Microsoft identity platform `/token` endpoint for an access token.

+1. After you replace the parameters, you can paste the request in the URL of any browser and select Enter.

+1. Sign in to your Azure portal if you aren't signed in already.

+1. You might see the "Hmmm...can't reach this page" error message in the browser. You can ignore it.

+ :::image type="content" source="media/how-to-generate-auth-token/localhost-redirection-error.png" alt-text="Screenshot of localhost redirection.":::

+1. The browser redirects to `http://localhost:8080/?code={authorization code}&state=...` upon successful authentication.

+1. Copy the response from the URL bar of the browser and fetch the text between `code=` and `&state`.

+1. Keep this authorization code handy for future use.

+|tenant-id|Required|Name of your Microsoft Entra tenant.|

+| redirect_uri |Required |The redirect URI of your app, where your app sends and receives the authentication responses. It must exactly match one of the redirect URIs that you registered in the portal, except that it must be URL encoded. |

+| scope |Required |A space-separated list of scopes. The `openid` scope indicates a permission to sign in the user and get data about the user in the form of ID tokens. The `offline_access` scope is optional for web applications. It indicates that your application needs a *refresh token* for extended access to resources. The client ID indicates the token issued is intended for use by an Azure Active Directory B2C registered client. The `https://{tenant-name}/{app-id-uri}/{scope}` indicates a permission to protected resources, such as a web API. |

+| state |Recommended |A value included in the request that can be a string of any content that you want to use. Usually, a randomly generated unique value is used to prevent cross-site request forgery (CSRF) attacks. The state also is used to encode information about the user's state in the app before the authentication request occurred. For example, the page the user was on, or the user flow that was being executed. |

+> [!NOTE]

+> The browser might say that the site can't be reached, but it should still have the authorization code in the URL bar.

+|code|The authorization code that the app requested. The app can use the authorization code to request an access token for the target resource. Authorization codes are short lived. Typically, they expire after about 10 minutes.|

+|state|If a state parameter is included in the request, the same value should appear in the response. The app should verify that the state values in the request and response are identical. This check helps to detect [CSRF attacks](https://tools.ietf.org/html/rfc6749#section-10.12) against the client.|

+|session_state|A unique value that identifies the current user session. This value is a GUID, but it should be treated as an opaque value that's passed without examination.|

+> Running the URL in Postman won't work because it requires extra configuration for token retrieval.

+### Get an auth token and a refresh token

+The second step is to get the auth token and the refresh token. Your app uses the authorization code received in the previous step to request an access token by sending a POST request to the `/token` endpoint.

+|tenant | Required | The `{tenant-id}` value in the path of the request can be used to control who can sign in to the application.|

+|client_id | Required | The application ID assigned to your app upon registration. |

+|scope | Required | A space-separated list of scopes. The scopes that your app requests in this leg must be equivalent to or a subset of the scopes that it requested in the first (authorization) leg. If the scopes specified in this request span multiple resource servers, the v2.0 endpoint returns a token for the resource specified in the first scope. |

+|code |Required |The authorization code that you acquired in the first step of the flow. |

+|redirect_uri | Required |The same redirect URI value that was used to acquire the authorization code. |

+|grant_type | Required | Must be `authorization_code` for the authorization code flow. |

+|client_secret | Required | The client secret that you created in the app registration portal for your app. It shouldn't be used in a native app because client secrets can't be reliably stored on devices. It's required for web apps and web APIs, which have the ability to store the client secret securely on the server side.|

+|scope |A space-separated list of the Microsoft Graph permissions that the access token is valid for. |

+|refresh_token |An OAuth 2.0 refresh token. Your app can use this token to acquire extra access tokens after the current access token expires. Refresh tokens are long-lived and can be used to retain access to resources for extended periods of time.|

+For more information on generating a user access token and using a refresh token to generate a new access token, see [Generate refresh tokens](/graph/auth-v2-user#2-get-authorization).

+To learn more about how to use the generated refresh token, see:

+ Title: Manage ACLs in Azure Data Manager for Energy

+description: This article describes how to manage ACLs in Azure Data Manager for Energy.

+# Manage ACLs of the data record

+Keep the record ID from the response handy for future references.

+The first `/acl/owners/0` operation removes ACL from 0th position in the array of ACL. When you delete the first entry with this operation, the system deletes it. The previous second entry then becomes the first entry. The second `/acl/owners/0` operation tries to remove the second entry.

+If you delete the last owner ACL from the data record, you get the error.

+After you add ACLs to the data records, you can:

+- [Manage legal tags](how-to-manage-legal-tags.md)

+- [Manage users](how-to-manage-users.md)

+You can also ingest data into your Azure Data Manager for Energy instance:

+ Title: Manage users in Azure Data Manager for Energy

+description: This article describes how to manage users in Azure Data Manager for Energy.

+# Manage users in Azure Data Manager for Energy

+In this article, you learn how to manage users and their memberships in OSDU groups in Azure Data Manager for Energy. [Entitlements APIs](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/tree/master/) are used to add or remove users to OSDU groups and to check the entitlements when the user tries to access the OSDU services or data. For more information about OSDU groups, see [Entitlement services](concepts-entitlements.md).

+- Create an Azure Data Manager for Energy instance. See [How to create Azure Data Manager for Energy instance](quickstart-create-microsoft-energy-data-services-instance.md).

+- Get various parameters of your instance, such as `client-id` and `client-secret`. See [How to generate auth token](how-to-generate-auth-token.md).

+- Generate the service principal access token that's needed to call the Entitlement APIs. See [How to generate auth token](how-to-generate-auth-token.md).

+- Keep all the parameter values handy. They're needed to run different user management requests via the Entitlements API.

+The object ID (OID) is the Microsoft Entra user OID.

+1. Find the OID of the users first. If you're managing an application's access, you must find and use the application ID (or client ID) instead of the OID.

+1. Input the OID of the users (or the application or client ID if managing access for an application) as parameters in the calls to the Entitlements API of your Azure Data Manager for Energy instance.

+ :::image type="content" source="media/how-to-manage-users/azure-active-directory-object-id.png" alt-text="Screenshot that shows finding the object ID from Microsoft Entra ID.":::

+ :::image type="content" source="media/how-to-manage-users/profile-object-id.png" alt-text="Screenshot that shows finding the OID from the profile.":::

+## First-time addition of users in a new data partition

+1. To add the first admin to a new data partition of an Azure Data Manager for Energy instance, use the access token of the OID that was used to provision the instance.

+1. Get the `client-id` access token by using [Generate client-id access token](how-to-generate-auth-token.md#generate-the-client-id-auth-token).

+ If you try to directly use your own access token for adding entitlements, it results in a 401 error. The `client-id` access token must be used to add the first set of users in the system. Those users (with admin access) can then manage more users with their own access token.

+1. Use the `client-id` access token to do the following steps by using the commands outlined in the following sections:

+1. The user becomes the admin of the data partition. The admin can then add or remove more users to the required entitlement groups:

+ 1. Get the admin's auth token by using [Generate user access token](how-to-generate-auth-token.md#generate-the-user-auth-token) and by using the same `client-id` and `client-secret` values.

+ 1. Get the OSDU group, such as `service.legal.editor@<data-partition-id>.<domain>`, to which you want to add more users by using the admin's access token.

+ 1. Add more users to that OSDU group by using the admin's access token.

+Run the following curl command in Azure Cloud Shell to get all the groups that are available for you or that you have access to in the specific data partition of the Azure Data Manager for Energy instance.

+1. Run the following curl command in Azure Cloud Shell to add the users to the users group by using the entitlement service.

+1. The value to be sent for the parameter `email` is the OID of the user and not the user's email address.

+ ```bash

+ curl --location --request POST 'https://<URI>/api/entitlements/v2/groups/<group-name>@<data-partition-id>.dataservices.energy/members' \

+ --header 'data-partition-id: <data-partition-id>' \

+ --header 'Authorization: Bearer <access_token>' \

+ --header 'Content-Type: application/json' \

+ --data-raw '{

+ "email": "<Object_ID>",

+ "role": "MEMBER"

+ }'

+ ```

+ **Sample request for users OSDU group**

+

+ Consider an Azure Data Manager for Energy instance named `medstest` with a data partition named `dp1`.

+

+ ```bash

+ curl --location --request POST 'https://medstest.energy.azure.com/api/entitlements/v2/groups/users@medstest-dp1.dataservices.energy/members' \

+ --header 'data-partition-id: medstest-dp1' \

+ --header 'Authorization: Bearer abcdefgh123456.............' \

+ --header 'Content-Type: application/json' \

+ --data-raw '{

+ "email": "90e0d063-2f8e-4244-860a-XXXXXXXXXX",

+ "role": "MEMBER"

+ }'

+ ```

+

+ **Sample response**

+

+ ```JSON

+ {

+ "email": "90e0d063-2f8e-4244-860a-XXXXXXXXXX",

+ "role": "MEMBER"

+ }

+ ```

+

+ **Sample request for legal service editor OSDU group**

+

+ ```bash

+ curl --location --request POST 'https://medstest.energy.azure.com/api/entitlements/v2/groups/service.legal.editor@medstest-dp1.dataservices.energy/members' \

+ --header 'data-partition-id: medstest-dp1' \

+ --header 'Authorization: Bearer abcdefgh123456.............' \

+ --header 'Content-Type: application/json' \

+ --data-raw '{

+ "email": "90e0d063-2f8e-4244-860a-XXXXXXXXXX",

+ "role": "MEMBER"

+ }'

+ ```

+ > [!IMPORTANT]

+ > The app ID is the default OWNER of all the groups.

+ :::image type="content" source="media/how-to-manage-users/appid.png" alt-text="Screenshot that shows the app ID in Microsoft Entra ID.":::

+1. Run the following curl command in Azure Cloud Shell to get all the groups associated with the user.

+ ```bash

+ curl --location --request GET 'https://<URI>/api/entitlements/v2/members/<OBJECT_ID>/groups?type=none' \

+ --header 'data-partition-id: <data-partition-id>' \

+ --header 'Authorization: Bearer <access_token>'

+ ```

+

+ **Sample request**

+

+ Consider an Azure Data Manager for Energy instance named `medstest` with a data partition named `dp1`.

+

+ ```bash

+ curl --location --request GET 'https://medstest.energy.azure.com/api/entitlements/v2/members/90e0d063-2f8e-4244-860a-XXXXXXXXXX/groups?type=none' \

+ --header 'data-partition-id: medstest-dp1' \

+ --header 'Authorization: Bearer abcdefgh123456.............'

+ ```

+ **Sample response**

+

+ ```JSON

+ "desId": "90e0d063-2f8e-4244-860a-XXXXXXXXXX",

+ "memberEmail": "90e0d063-2f8e-4244-860a-XXXXXXXXXX",

+ "groups": [

+ {

+ "name": "users",

+ "description": "Datalake users",

+ "email": "users@medstest-dp1.dataservices.energy"

+ },

+ {

+ "name": "service.search.user",

+ "description": "Datalake Search users",

+ "email": "service.search.user@medstest-dp1.dataservices.energy"

+ }

+ ]

+ ```

+## Delete OSDU groups of a specific user in a data partition

+1. Run the following curl command in Azure Cloud Shell to delete a specific user from a specific data partition.

+1. *Do not* delete the OWNER of a group unless you have another OWNER who can manage users in that group.

+

+ ```bash

+ curl --location --request DELETE 'https://<URI>/api/entitlements/v2/members/<OBJECT_ID>' \

+ --header 'data-partition-id: <data-partition-id>' \

+ --header 'Authorization: Bearer <access_token>'

+ ```

+

+ **Sample request**

+

+ Consider an Azure Data Manager for Energy instance named `medstest` with a data partition named `dp1`.

+

+ ```bash

+ curl --location --request DELETE 'https://medstest.energy.azure.com/api/entitlements/v2/members/90e0d063-2f8e-4244-860a-XXXXXXXXXX' \

+ --header 'data-partition-id: medstest-dp1' \

+ --header 'Authorization: Bearer abcdefgh123456.............'

+ ```

+

+ **Sample response**

+

+ No output for a successful response.

+

+## Next steps

+After you add users to the groups, you can:

+- [Manage legal tags](how-to-manage-legal-tags.md)

+- [Manage ACLs](how-to-manage-acls.md)

+You can also ingest data into your Azure Data Manager for Energy instance:

+You can create an Azure Event Grid subscription with an Event Hubs namespace as its source. The following tutorial shows you how to create an Event Grid subscription with an event hub as a source and an Azure Functions app as a sink: [Process and migrate captured Event Hubs data to an Azure Synapse Analytics using Event Grid and Azure Functions](store-captured-data-data-warehouse.md).

+* Italy North

+* Norway East

+* UAE North

+Yes, you can use up to two times the bandwidth limit you procured by spreading the traffic across both links of your ExpressRoute circuit and thereby using the redundant bandwidth available. The built-in redundancy of your circuit is configured using redundant links, each with procured bandwidth, to two Microsoft Enterprise Edge routers (MSEEs). The bandwidth available through your secondary link can be used for more traffic if necessary. Since the second link is meant for redundancy, it isn't guaranteed and shouldn't be used for extra traffic for a sustained period of time. To learn more about how to use both connections to transmit traffic, see [use AS PATH prepending](expressroute-optimize-routing.md#solution-use-as-path-prepending).

+If you plan to use only your primary link to transmit traffic, the bandwidth for the connection is fixed, and attempting to oversubscribe it results in increased packet drops. If traffic flows through an ExpressRoute Gateway, the bandwidth for the Gateway SKU is fixed and not burstable. For the bandwidth of each Gateway SKU, see [About ExpressRoute virtual network gateways](expressroute-about-virtual-network-gateways.md#aggthroughput).

+| **[Orange Poland](https://www.orange.pl/duze-firmy/rozwiazania-chmurowe)** | Supported | Supported | Warsaw |

+ Title: About rate limiting for ExpressRoute circuits over service provider ports

+description: This document discusses how rate limiting works for ExpressRoute circuits over service provider ports. You'll also learn how to monitor the throughput and traffic drop due to rate limiting.

+# About rate limiting for ExpressRoute circuits over service provider ports

+This article discusses how rate limiting works for ExpressRoute circuits created over service provider ports. You'll also learn how to monitor the throughput and traffic drop due to rate limiting.

+## How does rate limiting work over an ExpressRoute circuit?

+An ExpressRoute circuit consists of two links that connects the Customer/Provider edge to the Microsoft Enterprise Edge (MSEE) routers. If your circuit bandwidth is 1 Gbps and you distribute your traffic evenly across both links, you can achieve a maximum throughput of 2 Gbps (two times 1 Gbps). Rate limiting restricts your throughput to the configured bandwidth if you exceed it on either link. The ExpressRoute circuit SLA is only guaranteed for the bandwidth that you configured. For example, if you purchased a 1-Gbps circuit, your SLA is for a maximum throughput of 1 Gbps.

+## How can I determine what my circuit throughput is?

+You can monitor the ingress and egress throughput of your ExpressRoute circuit for both links through the Azure portal using ExpressRoute circuit metrics. For ingress, select `BitsInPerSecond` and for egress, select `BitsOutPerSecond`. The following screenshot shows the ExpressRoute circuit metrics for ingress and egress throughput.

+## How can I identify if traffic is being dropped due to rate limiting?

+You can monitor the traffic that is being dropped due to rate limiting through the Azure portal using the ExpressRoute circuit QOS metrics. For ingress, select `DroppedInBitsPerSecond` and for egress, select `DroppedOutBitsPerSecond`. The following screenshot shows the ExpressRoute circuit QOS metrics for ingress and egress throughput.

+## How can I increase my circuit bandwidth?

+You can seamlessly increase your circuit bandwidth through the Azure portal. For more information, see [About upgrading ExpressRoute circuit bandwidth](about-upgrade-circuit-bandwidth.md).

+## What are the causes of traffic drop when the throughput is below the configured bandwidth?

+ExpressRoute circuit throughput is monitored at an aggregate level of every few minutes, while the rate limiting is enforced at a granular level in milliseconds. Therefore, occasional traffic bursts exceeding the configured bandwidth might not get detected by the throughput monitoring. However, the rate limiting is still be enforced and traffic gets dropped.

+## Next steps

+For more frequently asked questions, see [ExpressRoute FAQ](expressroute-faqs.md).

+Azure Load Testing currently does not support other testing frameworks than Apache JMeter.

+Azure Load Testing currently does not support other testing frameworks than Apache JMeter.

+private static async Task SerializeToBlobAsync(BlobContainerClient container, RegistrationDescription[] descriptions)

+ await inputBlob.UploadAsync(stream);

+var job = await client.SubmitNotificationHubJobAsync(

+ new NotificationHubJob {

+ JobType = NotificationHubJobType.ImportCreateRegistrations,

+ OutputContainerUri = outputContainerSasUri,

+ ImportFileUri = inputFileSasUri

+ }

+ );

+ job = await client.GetNotificationHubJobAsync(job.JobId);

+ await Task.Delay(1000);

+using Azure.Storage.Blobs;

+using Azure.Storage.Sas;

+ static async Task Main(string[] args)

+ // Get a reference to a container named "sample-container" and then create it

+ await container.CreateIfNotExistsAsync();

+ await SerializeToBlobAsync(container, descriptions);

+ BlobContainerClient inputcontainer = new BlobContainerClient(STORAGE_ACCOUNT_CONNECTIONSTRING, STORAGE_ACCOUNT_CONNECTIONSTRING + "/" + INPUT_FILE_NAME);

+ var job = await client.SubmitNotificationHubJobAsync(

+ job = await client.GetNotificationHubJobAsync(job.JobId);

+ await Task.Delay(1000);

+ private static async Task SerializeToBlobAsync(BlobContainerClient container, RegistrationDescription[] descriptions)

+ await inputBlob.UploadAsync(stream);

+This action elevates a replica to the role of the primary server. In the process, the current primary server is demoted to a replica role, swapping their roles. For a successful promotion, it's necessary to have a [virtual endpoint](#virtual-endpoints-preview) configured for both the current primary as the writer endpoint, and the replica intended for promotion as the reader endpoint. The promotion will only be successful if the targeted replica is included in the reader endpoint configuration.

+- **Geo-redundant backup storage**: Geo-backup settings aren't transferred. Since replicas can't have geo-backup enabled, the promoted primary (formerly the replica) won't have it post-promotion. The feature can only be activated at the standard server's creation time (not a replica).

+If you want to configure UE usage tracking for your site, collect all the values in the following table to define the packet core instance's associated Event Hubs instance. See [Monitor UE usage with Event Hubs](ue-usage-event-hub.md) for more information.

+- Ensure you have at least one network switch with at least three ports available. You'll connect each Azure Stack Edge Pro device to the switch(es) in the same site as part of the instructions in [Order and set up your Azure Stack Edge Pro device(s)](#order-and-set-up-your-azure-stack-edge-pro-devices).

+- For every network where you decided not to enable NAPT (as described in [Allocate user equipment (UE) IP address pools](#allocate-user-equipment-ue-ip-address-pools)), configure the data network to route traffic destined for the UE IP address pools via the IP address you allocated to the packet core instance's user plane interface on the data network.

+ - If required, you can disable automatic backhauling of edge network function logs to Microsoft support using the checkbox.

+8. Go to the **Diagnostics** tab. If you want to enable UE Metric monitoring, select **Enable** from the **UE Metric monitoring** dropdown. Use the information collected in [Collect UE Usage Tracking values](collect-required-information-for-a-site.md#collect-ue-usage-tracking-values) to fill out the **Azure Event Hub Namespace**, **Event Hub name** and **User Assigned Managed Identity** values.

+1. If you decided you want to configure diagnostics packet collection or use a user assigned managed identity for HTTPS certificate for this site, select **Next : Identity >**.

+1. If you decided you want to provide a custom HTTPS certificate in [Collect local monitoring values](collect-required-information-for-a-site.md#collect-local-monitoring-values), select **Next : Local access >**. If you decided not to provide a custom HTTPS certificate at this stage, you can skip this step.

+- If you want to make changes to the packet core configuration or access network, refer to [Collect packet core configuration values](collect-required-information-for-a-site.md#collect-packet-core-configuration-values) and [Collect access network values](collect-required-information-for-a-site.md#collect-access-network-values) to collect the new values and make sure they're in the correct format. If you want to enable UE usage monitoring, refer to [Collect UE usage tracking values](collect-required-information-for-a-site.md#collect-ue-usage-tracking-values).

+- Enabling [monitoring UE usage with Event Hubs](ue-usage-event-hub.md).

+ - If you want to enable UE usage monitoring, use the information collected in [Collect UE usage tracking values](collect-required-information-for-a-site.md#collect-ue-usage-tracking-values) to fill out the **Azure Event Hub Namespace**, **Event Hub name** and **User Assigned Managed Identity** values.

+UE usage monitoring can be configured during [site creation](create-a-site.md) or at a later stage by [modifying the packet core](modify-packet-core.md).

+Once Event Hubs is receiving data from your AP5GC deployment you can write an application, using SDKs such as [.NET](/azure/event-hubs/event-hubs-dotnet-standard-getstarted-send?tabs=passwordless%2Croles-azure-portal), to consume event data and produce useful metric data.

+The new Edge Log Backhaul feature provides Microsoft support personnel with easy access to customer network function logs to help them troubleshoot and find root cause for customer issues. This is enabled by default. To disable this feature, [modify the packet core configuration](modify-packet-core.md).

+It's expected for a debug session to take longer to execute than the indexer since it goes through extra processing.

+The following table summarizes processing for each document format, and describes the metadata properties extracted by a blob indexer and the SharePoint indexer.

+[**Azure AI Search**](search-what-is-azure-search.md) is an Azure resource used for adding a full text search experience to custom apps or for providing enterprise information retrieval for chat-style search solutions.

+If you have an Azure subscription, including a [trial subscription](https://azure.microsoft.com/pricing/free-trial/?WT.mc_id=A261C142F), you can create a search service for free. Free services have limitations, but you can complete all of the quickstarts and most tutorials, except for those featuring semantic ranking (it requires a billable service).

+The easiest way to create a service is using the [Azure portal](https://portal.azure.com/), which is covered in this article. You can also use [Azure PowerShell](search-manage-powershell.md#create-or-delete-a-service), [Azure CLI](search-manage-azure-cli.md#create-or-delete-a-service), the [Management REST API](search-manage-rest.md#create-or-update-a-service), an [Azure Resource Manager service template](search-get-started-arm.md), a [Bicep file](search-get-started-bicep.md), or [Terraform](search-get-started-terraform.md).

+To try search for free, [open a free Azure account](https://azure.microsoft.com/pricing/free-trial/?WT.mc_id=A261C142F) and then create your search service by choosing the **Free** tier. You can have one free search service per Azure subscription. Free search services are intended for short-term evaluation of the product for non-production applications. If you want to move forward with a production application, create a new search service on a billable tier.

+Alternatively, you can use free credits to try out paid Azure services. With this approach, you can create your search service at **Basic** or above to get more capacity. Your credit card is never charged unless you explicitly change your settings and ask to be charged. Another approach is to [activate Azure credits in a Visual Studio subscription](https://azure.microsoft.com/pricing/member-offers/msdn-benefits-details/?WT.mc_id=A261C142F). A Visual Studio subscription gives you credits every month you can use for paid Azure services.

+description: Lists data source connectors for importing into an Azure AI Search index.

+Find a data connector from Microsoft or a partner that works with [an indexer](search-indexer-overview.md) to simplify data ingestion into a search index. This article has the following sections:

+The SharePoint connector will crawl content from any SharePoint site collection URL. The connector will retrieve Sites, Lists, Folders, List Items and Attachments, as well as other pages (in .aspx format). Supports SharePoint running in the Microsoft 365 offering.

+ Title: Move a search service across regions

+description: Learn how to move your Azure AI Search resources from one region to another in the Azure cloud.

+Occasionally, customers ask about moving a search service to another region. Currently, there's no built-in mechanism or tooling to help with that task, but this article can help you understand the manual steps for recreating indexes and other objects on a new search service in a different region.

+ Azure Storage is used for logging, creating a knowledge store, and is a commonly used external data source for AI enrichment and indexing. Azure AI services are used to power built-in skills during AI enrichment. Both Azure AI services and your search service are required to be in the same region if you're using AI enrichment.

+1. Check pricing and availability in the new region to ensure availability of Azure AI Search plus any related services in the new region. Most features are available in all regions, but some preview features have restricted availability.

+1. Create a service in the new region and republish from source code any existing indexes, synonym maps, indexers, data sources, and skillsets. Remember that service names must be unique so you can't reuse the existing name. Check each skillset to see if connections to Azure AI services are still valid in terms of the same-region requirement. Also, if knowledge stores are created, check the connection strings for Azure Storage if you're using a different service.

+1. Enable logging, and if you're using them, re-create security roles.

+ Title: Indexer troubleshooting

+description: Provides indexer problem and resolution guidance for cases when no error messages are returned from the service search.

+Occasionally, indexers run into problems that don't produce errors or that occur on other Azure services, such as during authentication or when connecting. This article focuses on troubleshooting indexer problems when there are no messages to guide you. It also provides troubleshooting for errors that come from non-search resources used during indexing.

+> [!NOTE]

+> If you have an Azure AI Search error to investigate, see [Troubleshooting common indexer errors and warnings](cognitive-search-common-errors-warnings.md) instead.

+For data sources under Azure network security, indexers are limited in how they make the connection. Currently, indexers can access restricted data sources [behind an IP firewall](search-indexer-howto-access-ip-restricted.md) or on a virtual network through a [private endpoint](search-indexer-howto-access-private.md) using a shared private link.

+Azure Storage, Azure Cosmos DB and Azure SQL provide a configurable firewall. There's no specific error message when the firewall blocks the request. Typically, firewall errors are generic. Some common errors include:

+* Configure an inbound rule for the IP address of your search service and the IP address range of `AzureCognitiveSearch` [service tag](../virtual-network/service-tags-overview.md#available-service-tags). Details for configuring IP address range restrictions for each data source type can be found from the following links:

+ * [Azure Storage](../storage/common/storage-network-security.md#grant-access-from-an-internet-ip-range)

+ * [Azure Cosmos DB](../storage/common/storage-network-security.md#grant-access-from-an-internet-ip-range)

+ * [Azure SQL](/azure/azure-sql/database/firewall-configure#create-and-manage-ip-firewall-rules)

+* As a last resort or as a temporary measure, disable the firewall by allowing access from **All Networks**.

+**Limitation**: IP address range restrictions only work if your search service and your storage account are in different regions.

+In addition to data retrieval, indexers also send outbound requests through skillsets and [custom skills](cognitive-search-custom-skill-web-api.md). For custom skills based on an Azure function, be aware that Azure functions also have [IP address restrictions](/azure/azure-functions/ip-addresses#ip-address-restrictions). The list of IP addresses to allow through for custom skill execution include the IP address of your search service and the IP address range of `AzureCognitiveSearch` service tag.

+### Network security group (NSG) rules

+When an indexer accesses data on a SQL managed instance, or when an Azure VM is used as the web service URI for a [custom skill](cognitive-search-custom-skill-web-api.md), the network security group determines whether requests are allowed in.

+For external resources residing on a virtual network, [configure inbound NSG rules](/azure/virtual-network/manage-network-security-group#work-with-security-rules) for the `AzureCognitiveSearch` service tag.

+When you receive any of those errors:

+* Check your resource in the Azure portal for any current errors or outages

+* Verify you're using a public DNS for name resolution and not an [Azure Private DNS](/azure/dns/private-dns-overview)

+If the database is paused, the first sign in from your search service is expected to auto-resume the database, but instead returns an error stating that the database is unavailable, giving error code 40613. After the database is running, retry the sign in to establish connectivity.

+When you create a SharePoint indexer, there's a step requiring you to sign in to your Microsoft Entra app after providing a device code. If you receive a message that says `"Your sign-in was successful but your admin requires the device requesting access to be managed"`, the indexer is probably blocked from the SharePoint document library by a [Conditional Access](../active-directory/conditional-access/overview.md) policy.

+To update the policy and allow indexer access to the document library:

+1. Open the Azure portal and search for **Microsoft Entra Conditional Access**.

+1. Select **Policies** on the left menu. If you don't have access to view this page, you need to either find someone who has access or get access.

+1. Once you've confirmed which policy is blocking the indexer, make an exemption for the indexer. Start by retrieving the search service IP address.

+ First, obtain the fully qualified domain name (FQDN) of your search service. The FQDN looks like `<your-search-service-name>.search.windows.net`. You can find the FQDN in the Azure portal.

+ 

+ Now that you have the FQDN, get the IP address of the search service by performing a `nslookup` (or a `ping`) of the FQDN. In the following example, you would add "150.0.0.1" to an inbound rule on the Azure Storage firewall. It might take up to 15 minutes after the firewall settings have been updated for the search service indexer to be able to access the Azure Storage account.

+ Extra IP addresses are used for requests that originate from the indexer's [multitenant execution environment](search-indexer-securing-resources.md#indexer-execution-environment). You can get this IP address range from the service tag.

+ For this exercise, assuming the search service is the Azure Public cloud, the [Azure Public JSON file](https://www.microsoft.com/download/details.aspx?id=56519) should be downloaded.

+ From the JSON file, assuming the search service is in West Central US, the list of IP addresses for the multitenant indexer execution environment are listed below.

+1

+ * For your search service IP address, you might need to add "/32" to the end of the IP address since it only accepts valid IP ranges.

+1. Exclude the new Named location from the policy:

+1. Attempt to create the indexer again:

+If you're indexing content from Azure Blob Storage, and the container includes blobs of an [unsupported content type](search-howto-indexing-azure-blob-storage.md#SupportedFormats), the indexer skips that document. In other cases, there might be problems with individual documents.

+In this situation, you can [set configuration options](search-howto-indexing-azure-blob-storage.md#DealingWithErrors) to allow indexer processing to continue in the event of problems with individual documents.

+PUT https://[service name].search.windows.net/indexers/[indexer name]?api-version=2023-11-01

+* [Change tracking](/rest/api/searchservice/create-data-source#data-change-detection-policies) values are erroneous or prerequisites are missing. If your high watermark value is a date set to a future time, then any documents that have an earlier date are skipped by the indexer. You can determine your indexer's change tracking state using the 'initialTrackingState' and 'finalTrackingState' fields in the [indexer status](/rest/api/searchservice/get-indexer-status#indexer-execution-result). Indexers for Azure SQL and MySQL must have an index on the high water mark column of the source table, or queries used by the indexer might time out.

+## Document count discrepancy between the data source and index

+An indexer might show a different document count than either the data source, the index itself, or count in your code. Here are some possible reasons why this behavior can occur:

+- The index can lag in showing the real document count, especially in the portal.

+- The indexer has a Deleted Document Policy. The deleted documents get counted by the indexer if the documents are indexed before they get deleted.

+- If the ID column in the data source isn't unique. This applies to data sources that have the concept of columns, such as Azure Cosmos DB.

+- If the data source definition has a different query than the one you're using to estimate the number of records. In example, in your database, you're querying the database record count, while in the data source definition query, you might be selecting just a subset of records to index.

+- The counts are being checked at different intervals for each component of the pipeline: data source, indexer and index.

+Indexers use a conservative buffering strategy to ensure that every new and changed document in the data source is picked up during indexing. In certain situations, these buffers can overlap, causing an indexer to index a document two or more times resulting in the processed documents count to be more than actual number of documents in the data source. This behavior does **not** affect the data stored in the index, such as duplicating documents, only that it can take longer to reach eventual consistency. This condition is especially prevalent if any of the following criteria are true:

+Indexers aren't intended to be invoked multiple times in quick succession. If you need updates quickly, the supported approach is to push updates to the index while simultaneously updating the data source. For on-demand processing, we recommend that you pace your requests in five-minute intervals or more, and run the indexer on a schedule.

+In practice, this scenario only happens when on-demand indexers are manually invoked within minutes of each other, for certain data sources. It can result in mismatched numbers (like the indexer processed 345 documents total according to the indexer execution stats, but there are 340 documents in the data source and index) or potentially increased billing if you're running the same skills for the same document multiple times. Running an indexer using a schedule is the preferred recommendation.

+If you have strings in multiple languages, you can attach [language analyzers](index-add-language-analyzers.md#supported-language-analyzers) that analyze strings using linguistic rules of a specific language during indexing and query execution. With a language analyzer, you get better handling of character variations, punctuation, and word root forms.

+Azure AI Search supports Microsoft and Lucene analyzers. By default, the search engine uses Standard Lucene, which is language agnostic. If testing indicates that the default analyzer is insufficient, replace it with a language analyzer.

+In Azure AI Search, the two patterns for supporting multiple languages include:

+This article focuses on steps and best practices for configuring and querying language-specific fields in a blended index:

+> + Define a string field for each language variant.

+> + Set a language analyzer on each field.

+> + On the query request, set the `searchFields` parameter to specific fields, and then use `select` to return just those fields that have compatible content.

+> [!NOTE]

+> If you're using large language models in a retrieval augmented generated (RAG) pattern, you can engineer the prompt to return translated strings. That scenario is out of scope for this article.

+Language analysis applies to fields of type `Edm.String` that are `searchable`, and that contain localized text. If you also need text translation, review the next section to see if AI enrichment meets your needs.

+Non-string fields and non-searchable string fields don't undergo lexical analysis and aren't tokenized. Instead, they're stored and returned verbatim.

+This article assumes translated strings alreach exist. If that's not the case, you can attach Azure AI services to an [enrichment pipeline](cognitive-search-concept-intro.md), invoking text translation during indexing. Text translation takes a dependency on the indexer feature and Azure AI services, but all setup is done within Azure AI Search.

+The `analyzer` property on a field definition is used to set the [language analyzer](index-add-language-analyzers.md). It's used for both indexing and query execution.

+ }

+ ]

+}

+| `searchFields` | Limits full text search to the list of named fields. |

+| `select` | Trims the response to include only the fields you specify. By default, all retrievable fields are returned. The `select` parameter lets you choose which ones to return. |

+Given a goal of constraining search to fields containing French strings, you would use `searchFields` to target the query at fields containing strings in that language.

+Specifying the analyzer on a query request isn't necessary. A language analyzer on the field definition determines text analysis during query execution. For queries that specify multiple fields, each invoking different language analyzers, the terms or phrases are processed concurrently by the assigned analyzers for each field.

+By default, a search returns all fields that are marked as retrievable. As such, you might want to exclude fields that don't conform to the language-specific search experience you want to provide. Specifically, if you limited search to a field with French strings, you probably want to exclude fields with English strings from your results. Using the `select` query parameter gives you control over which fields are returned to the calling application.

+Sometimes the language of the agent issuing a query isn't known, in which case the query can be issued against all fields simultaneously. IA preference for results in a certain language can be defined using [scoring profiles](index-add-scoring-profiles.md). In the example below, matches found in the description in French are scored higher relative to matches in other languages:

+POST /indexes/hotels/docs/search?api-version=2023-11-01

+This article explains the billing model and billable events of Azure AI Search, and provides guidance for managing the costs.

+² In an [indexer configuration](/rest/api/searchservice/create-indexer#indexer-parameters), `imageAction` is the parameter that triggers image extraction. If `imageAction` is set to "none" (the default), you won't be charged for image extraction. Costs are incurred when `imageAction` parameter is set *and* you include OCR, Image Analysis, or Document Extraction in a skillset.

+You aren't billed on the number of full text or vector queries, query responses, or documents ingested, although [service limits](search-limits-quotas-capacity.md) do apply at each tier.

+Several premium features such as [knowledge store](knowledge-store-concept-intro.md), [debug sessions](cognitive-search-debug-session.md), and [enrichment cache](cognitive-search-incremental-indexing-conceptual.md) have a dependency on Azure Storage. The meters for Azure Storage apply in this case, and the associated storage costs of using these features are included in the Azure Storage bill.

+Skillsets can include [billable built-in skills](cognitive-search-predefined-skills.md), non-billable built-in utility skills, and custom skills. Non-billable utility skills include Conditional, Shaper, Text Merge, Text Split. You aren't charged for using them. There's no API key requirement, and no 20 document limit.

+A custom skill is functionality you provide. The cost of using a custom skill depends entirely on whether custom code is calling other billable services. There's no API key requirement and no 20 document limit on custom skills.

+1. If you're using [AI enrichment](cognitive-search-concept-intro.md), there's an extra charge for blob storage, but the cumulative cost goes down if you enable [enrichment caching](cognitive-search-incremental-indexing-conceptual.md).

+In-place upgrade or downgrade isn't supported. Changing a service tier requires provisioning a new service at the desired tier.

+ Title: Synonyms for query expansion

+description: Create a synonym map to expand the scope of a search query over an Azure AI Search index. Scope is broadened to include equivalent terms you provide in the synonym map.

+On a search service, synonym maps are a global resource that associate equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming "dog", "canine", and "puppy" are mapped synonyms, a query on "canine" will match on a document containing "dog".

+You might create multiple synonym maps for different languages, such as English and French versions, or lexicons if your content includes technical jargon, slang, or obscure terminology. Although you can create multiple synonym maps in your search service, within an index, a field definition can only have one synonym map assignment.

+POST /synonymmaps?api-version=2023-11-01

+Query parsers automatically lower-case any upper or mixed case terms, but if you want to preserve special characters in the string, such as a comma or dash, add the appropriate escape characters when creating the synonym map.

+Rules for equivalent terms are comma-delimited within the same rule. In the first example, a query on `USA` expands to `USA` OR `"United States"` OR `"United States of America"`. Notice that if you want to match on a phrase, the query itself must be a quote-enclosed phrase query.

+In the equivalence case, a query for `dog` expands the query to also include `puppy` and `canine`.

+Rules for an explicit mapping are denoted by an arrow `=>`. When specified, a term sequence of a search query that matches the left-hand side of `=>` is replaced with the alternatives on the right-hand side at query time.

+In the explicit case, a query for `Washington`, `Wash.` or `WA` is rewritten as `WA`, and the query engine only looks for matches on the term `WA`. Explicit mapping only applies in the direction specified, and doesn't rewrite the query `WA` to `Washington` in this case.

+ "format": "solr",

+ "synonyms": "WA\, USA, WA, Washington\n"

+ "format":"solr",

+ "synonyms": "WA\\, USA, WA, Washington"

+Azure AI Search supports IP rules for inbound access through a firewall, similar to the IP rules found in an Azure virtual network security group. By applying IP rules, you can restrict service access to an approved set of devices and cloud services. An IP rule only allows the request through. Access to data and operations will still require the caller to present a valid authorization token.

+ The Azure portal supports IP addresses and IP address ranges in the CIDR format. An example of CIDR notation is 8.8.8.0/24, which represents the IPs that range from 8.8.8.0 to 8.8.8.255.

+When IP rules are configured, some features of the Azure portal are disabled. You can view and manage service level information, but portal access to indexes, indexers, and other top-level resources is restricted. You can restore portal access to the full range of search service operations by allowing access from the portal IP address and your client IP address.

+In the following example, the IP address that you should copy is `52.252.175.48`.

+When services run in different regions, they connect to different traffic managers. Regardless of the domain name, the IP address returned from the ping is the correct one to use when defining an inbound firewall rule for the Azure portal in your region.

+For ping, the request will time out, but the IP address is visible in the response. For example, in the message `"Pinging azsyrie.northcentralus.cloudapp.azure.com [52.252.175.48]"`, the IP address is `52.252.175.48`.

+| [Storage Analytics metrics (classic)](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json)³ | &nbsp;&#x2B24; | &nbsp;&#x2B24; | &nbsp;&#x2B24; | &nbsp;&#x2B24; |

+³ Storage Analytics metrics is retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json).

+| [Storage Analytics metrics (classic)](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json)³ | &nbsp;&#x2B24; | &nbsp;&#x2B24; | &nbsp;&#x2B24; | &nbsp;&#x2B24; |

+³ Storage Analytics metrics is retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json).

+To monitor your existing storage accounts and gather this data, you can make use of storage metrics in Azure Monitor. Azure Monitor stores metrics that include aggregated transaction statistics and capacity data about requests to the storage service. Azure Storage sends metric data to the Azure Monitor back end. Azure Monitor provides a unified monitoring experience that includes data from the Azure portal as well as data that is ingested. For more information, see any of these articles:

+- [Monitoring Azure Blob Storage](../blobs/monitor-blob-storage.md)

+- [Monitoring Azure Files](../files/storage-files-monitoring.md)

+- [Monitoring Azure Queue Storage](../queues/monitor-queue-storage.md)

+- [Monitoring Azure Table storage](../tables/monitor-table-storage.md)

+- The amount of data retrieved from the storage account can be estimated by looking at the sum of the *'Ingress'* metric for primarily the *'GetBlob'* and *'CopyBlob'* operations.

+- The amount of data written to the storage account can be estimated by looking at the sum of *'Egress'* metrics for primarily the *'PutBlob'*, *'PutBlock'*, *'CopyBlob'* and *'AppendBlock'* operations.

+To determine the price of each operation against the blob storage service, see [Map each REST operation to a price](../blobs/map-rest-apis-transaction-categories.md).

+ Title: Use Azure Storage analytics to collect logs data

+description: Storage Analytics enables you to collect logs for Blob, Queue, and Table storage.

+Azure Storage Analytics performs logging for a storage account. You can use this data to trace requests, analyze usage trends, and diagnose issues with your storage account.

+> [!NOTE]

+> Storage Analytics supports only logs. Storage Analytics metrics are retired. See [Transition to metrics in Azure Monitor](../common/storage-analytics-metrics.md?toc=/azure/storage/blobs/toc.json). While Storage Analytics logs are still supported, we recommend that you use Azure Storage logs in Azure Monitor instead of Storage Analytics logs. To learn more, see any of the following articles:

+>

+> - [Monitoring Azure Blob Storage](../blobs/monitor-blob-storage.md)

+> - [Monitoring Azure Files](../files/storage-files-monitoring.md)

+> - [Monitoring Azure Queue Storage](../queues/monitor-queue-storage.md)

+> - [Monitoring Azure Table storage](../tables/monitor-table-storage.md)

+The aggregated log data is stored in a well-known blob, which may be accessed using the Blob service and Table service APIs.

+The amount of storage used by logs data is billable. You're also billed for requests to create blobs for logging.

+If you have configured a data retention policy, you can reduce the spending by deleting old log data. For more information about retention policies, see [Setting a Storage Analytics Data Retention Policy](/rest/api/storageservices/Setting-a-Storage-Analytics-Data-Retention-Policy).

+Every request made to an account's storage service is either billable or non-billable. Storage Analytics logs each individual request made to a service, including a status message that indicates how the request was handled. See [Understanding Azure Storage Billing - Bandwidth, Transactions, and Capacity](/archive/blogs/windowsazurestorage/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity).

+When looking at Storage Analytics data, you can use the tables in the [Storage Analytics Logged Operations and Status Messages](/rest/api/storageservices/storage-analytics-logged-operations-and-status-messages) topic to determine what requests are billable. Then you can compare your log data to the status messages to see if you were charged for a particular request. You can also use the tables in the previous topic to investigate availability for a storage service or individual AP

+- [Azure Storage client libraries for .NET](/dotnet/api/overview/azure/storage)

+- [Azure Storage client libraries for Java](/java/api/overview/azure/storage)

+- [Azure Storage client libraries for JavaScript](/javascript/api/overview/azure/storage)

+- [Azure Storage client libraries for Python](/python/api/overview/azure/storage)

+- [Azure Storage client libraries for Go](https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage/)

+- [Azure Storage client libraries for C++](https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/storage)

+- [Storage Data Movement Client Library for .NET](storage-use-data-movement-library.md)

+- [AzCopy Command-Line Utility](storage-use-azcopy-v10.md)

+On **January 9, 2024** Storage Analytics metrics, also referred to as *classic metrics* retired. If you used classic metrics, this article helps you transition to metrics in Azure Monitor.

+Azure File Sync enables you to centralize your organization's file shares in Azure Files, while keeping the flexibility, performance, and compatibility of a Windows file server. While some users might opt to keep a full copy of their data locally, Azure File Sync additionally has the ability to transform Windows Server into a quick cache of your Azure file share. You can use any protocol that's available on Windows Server to access your data locally, including SMB, NFS, and FTPS. You can have as many caches as you need across the world.

+Azure File Sync is backed by Azure Files, which offers several redundancy options for highly available storage. Because Azure contains resilient copies of your data, your local server becomes a disposable caching device. You can recover from a failed server by adding a new server to your Azure File Sync deployment. Rather than restoring from a local backup, you provision another Windows Server, install the Azure File Sync agent on it, and then add it to your Azure File Sync deployment. Azure File Sync downloads your file namespace before downloading data, so that your server can be up and running as soon as possible. For even faster recovery, you can have a warm standby server as part of your deployment, or you can use Azure File Sync with Windows Clustering.

+| V14.1 Release - [KB5001873](https://support.microsoft.com/topic/d06b8723-c4cf-4c64-b7ec-3f6635e044c5)| 14.1.0.0 | December 1, 2021 | Supported - Agent version will expire on February 8, 2024 |

+| V14 Release - [KB5001872](https://support.microsoft.com/topic/92290aa1-75de-400f-9442-499c44c92a81)| 14.0.0.0 | October 29, 2021 | Supported - Agent version will expire on February 8, 2024 |

+ Azure Files can be used to replace or supplement traditional on-premises file servers or network-attached storage (NAS) devices. Popular operating systems such as Windows, macOS, and Linux can directly mount Azure file shares wherever they are in the world. SMB Azure file shares can also be replicated with Azure File Sync to Windows servers, either on-premises or in the cloud, for performance and distributed caching of the data. With [Azure Files AD Authentication](storage-files-active-directory-overview.md), SMB Azure file shares can work with Active Directory Domain Services (AD DS) hosted on-premises for access control.

+ Azure Files makes it easy to "lift and shift" applications to the cloud that expect a file share to store file application or user data. Azure Files enables both the "classic" lift and shift scenario, where both the application and its data are moved to Azure, and the "hybrid" lift and shift scenario, where the application data is moved to Azure Files, and the application continues to run on-premises.

+* **Easy to use**. When an Azure file share is mounted on your computer, you don't need to do anything special to access the data: just navigate to the path where the file share is mounted and open/modify a file.

+* **Shared access**. Azure file shares support the industry standard SMB and NFS protocols, meaning you can seamlessly replace your on-premises file shares with Azure file shares without worrying about application compatibility. Being able to share a file system across multiple machines, applications, and application instances is a significant advantage for applications that need shareability.

+* **Scripting and tooling**. PowerShell cmdlets and Azure CLI can be used to create, mount, and manage Azure file shares as part of the administration of Azure applications. You can create and manage Azure file shares using Azure portal and Azure Storage Explorer.

+* **Resiliency**. Azure Files has been built from the ground up to be always available. Replacing on-premises file shares with Azure Files means you no longer have to wake up to deal with local power outages or network issues.

+ Title: Microsoft Entra joined session hosts in Azure Virtual Desktop

+description: Learn about using Microsoft Entra joined session hosts in Azure Virtual Desktop.

+# Microsoft Entra joined session hosts in Azure Virtual Desktop

+- For the full PowerShell reference documentation, see [Az.DesktopVirtualization](/powershell/module/az.desktopvirtualization).

+ - For **Size (TiB)**, enter the capacity pool size that best fits your needs.

+Custom images can be stored in [Azure Compute Gallery](../virtual-machines/azure-compute-gallery.md) or as a [managed image](../virtual-machines/windows/capture-image-resource.md), or both. Azure Compute Gallery allows you to manage region replication, versioning, and sharing of custom images. See [Create a legacy managed image of a generalized VM in Azure](../virtual-machines/capture-image-resource.md) to review limitations for managed images.

+|Maximum session limit| One. | As configured by the [maximum session limit](configure-host-pool-load-balancing.md#configure-breadth-first-load-balancing) value of the properties of a host pool. Under high concurrent connection load when multiple users connect to the host pool at the same time, the number of sessions created on a session host can exceed the maximum session limit. |

+|User assignment process| Users can either be directly assigned to session hosts or be automatically assigned to the first available session host. Users always have sessions on the session hosts they are assigned to. | Users aren't assigned to session hosts. After a user signs out and signs back in, their user session might get load balanced to a different session host. To learn more, see [Configure personal desktop assignment](configure-host-pool-personal-desktop-assignment-type.md). |

+You can set a host pool to be a [validation environment](configure-validation-environment.md). Validation environments let you monitor service updates before the service applies them to your production or non-validation environment. Without a validation environment, you may not discover changes that introduce errors, which could result in downtime for users in your production environment.

+An [application group](deploy-azure-virtual-desktop.md#create-an-application-group) is a logical grouping of applications installed on session hosts in the host pool.

+A [workspace](deploy-azure-virtual-desktop.md#create-a-workspace) is a logical grouping of application groups in Azure Virtual Desktop. Each Azure Virtual Desktop application group must be associated with a workspace for users to see the desktops and applications published to them.

+> Application Health Extension expects to receive a consistent probe response at the configured port `tcp` or request path `http/https` in order to label a VM as *Healthy*. If no application is running on the VM, or you're unable to configure a probe response, your VM is going to show up as *Unhealthy* (Binary Health States) or *Unknown* (Rich Health States).

+ "location": "<location>",

+ "location": "<location>",

+Generally, if you're using customer-managed keys, you should enable automatic key rotation to the latest key version. Automatic key rotation helps ensure your keys are secure. A disk references a key via its disk encryption set. When you enable automatic rotation for a disk encryption set, the system will automatically update all managed disks, snapshots, and images referencing the disk encryption set to use the new version of the key within one hour. To learn how to enable customer-managed keys with automatic key rotation, see [Set up an Azure Key Vault and DiskEncryptionSet with automatic key rotation](windows/disks-enable-customer-managed-keys-powershell.md#set-up-an-azure-key-vault-and-diskencryptionset-optionally-with-automatic-key-rotation).

+ Title: Use Application Health extension with Azure Virtual Machines

+description: Learn how to use the Application Health extension to monitor the health of your applications deployed on Azure virtual machines.

+# Using Application Health extension with Azure Virtual Machines

+Monitoring your application health is an important signal for managing your VMs. Azure Virtual Machines provides support for [Automatic VM Guest Patching](../automatic-vm-guest-patching.md), which rely on health monitoring of the individual instances to safely update your VMs.

+This article describes how you can use the two types of Application Health extension, **Binary Health States** or **Rich Health States**, to monitor the health of your applications deployed on Azure virtual machines.

+Application health monitoring is also available on virtual machine scale sets and helps enable functionalities such as [Rolling Upgrades](../../virtual-machine-scale-sets/virtual-machine-scale-sets-upgrade-policy.md), [Automatic OS-Image Upgrades](../../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-upgrade.md), and [Automatic Instance Repairs](../../virtual-machine-scale-sets/virtual-machine-scale-sets-automatic-instance-repairs.md). To experience these capabilities with the added benefits of scale, availability, and flexibility on scale sets, you can [attach your VM to an existing scale set](../../virtual-machine-scale-sets/virtual-machine-scale-sets-attach-detach-vm.md) or [create a new scale set](../../virtual-machine-scale-sets/flexible-virtual-machine-scale-sets-portal.md).

+## Prerequisites

+This article assumes that you're familiar with [Azure virtual machine extensions](overview.md).

+> [!CAUTION]

+> Application Health Extension expects to receive a consistent probe response at the configured port `tcp` or request path `http/https` in order to label a VM as *Healthy*. If no application is running on the VM, or you're unable to configure a probe response, your VM is going to show up as *Unhealthy* (Binary Health States) or *Unknown* (Rich Health States).

+## When to use the Application Health extension

+Application Health Extension reports on application health from inside the Virtual Machine. The extension probes on a local application endpoint and updates the health status based on TCP/HTTP(S) responses received from the application. This health status is used by Azure to monitor and detect patching failures during [Automatic VM Guest Patching](../automatic-vm-guest-patching.md).

+The extension reports health from within a VM and can be used in situations where an external probe such as the [Azure Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) canΓÇÖt be used.

+Application health is a customer-provided signal on the status of your application running inside the VM. Application health is different from [resource health](../../service-health/resource-health-overview.md), which is a platform-provided signal used to report service-level events impacting the performance of your VM.

+## Binary versus Rich Health States

+Application Health Extensions has two options available: **Binary Health States** and **Rich Health States**. The following table highlights some key differences between the two options. See the end of this section for general recommendations.

+| Features | Binary Health States | Rich Health States |

+| -- | -- | |

+| Available Health States | Two available states: *Healthy*, *Unhealthy* | Four available states: *Healthy*, *Unhealthy*, *Initializing*, *Unknown*¹ |

+| Sending Health Signals | Health signals are sent through HTTP/HTTPS response codes or TCP connections. | Health signals on HTTP/HTTPS protocol are sent through the probe response code and response body. Health signals through TCP protocol remain unchanged from Binary Health States. |

+| Identifying *Unhealthy* Instances | Instances automatically fall into *Unhealthy* state if a *Healthy* signal isn't received from the application. An *Unhealthy* instance can indicate either an issue with the extension configuration (for example, unreachable endpoint) or an issue with the application (for example, non-200 status code). | Instances only go into an *Unhealthy* state if the application emits an *Unhealthy* probe response. Users are responsible for implementing custom logic to identify and flag instances with *Unhealthy* applications². Instances with incorrect extension settings (for example, unreachable endpoint) or invalid health probe responses will fall under the *Unknown* state². |

+| *Initializing* state for newly created instances | *Initializing* state isn't available. Newly created instances may take some time before settling into a steady state. | *Initializing* state allows newly created instances to settle into a steady Health State before surfacing the health state as _Healthy_, _Unhealthy_, or _Unknown_. |

+| HTTP/HTTPS protocol | Supported | Supported |

+| TCP protocol | Supported | Limited Support ΓÇô *Unknown* state is unavailable on TCP protocol. See [Rich Health States protocol table](#rich-health-states) for Health State behaviors on TCP. |

+¹ The *Unknown* state is unavailable on TCP protocol.

+² Only applicable for HTTP/HTTPS protocol. TCP protocol follows the same process of identifying *Unhealthy* instances as in Binary Health States.

+Use **Binary Health States** if:

+- You're not interested in configuring custom logic to identify and flag an unhealthy instance

+- You don't require an *initializing* grace period for newly created instances

+Use **Rich Health States** if:

+- You send health signals through HTTP/HTTPS protocol and can submit health information through the probe response body

+- You would like to use custom logic to identify and mark unhealthy instances

+- You would like to set an *initializing* grace period allowing newly created instances to settle into a steady health state

+## Binary Health States

+Binary Health State reporting contains two Health States, *Healthy* and *Unhealthy*. The following tables provide a brief description for how the Health States are configured.

+**HTTP/HTTPS Protocol**

+| Protocol | Health State | Description |

+| -- | | -- |

+| http/https | Healthy | To send a *Healthy* signal, the application is expected to return a 200 response code. |

+| http/https | Unhealthy | The instance is marked as *Unhealthy* if a 200 response code isn't received from the application. |

+**TCP Protocol**

+| Protocol | Health State | Description |

+| -- | | -- |

+| TCP | Healthy | To send a *Healthy* signal, a successful handshake must be made with the provided application endpoint. |

+| TCP | Unhealthy | The instance is marked as *Unhealthy* if a failed or incomplete handshake occurred with the provided application endpoint. |

+Some common scenarios that result in an *Unhealthy* state include:

+- When the application endpoint returns a non-200 status code

+- When there's no application endpoint configured inside the virtual machine to provide application health status

+- When the application endpoint is incorrectly configured

+- When the application endpoint isn't reachable

+## Rich Health States

+Rich Health States reporting contains four Health States, *Initializing*, *Healthy*, *Unhealthy*, and *Unknown*. The following tables provide a brief description for how each Health State is configured.

+**HTTP/HTTPS Protocol**

+| Protocol | Health State | Description |

+| -- | | -- |

+| http/https | Healthy | To send a *Healthy* signal, the application is expected to return a probe response with: **Probe Response Code**: Status 2xx, **Probe Response Body**: `{"ApplicationHealthState": "Healthy"}` |

+| http/https | Unhealthy | To send an *Unhealthy* signal, the application is expected to return a probe response with: **Probe Response Code**: Status 2xx, **Probe Response Body**: `{"ApplicationHealthState": "Unhealthy"}` |

+| http/https | Initializing | The instance automatically enters an *Initializing* state at extension start time. For more information, see [Initializing state](#initializing-state). |

+| http/https | Unknown | An *Unknown* state may occur in the following scenarios: when a non-2xx status code is returned by the application, when the probe request times out, when the application endpoint is unreachable or incorrectly configured, when a missing or invalid value is provided for `ApplicationHealthState` in the response body, or when the grace period expires. For more information, see [Unknown state](#unknown-state). |

+**TCP Protocol**

+| Protocol | Health State | Description |

+| -- | | -- |

+| TCP | Healthy | To send a *Healthy* signal, a successful handshake must be made with the provided application endpoint. |

+| TCP | Unhealthy | The instance is marked as *Unhealthy* if a failed or incomplete handshake occurred with the provided application endpoint. |

+| TCP | Initializing | The instance automatically enters an *Initializing* state at extension start time. For more information, see [Initializing state](#initializing-state). |

+## Initializing state

+This state only applies to Rich Health States. The *Initializing* state only occurs once at extension start time and can be configured by the extension settings `gracePeriod` and `numberOfProbes`.

+At extension startup, the application health remains in the *Initializing* state until one of two scenarios occurs:

+- The same Health State (*Healthy* or *Unhealthy*) is reported a consecutive number of times as configured through *numberOfProbes*

+- The `gracePeriod` expires

+If the same Health State (*Healthy* or *Unhealthy*) is reported consecutively, the application health transitions out of the *Initializing* state and into the reported Health State (*Healthy* or *Unhealthy*).

+### Example

+If `numberOfProbes` = 3, that would mean:

+- To transition from *Initializing* to *Healthy* state: Application health extension must receive three consecutive *Healthy* signals via HTTP/HTTPS or TCP protocol

+- To transition from *Initializing* to *Unhealthy* state: Application health extension must receive three consecutive *Unhealthy* signals via HTTP/HTTPS or TCP protocol

+If the `gracePeriod` expires before a consecutive health status is reported by the application, the instance health is determined as follows:

+- HTTP/HTTPS protocol: The application health transitions from *Initializing* to *Unknown*

+- TCP protocol: The application health transitions from *Initializing* to *Unhealthy*

+## Unknown state

+The *Unknown* state only applies to Rich Health States. This state is only reported for `http` or `https` probes and occurs in the following scenarios:

+- When a non-2xx status code is returned by the application

+- When the probe request times out

+- When the application endpoint is unreachable or incorrectly configured

+- When a missing or invalid value is provided for `ApplicationHealthState` in the response body

+- When the grace period expires

+## Extension schema for Binary Health States

+The following JSON shows the schema for the Application Health extension. The extension requires at a minimum either a "tcp", "http" or "https" request with an associated port or request path respectively.

+```json

+{

+ "type": "extensions",

+ "name": "HealthExtension",

+ "apiVersion": "2018-10-01",

+ "location": "<location>",

+ "properties": {

+ "publisher": "Microsoft.ManagedServices",

+ "type": "<ApplicationHealthLinux or ApplicationHealthWindows>",

+ "autoUpgradeMinorVersion": true,

+ "typeHandlerVersion": "1.0",

+ "settings": {

+ "protocol": "<protocol>",

+ "port": <port>,

+ "requestPath": "</requestPath>",

+ "intervalInSeconds": 5,

+ "numberOfProbes": 1

+ }

+ }

+}

+```

+### Property values

+| Name | Value / Example | Data Type |

+| - | | |

+| apiVersion | `2018-10-01` or above | date |

+| publisher | `Microsoft.ManagedServices` | string |

+| type | `ApplicationHealthLinux` (Linux), `ApplicationHealthWindows` (Windows) | string |

+| typeHandlerVersion | `1.0` | string |

+### Settings

+| Name | Value / Example | Data Type |

+| - | | |

+| protocol | `http` or `https` or `tcp` | string |

+| port | Optional when protocol is `http` or `https`, mandatory when protocol is `tcp` | int |

+| requestPath | Mandatory when protocol is `http` or `https`, not allowed when protocol is `tcp` | string |

+| intervalInSeconds | Optional, default is 5 seconds. This setting is the interval between each health probe. For example, if intervalInSeconds == 5, a probe is sent to the local application endpoint once every 5 seconds. | int |

+| numberOfProbes | Optional, default is 1. This setting is the number of consecutive probes required for the health status to change. For example, if numberOfProbles == 3, you will need 3 consecutive "Healthy" signals to change the health status from "Unhealthy" into "Healthy" state. The same requirement applies to change health status into "Unhealthy" state. | int |

+## Extension schema for Rich Health States

+The following JSON shows the schema for the Rich Health States extension. The extension requires at a minimum either an "http" or "https" request with an associated port or request path respectively. TCP probes are also supported, but cannot set the `ApplicationHealthState` through the probe response body and do not have access to the *Unknown* state.

+```json

+{

+ "type": "extensions",

+ "name": "HealthExtension",

+ "apiVersion": "2018-10-01",

+ "location": "<location>",

+ "properties": {

+ "publisher": "Microsoft.ManagedServices",

+ "type": "<ApplicationHealthLinux or ApplicationHealthWindows>",

+ "autoUpgradeMinorVersion": true,

+ "typeHandlerVersion": "2.0",

+ "settings": {

+ "protocol": "<protocol>",

+ "port": <port>,

+ "requestPath": "</requestPath>",

+ "intervalInSeconds": 5,

+ "numberOfProbes": 1,

+ "gracePeriod": 600

+ }

+ }

+}

+```

+### Property values

+| Name | Value / Example | Data Type |

+| - | | |

+| apiVersion | `2018-10-01` or above | date |

+| publisher | `Microsoft.ManagedServices` | string |

+| type | `ApplicationHealthLinux` (Linux), `ApplicationHealthWindows` (Windows) | string |

+| typeHandlerVersion | `2.0` | string |

+### Settings

+| Name | Value / Example | Data Type |

+| - | | |

+| protocol | `http` or `https` or `tcp` | string |

+| port | Optional when protocol is `http` or `https`, mandatory when protocol is `tcp` | int |

+| requestPath | Mandatory when protocol is `http` or `https`, not allowed when protocol is `tcp` | string |

+| intervalInSeconds | Optional, default is 5 seconds. This setting is the interval between each health probe. For example, if intervalInSeconds == 5, a probe is sent to the local application endpoint once every 5 seconds. | int |

+| numberOfProbes | Optional, default is 1. This setting is the number of consecutive probes required for the health status to change. For example, if numberOfProbles == 3, you will need 3 consecutive "Healthy" signals to change the health status from "Unhealthy"/"Unknown" into "Healthy" state. The same requirement applies to change health status into "Unhealthy" or "Unknown" state. | int |

+| gracePeriod | Optional, default = `intervalInSeconds` * `numberOfProbes`; maximum grace period is 7200 seconds | int |

+## Deploy the Application Health extension

+There are multiple ways of deploying the Application Health extension to your VMs as detailed in the following examples.

+### Binary Health States

+# [REST API](#tab/rest-api)

+The following example adds the Application Health extension named *myHealthExtension* to a Windows-based virtual machine.

+You can also use this example to change an existing extension from Rich Health States to Binary Health by making a PATCH call instead of a PUT.

+```

+PUT on `/subscriptions/subscription_id/resourceGroups/myResourceGroup/providers/Microsoft.Compute/virtualMachines/myVM/extensions/myHealthExtension?api-version=2018-10-01`

+```

+```json

+{

+ "name": "myHealthExtension",

+ "location": "<location>",

+ "properties": {

+ "publisher": "Microsoft.ManagedServices",

+ "type": "ApplicationHealthWindows",

+ "autoUpgradeMinorVersion": true,

+ "typeHandlerVersion": "1.0",

+ "settings": {

+ "protocol": "<protocol>",

+ "port": <port>,

+ "requestPath": "</requestPath>"

+ }

+ }

+}

+```

+Use `PATCH` to edit an already deployed extension.

+# [Azure PowerShell](#tab/azure-powershell)

+Use the [Set-AzVmExtension](/powershell/module/az.compute/set-azvmextension) cmdlet to add or update Application Health extension to your virtual machine.

+The following example adds the Application Health extension to a Windows-based virtual machine.

+You can also use this example to change an existing extension from Rich Health States to Binary Health.

+```azurepowershell-interactive

+# Define the Application Health extension properties

+$publicConfig = @{"protocol" = "http"; "port" = 80; "requestPath" = "/healthEndpoint"};

+# Add the Application Health extension to the virtual machine

+Set-AzVMExtension -Name "myHealthExtension" `

+ -ResourceGroupName "<myResourceGroup>" `

+ -VMName "<myVM>" `

+ -Publisher "Microsoft.ManagedServices" `

+ -ExtensionType "ApplicationHealthWindows" `

+ -TypeHandlerVersion "1.0" `

+ -Location "<location>" `

+ -Settings $publicConfig

+

+```

+# [Azure CLI 2.0](#tab/azure-cli)

+Use [az vm extension set](/cli/azure/vm/extension#az-vm-extension-set) to add the Application Health extension to a virtual machine.

+The following example adds the Application Health extension to a Linux-based virtual machine.

+You can also use this example to change an existing extension from Rich Health States to Binary Health.

+```azurecli-interactive

+az vm extension set \

+ --name ApplicationHealthLinux \

+ --publisher Microsoft.ManagedServices \

+ --version 1.0 \

+ --resource-group <myResourceGroup> \

+ --vm-name <myVM> \

+ --settings ./extension.json

+```

+The extension.json file content.

+```json

+{

+ "protocol": "<protocol>",

+ "port": <port>,

+ "requestPath": "</requestPath>"

+}

+```

+# [Azure portal](#tab/azure-portal)

+The following example adds the Application Health extension to an existing virtual machine on [Azure portal](https://portal.azure.com).

+1. Navigate to your existing Virtual Machine

+2. On the left sidebar, go to the **Health monitoring** blade

+3. Click on **Enable application health monitoring**, select **Binary** for Health States. Configure your protocol, port, and more to set up the health probes.

+4. Click **Save** to save your settings

+### Rich Health States

+# [REST API](#tab/rest-api)

+The following example adds the **Application Health - Rich States** extension (with name myHealthExtension) to a Windows-based virtual machine.

+You can also use this example to upgrade an existing extension from Binary to Rich Health States by making a PATCH call instead of a PUT.

+```

+PUT on `/subscriptions/subscription_id/resourceGroups/myResourceGroup/providers/Microsoft.Compute/virtualMachines/myVM/extensions/myHealthExtension?api-version=2018-10-01`

+```

+```json

+{

+ "name": "myHealthExtension",

+ "location": "<location>",

+ "properties": {

+ "publisher": "Microsoft.ManagedServices",

+ "type": "ApplicationHealthWindows",

+ "autoUpgradeMinorVersion": true,

+ "typeHandlerVersion": "2.0",

+ "settings": {

+ "requestPath": "</requestPath>",

+ "intervalInSeconds": <intervalInSeconds>,

+ "numberOfProbes": <numberOfProbes>,

+ "gracePeriod": <gracePeriod>

+ }

+ }

+}

+```

+Use `PATCH` to edit an already deployed extension.

+# [Azure PowerShell](#tab/azure-powershell)

+Use the [Set-AzVmExtension](/powershell/module/az.compute/set-azvmextension) cmdlet to add or update Application Health extension to your virtual machine.

+The following example adds the **Application Health - Rich States** extension to a Windows-based virtual machine.

+You can also use this example to upgrade an existing extension from Binary to Rich Health States.

+```azurepowershell-interactive

+# Define the Application Health extension properties

+$publicConfig = @{"protocol" = "http"; "port" = 80; "requestPath" = "/healthEndpoint"; "gracePeriod" = 600};

+# Add the Application Health extension to the virtual machine

+Set-AzVMExtension -Name "myHealthExtension" `

+ -ResourceGroupName "<myResourceGroup>" `

+ -VMName "<myVM>" `

+ -Publisher "Microsoft.ManagedServices" `

+ -ExtensionType "ApplicationHealthWindows" `

+ -TypeHandlerVersion "2.0" `

+ -Location "<location>" `

+ -Settings $publicConfig

+

+```

+# [Azure CLI 2.0](#tab/azure-cli)

+Use [az vm extension set](/cli/azure/vm/extension#az-vm-extension-set) to add the Application Health extension to a virtual machine.

+The following example adds the **Application Health - Rich States** extension to a Linux-based virtual machine.

+You can also use this example to upgrade an existing extension from Binary to Rich Health States.

+```azurecli-interactive

+az vm extension set \

+ --name ApplicationHealthLinux \

+ --publisher Microsoft.ManagedServices \

+ --version 2.0 \

+ --resource-group <myResourceGroup> \

+ --vm-name <myVM> \

+ --settings ./extension.json

+```

+The extension.json file content.

+```json

+{

+ "protocol": "<protocol>",

+ "port": <port>,

+ "requestPath": "</requestPath>",

+ "gracePeriod": <healthExtensionGracePeriod>

+}

+```

+# [Azure portal](#tab/azure-portal)

+The following example adds the Application Health extension to an existing virtual machine on [Azure portal](https://portal.azure.com).

+1. Navigate to your existing Virtual Machine

+2. On the left sidebar, go to the **Health monitoring** blade

+3. Click on **Enable application health monitoring**, select **Rich (advanced)** for Health States. Configure your protocol, port, and more to set up the health probes.

+4. Click **Save** to save your settings

+## Troubleshoot

+### View VMHealth

+# [REST API](#tab/rest-api)

+```

+GET https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/myResourceGroup/providers/Microsoft.Compute/virtualMachines/myVM/instanceView?api-version=2023-07-01

+```

+Sample Response (see "vmHealth" object for the latest VM health status)

+```

+"vmHealth": {

+ "status": {

+ "code": "HealthState/unknown",

+ "level": "Warning",

+ "displayStatus": "The VM health is unknown",

+ "time": "2023-12-04T22:25:39+00:00"

+ }

+}

+```

+# [Azure PowerShell](#tab/azure-powershell)

+```azurepowershell-interactive

+Get-AzVM

+ -ResourceGroupName "<rgName>" `

+ -Name "<vmName>" `

+ -Status

+```

+# [Azure CLI 2.0](#tab/azure-cli)

+```azurecli-interactive

+az vm get-instance-view --name <vmName> --resource-group <rgName>

+```

+# [Azure portal](#tab/azure-portal)

+1. Navigate to your existing Virtual Machine

+2. On the left sidebar, go to the **Overview** blade

+3. Your application health can be observed under the **Health State** field

+### Extension execution output log

+Extension execution output is logged to files found in the following directories:

+```Windows

+C:\WindowsAzure\Logs\Plugins\Microsoft.ManagedServices.ApplicationHealthWindows\<version>\

+```

+```Linux

+/var/lib/waagent/Microsoft.ManagedServices.ApplicationHealthLinux-<extension_version>/status

+/var/log/azure/applicationhealth-extension

+```

+The logs also periodically capture the application health status.

+| [General purpose](sizes-general.md) | B, Dsv3, Dv3, Dasv4, Dav4, DSv2, Dv2, Av2, Dpdsv5, Dpldsv5, Dpsv5, Dplsv5, Dv4, Dsv4, Ddv4, Ddsv4, Dv5, Dsv5, Ddv5, Ddsv5, Dasv5, Dadsv5, DCasv5, DCadsv5, DCesv5, DCedsv5 | Balanced CPU-to-memory ratio. Ideal for testing and development, small to medium databases, and low to medium traffic web servers. |

+| [Memory optimized](sizes-memory.md) | Esv3, Ev3, Easv4, Eav4, Epdsv5, Epsv5, Ev4, Esv4, Edv4, Edsv4, Ev5, Esv5, Edv5, Edsv5, Easv5, Eadsv5, Mv2, M, DSv2, Dv2, ECasv5, ECadsv5, ECesv5, ECedsv5 | High memory-to-CPU ratio. Great for relational database servers, medium to large caches, and in-memory analytics. |

+Migrations can get a bit challenging when the above requirement is coupled with an extra requirement to keep some applications on-premises. In such a situation, you have to split the applications between Azure and on-premises, without renumbering the IP addresses on either side. Additionally, you have to allow the applications to communicate as if they are in the same network.

+[Extend your on-premises subnets into Azure using Azure Extended Network](/windows-server/manage/windows-admin-center/azure/azure-extended-network).

+|Create alert rule for number of routes advertised to peer. |**Count of Routes Advertised to Peers** monitors the number of routes advertised from the ExpressRoute gateway to the virtual hub router and to the Microsoft Enterprise Edge Devices.<br><br>We recommend that you **add a filter** to **only** select the two BGP peers displayed as **ExpressRoute Device** and create an alert to identify when the count of advertised routes approaches the documented limit of **1000**. For example, configure the alert to be triggered when the number of routes advertised is **greater than 950**.<br><br>We also recommend that you configure an alert when the number of routes advertised to the Microsoft Edge Devices is **zero** in order to proactively detect any connectivity issues.<br><br>To add these alerts, select the **Count of Routes Advertised to Peers** metric, and then select the **Add filter** option and the **ExpressRoute** devices.|

+|Create alert rule for number of routes learned from peer.|**Count of Routes Learned from Peers** monitors the number of routes the ExpressRoute gateway learns from the virtual hub router and from the Microsoft Enterprise Edge Device.<br><br>We recommend that you add a filter to **only** select the two BGP peers displayed as **ExpressRoute Device** and create an alert to identify when the count of learned routes approaches the [documented limit](../expressroute/expressroute-faqs.md#are-there-limits-on-the-number-of-routes-i-can-advertise) of 4000 for Standard SKU and 10,000 for Premium SKU circuits.<br><br>We also recommend that you configure an alert when the number of routes advertised to the Microsoft Edge Devices is **zero**. This can help in detecting when your on-premises has stopped advertising routes.

+* See [Create diagnostic settings in Azure Monitor](../azure-monitor/essentials/diagnostic-settings.md) for more information and troubleshooting when creating diagnostic settings via the Azure portal, CLI, PowerShell, etc.