+| **Cost** | [Global deployment pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) | [Regional pricing](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) | May experience cost savings for consistent usage |

+- Video translation

+- [Fast transcription REST API reference](/rest/api/speechtotext/transcriptions/transcribe)

+- [Speech to text supported languages](./language-support.md?tabs=stt)

+- [Batch transcription](./batch-transcription.md)

+### Fast transcription

+The supported locales for the [fast transcription API](fast-transcription-create.md) are: en-US, es-ES, es-MX, fr-FR, hi-IN, it-IT, ja-JP, ko-KR, pt-BR, and zh-CN. You can only specify one locale per transcription request.

+- Video translation

+> [See the Speech to text REST API 2024-05-15 reference documentation](/rest/api/speechtotext/operation-groups?view=rest-speechtotext-2024-05-15-preview&preserve-view=true)

+> [See the Speech to text REST API v3.2 reference documentation](/rest/api/speechtotext/operation-groups?view=rest-speechtotext-v3.2&preserve-view=true)

+> [See the Speech to text REST API v3.1 reference documentation](/rest/api/speechtotext/operation-groups?view=rest-speechtotext-v3.1&preserve-view=true)

+- [Fast transcription](fast-transcription-create.md): Transcribe audio files with returning results synchronously and much faster than real-time audio. Use the fast transcription API ([/speechtotext/transcriptions:transcribe](/rest/api/speechtotext/transcriptions/transcribe)) in the scenarios that you need the transcript of an audio recording as quickly as possible with predictable latency, such as quick audio or video transcription or video translation.

+- [Custom speech](custom-speech-overview.md): Upload your own data, test and train a custom model, compare accuracy between models, and deploy a model to a custom endpoint. Copy models to other subscriptions if you want colleagues to have access to a model that you built, or if you want to deploy a model to more than one region.

+|`style="documentary-narration"`|Narrates documentaries in a relaxed, interested, and informative style suitable for documentaries, expert commentary, and similar content.|

+- Video translation

+The process of replacing the original language of a video with audio recorded in a different language is often relied upon to cater to diverse audiences. Traditionally achieved through human recording and manual post-production, translation is essential for ensuring that viewers can enjoy video content in their native language. However, this process comes with key pain points, including its high cost, lengthy duration, and inability to replicate the original speaker's voice accurately. Video translation in Azure AI Speech addresses these challenges by providing an automated, efficient, and cost-effective solution for creating localized videos.

+- **Automatic translation ΓÇô voice generation in other language.**

+ Utilizes AI-powered text-to-speech technology to automatically generate human-like voices in the target language. These voices are precisely synchronized with the video, ensuring a flawless translation experience. This includes utilizing prebuilt neural voices for high-quality output and offering options for personal voice.

+Features | Managed compute | Serverless API (pay-as-you-go)

+Deployment experience and billing | Model weights are deployed to dedicated Virtual Machines with Managed Online Endpoints. The managed online endpoint, which can have one or more deployments, makes available a REST API for inference. You're billed for the Virtual Machine core hours used by the deployments. | Access to models is through a deployment that provisions an API to access the model. The API provides access to the model hosted and managed by Microsoft, for inference. You're billed for inputs and outputs to the APIs, typically in tokens; pricing information is provided before you deploy.

+Content safety | Use Azure Content Safety service APIs. | Azure AI Content Safety filters are available integrated with inference APIs. Azure AI Content Safety filters is billed separately.

+Network isolation | [Configure managed networks for Azure AI Studio hubs.](configure-managed-network.md) | Endpoints will follow your hub's public network access (PNA) flag setting. For more information, see the [Network isolation for models deployed via Serverless APIs](#network-isolation-for-models-deployed-via-serverless-apis) section.

+Certain models in the Model Catalog can be deployed as serverless APIs with pay-as-you-go billing, providing a way to consume them as an API without hosting them on your subscription. Models are hosted in infrastructure managed by Microsoft, which enables API-based access to the model provider's model. API based access can dramatically reduce the cost of accessing a model and significantly simplify the provisioning experience.

+Models that are available for deployment as serverless APIs with pay-as-you-go billing are offered by the model provider but hosted in Microsoft-managed Azure infrastructure and accessed via API. Model providers define the license terms and set the price for use of their models, while Azure Machine Learning service manages the hosting infrastructure, makes the inference APIs available, and acts as the data processor for prompts submitted and content output by models deployed via MaaS. Learn more about data processing for MaaS at the [data privacy](concept-data-privacy.md) article.

+### Billing

+### Fine-tune models

+Certain models support also serverless fine-tuning where users can take advantage of hosted fine-tuning with pay-as-you-go billing to tailor the models using data they provide. For more information, see the [fine-tuning overview](../concepts/fine-tuning-overview.md).

+ Title: Dapr extension for Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes

+# Dapr extension for Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes

+[Using the Dapr extension to provision Dapr on your AKS or Arc-enabled Kubernetes cluster][dapr-create-extension] eliminates the overhead of:

+You can run Azure CLI commands to retrieve a list of available versions in [a cluster](/cli/azure/k8s-extension/extension-types#az-k8s-extension-extension-types-list-versions-by-cluster) or [a location](/cli/azure/k8s-extension/extension-types#az-k8s-extension-extension-types-list-versions-by-location).

+> [!div class="nextstepaction"]

+> [Walk through the Dapr extension quickstart to demo how it works][dapr-quickstart]

+[dapr-create-extension]: ./dapr.md

+[dapr-alpha-beta]: https://docs.dapr.io/operations/support/alpha-beta-apis/

+- [Walk through the tutorial for deploying Dapr Workflow via the extension][dapr-workflow]

+- [Determine if you need to migrate from Dapr open source to the Dapr extension][dapr-migration].

+[dapr-workflow]: ./dapr-workflow.md

+- [Configure the Dapr extension on your AKS cluster][dapr-config].

+- [Determine if you need to migrate from Dapr open source to the Dapr extension][dapr-migration].

+[dapr-migration]: ./dapr-migration.md

+ Title: Install the Dapr extension for Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes

+# Install the Dapr extension for Azure Kubernetes Service (AKS) and Arc-enabled Kubernetes

+> [!div class="nextstepaction"]

+> [Configure the Dapr extension for your unique scenario][dapr-settings]

+## Support

+### Does AKS offer a service-level agreement?

+AKS provides SLA guarantees in the [Standard pricing tier with the Uptime SLA feature][pricing-tiers].

+The Free pricing tier doesn't have an associated Service Level *Agreement*, but has a Service Level *Objective* of 99.5%. Transient connectivity issues are observed if there's an upgrade, unhealthy underlay nodes, platform maintenance, an application overwhelms the API Server with requests, etc. For mission-critical and production workloads, or if your workload doesn't tolerate API Server restarts, we recommend using the Standard tier, which includes Uptime SLA.

+### What is platform support, and what does it include?

+Platform support is a reduced support plan for unsupported "N-3" version clusters. Platform support only includes Azure infrastructure support. Platform support doesn't include anything related to the following:

+- Kubernetes functionality and components

+- Cluster or node pool creation

+- Hotfixes

+- Bug fixes

+- Security patches

+- Retired components

+For more information on restrictions, see the [platform support policy][supported-kubernetes-versions].

+AKS relies on the releases and patches from [Kubernetes](https://kubernetes.io/releases/), which is an Open Source project that only supports a sliding window of *three* minor versions. AKS can only guarantee [full support](./supported-kubernetes-versions.md#kubernetes-version-support-policy) while those versions are being serviced upstream. Since there's no more patches being produced upstream, AKS can either leave those versions unpatched or fork. Due to this limitation, platform support doesn't support anything from relying on kubernetes upstream.

+### Does AKS automatically upgrade my unsupported clusters?

+AKS initiates auto-upgrades for unsupported clusters. When a cluster in an n-3 version (where n is the latest supported AKS GA minor version) is about to drop to n-4, AKS automatically upgrades the cluster to n-2 to remain in an AKS support [policy][supported-kubernetes-versions]. Automatically upgrading a platform supported cluster to a supported version is enabled by default.

+For example, Kubernetes v1.25 upgrades to v1.26 during the v1.29 GA release. To minimize disruptions, set up [maintenance windows][planned-maintenance]. See [auto-upgrade][auto-upgrade-cluster] for details on automatic upgrade channels.

+### Can I run Windows Server containers on AKS?

+Yes, Windows Server containers are available on AKS. To run Windows Server containers in AKS, you create a node pool that runs Windows Server as the guest OS. Windows Server containers can use only Windows Server 2019. To get started, see [Create an AKS cluster with a Windows Server node pool](./learn/quick-windows-container-deploy-cli.md).

+Windows Server support for node pool includes some limitations that are part of the upstream Windows Server in Kubernetes project. For more information on these limitations, see [Windows Server containers in AKS limitations][aks-windows-limitations].

+### Can I apply Azure reservation discounts to my AKS agent nodes?

+AKS agent nodes are billed as standard Azure virtual machines. If you purchased [Azure reservations][reservation-discounts] for the VM size that you're using in AKS, those discounts are automatically applied.

+## Operations

+### Can I move/migrate my cluster between Azure tenants?

+Moving your AKS cluster between tenants is currently unsupported.

+### Can I move/migrate my cluster between subscriptions?

+Movement of clusters between subscriptions is currently unsupported.

+### Can I move my AKS clusters from the current Azure subscription to another?

+Moving your AKS cluster and its associated resources between Azure subscriptions isn't supported.

+### Can I move my AKS cluster or AKS infrastructure resources to other resource groups or rename them?

+Moving or renaming your AKS cluster and its associated resources isn't supported.

+### Can I restore my cluster after deleting it?

+No, you cannot restore your cluster after deleting it. When you delete your cluster, the node resource group and all its resources are also deleted. An example of the second resource group is *MC_myResourceGroup_myAKSCluster_eastus*.

+If you want to keep any of your resources, move them to another resource group before deleting your cluster. If you want to protect against accidental deletes, you can lock the AKS managed resource group hosting your cluster resources using [Node resource group lockdown][node-resource-group-lockdown].

+### Can I scale my AKS cluster to zero?

+You can completely [stop a running AKS cluster](start-stop-cluster.md), saving on the respective compute costs. Additionally, you may also choose to [scale or autoscale all or specific `User` node pools](scale-cluster.md#scale-user-node-pools-to-0) to 0, maintaining only the necessary cluster configuration.

+You can't directly scale [system node pools](use-system-pools.md) to zero.

+### Can I use the Virtual Machine Scale Set APIs to scale manually?

+No, scale operations by using the Virtual Machine Scale Set APIs aren't supported. Use the AKS APIs (`az aks scale`).

+### Can I use Virtual Machine Scale Sets to manually scale to zero nodes?

+No, scale operations by using the Virtual Machine Scale Set APIs aren't supported. You can use the AKS API to scale to zero nonsystem node pools or [stop your cluster](start-stop-cluster.md) instead.

+### Can I stop or de-allocate all my VMs?

+While AKS has resilience mechanisms to withstand such a config and recover from it, it isn't a supported configuration. [Stop your cluster](start-stop-cluster.md) instead.

+### Why are two resource groups created with AKS?

+### Can I provide my own name for the AKS node resource group?

+### Can I modify tags and other properties of the AKS resources in the node resource group?

+## Quotas, limits, and region availability

+### Which Azure regions currently provide AKS?

+For a complete list of available regions, see [AKS regions and availability][aks-regions].

+### Can I spread an AKS cluster across regions?

+No. AKS clusters are regional resources and can't span regions. See [best practices for business continuity and disaster recovery][bcdr-bestpractices] for guidance on how to create an architecture that includes multiple regions.

+### Can I spread an AKS cluster across availability zones?

+Yes. You can deploy an AKS cluster across one or more [availability zones][availability-zones] in [regions that support them][az-regions].

+### Can I have different VM sizes in a single cluster?

+Yes, you can use different virtual machine sizes in your AKS cluster by creating [multiple node pools][multi-node-pools].

+### What's the size limit on a container image in AKS?

+AKS doesn't set a limit on the container image size. However, it's important to understand that the larger the image, the higher the memory demand. A larger size could potentially exceed resource limits or the overall available memory of worker nodes. By default, memory for VM size Standard_DS2_v2 for an AKS cluster is set to 7 GiB.

+When a container image is excessively large, as in the Terabyte (TBs) range, kubelet might not be able to pull it from your container registry to a node due to lack of disk space.

+#### Windows Server nodes

+For Windows Server nodes, Windows Update doesn't automatically run and apply the latest updates. On a regular schedule around the Windows Update release cycle and your own validation process, you should perform an upgrade on the cluster and the Windows Server node pool(s) in your AKS cluster. This upgrade process creates nodes that run the latest Windows Server image and patches, then removes the older nodes. For more information on this process, see [Upgrade a node pool in AKS][nodepool-upgrade].

+### Are AKS images required to run as root?

+The following images have functional requirements to "Run as Root" and exceptions must be filed for any policies:

+- *mcr.microsoft.com/oss/kubernetes/coredns*

+- *mcr.microsoft.com/azuremonitor/containerinsights/ciprod*

+- *mcr.microsoft.com/oss/calico/node*

+- *mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi*

+## Security, access, and identity

+### Can I limit who has access to the Kubernetes API server?

+Yes. There are two options for limiting access to the API server:

+- Use [API Server Authorized IP Ranges][api-server-authorized-ip-ranges] if you want to maintain a public endpoint for the API server but restrict access to a set of trusted IP ranges.

+- Use a [private cluster][private-clusters] if you want to limit the API server to *only* be accessible from within your virtual network.

+### Are security updates applied to AKS agent nodes?

+AKS patches CVEs that have a "vendor fix" every week. CVEs without a fix are waiting on a "vendor fix" before they can be remediated. The AKS images are automatically updated inside of 30 days. We recommend you apply an updated Node Image on a regular cadence to ensure that latest patched images and OS patches are all applied and current. You can do this using one of the following methods:

+- Manually, through the Azure portal or the Azure CLI.

+- By upgrading your AKS cluster. The cluster upgrades [cordon and drain nodes][cordon-drain] automatically and then bring a new node online with the latest Ubuntu image and a new patch version or a minor Kubernetes version. For more information, see [Upgrade an AKS cluster][aks-upgrade].

+- By using [node image upgrade](node-image-upgrade.md).

+### Are there security threats targeting AKS that I should be aware of?

+Microsoft provides guidance for other actions you can take to secure your workloads through services like [Microsoft Defender for Containers](../defender-for-cloud/defender-for-containers-introduction.md?tabs=defender-for-container-arch-aks). The following security threat is related to AKS and Kubernetes that you should be aware of:

+- [New large-scale campaign targets Kubeflow](https://techcommunity.microsoft.com/t5/azure-security-center/new-large-scale-campaign-targets-kubeflow/ba-p/2425750) (June 8, 2021).

+### Does AKS store any customer data outside of the cluster's region?

+No, all data is stored in the cluster's region.

+### How to avoid permission ownership setting slow issues when the volume has numerous files

+Traditionally if your pod is running as a nonroot user (which you should), you must specify a `fsGroup` inside the pod's security context so the volume can be readable and writable by the Pod. This requirement is covered in more detail [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).

+A side effect of setting `fsGroup` is that each time a volume is mounted, Kubernetes must recursively `chown()` and `chmod()` all the files and directories inside the volume (with a few exceptions noted below). This scenario happens even if group ownership of the volume already matches the requested `fsGroup`. It can be expensive for larger volumes with lots of small files, which can cause pod startup to take a long time. This scenario has been a known problem before v1.20, and the workaround is setting the Pod run as root:

+```yaml

+apiVersion: v1

+kind: Pod

+metadata:

+ name: security-context-demo

+spec:

+ securityContext:

+ runAsUser: 0

+ fsGroup: 0

+```

+The issue has been resolved with Kubernetes version 1.20. For more information, see [Kubernetes 1.20: Granular Control of Volume Permission Changes](https://kubernetes.io/blog/2020/12/14/kubernetes-release-1.20-fsgroupchangepolicy-fsgrouppolicy/).

+## Networking

+### How does the managed Control Plane communicate with my Nodes?

+AKS uses a secure tunnel communication to allow the api-server and individual node kubelets to communicate even on separate virtual networks. The tunnel is secured through mTLS encryption. The current main tunnel that is used by AKS is [Konnectivity, previously known as apiserver-network-proxy](https://kubernetes.io/docs/tasks/extend-kubernetes/setup-konnectivity/). Verify all network rules follow the [Azure required network rules and FQDNs](limit-egress-traffic.md).

+### Can my pods use the API server FQDN instead of the cluster IP?

+Yes, you can add the annotation `kubernetes.azure.com/set-kube-service-host-fqdn` to pods to set the `KUBERNETES_SERVICE_HOST` variable to the domain name of the API server instead of the in-cluster service IP. This is useful in cases where your cluster egress is done via a layer 7 firewall, such as when using Azure Firewall with Application Rules.

+### Can I configure NSGs with AKS?

+AKS doesn't apply Network Security Groups (NSGs) to its subnet and doesn't modify any of the NSGs associated with that subnet. AKS only modifies the network interfaces NSGs settings. If you're using CNI, you also must ensure the security rules in the NSGs allow traffic between the node and pod CIDR ranges. If you're using kubenet, you must also ensure the security rules in the NSGs allow traffic between the node and pod CIDR. For more information, see [Network security groups](concepts-network.md#network-security-groups).

+### How does Time synchronization work in AKS?

+AKS nodes run the "chrony" service, which pulls time from the localhost. Containers running on pods get the time from the AKS nodes. Applications launched inside a container use time from the container of the pod.

+### What is Azure CNI Transparent Mode vs. Bridge Mode?

+#### Bridge mode

+#### Transparent mode

+#### Benefits of Transparent mode

+## Add-ons, extensions, and integrations

+### Can I use custom VM extensions?

+No, AKS is a managed service, and manipulation of the IaaS resources isn't supported. To install custom components, use the Kubernetes APIs and mechanisms. For example, use DaemonSets to install required components.

+### What Kubernetes admission controllers does AKS support? Can admission controllers be added or removed?

+AKS supports the following [admission controllers][admission-controllers]:

+- *NamespaceLifecycle*

+- *LimitRanger*

+- *ServiceAccount*

+- *DefaultIngressClass*

+- *DefaultStorageClass*

+- *DefaultTolerationSeconds*

+- *MutatingAdmissionWebhook*

+- *ValidatingAdmissionWebhook*

+- *ResourceQuota*

+- *PodNodeSelector*

+- *PodTolerationRestriction*

+- *ExtendedResourceToleration*

+Currently, you can't modify the list of admission controllers in AKS.

+### Can I use admission controller webhooks on AKS?

+Yes, you can use admission controller webhooks on AKS. It's recommended you exclude internal AKS namespaces, which are marked with the **control-plane label.** For example:

+```output

+namespaceSelector:

+ matchExpressions:

+ - key: control-plane

+ operator: DoesNotExist

+AKS firewalls the API server egress so your admission controller webhooks need to be accessible from within the cluster.

+### Can admission controller webhooks impact kube-system and internal AKS namespaces?

+To protect the stability of the system and prevent custom admission controllers from impacting internal services in the kube-system, namespace AKS has an **Admissions Enforcer**, which automatically excludes kube-system and AKS internal namespaces. This service ensures the custom admission controllers don't affect the services running in kube-system.

+If you have a critical use case for deploying something on kube-system (not recommended) in support of your custom admission webhook, you may add the following label or annotation so that Admissions Enforcer ignores it.

+Label: ```"admissions.enforcer/disabled": "true"``` or Annotation: ```"admissions.enforcer/disabled": true```

+### Is Azure Key Vault integrated with AKS?

+[Azure Key Vault Provider for Secrets Store CSI Driver][aks-keyvault-provider] provides native integration of Azure Key Vault into AKS.

+### Can I use FIPS cryptographic libraries with deployments on AKS?

+FIPS-enabled nodes are now supported on Linux-based node pools. For more information, see [Add a FIPS-enabled node pool](create-node-pools.md#fips-enabled-node-pools).

+### How are AKS addons updated?

+### What is the purpose of the AKS Linux Extension I see installed on my Linux Virtual Machine Scale Sets instances?

+## Troubleshooting cluster issues

+### Why is my cluster delete taking so long?

+Most clusters are deleted upon user request. In some cases, especially cases where you bring your own Resource Group or perform cross-RG tasks, deletion can take more time or even fail. If you have an issue with deletes, double-check that you don't have locks on the RG, that any resources outside of the RG are disassociated from the RG, and so on.

+### Why is my cluster create/update taking so long?

+If you have issues with create and update cluster operations, make sure you don't have any assigned policies or service constraints that may block your AKS cluster from managing resources like VMs, load balancers, tags, etc.

+### If I have pod / deployments in state 'NodeLost' or 'Unknown' can I still upgrade my cluster?

+You can, but we don't recommend it. You should perform updates when the state of the cluster is known and healthy.

+### If I have a cluster with one or more nodes in an Unhealthy state or shut down, can I perform an upgrade?

+No, delete/remove any nodes in a failed state or otherwise from the cluster before upgrading.

+### I ran a cluster delete, but see the error `[Errno 11001] getaddrinfo failed`

+Most commonly, this error arises if you have one or more Network Security Groups (NSGs) still in use that are associated with the cluster. Remove them and attempt the delete again.

+### I ran an upgrade, but now my pods are in crash loops, and readiness probes fail

+Confirm your service principal hasn't expired. See: [AKS service principal](./kubernetes-service-principal.md) and [AKS update credentials](./update-credentials.md).

+### My cluster was working, but suddenly can't provision LoadBalancers, mount PVCs, etc.

+Confirm your service principal hasn't expired. See: [AKS service principal](./kubernetes-service-principal.md) and [AKS update credentials](./update-credentials.md).

+> [Learn how to create the Dapr extension][dapr-create-extension]

+[dapr-overview]: ./dapr-overview.md

+[dapr-create-extension]: ./dapr.md

+> [!NOTE]

+> AKS start operations will restore all objects from ETCD with the exception of standalone pods with the same names and ages. meaning that a pod's age will continue to be calculated from its original creation time. This count will keep increasing over time, regardless of whether the cluster is in a stopped state.

+ Title: Discover shadow APIs using Dev Proxy

+description: Learn how to discover shadow APIs in your apps using Dev Proxy and onboard them to API Center.

+# Discover shadow APIs using Dev Proxy

+* **API governance** - Organize and filter APIs and related resources using built-in and custom metadata, to help with API governance and discovery by API consumers. Set up [linting and analysis](enable-api-analysis-linting.md) to enforce API definition quality. Integrate with tools such as Dev Proxy to ensure that apps don't use unregistered [shadow APIs](discover-shadow-apis-dev-proxy.md) or APIs that don't meet organizational standards.

+For Linux/custom containers, the ZIP file contains console output logs for both the docker host and the docker container. For a scaled-out app, the ZIP file contains one set of logs for each instance. In the App Service file system, these log files are the contents of the */home/LogFiles* directory. Deployment logs are stored in */site/deployments/*.

+ If you're using **Azure role-based access control** follow the article [Assign a managed identity access to a resource](/entra/identity/managed-identities-azure-resources/how-to-assign-access-azure-resource) and assign the user-assigned managed identity the **Key Vault Secrets User** role to the Azure Key Vault.

+You can use variants to override the enabled state of a feature flag. This gives variants an opportunity to extend the evaluation of a feature flag. If a caller is checking whether a flag that has variants is enabled, the feature manager will check if the variant assigned to the current user is set up to override the result. This is done using the optional variant property `status_override`. By default, this property is set to `None`, which means the variant doesn't affect whether the flag is considered enabled or disabled. Setting `status_override` to `Enabled` allows the variant, when chosen, to override a flag to be enabled. Setting `status_override` to `Disabled` provides the opposite functionality, therefore disabling the flag when the variant is chosen. A feature with an `enabled` state of `false` can't be overridden.

+## Deployment .zip file requirements

+A zip deployment process extracts the zip archive's files and folders in the `wwwroot` directory. If you include the parent directory when creating the archive, the system will not find the files it expects to see in `wwwroot`.

+### Publish your application

+#### Deployment payload

+Many of the deployment methods make use of a zip archive. If you are creating the zip archive yourself, it must follow the structure outlined in this section. If it does not, your app may experience errors at startup.

+The deployment payload should match the output of a `dotnet publish` command, though without the enclosing parent folder. The zip archive should be made from the following files:

+- `.azurefunctions/`

+- `extensions.json`

+- `functions.metadata`

+- `host.json`

+- `worker.config.json`

+- Your project executable (a console app)

+- Other supporting files and directories peer to that executable

+These files are generated by the build process, and they are not meant to be edited directly.

+When preparing a zip archive for deployment, you should only compress the contents of the output directory, not the enclosing directory itself. When the archive is extracted into the current working directory, the files listed above need to be immediately visible.

+### Creating the zip archive

+- A basic understanding of Creator. For an overview, see [Creator for indoor maps].

+[Azure Maps Creator onboarding tool]: creator-onboarding-tool.md

+[Creator for indoor maps]: creator-indoor-maps.md

+- Understanding of [Creator concepts]

+- A [dataset]

+2. Now that the drawing package is uploaded, use `udid` for the uploaded package to convert the package into map data.

+Each floor of a facility is provided as an individual DWG file. As a result, it's possible that the floors aren't perfectly aligned when stacked on top of each other. Azure Maps Conversion service requires that all drawings be aligned with the physical space. To verify alignment, use a reference point that can span across floors, such as an elevator or column that spans multiple floors. You can view all the floors by opening a new drawing, and then use the `XATTACH` command to load all floor drawings. If you need to fix any alignment issues, you can use the reference points and the `MOVE` command to realign the floors that require it.

+[Azure Maps Creator onboarding tool]: creator-onboarding-tool.md

+- An Azure Maps Creator [tileset].

+- An Azure Maps Creator [dataset] and [tileset].

+> - Replace `{datasetId`} with your `datasetId`.

+* A [dataset]

+[dataset]: creator-indoor-maps.md#datasets

+| Identifier | Description |

+|--||

+| conversionId | The ID returned when converting your drawing package. |

+1. A `tilesetId`. The tileset ID is used to render indoor maps with the Azure Maps iOS SDK.

+[Creator]: creator-indoor-maps.md

+The Azure Maps Get Map Tile API provides map tiles in vector or raster formats to be used in the [Azure Maps Web SDK] or 3rd party map controls. Some example tiles that can be requested are Azure Maps road, satellite/aerial, or weather radar.

+| imageUrl (Json)<BR>ImageUrl (XML)       | Not supported  | Azure Maps Get Map Tile API provides the map tile image directly in the HTML response (binary image string), as opposed to an image URL. |

+| imageUrlSubdomains (Json)<BR>ImageUrlSubdomains (XML)  | Not supported  | Azure Maps Get Map Tile API provides the map tile image directly in the HTML response (binary image string), as opposed to an image URL. |

+> Note: Once logs are migrated, MMA will not be able to write to the destination table. This is an issue for the migration of production system that we are actively working.

+>

+> [!NOTE]

+>

+> If you wish to add labels to all the jobs in your custom configuration, explicitly add labels using metrics_relabel_configs for each job. Global external labels are not supported via configmap based prometheus configuration.

+> ```yaml

+> relabel_configs:

+> - source_labels: [__address__]

+> target_label: example_label

+> replacement: 'example_value'

+> ```

+>

+[You can review your Code Optimizations in the Azure portal.](https://aka.ms/codeoptimizations)

+[Code Optimizations](https://aka.ms/codeoptimizations) analyzes the profiling data collected by the Application Insights Profiler. As the Profiler uploads data to Application Insights, our machine learning model analyzes some of the data to find where the application's code can be optimized. Code Optimizations:

+## Cost and overhead

+Code Optimizations are generated automatically after [Application Insights Profiler is enabled](../profiler/profiler-overview.md#sampling-rate-and-overhead). It incurs no extra cost to you as it analyzes performance issues and generates performance recommendations. Some features (such as code-level fix suggestions) require [Copilot for GitHub](https://docs.github.com/copilot/about-github-copilot/what-is-github-copilot) and/or [Copilot for Azure](../../copilot/overview.md).

+[You can review your Code Optimizations in the Azure portal.](https://aka.ms/codeoptimizations)

+Now that you set up and configured Code Optimizations on your app, [access and view any insights you received directly via the Azure portal.](https://aka.ms/codeoptimizations)

+You can also access Code Optimizations through any of your Application Insights resources from **Performance** pane and select **Code Optimizations (preview)** button from the top menu.

+> [Review Code Optimizations in Azure portal](https://aka.ms/codeoptimizations)

+> The daily cap can't stop data collection at precisely the specified cap level and some excess data is expected. The data collection beyond the daily cap can be particularly large if the workspace is receiving high rates of data. If data is collected above the cap, it's still billed. See [View the effect of the Daily Cap](#view-the-effect-of-the-daily-cap) for a query that is helpful in studying the daily cap behavior.

+ms.assetid:

+ Title: Monitor Linux machines

+description: This article describes how it monitor Linux machines.

+# Monitor Linux machines

+Azure Monitor SCOM Managed Instance provides a cloud-based alternative for Operations Manager users providing monitoring continuity for cloud and on-premises environments across the cloud adoption journey.

+>[!NOTE]

+>- Linux monitoring is supported only via Managed Gateways.

+>- Azure and Arc-enabled Linux machines aren't supported.

+With SCOM Managed Instance, you can monitor Linux workloads that are on-premises and behind a gateway server. At this stage, we don't support monitoring Linux VMs hosted in Azure. For more information, see [How to monitor on-premises Linux VMs](/system-center/scom/manage-deploy-crossplat-agent-console).

+ Title: Monitor Off-Azure Virtual machines with Azure Monitor SCOM Managed Instance

+# Monitor Off-Azure Virtual machines with Azure Monitor SCOM Managed Instance

+## Support for Azure and Off-Azure workloads

+One of the most important monitoring scenarios is that of on-premises (off-Azure) workloads that unlock SCOM Managed Instance as a true **Hybrid monitoring solution**.

+The following are the supported monitoring scenarios:

+|Type of endpoint|Trust|Experience|

+||||

+|Line of sight on-premises agent|Trusted|OpsConsole|

+|Line of sight on-premises agent|Untrusted|Managed Gateway and OpsConsole|

+|No Line of sight on-premises agent|Trusted/Untrusted|Managed Gateway and OpsConsole|

+SCOM Managed Instance users will be able to:

+- Set up and manage Gateways seamlessly from SCOM Managed Instance portal on Arc-enabled servers.

+- Set high availability at Gateway plane for agent failover as described in [Designing for High Availability and Disaster Recovery](/system-center/scom/plan-hadr-design).

+- On-premises virtual machines with no Line of sight connectivity (must use managed Gateway) to Azure

+- On-premises virtual machines that have Line of sight connectivity to Azure

+1. Confirm the Line of sight between SCOM Managed Instance and desired monitoring endpoints by running the following command. Obtain LB DNS (Load balancer DNS) information by navigating to SCOM Managed Instance > **Overview** > **Properties** > **Load balancer** > **DNS Name**.

+2. Ensure to install [.NET Framework 4.7.2](https://support.microsoft.com/topic/microsoft-net-framework-4-7-2-offline-installer-for-windows-05a72734-2127-a15d-50cf-daf56d5faec2) or higher on desired monitoring endpoints.

+3. Ensure TLS 1.2 or higher is enabled.

+To Troubleshooting connectivity problems, see [Troubleshoot issues with Azure Monitor SCOM Managed Instance](troubleshoot-scom-managed-instance.md).

+## Install SCOM Managed Instance Gateway

+Managed Gateway can be installed on Arc-enabled servers enabling it to relay monitoring data from air-gapped and network isolated servers to SCOM Managed Instance.

+To install SCOM Managed Instance gateway, follow these steps:

+1. Sign in to the [Azure portal](https://portal.azure.com/). Search and select **SCOM Managed Instance**.

+2. On the **Overview** page, under **Manage**, select **SCOM managed instances**.

+3. On the **SCOM managed instances** page, select the desired SCOM managed instance.

+4. On the desired SCOM managed instance **Overview** page, under **Manage**, select **Managed Gateway**.

+5. On the **Managed Gateways** page, select **New Managed Gateway**.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/new-managed-gateway-inline.png" alt-text="Screenshot that shows new managed gateway." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/new-managed-gateway-expanded.png":::

+ **Add a Managed Gateway** page opens listing all the Azure arc virtual machines.

+ >[!NOTE]

+ >SCOM Managed Instance Managed Gateway can be configured on Arc-enabled machines only.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/add-managed-gateway-inline.png" alt-text="Screenshot that shows add a managed gateway option." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/add-managed-gateway-expanded.png":::

+6. Select the desired virtual machine and then select **Add**.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/add-inline.png" alt-text="Screenshot that shows Add managed gateway." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/add-expanded.png":::

+7. On the **Add Monitored Resources** window, review the selections and select **Add**.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/install-gateway-inline.png" alt-text="Screenshot that shows Install managed gateway page." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/install-gateway-expanded.png":::

+### Delete a Gateway

+To delete a Gateway, follow these steps:

+1. Sign in to the [Azure portal](https://portal.azure.com/). Search and select **SCOM Managed Instance**.

+2. On the **Overview** page, under **Manage**, select **SCOM managed instances**.

+3. On the **SCOM managed instances** page, select the desired SCOM managed instance.

+4. On the desired SCOM managed instance **Overview** page, under **Manage**, select **Managed Gateways**.

+5. On the **Managed Gateways** page, select Ellipsis button **(…)**, which is next to your desired gateway, and select **Delete**.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/delete-gateway-inline.png" alt-text="Screenshot that shows delete gateway option." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/delete-gateway-expanded.png":::

+6. On the **Delete SCOM MI Gateway** page, check **Are you sure that you want to delete Managed Gateway?** and then select **Delete**.

+ :::image type="content" source="media/monitor-off-azure-vm-with-scom-managed-instance/delete-managed-gateway-inline.png" alt-text="Screenshot that shows delete managed gateway option." lightbox="media/monitor-off-azure-vm-with-scom-managed-instance/delete-managed-gateway-expanded.png":::

+## Managed Gateway configuration

+### Configure monitoring of servers via SCOM Managed Instance Gateway

+To configure monitoring of air-gapped and network isolated servers through Managed Gateway, follow the steps mentioned in [Install an agent on a computer running Windows by using the Discovery Wizard](/system-center/scom/manage-deploy-windows-agent-console#install-an-agent-on-a-computer-running-windows-by-using-the-discovery-wizard) section. Download and install agent from [here](https://go.microsoft.com/fwlink/?linkid=2251996).

+>[!NOTE]

+>Operations Manager Console is required for this action. For more information, see [Connect the Azure Monitor SCOM Managed Instance to Ops console](connect-managed-instance-ops-console.md).

+7. On the **Agent Setup Options** page, you can choose whether you want to **connect the agent to Operations Manager**.

+## Configure monitoring of on-premises servers

+To configure monitoring of on-premises servers that have direct connectivity (VPN/ER) with Azure, follow the steps mentioned in [Install an agent on a computer running Windows by using the Discovery Wizard](/system-center/scom/manage-deploy-windows-agent-console#install-an-agent-on-a-computer-running-windows-by-using-the-discovery-wizard) section.

+>[!NOTE]

+>Operations Manager Console is required for this action. For more information, see [Connect the Azure Monitor SCOM Managed Instance to Ops console](connect-managed-instance-ops-console.md).

+## Snapshot Debugger overhead scenarios

+The Snapshot Debugger is designed for use in production environments. The default settings include rate limits to minimize the impact on your applications.

+However, you may experience small CPU, memory, and I/O overhead associated with the Snapshot Debugger, like in the following scenarios.

+**When an exception is thrown in your application:**

+- Creating a signature for the problem type and deciding whether to create a snapshot adds a very small CPU and memory overhead.

+- If de-optimization is enabled, there is an overhead for re-JITting the method that threw the exception. This will be incurred the next time that method executes. Depending on the size of the method, this could be between 1ms and 100ms of CPU time.

+**If the exception handler decides to create a snapshot:**

+- Creating the process snapshot takes about half a second (P50=0.3s, P90=1.2s, P95=1.9s) during which time, the thread that threw the exception is paused. Other threads are not blocked.

+- Converting the process snapshot to a minidump and uploading it to Application Insights takes several minutes.

+ - Convert: P50=63s, P90=187s, P95=275s.

+ - Upload: P50=31s, P90=75s, P95=98s.

+ This is done in Snapshot Uploader, which runs in a separate process. The Snapshot Uploader process runs at below normal CPU priority and uses low priority I/O.

+ The minidump is first written to disk and the amount of disk spaced is roughly the same as the working set of the original process. Writing the minidump can induce page faults as memory is read.

+ The minidump is compressed during upload, which consumes both CPU and memory in the Snapshot Uploader process. The CPU, memory, and disk overhead for this is be proportional to the size of the process snapshot. Snapshot Uploader processes snapshots serially.

+**When `TrackException` is called:**

+The Snapshot Debugger checks if the exception is new or if a snapshot has been created for it. This adds a very small CPU overhead.

+## Overhead

+The Snapshot Debugger is designed for use in production environments. The default settings include rate limits to minimize the impact on your applications.

+However, you may experience small CPU, memory, and I/O overhead associated with the Snapshot Debugger, such as:

+- When an exception is thrown in your application

+- If the exception handler decides to create a snapshot

+- When `TrackException` is called

+There is **no additional cost** for storing data captured by Snapshot Debugger.

+[See example scenarios in which you may experience Snapshot Debugger overhead.](./snapshot-debugger-troubleshoot.md#snapshot-debugger-overhead-scenarios)

+## Definitions

+Understanding the terminology related to performance and capacity in Azure NetApp Files is essential to understanding the metrics available:

+- **Capacity pool**: A capacity pool is how capacity is billed in Azure NetApp Files. Capacity pools contain volume.

+- **Volume quota**: The amount of capacity provisioned to an Azure NetApp Files volume. Volume quota is directly tied to automatic Quality of Service (QoS), which impacts the volume performance. For more information, see [QoS types for capacity pools](azure-netapp-files-understand-storage-hierarchy.md#qos_types).

+- **Throughput**: The amount of data transmitted across the wire (read/write/other) between Azure NetApp Files and the client. Throughput in Azure NetApp Files is measured in bytes per second.

+- **Latency**: Latency is the amount of time for a storage operation to complete within storage from the time it arrives to the time it's processed and is ready to be sent back to the client. Latency in Azure NetApp Files is measured in milliseconds (ms).

+## About storage performance operation metrics

+An operation in Azure NetApp Files is defined as _something_ that happens during a client/server conversation. For instance, when a client requests a file to be read from Azure NetApp Files, read and other operations are sent and received between the client and server.

+When monitoring the Azure NetApp Files volume, read and write operations are self-explanatory. Also included in the metrics is a metric called **Other IOPS**, meaning any operation that isn't a read or write. **Other IOPS** encompasses operations such as metadata, which is present alongside most read and write operations.

+The following types of metadata operations are included in the **Other IOPS** metric:

+**NFSv3**

+NFSv3 metadata calls included in **Other IOPS** as covered in [RFC-1813](https://www.rfc-editor.org/rfc/rfc1813):

+- Procedure 0: NULL - Do nothing

+- Procedure 1: GETATTR - Get file attributes

+- Procedure 2: SETATTR - Set file attributes

+- Procedure 3: LOOKUP - Lookup filename

+- Procedure 4: ACCESS - Check Access Permission

+- Procedure 5: READLINK - Read from symbolic link

+- Procedure 8: CREATE - Create a file

+- Procedure 9: MKDIR - Create a directory

+- Procedure 10: SYMLINK - Create a symbolic link

+- Procedure 11: MKNOD - Create a special device

+- Procedure 12: REMOVE - Remove a File

+- Procedure 13: RMDIR - Remove a Directory

+- Procedure 14: RENAME - Rename a File or Directory

+- Procedure 15: LINK - Create Link to an object

+- Procedure 16: READDIR - Read From Directory

+- Procedure 17: READDIRPLUS - Extended read from directory

+- Procedure 18: FSSTAT - Get dynamic file system information

+- Procedure 19: FSINFO - Get static file system Information

+- Procedure 20: PATHCONF - Retrieve POSIX information

+- Procedure 21: COMMIT - Commit cached data on a server to stable storage

+**NFSv4.1**

+NFSv4.1 metadata calls included in **Other IOPS** as covered in [RFC-7530](https://www.rfc-editor.org/rfc/rfc7530):

+- Procedure 0: NULL ΓÇô Do nothing

+- Procedure 1: COMPOUND ΓÇô Combining multiple NFS operations into a single request

+- Operation 3: ACCESS ΓÇô Check access rights

+- Operation 4: CLOSE ΓÇô Close file

+- Operation 5: COMMIT ΓÇô Commit cached data

+- Operation 6: CREATE - Create a nonregular file object

+- Operation 7: DELEGPURGE - Purge delegations awaiting recovery

+- Operation 8: DELEGRETURN - Return delegation

+- Operation 9: GETATTR - Get attributes

+- Operation 10: GETFH - Get current filehandle

+- Operation 11: LINK - Create link to a file

+- Operation 12: LOCK - Create lock

+- Operation 13: LOCKT - Test for Lock

+- Operation 14: LOCKU - Unlock file

+- Operation 15: LOOKUP - Look Up filename

+- Operation 16: LOOKUPP - Look Up parent directory

+- Operation 17: NVERIFY - Verify difference in attributes

+- Operation 18: OPEN - Open a regular file

+- Operation 19: OPENATTR - Open named attribute directory

+- Operation 20: OPEN_CONFIRM - Confirm open

+- Operation 21: OPEN_DOWNGRADE - Reduce open file access

+- Operation 22: PUTFH - Set current filehandle

+- Operation 23: PUTPUBFH - Set public filehandle

+- Operation 24: PUTROOTFH - Set root filehandle

+- Operation 26: READDIR - Read directory

+- Operation 27: READLINK - Read symbolic link

+- Operation 28: REMOVE - Remove file system object

+- Operation 29: RENAME - Rename directory entry

+- Operation 30: RENEW - Renew a lease

+- Operation 32: SAVEFH - Save current filehandle

+- Operation 33: SECINFO - Obtain available security

+- Operation 34: SETATTR - Set attributes

+- Operation 35: SETCLIENTID - Negotiate client ID

+- Operation 36: SETCLIENTID_CONFIRM - Confirm client ID

+- Operation 37: VERIFY - Verify same attributes

+- Operation 39: RELEASE_LOCKOWNER ΓÇô Release lock-owner state

+**SMB (includes SMB2 and SMB3.x)**

+SMB commands included in **Other IOPS** with opcode value:

+| SMB command | Opcode value |

+| - | - |

+| SMB2 NEGOTIATE | 0x0000 |

+| SMB2 SESSION_SETUP | 0x0001 |

+| SMB2 LOGOFFΓÇ»| 0x0002 |

+| SMB2 TREE_CONNECTΓÇ»| 0x0003 |

+| SMB2 TREE_DISCONNECTΓÇ»| 0x0004 |

+| SMB2 CREATEΓÇ»| 0x0005 |

+| SMB2 CLOSEΓÇ»| 0x0006 |

+| SMB2 FLUSHΓÇ»| 0x0007 |

+| SMB2 LOCK | 0x000A |

+| SMB2 IOCTLΓÇ»| 0x000B |

+| SMB2 CANCELΓÇ»| 0x000C |

+| SMB2 ECHOΓÇ»| 0x000D |

+| SMB2 QUERY_DIRECTORYΓÇ»| 0x000E |

+| SMB2 CHANGE_NOTIFY | 0x000F |

+| SMB2 QUERY_INFOΓÇ»| 0x0010 |

+| SMB2 SET_INFO | 0x0011 |

+| SMB2 OPLOCK_BREAK | 0x0012 |

+ The average roundtrip time (RTT) for reads from the volume in milliseconds.

+ The average roundtrip time (RTT) for writes from the volume in milliseconds.

+ The number of read operations to the volume per second.

+ The number of write operations to the volume per second.

+- ***Other IOPS***

+ The number of [other operations](#about-storage-performance-operation-metrics) to the volume per second.

+- *Total IOPS*

+ A sum of the write, read, and other operations to the volume per second.

+ Whether the status of the volume replication is transferring.

+* *Total throughput*

+ Sum of all throughput (read, write, and other) in bytes per second.

+* [Azure NetApp Files backup](backup-introduction.md) is now available in Azure [US Gov regions](backup-introduction.md#supported-regions).

+* [Metrics enhancement:](azure-netapp-files-metrics.md) New performance metrics for volumes

+ New counters have been added to Azure NetApp Files performance metrics to increase visibility into your volumes' workloads:

+ - Other IOPS: any operations other than read or write.

+ - Total IOPS: a summation of all IOPS (read, write, and other)

+ - Other throughput: any operations other than read or write.

+ - Total throughput: Total throughput is a summation of all throughput (read, write, and other)

+> [!NOTE]

+> A private endpoint should be in succeeded state prior to attempting to move the resource.

+ Title: Architecture of BareMetal Infrastructure for NC2 on Azure

+description: Learn about the architecture of several configurations of BareMetal Infrastructure for NC2 on Azure.

+# Nutanix Cloud Clusters (NC2) on Azure architectural concepts

+NC2 provides Nutanix-based private clouds in Azure. The private cloud hardware and software deployments are fully integrated and automated in Azure. Deploy and manage the private cloud through the Azure portal, CLI, or PowerShell.

+A private cloud includes clusters with:

+- Dedicated bare-metal server hosts provisioned with Nutanix AHV hypervisor

+- Nutanix Prism Central for managing Nutanix Prism Element, Nutanix AHV and Nutanix AOS.

+- Nutanix Flow software-defined networking for Nutanix AHV workload VMs

+- Nutanix AOS software-defined storage for Nutanix AHV workload VMs

+- Nutanix Move for workload mobility

+- Resources in the Azure underlay (required for connectivity and to operate the private cloud)

+Private clouds are installed and managed within an Azure subscription. The number of private clouds within a subscription is scalable.

+The following diagram describes the architectural components of the Azure VMware Solution.

+Each NC2 on Azure architectural component has the following function:

+- Azure Subscription: Used to provide controlled access, budget, and quota management for the NC2 on Azure service.

+- Azure Region: Physical locations around the world where we group data centers into Availability Zones (AZs) and then group AZs into regions.

+- Azure Resource Group: Container used to place Azure services and resources into logical groups.

+- NC2 on Azure: Uses Nutanix software, including Prism Central, Prism Element, Nutanix Flow software-defined networking, Nutanix Acropolis Operating System (AOS) software-defined storage, and Azure bare-metal Acropolis Hypervisor (AHV) hosts to provide compute, networking, and storage resources.

+- Nutanix Move: Provides migration services.

+- Nutanix Disaster Recovery: Provides disaster recovery automation and storage replication services.

+- Nutanix Files: Provides filer services.

+- Nutanix Self Service: Provides application lifecycle management and cloud orchestration.

+- Nutanix Cost Governance: Provides multi-cloud optimization to reduce cost & enhance cloud security.

+- Azure Virtual Network (VNet): Private network used to connect AHV hosts, Azure services and resources together.

+- Azure Route Server: Enables network appliances to exchange dynamic route information with Azure networks.

+- Azure Virtual Network Gateway: Cross premises gateway for connecting Azure services and resources to other private networks using IPSec VPN, ExpressRoute, and VNet to VNet.

+- Azure ExpressRoute: Provides high-speed private connections between Azure data centers and on-premises or colocation infrastructure.

+- Azure Virtual WAN (vWAN): Aggregates networking, security, and routing functions together into a single unified Wide Area Network (WAN).

+## Supported topologies

+The following table describes the network topologies supported by each network features configuration of NC2 on Azure.

+|Topology |Supported |

+| :- |::|

+|Connectivity to BareMetal Infrastructure (BMI) in a local VNet| Yes |

+|Connectivity to BMI in a peered VNet (Same region)|Yes |

+|Connectivity to BMI in a peered VNet\* (Cross region or global peering) with VWAN\*|Yes |

+|Connectivity to BM in a peered VNet* (Cross region or global peering)* without VWAN| No|

+|On-premises connectivity to Delegated Subnet via Global and Local Expressroute |Yes|

+|ExpressRoute (ER) FastPath |No |

+|Connectivity from on-premises to BMI in a spoke VNet over ExpressRoute gateway and VNet peering with gateway transit|Yes |

+|On-premises connectivity to Delegated Subnet via VPN GW| Yes |

+|Connectivity from on-premises to BMI in a spoke VNet over VPN gateway and VNet peering with gateway transit| Yes |

+|Connectivity over Active/Passive VPN gateways| Yes |

+|Connectivity over Active/Active VPN gateways| No |

+|Connectivity over Active/Active Zone Redundant gateways| No |

+|Transit connectivity via vWAN for Spoke Delegated VNETS| Yes |

+|On-premises connectivity to Delegated subnet via vWAN attached SD-WAN| No|

+|On-premises connectivity via Secured HUB(Az Firewall NVA) | No|

+|Connectivity from UVMs on NC2 nodes to Azure resources|Yes|

+\* You can overcome this limitation by setting Site-to-Site VPN.

+## Constraints

+The following table describes whatΓÇÖs supported for each network features configuration:

+|Features |Basic network features |

+| :- | -: |

+|Delegated subnet per VNet |1|

+|[Network Security Groups](../../../virtual-network/network-security-groups-overview.md) on NC2 on Azure-delegated subnets|No|

+|VWAN enables traffic inspection via NVA (Virtual WAN Hub routing intent)|Yes|

+[User-defined routes (UDRs)](../../../virtual-network/virtual-networks-udr-overview.md#user-defined) on NC2 on Azure-delegated subnets without VWAN| No|

+|Connectivity from BareMetal to [private endpoints](../../../private-link/private-endpoint-overview.md) in the same Vnet on Azure-delegated subnets|No|

+|Connectivity from BareMetal to [private endpoints](../../../private-link/private-endpoint-overview.md) in a different spoke Vnet connected to vWAN|Yes|

+|Load balancers for NC2 on Azure traffic|No|

+|Dual stack (IPv4 and IPv6) virtual network|IPv4 only supported|

+> [Architecture](architecture.md)

+| Windows¹, Linux² | [Network Isolation](#network-isolation) | Network disruption |

+### Network Isolation

+| Property | Value |

+|-|-|

+| Capability name | NetworkIsolation-1.0 |

+| Target type | Microsoft-Agent |

+| Supported OS types | Windows, Linux (outbound traffic only) |

+| Description | Fully isolate the virtual machine from network connections by dropping all IP-based inbound (on Windows) and outbound (on Windows and Linux) packets for the specified duration. At the end of the duration, network connections will be re-enabled. Because the agent depends on network traffic, this action cannot be cancelled and will run to the specified duration. |

+| Prerequisites | **Windows:** The agent must run as administrator, which happens by default if installed as a VM extension. |

+| | **Linux:** The `tc` (Traffic Control) package is used for network faults. If it isn't already installed, the agent automatically attempts to install it from the default package manager. |

+| Urn | urn:csci:microsoft:agent:networkIsolation/1.0 |

+| Fault type | Continuous. |

+| Parameters (key, value) | |

+| virtualMachineScaleSetInstances | An array of instance IDs when you apply this fault to a virtual machine scale set. Required for virtual machine scale sets in uniform orchestration mode, optional otherwise. [Learn more about instance IDs](../virtual-machine-scale-sets/virtual-machine-scale-sets-instance-ids.md#scale-set-instance-id-for-uniform-orchestration-mode). |

+#### Sample JSON

+```json

+{

+ "name": "branchOne",

+ "actions": [

+ {

+ "type": "continuous",

+ "name": "urn:csci:microsoft:agent:networkIsolation/1.0",

+ "parameters": [],

+ "duration": "PT10M",

+ "selectorid": "myResources"

+ }

+ ]

+}

+```

+#### Limitations

+* Because the agent depends on network traffic, **this action cannot be cancelled** and will run to the specified duration. Use with caution.

+* The agent-based network faults currently only support IPv4 addresses.

+* When running on Windows, the network packet loss fault currently only works with TCP or UDP packets.

+* When running on Linux, this fault only affects **outbound** traffic, not inbound traffic. The fault affects **both inbound and outbound** traffic on Windows environments.

+* This fault currently only affects new connections. Existing active connections are unaffected. You can restart the service or process to force connections to break.

+##### Working with MongoDB `id` field

+The `id` property in MongoDB containers is automatically overridden with the Base64 representation of the "_id" property both in analytical store. The "id" field is intended for internal use by MongoDB applications. Currently, the only workaround is to rename the "id" property to something other than "id".

+description: Review various options to migrate your data from other MongoDB sources to vCore-based Azure Cosmos DB for MongoDB.

+# CustomerIntent: As a MongoDB user, I want to understand the various options available to migrate my data to vCore-based Azure Cosmos DB for MongoDB, so that I can make an informed decision about which option is best for my use case.

+# What are the options to migrate data from MongoDB to vCore-based Azure Cosmos DB for MongoDB?

+This document describes the various options to lift and shift your MongoDB workloads to vCore-based Azure Cosmos DB for MongoDB offering.

+Migrations can be done in two ways:

+- Offline Migration: A snapshot based bulk copy from source to target. New data added/updated/deleted on the source after the snapshot isn't copied to the target. The application downtime required depends on the time taken for the bulk copy activity to complete.

+- Online Migration: Apart from the bulk data copy activity done in the offline migration, a change stream monitors all additions/updates/deletes. After the bulk data copy is completed, the data in the change stream is copied to the target to ensure that all updates made during the migration process are also transferred to the target. The application downtime required is minimal.

+The [MongoDB migration extension for Azure Data Studio](/azure-data-studio/extensions/database-migration-for-mongo-extension) is the preferred tool in migrating your MongoDB workloads to the vCore-based Azure Cosmos DB for MongoDB.

+You can use the native MongoDB tools such as *mongodump/mongorestore*, *mongoexport/mongoimport* to migrate datasets offline (without replicating live changes) to vCore-based Azure Cosmos DB for MongoDB offering.

+ - *mongoimport* opens a JSON or CSV file and inserts the content into the target database instance (vCore-based Azure Cosmos DB for MongoDB in this case.).

+ - JSON and CSV aren't a compact format; you could incur excess network charges as *mongoimport* sends data to vCore-based Azure Cosmos DB for MongoDB.

+- *mongodump/mongorestore* is the best pair of migration tools for migrating your entire MongoDB database. The compact BSON format makes more efficient use of network resources as the data is inserted into vCore-based Azure Cosmos DB for MongoDB.

+ - *mongorestore* imports your BSON file dump into vCore-based Azure Cosmos DB for MongoDB.

+Migrating using Azure Databricks offers full control of the migration rate and data transformation. This method can also support large datasets that are in TBs in size. The spark migration utility operates as a job within Databricks.

+This tool supports the following MongoDB sources:

+- MongoDB VM

+- MongoDB Atlas

+- AWS DocumentDB

+- Azure Cosmos DB MongoDB RU (Offline only)

+[Sign up for Azure Cosmos DB for MongoDB Spark Migration](https://forms.office.com/r/cLSRNugFSp) to gain access to the Spark Migration Tool GitHub repository. The repository offers detailed, step-by-step instructions for migrating your workloads from various Mongo sources to vCore-based Azure Cosmos DB for MongoDB.

+- Migrate data to vCore-based Azure Cosmos DB for MongoDB using [native MongoDB tools](how-to-migrate-native-tools.md).

+- Migrate data to vCore-based Azure Cosmos DB for MongoDB using the [MongoDB migration extension for Azure Data Studio](/azure-data-studio/extensions/database-migration-for-mongo-extension).

+- [Connect with Microsoft Entra ID (recommended)](#connect-using-the-microsoft-identity-platform-recommended)

+### Connect using the Microsoft identity platform (recommended)

+For more information about forecasting costs, see [Forecasting costs in Cost Analysis](quick-acm-cost-analysis.md#forecasting-costs-in-cost-analysis).

+Before you can control and optimize your costs, you first need to understand where they originated ΓÇô from the underlying resources used to support your cloud projects to the environments they get deployed in and the owners who manage them. Full visibility backed by a thorough tagging strategy is critical to accurately understand your spending patterns and enforce cost control mechanisms.

+Views in the **Recommended** list might vary based on what users most commonly use across Azure.

+Depending on the view and scope you're using, you might also see cost insights below the KPIs. Cost insights show important datapoints about your cost ΓÇô from discovering top cost contributors to identifying anomalies based on usage patterns. Select the **See insights** link to review and provide feedback on all insights. Here's an insights example.

+2. Select the name to drill down and see the next level details in a full view. From there, you can drill down again and again, to get down to the finest level of detail, based on what you have interest in. Examples include selecting a subscription, then a resource group, and then a resource to view the specific product meters for that resource.

+3. To see related costs, select the shortcut menu (Γï»). Examples include filtering the list of resource groups to a subscription or filtering resources to a specific location or tag.

+>[!NOTE]

+>If you want to visualize and monitor daily trends within the period, enable the [chart preview feature](enable-preview-features-cost-management-labs.md#chartsfeature) in Cost Management Labs, available from the **Try preview** command.

+After you customize your view to meet your needs, you might want to save and share it with others. To share views with others:

+1. Forecasting employs a *time series linear regression* model, which adjusts to factors such as reserved instance purchases that temporarily affect forecasted costs. Following such purchases, the forecasted costs typically stabilize in alignment with usage trends within a few days. You can filter out these temporary spikes to obtain a more normalized forecasted cost.

+1. For accurate long-term forecasting, it's essential to have sufficient historical data. New subscriptions or contracts with limited historical data might result in less accurate forecasts. At least 90 days of historical data are recommended for a more precise annual forecast.

+Here's a table to help you understand how the forecast duration and lookback period are calculated based on the forecast period:

+| Forecast Duration | Lookback Period |

+|--||

+| Up to 28 days | 28 days |

+| Above 28 days | Same as Forecast Duration |

+| Above 90 days | 90 days |

+To help drive accountability and cost control, [configure subscription anomaly alerts](../understand/analyze-unexpected-charges.md#create-an-anomaly-alert) and set up a [budget](tutorial-acm-create-budgets.md).

+- Cost and usage details (FOCUS) - Select this option to export cost and usage details using the open-source FinOps Open Cost and Usage Specification ([FOCUS](https://focus.finops.org/)) format. It combines actual and amortized costs.

+ - This format reduces data processing time and storage and compute charges for exports.

+ - The management group scope isn't supported for Cost and usage details (FOCUS) exports.

+ - You can use the FOCUS-formatted export as the input for a Microsoft Fabric workspace for FinOps. For more information, see [Create a Fabric workspace for FinOps](/cloud-computing/finops/fabric/create-fabric-workspace-finops).

+To determine the version of the Snowflake connector used in your existing Snowflake linked service, check the ```type``` property. The legacy version is identified by ```"type": "Snowflake"```, while the latest V2 version is identified by ```"type": "SnowflakeV2"```.

+The V2 version offers several enhancements over the legacy version, including:

+Autoscaling: Automatically adjusts resources based on traffic load.

+Multi-Availability Zone Operation: Provides resilience by operating across multiple availability zones.

+Static IP Support: Enhances security by allowing the use of static IP addresses.

+#### Error code: 20000

+#### Error code: 20002

+- **Message**: `An error occurred when invoking Java Native Interface.`

+- **Cause**: If the error message contains "Cannot create JVM: JNI return code [-6][JNI call failed: Invalid arguments.]", the possible cause is that JVM can't be created because some illegal (global) arguments are set.

+- **Recommendation**: Log in to the machine that hosts *each node* of your self-hosted integration runtime. Check to ensure that the system variable is set correctly, as follows: `_JAVA_OPTIONS "-Xms256m -Xmx16g" with memory bigger than 8G`. Restart all the integration runtime nodes, and then rerun the pipeline.

+#### Error code: 20020

+#### Error code: 20150

+#### Error code: 20151

+#### Error code: 20152

+#### Error code: 20153

+#### Error code: 20523

+#### Error code: 20551

+#### Error code: 20552

+#### Error code: 20701

+#### Error code: 20703

+#### Error code: 20704

+#### Error code: 20705

+#### Error code: 20741

+#### Error code: 20742

+#### Error code: 20743

+#### Error code: 20744

+#### Error code: 20745

+#### Error code: 20746

+#### Error code: 20747

+#### Error code: 20748

+#### Error code: 20771

+#### Error code: 20772

+#### Error code: 27002

+#### Error code: 9611

+#### Error code: 11775

+- **Message**: `Failed to connect to your instance of Azure Database for PostgreSQL flexible server.`

+- **Cause**: User or password provided are incorrect. The encryption method selected is not compatible with the configuration of the server. The network connectivity method configured for your instance doesn't allow connections from the Integration Runtime selected.

+- **Recommendation**: Confirm that the user provided exists in your instance of PostgreSQL and that the password corresponds to the one currently assigned to that user. Make sure that the encryption method selected is accepted by your instance of PostgreSQL, based on its current configuration. If the network connectivity method of your instance is configured for Private access (VNet integration), use a Self-Hosted Integration Runtime (IR) to connect to it. If it is configured for Public access (allowed IP addresses), it is recommended to use an Azure IR with managed virtual network and deploy a managed private endpoint to connect to your instance. When it is configured for Public access (allowed IP addresses) a less recommended alternative consists in creating firewall rules in your instance to allow traffic originating on the IP addresses used by the Azure IR you're using.

+## Error code: 23704

+## Error code: 23705

+- **Message**: `The data type of the chosen Partition Column, '%partitionColumn;', is '%dataType;' and this data type is not supported for partitioning.`

+- **Recommendation**: Pick a partition column with int, bigint, smallint, serial, bigserial, smallserial, timestamp with or without time zone, time without time zone or date data type.

+## Related content

+[Store credentials in Azure Key Vault](store-credentials-in-key-vault.md)

+- Contributor level permission for the relevant Azure subscription.

+- An Entra ID account that has an Application Administrator or Cloud Application Administrator directory role for your tenant (or equivalent administrator rights to create app registrations).

+- Contributor level permission for the relevant Azure subscription.

+- An Entra ID account that has an Application Administrator or Cloud Application Administrator directory role for your tenant (or equivalent administrator rights to create app registrations).

+ If you exceed limitations, such as using over 20,000 unique source/destination combinations in rules, it can affect firewall traffic processing and cause latency. Even though this is a soft limit, if you surpass this value it can affect overall firewall performance. For more information, see the [documented limits](../azure-resource-manager/management/azure-subscription-service-limits.md#azure-firewall-limits).

+* Application Gateway (Preview only. Please do not put production workloads)

+The above metrics are checked every 60 seconds. Autoscale makes scale-up and scale-down decisions based on these metrics.

+For a complete list of cluster metrics, see [Supported metrics for Microsoft.HDInsight/clusters](monitor-hdinsight-reference.md#supported-metrics-for-microsofthdinsightclusters).

+* `ranger_audit_logs_CL` - this table provides audit logs from Apache Ranger on ESP clusters.

+For the log table mappings from the classic Azure Monitor integration to the new one, see [Log table mapping](monitor-hdinsight-reference.md#log-table-mapping).

+* `ranger_audit_logs_CL` - this table provides audit logs from Apache Ranger on ESP clusters.

+For the log table mappings from the classic Azure Monitor integration to the new one, see [Log table mapping](monitor-hdinsight-reference.md#log-table-mapping).

+Use external storage account via [SAS URIs](hdinsight-storage-sharedaccesssignature-permissions.md) for script actions or make the scripts publicly accessible.

+ You can also enter `*` to search all types logged. For a list of logs that are available for queries, see [Kafka workload](../monitor-hdinsight-reference.md#kafka-workload).

+We provide a [mapping table](monitor-hdinsight-reference.md#log-table-mapping) between the old table to the new table to help you quickly find the new fields you need to use to migrate your dashboards and queries.

+Refer to the [mapping table](monitor-hdinsight-reference.md#log-table-mapping) between the old table/schema to the new table/schema to update the query behind the dashboards.

+For an example that shows how to create an alert, see [Azure Monitor alerts](cluster-availability-monitor-logs.md#azure-monitor-alerts).

+For a complete listing of table names for different log types (sources), see [Azure Monitor Logs tables](monitor-hdinsight-reference.md#azure-monitor-logs-tables).

+| [Debian 11](https://www.debian.org/releases/bullseye/) | |  | | [June 2026](https://wiki.debian.org/LTS) |

+| [Red Hat Enterprise Linux 9](https://access.redhat.com/articles/3078) |  | | | [May 2032](https://access.redhat.com/product-life-cycles?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204) |

+| [Red Hat Enterprise Linux 8](https://access.redhat.com/articles/3078) |  | | | [May 2029](https://access.redhat.com/product-life-cycles?product=Red%20Hat%20Enterprise%20Linux,OpenShift%20Container%20Platform%204) |

+| [Ubuntu Server 22.04](https://wiki.ubuntu.com/JammyJellyfish/ReleaseNotes) |  | |  | [June 2027](https://wiki.ubuntu.com/Releases) |

+| [Ubuntu Server 20.04](https://wiki.ubuntu.com/FocalFoss64](./media/support/green-check.png) | |  | [April 2025](https://wiki.ubuntu.com/Releases) |

+| [Windows 10/11](iot-edge-for-linux-on-windows.md#prerequisites) |  for supported Windows OS versions. |

+| [Windows Server 2019/2022](iot-edge-for-linux-on-windows.md#prerequisites) |  for supported Windows OS versions. |

+| [Ubuntu Server 22.04 ²](https://wiki.ubuntu.com/JammyJellyfish/ReleaseNotes) | |  | | [June 2027](https://wiki.ubuntu.com/Releases) |

+| [Ubuntu Server 20.04 ²](https://wiki.ubuntu.com/FocalFossa/ReleaseNotes) | |  | | [April 2025](https://wiki.ubuntu.com/Releases) |

+| [Debian 11 ](https://www.debian.org/releases/bullseye/) |  | |  | [June 2026](https://wiki.debian.org/LTS) |

+| [Ubuntu Server 22.04 ¹](https://wiki.ubuntu.com/JammyJellyfish/ReleaseNotes) | |  | | [June 2027](https://wiki.ubuntu.com/Releases) |

+| [Ubuntu Server 20.04 ¹](https://wiki.ubuntu.com/FocalFossa/ReleaseNotes) | |  | | [April 2025](https://wiki.ubuntu.com/Releases) |

+ |Azure Key Vault (when using [access policies permission model](../../key-vault/general/assign-access-policy.md))|Contributor + any access policy permissions besides **purge** operations, this is **default mode** for linked Azure Key Vault.|

+

+ > [!NOTE]

+ > The job submitter need have `assign` permission on user assigned managed identity, you can assign `Managed Identity Operator` role, as every time create serverless compute session, it will assign user assigned managed identity to compute.

+ > The idle shutdown is one hour if you are using CLI/SDK to submit a flow run. You can go to compute page to release compute.

+> You can inspect the property `detail.loc` to understand the location of the offending parameter and `detail.input` to see the value that was passed in the request.

+ Title: Azure Operator Nexus - Networking concepts

+description: Get an overview of networking in Azure Operator Nexus.

+# Networking in Azure Operator Nexus Kubernetes

+An Azure Operator Nexus, or simply Operator Nexus, instance comprises compute

+and networking hardware installed at the customer premises. Multiple layers of

+physical and virtual devices provide network connectivity and routing services

+to the workloads running on this compute hardware. This document provides a

+detailed description of each of these networking layers.

+## Topology

+Here we describe the topology of hardware in an Operator Nexus instance.

+Customers own and manage Provider edge (PE) routers. These routers represent

+the edge of the customerΓÇÖs backbone network.

+Operator Nexus manages the customer edge (CE) routers. These routers are part

+of the Operator Nexus instance and are included in near-edge hardware

+[bill of materials][bom] (BOM). There are two CE routers in each multi-rack

+Operator Nexus instance. Each CE router has an uplink to each of the PE

+routers. The CE routers are the only Operator Nexus devices that are physically

+connected to the customerΓÇÖs network.

+Each rack of compute servers in a multi-rack Azure Operator Nexus instance has

+two top-of-rack (TOR) switches. Each TOR has an uplink to each of the CE

+routers. Each TOR is connected to each bare metal compute server in the rack

+and is configured as a simple [layer 2 switch][layer2-switch].

+[bom]: ./reference-operator-nexus-fabric-skus.md

+[layer2-switch]: https://en.wikipedia.org/wiki/Multilayer_switch#Layer-2_switching

+## Bare metal

+Tenant workloads running on this compute infrastructure are typically virtual

+or containerized network functions. Virtual network functions (VNFs) run as

+virtual machines (VMs) on the compute server hardware. Containerized network

+functions (CNFs) run inside containers. These containers run on VMs that

+themselves run on the compute server hardware.

+Network functions that provide end-user data plane services require high

+performance network interfaces that offer advanced features and high I/O rates.

+In near-edge multi-rack Operator Nexus instances, each bare metal compute

+server is a dual-socket machine with [Non-Uniform Memory Access][numa] (NUMA)

+architecture.

+A bare metal compute server in a near-edge multi-rack Azure Operator Nexus

+instance contains one dual-port network interface card (pNIC) for each NUMA

+cell. These pNICs support [Single-Root I/O Virtualization][sriov] (SR-IOV) and

+other high-performance features. One NUMA cell is memory and CPU-aligned with a

+one pNIC.

+All network interfaces assigned to tenant workloads are host passthrough

+devices and use SR-IOV virtual functions (VFs) allocated from the pNIC aligned

+to the NUMA cell housing the workload VMΓÇÖs CPU and memory resources. This

+arrangement ensures optimal performance of the networking stack inside the VMs

+and containers that are assigned those VFs.

+Compute racks are deployed with a pair of Top-of-Rack (TOR) switches. Each pNIC

+on each bare metal compute server is connected to both of those TORs.

+[Multi-chassis link aggregation group][mlag] (MLAG) provides high availability

+and [link aggregation control protocol][lacp] (LACP) provides increased

+aggregate throughput for the link.

+Each bare metal compute server has a storage network interface that is provided

+by a bond that aggregates two *host-local* virtual functions (VF)s (as opposed

+to VM-local VFs) connected to *both* pNICs. These two VFs are aggregated in an

+active-backup bond to ensure if one of the pNICs fails, network storage

+connectivity remains available.

+[sriov]: https://en.wikipedia.org/wiki/Single-root_input/output_virtualization

+[mlag]: https://en.wikipedia.org/wiki/Multi-chassis_link_aggregation_group

+[lacp]: https://www.cisco.com/c/en/us/td/docs/ios/12_2sb/feature/guide/gigeth.html

+[numa]: https://en.wikipedia.org/wiki/Non-uniform_memory_access

+## Logical network resources

+When interacting with the Operator Nexus Network Cloud API and Managed Network

+Fabric APIs, users create and modify a set of logical resources.

+Logical resources in the Managed Network Fabric API correspond to the networks

+and access control configuration on the underlying networking hardware (the

+TORs and CEs). Notably, `ManagedNetworkFabric.L2IsolationDomain` and

+`ManagedNetworkFabric.L3IsolationDomain` resources contain low-level switch and

+network configuration. A `ManagedNetworkFabric.L2IsolationDomain` represents a

+[virtual local area network][vlan] identifier (VLAN). A

+`ManagedNetworkFabric.L3IsolationDomain` represents a

+[virtual routing and forwarding][vrf] configuration (VRF) on the CE routers.

+Read about the [concept of an Isolation Domain][isd].

+Logical resources in the Network Cloud API correspond to compute

+infrastructure. There are resources for physical racks and bare metal hardware.

+Likewise, there are resources for Kubernetes clusters and virtual machines that

+run on that hardware and the logical networks that connect them.

+`NetworkCloud.L2Network`, `NetworkCloud.L3Network`, and

+`NetworkCloud.TrunkedNetwork` all represent workload networks, meaning traffic

+on these networks is meant for tenant workloads.

+A `NetworkCloud.L2Network` represents a layer-2 network and contains little

+more than a link to a `ManagedNetworkFabric.L2IsolationDomain`. This

+L2IsolationDomain contains a VLAN identifier and a maximum transmission unit

+(MTU) setting.

+A `NetworkCloud.L3Network` represents a layer-3 network and contains a VLAN

+identifier, information about IP address assignment for endpoints on the

+network and a link to a `ManagedNetworkFabric.L3IsolationDomain`.

+> [!NOTE]

+> Why does a `NetworkCloud.L3Network` resource contain a VLAN identifier?

+> Aren't VLANs a layer-2 concept?

+>

+> Yes, yes they are! The reason for this is due

+> to the fact that the `NetworkCloud.L3Network` must be able to refer to a

+> specific [`ManagedNetworkFabric.InternalNetwork`][internal-net].

+> `ManagedNetworkFabric.InternalNetwork`s are created within a specific

+> `ManagedNetworkFabric.L3IsolationDomain` and are given a VLAN identifier.

+> Therefore, in order to reference a specific

+> `ManagedNetworkFabric.InternalNetwork`, the `NetworkCloud.L3Network` must

+> contain both an L3IsolationDomain identifier and a VLAN identifier.

+Logical *network resources* in the Network Cloud API such as

+`NetworkCloud.L3Network` *reference* logical resources in the Managed Network

+Fabric API and in doing so provide a logical connection between the physical

+compute infrastructure and the physical network infrastructure.

+When creating a Nexus Virtual Machine, you may specify zero or more L2, L3, and

+Trunked Networks in the Nexus Virtual Machine's

+[`NetworkAttachments`][vm-netattach]. When creating a Nexus Kubernetes Cluster,

+you may specify zero or more L2, L3, and Trunked Networks in the Nexus

+Kubernetes Cluster's

+[`NetworkConfiguration.AttachedNetworkConfiguration`][attachednetconf] field.

+AgentPools are collections of similar Kubernetes worker nodes within a Nexus

+Kubernetes Cluster. You can configure each Agent Pool's attached L2, L3, and

+Trunked Networks in the AgentPool's

+[`AttachedNetworkConfiguration`][attachednetconf] field.

+You can share networks across standalone Nexus Virtual Machines and Nexus

+Kubernetes Clusters. This composability allows you to stitch together CNFs and

+VNFs working in concert across the same logical networks.

+The diagram shows an example of a Nexus Kubernetes cluster with two agent pools

+and a standalone Nexus Virtual Machine connected to different workload

+networks. Agent Pool "AP1" has no extra network configuration and therefore it

+inherits the KubernetesCluster's network information. Also note that all

+Kubernetes Nodes and all standalone Nexus Virtual Machines are configured to

+connect to the same Cloud Services Network. Finally, Agent Pool "AP2" and the

+stand-alone VM are configured to connect to a "Shared L3 Network".

+### The CloudServicesNetwork

+Nexus Virtual Machines and Nexus Kubernetes Clusters always reference something

+called the "Cloud Services Network" (CSN). The CSN is a special network used

+for traffic between on-premises workloads and a set of external or Azure-hosted

+endpoints.

+Traffic on the CloudServicesNetwork is routed through a proxy, where egress

+traffic is controlled via the use of an allowlist. Users can tune this

+allowlist [using the Network Cloud API][csnapi].

+[csnapi]: ./quickstarts-tenant-workload-prerequisites.md#create-a-cloud-services-network

+<!

+TODO(jaypipes): Expand and explain this more. There's no good information about

+CSN in our public docs.

+TODO(jaypipes): A diagram showing CSN traffic flow and proxy.

+>

+### The CNI Network

+When creating a Nexus Kubernetes Cluster, you provide the resource identifier

+of a `NetworkCloud.L3Network` in the `NetworkConfiguration.CniNetworkId` field.

+This "CNI network", sometimes referred to as "DefaultCNI Network", specifies

+the layer-3 network that provides IP addresses for Kubernetes Nodes in the

+Nexus Kubernetes cluster.

+The diagram shows the relationships between some of the Network Cloud, Managed

+Network Fabric, and Kubernetes logical resources. In the diagram, a

+`NetworkCloud.L3Network` is a logical resource in the Network Cloud API that

+represents a layer 3 network. The `NetworkCloud.KubernetesCluster` resource has

+a field `networkConfiguration.cniNetworkId` that contains a reference to the

+`NetworkCloud.L3Network` resource.

+The `NetworkCloud.L3Network` resource is associated with a single

+`ManagedNetworkFabric.InternalNetwork` resource via its `l3IsolationDomainId`

+and `vlanId` fields. The `ManagedNetworkFabric.L3IsolationDomain` resource

+contains one or more `ManagedNetworkFabric.InternalNetwork` resources, keyed by

+`vlanId`. When the user creates the `NetworkCloud.KubernetesCluster` resource,

+one or more `NetworkCloud.AgentPool` resources are created.

+Each of these `NetworkCloud.AgentPool` resources comprises one or more virtual

+machines. A Kubernetes `Node` resource represents each of those virtual

+machines. These Kubernetes `Node` resources must get an IP address and the

+Container Networking Interface (CNI) plugins on the virtual machines grab an IP

+address from the pool of IP addresses associated with the

+`NetworkCloud.L3Network`. The `NetworkCloud.KubernetesCluster` resource

+references the `NetworkCloud.L3Network` via its `cniNetworkId` field. The

+routing and access rules for those node-level IP addresses are contained in the

+`ManagedNetworkFabric.L3IsolationDomain`. The `NetworkCloud.L3Network` refers

+to the `ManagedNetworkFabric.L3IsolationDomain` via its `l3IsoldationDomainId`

+field.

+[netfabric]: ./concepts-network-fabric.md

+[vlan]: https://en.wikipedia.org/wiki/VLAN

+[vrf]: https://en.wikipedia.org/wiki/Virtual_routing_and_forwarding

+[isd]: ./howto-configure-isolation-domain.md

+[internal-net]: ./howto-configure-isolation-domain.md#create-internal-network

+[vm-netattach]: https://learn.microsoft.com/rest/api/networkcloud/virtual-machines/create-or-update?view=rest-networkcloud-2023-07-01&tabs=HTTP#networkattachment

+[attachednetconf]: https://learn.microsoft.com/rest/api/networkcloud/kubernetes-clusters/create-or-update?view=rest-networkcloud-2023-07-01&tabs=HTTP#attachednetworkconfiguration

+## Operator Nexus Kubernetes networking

+There are three logical layers of networking in Kubernetes:

+* Node networking layer

+* Pod networking layer

+* Service networking layer

+The *Node networking layer* provides connectivity between the Kubernetes

+control plane and the kubelet worker node agent.

+The *Pod networking layer* provides connectivity between containers (Pods)

+running inside the Nexus Kubernetes cluster and connectivity between a Pod and

+one or more tenant-defined networks.

+The *Service networking layer* provides load balancing and ingress

+functionality for sets of related Pods.

+### Node networking

+Operator Nexus Kubernetes clusters house one or more containerized network

+functions (CNFs) that run on a virtual machines (VM). A Kubernetes *Node*

+represents a single VM. Kubernetes Nodes may be either *Control Plane* Nodes or

+*Worker* Nodes. Control Plane Nodes contain management components for the

+Kubernetes Cluster. Worker Nodes house tenant workloads.

+Groups of Kubernetes Worker Nodes are called *Agent Pools*. Agent Pools are an

+Operator Nexus construct, *not* a Kubernetes construct.

+Each bare metal compute server in an Operator Nexus instance has a

+[switchdev][switchdev] that is affined to a single NUMA cell on the bare metal

+server. The switchdev houses a set of SR-IOV VF representor ports that provide

+connectivity to a set of bridge devices that are used to house routing tables

+for different networks.

+In addition to the `defaultcni` interface, Operator Nexus establishes a

+`cloudservices` network interface on every Node. The `cloudservices` network

+interface is responsible for routing traffic destined for external (to the

+customer's premises) endpoints. The `cloudservices` network interface

+corresponds to the `NetworkCloud.CloudServicesNetwork` API resource that the

+user defines before creating a Nexus Kubernetes cluster. The IP address

+assigned to the `cloudservices` network interface is a

+[link-local address][lladdr], ensuring that external network traffic always

+traverses this specific interface.

+In addition to the `defaultcni` and `cloudservices` network interfaces,

+Operator Nexus creates one or more network interfaces on each Kubernetes Node

+that correspond to `NetworkCloud.L2Network`, `NetworkCloud.L3Network`, and

+`NetworkCloud.TrunkedNetwork` associations with the Nexus Kubernetes cluster

+or AgentPool.

+Only Agent Pool VMs have these extra network interfaces. Control Plane VMs

+only have the `defaultcni` and `cloudservices` network interfaces.

+#### Node IP Address Management (IPAM)

+Nodes in an Agent Pool receive an IP address from a pool of IP addresses

+associated with the `NetworkCloud.L3Network` resource referred to in the

+`NetworkCloud.KubernetesCluster` resource's `networkConfiguration.cniNetworkId`

+field. This `defaultcni` network is the default gateway for all Pods that run

+on that Node and serves as the default network for east-west Pod to Pod

+communication within the Nexus Kubernetes cluster.

+[lladdr]: https://en.wikipedia.org/wiki/Link-local_address

+### Pod networking

+Kubernetes Pods are collections of one or more container images that run in a

+[Linux namespace][linux-ns]. This Linux namespace isolates the containerΓÇÖs

+processes and resources from other containers and processes on the host. For

+Nexus Kubernetes clusters, this "host" is a VM that is represented as a

+Kubernetes Worker Node.

+Before creating an Operator Nexus Kubernetes Cluster, users first create a set

+of resources that represent the virtual networks from which tenant workloads

+are assigned addresses. These virtual networks are then referenced in the

+`cniNetworkId`, `cloudServicesNetworkId`, `agentPoolL2Networks`,

+`agentPoolL3Networks`, and `agentPoolTrunkedNetworks` fields when creating the

+Operator Nexus Kubernetes Cluster.

+Pods can run on any compute server in any rack in an Operator Nexus instance.

+By default all Pods in a Nexus Kubernetes cluster can communicate with each

+other over what is known as the [*pod network*][podnetwork]. Several

+[Container Networking Interface][cni] (CNI) plugins that are installed in each

+Nexus Kubernetes Worker Node manage the Pod networking.

+#### Extra Networks

+When creating a Pod in a Nexus Kubernetes Cluster, you declare any extra

+networks that the Pod should attach to by [specifying][specify-net-anno] a

+`k8s.v1.cni.cnf.io/networks` annotation. The annotation's value is a

+comma-delimited list of network names. These network names correspond to names

+of any Trunked, L3 or L2 Networks associated with the Nexus Kubernetes Cluster

+or Agent Pool.

+Operator Nexus configures the Agent Pool VM with

+[NetworkAttachmentDefinition][nad] (NAD) files that contain network

+configuration for a single extra network.

+For each Trunked Network listed in the Pod's associated networks, the Pod gets

+a single network interface. The workload is responsible for sending raw tagged

+traffic through this interface or constructing tagged interfaces on top of the

+network interface.

+For each L2 Network listed in the Pod's associated networks, the Pod gets a

+single network interface. The workload is responsible for their own static MAC

+addressing.

+#### Pod IP Address Management

+When you create a Nexus Kubernetes cluster, you specify the IP address ranges

+for the pod network in the `podCidrs` field. When Pods launch, the CNI plugin

+establishes an `eth0@ifXX` interface in the Pod and assigns an IP address from

+a range of IP addresses in that `podCidrs` field.

+For L3 Networks, if the network has been configured to use Nexus IPAM, the

+Pod's network interface associated with the L3 Network receives an IP address

+from the IP address range (CIDR) configured for that network. If the L3 Network

+isn't configured to use Nexus IPAM, the workload is responsible for statically

+assigning an IP address to the Pod's network interface.

+#### Routing

+Inside each Pod, the `eth0` interface's traffic traverses a

+[virtual ethernet device][veth] (veth) that connects to a

+[switchdev][switchdev] on the host (the VM) that houses the `defaultcni`,

+`cloudservices`, and other Node-level interfaces.

+The `eth0` interface inside a Pod has a simple route table that effectively

+uses the worker node VM's route table for any of the following traffic.

+* Pod to pod traffic: Traffic destined for an IP in the `podCidrs` address

+ ranges flows to the switchdev on the host VM and over the Node-level

+ `defaultcni` interface where it is routed to the appropriate destination agent

+ pool VM's IP address.

+* L3 OSDevice network traffic: Traffic destined for an IP in an associated L3

+ Network with the `OSDevice` plugin type flows to the switchdev on the host VM

+ and over the Node-level interface associated with that L3 Network.

+* All other traffic passes to the default gateway in the Pod, which routes to the

+ Node-level `cloudservices` interface. Egress rules configured on the

+ CloudServicesNetwork associated with the Nexus Kubernetes cluster then

+ determine how the traffic should be routed.

+Additional network interfaces inside a Pod will use the Pod's route table to

+route traffic to additional L3 Networks that use the `SRIOV` and `DPDK` plugin

+types.

+[linux-ns]: https://en.wikipedia.org/wiki/Linux_namespaces

+[podnetwork]: https://kubernetes.io/docs/concepts/cluster-administration/networking/

+[cni]: https://www.cni.dev/

+[veth]: https://www.man7.org/linux/man-pages/man4/veth.4.html

+[switchdev]: https://www.kernel.org/doc/html/latest/networking/switchdev.html

+[specify-net-anno]: https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/docs/how-to-use.md#run-pod-with-network-annotation

+[nad]: https://github.com/k8snetworkplumbingwg/multi-net-spec/blob/master/v1.3/%5Bv1.3%5D%20Kubernetes%20Network%20Custom%20Resource%20Definition%20De-facto%20Standard.pdf

+<!

+### Service network configuration

+TODO(jaypipes)

+## Default Routing and BGP Configuration

+CNFs are typically a collection of Kubernetes Pods that are connected to one or

+more virtual networks. Those virtual networks are routed across the physical

+network infrastructure via [Border Gateway Protocol][bgp] (BGP).

+TODO(jaypipes)

+[bgp]: https://en.wikipedia.org/wiki/Border_Gateway_Protocol

+-->

+ Title: "Azure Operator Nexus: Nexus Workload Network"

+description: Introduction to Workload networks core concepts.

+# Nexus workload Network Overview

+This article describes core concepts of Nexus workload networks, and introduction to options to configure nexus workload networks with critical properties.

+Nexus workload network enables application connect with on-premises network and other services over Azure public cloud. It supports operator use cases with

+standard industry technologies that are reliable, predictable, and familiar to operators and network equipment providers.

+Nexus offers several top-level API resources that categorically represent different types of networks with different input expectations.

+These network types represent logical attachments but also Layer3 information as well. Essentially, they encapsulate how the customer

+wishes those networks are to be exposed within their cluster.

+## Nexus workload network types

+ * L3Network: The Nexus workload L3Network resource can be shared and reused across standalone virtual machines and Nexus Kubernetes clusters. Its primary purpose is to define a network that supports Layer 3 properties, which are coordinated between the virtualized workloads and the integrated Nexus Managed Fabric L3IsolationDomain. Additionally, it provides DualStack allocation capabilities (both IPv4 and IPv6) and directly references Azure Managed Network Fabric resources representing the VRF and VLAN associated with this network

+ * L2Network: The Nexus workload L2Network resource can be shared and reused across standalone virtual machines and Nexus AKS clusters. Its primary purpose is to grant direct access to a specific Nexus Managed Fabric L2IsolationDomain, enabling isolated network attachment within the Nexus Cluster. Customers utilize L2Network resources when they want the fabric to carry a VLAN across workloads without participating in Layer 3 on that network.

+ * TrunkedNetwork: The Nexus workload TrunkedNetwork resource allows association with multiple IsolationDomains, enabling customers to create a custom VLAN trunk range that workloads can access. The TrunkedNetwork defines the allowable VLAN set that workloads can directly tag traffic on. Tagged traffic for VLANs not specified in the TrunkedNetwork resource will be dropped. This custom VLAN trunk range can span across the same IsolationDomain or multiple L3IsolationDomains and/or L2IsolationDomains.

+## Nexus workload network plugins

+Network plugin is the feature to configuration how applications use the underlying Networks when attaching networks to application VMs or Pods.

+The type of plugins supported for different network types.

+| Plugin Name | Available Network Types |

+|||

+|SRIOV|L2Network, L3Network, TrunkedNetwork|

+|DPDK|L2Network, L3Network, TrunkedNetwork|

+|MACVLAN|L2Network, L3Network, TrunkedNetwork|

+|IPVLAN|L3Network, TrunkedNetwork|

+|OSDev|L2Network, L3Network, TrunkedNetwork|

+ * SRIOV: The SRIOV plugin generates a network attachment definition named after the corresponding network resource. This interface is integrated into a sriov-dp-config resource,

+which is linked to by the network attachment definition. If a network is connected to the cluster multiple times, all interfaces will be available for scheduling via the network

+attachment definition. No IP assignment is made to this type of interface within the node operating system.

+ * DPDK: Configured specifically for DPDK workloads, the DPDK plugin type creates a network attachment definition that mirrors the associated network resource. This interface is

+placed within a sriov-dp-config resource, which the network attachment definition references. Multiple connections of the same network to the cluster make all interfaces schedulable

+through the network attachment definition. Depending on the hardware of the platform, the interface might be linked to a specific driver to support DPDK processing. Like SRIOV, this

+interface doesn't receive an IP assignment within the node operating system.

+ * OSDevice: The OSDevice plugin type is tailored for direct use within the node operating system, rather than Kubernetes. It acquires a network configuration that is visible and

+functional within the nodeΓÇÖs operating system network namespace. This plugin is suitable for instances where direct communication over this network from the nodeΓÇÖs OS is required.

+ * IPVLAN: The IPVLAN plugin type helps the creation of a network attachment definition named according to the associated network resource. This interface allows for the efficient

+routing of traffic in environments where network isolation is required without the need for multiple physical network interfaces. It operates by assigning multiple IP addresses to a

+single network interface, each behaving as if it is on a separate physical device. Despite the separation at the IP layer, this type doesn't handle separate MAC addresses, and it doesn't provide IP assignments within the node operating system.

+ * MACVLAN: The MACVLAN plugin type generates a network attachment definition reflective of the linked network resource. This interface type creates multiple virtual networks interfaces

+each with a unique MAC address over a single physical network interface. It's useful in scenarios where applications running in containers need to appear as physically

+separate on the network for security or compliance reasons. Each interface behaves as if it's directly connected to the physical network, which allows for IP assignments within the

+node operating system.

+## Nexus Network IPAM

+Nexus Kubernetes offers IP Address Management (IPAM) solutions in various forms. For standalone virtual machines(VNF workload) or Nexus kubernetes nodes(CNF workload) connected to a Nexus network supporting Layer 3,

+an IPAM system is employed that covers multiple clusters. This system ensures unique IP addresses across both VMs and Nexus Kubernetes nodes within network VM interfaces inside VM operating

+systems. Additionally, when these networks are utilized for containerized workloads, Network Attachment Definitions (NADs) automatically generated by Nexus kubernetes cluster incorporate this IPAM feature.

+This same cross-cluster IPAM capability is used to guarantee that containers connected to the same networks receive unique IP addresses as well.

+## Nexus Relay

+Nexus Kubernetes utilizes the [Arc](../azure-arc/overview.md) [Azure Relay](../azure-relay/relay-what-is-it.md) functionality by integrating the Nexus kubernetes Hybrid Relay infrastructure in each region where the Nexus Cluster service operates.

+This setup uses dedicated Nexus relay infrastructure within Nexus owned subscriptions, ensuring that Nexus kubernetes cluster Arc Connectivity doesn't rely on shared public relay networks.

+Each Nexus kubernetes cluster and node instance is equipped with its own relay, and customers can manage Network ACL rules through the Nexus Cluster Azure Resource Manager APIs. These rules determine which networks can access both the az connectedk8s proxy and az ssh for their Nexus Arc resources within that specific on-premises Nexus Cluster. This feature enhances operator security by adhering to security protocols established after previous Arc/Relay security incidents, requiring remote Arc connectivity to have customer-defined network filters or ACLs.

+To get started with Operator Nexus, you need to create a Network Fabric Controller (NFC) and then a Cluster Manager (CM) in your target Azure region.

+## Install CLI Extensions and sign-in to your Azure subscription

+Install latest version of the

+[necessary CLI extensions](./howto-install-cli-extensions.md).

+### Azure subscription sign-in

+```azurecli

+ az login

+ az account set --subscription $SUBSCRIPTION_ID

+ az account show

+```

+>[!NOTE]

+>Your account must have permissions to read/write/publish in the subscription

+## Resource Provider registration

+Ensure access to the necessary Azure Resource Providers for the Azure Subscription for Operator Nexus resources. Register the following providers:

+```Azure CLI

+az provider register --namespace Microsoft.Compute

+az provider register --namespace Microsoft.ContainerService

+az provider register --namespace Microsoft.ExtendedLocation

+az provider register --namespace Microsoft.HybridCompute

+az provider register --namespace Microsoft.HybridConnectivity

+az provider register --namespace Microsoft.HybridContainerService

+az provider register --namespace Microsoft.HybridNetwork

+az provider register --namespace Microsoft.Insights

+az provider register --namespace Microsoft.Keyvault

+az provider register --namespace Microsoft.Kubernetes

+az provider register --namespace Microsoft.KubernetesConfiguration

+az provider register --namespace Microsoft.ManagedIdentity

+az provider register --namespace Microsoft.ManagedNetworkFabric

+az provider register --namespace Microsoft.Network

+az provider register --namespace Microsoft.NetworkCloud

+az provider register --namespace Microsoft.OperationalInsights

+az provider register --namespace Microsoft.OperationsManagement

+az provider register --namespace Microsoft.ResourceConnector

+az provider register --namespace Microsoft.Resources

+az provider register --namespace Microsoft.Storage

+```

+## EncryptionAtHost feature registration

+You must enable [EncryptionAtHost](/azure/virtual-machines/linux/disks-enable-host-based-encryption-cli) feature for your subscription. Use the following steps to enable the feature for your subscription:

+### Register the EncryptionAtHost feature:

+Execute the following command to register the feature for your subscription

+```Azure CLI

+az feature register --namespace Microsoft.Compute --name EncryptionAtHost

+```

+### Verify the registration State:

+Confirm that the registration state is Registered (registration might take a few minutes) using the following command before trying out the feature.

+```Azure CLI

+az feature show --namespace Microsoft.Compute --name EncryptionAtHost

+```

+### Register the Resource Provider:

+```Azure CLI

+az provider register --namespace Microsoft.Compute

+```

+Ensure that the registration state is Registered.

+description: Best practices for migration into Azure Database for PostgreSQL, including premigration validation, target server configuration, migration timeline, and migration speed benchmarking.

+As a first step in the migration, run the premigration validation before you perform a migration. You can use the **Validate** and **Validate and Migrate** options on the migration **Setup** page. Premigration validation conducts thorough checks against a predefined rule set. The goal is to identify potential problems and provide actionable insights for remedial actions. Keep running premigration validation until it results in a **Succeeded** state. To learn more, see [Premigration validations](concepts-premigration-migration-service.md).

+## Target Flexible Server configuration

+During the initial base copy of data, multiple insert statements are executed on the target, which generates write-ahead logs (WALs). Until these WALs are archived, the logs consume storage at the target and the storage required by the database.

+To calculate the number, sign in to the source instance and run this command for all the databases to be migrated:

+We recommend that you allocate sufficient storage on the flexible server, equivalent to 1.25 times or 25% more storage than what's being used per the output to the preceding command. You can also use [Storage Autogrow](../../flexible-server/how-to-auto-grow-storage-portal.md).

+> Storage size can't be reduced in manual configuration or Storage Autogrow. Each step in the storage configuration spectrum doubles in size, so estimating the required storage beforehand is prudent.

+The quickstart to [create an Azure Database for PostgreSQL - Flexible Server instance by using the portal](../../flexible-server/quickstart-create-server-portal.md) is an excellent place to begin. For more information about each server configuration, see [Compute and storage options in Azure Database for PostgreSQL - Flexible Server](../../flexible-server/concepts-compute-storage.md).

+Each migration has a maximum lifetime of seven days (168 hours) after it starts, and it times out after seven days. You can complete your migration and application cutover after the data validation and all checks are complete to avoid the migration from timing out. In online migrations, after the initial base copy is complete, the cutover window lasts three days (72 hours) before timing out. In offline migrations, the applications should stop writing to the database to prevent data loss. Similarly, for online migration, keep traffic low throughout the migration.

+Most nonproduction servers (dev, UAT, test, and staging) are migrated by using offline migrations. Because these servers have less data than the production servers, the migration is fast. For production server migration, you need to know the time it would take to complete the migration to plan for it in advance.

+The time taken for a migration to complete depends on several factors. It includes the number of databases, size, number of tables inside each database, number of indexes, and data distribution across tables. It also depends on the SKU of the target server and the IOPS available on the source instance and target server. With so many factors that can affect the migration time, it's hard to estimate the total time for a migration to complete. The best approach is to perform a test migration with your workload.

+The following phases are considered for calculating the total downtime to perform production server migration:

+- **Migration of PITR**: The best way to get a good estimate on the time taken to migrate your production database server is to take a point-in time restore (PITR) of your production server and run the offline migration on this newly restored server.

+- **Migration of buffer**: After you finish the preceding step, you can plan for actual production migration during a time period when application traffic is low. This migration can be planned on the same day or probably a week away. By this time, the size of the source server might have increased. Update your estimated migration time for your production server based on the amount of this increase. If the increase is significant, consider doing another test by using the PITR server. But for most servers, the size increase shouldn't be significant enough.

+- **Data validation**: After the migration is finished for the production server, you need to verify if the data in the flexible server is an exact copy of the source instance. You can use open-source or third-party tools or you can do the validation manually. Prepare the validation steps you want to do before the actual migration. Validation can include:

+ - Row count match for all the tables involved in the migration.

+ - Matching counts for all the database objects (tables, sequences, extensions, procedures, and indexes).

+ - Comparing maximum or minimum IDs of key application-related columns.

+ > [!NOTE]

+ > The size of databases needs to be the right metric for validation. The source instance might have bloats or dead tuples, which can bump up the size of the source instance. It's normal to have size differences between source instances and target servers. An issue in the first three steps of validation indicates a problem with the migration.

+- **Migration of server settings**: Any custom server parameters, firewall rules (if applicable), tags, and alerts must be manually copied from the source instance to the target.

+- **Changing connection strings**: The application should change its connection strings to a flexible server after successful validation. This activity is coordinated with the application team to change all the references of connection strings pointing to the source instance. In the flexible server, the user parameter can be used in the **user=username** format in the connection string.

+For example: `psql -h myflexserver.postgres.database.azure.com -u user1 -d db1`

+Although a migration often runs without any problems, it's good practice to plan for contingencies if more time is required for debugging or if a migration needs to be restarted.

+The following table shows the time it takes to perform migrations for databases of various sizes by using the migration service. The migration was performed by using a flexible server with the SKU Standard_D4ds_v4 (4 cores, 16-GB memory, 128-GB disk, and 500 IOPS).

+The preceding numbers give you an approximation of the time taken to complete the migration. We strongly recommend running a test migration with your workload to get a precise value for migrating your server.

+> Choose a higher SKU for your flexible server to perform faster migrations. Azure Database for PostgreSQL - Flexible Server supports near-zero downtime compute and IOPS scaling, so the SKU can be updated with minimal downtime. You can always change the SKU to match the application needs post-migration.

+### Improve migration speed: Parallel migration of tables

+We recommend a powerful SKU for the target because the PostgreSQL migration service runs out of a container on the flexible server. A powerful SKU enables more tables to be migrated in parallel. You can scale the SKU back to your preferred configuration after the migration. This section contains steps to improve the migration speed if the data distribution among the tables needs to be more balanced or a more powerful SKU doesn't significantly affect the migration speed.

+If the data distribution on the source is highly skewed, with most of the data present in one table, the allocated compute for migration needs to be fully utilized, which creates a bottleneck. So, split large tables into smaller chunks, which are then migrated in parallel. This feature applies to tables with more than 10,000,000 (10 m) tuples. Splitting the table into smaller chunks is possible if one of the following conditions is satisfied:

+- The table must have a column with a simple (not composite) primary key or unique index of type `int` or `significant int`.

+ > [!NOTE]

+ > In the case of the first or second approaches, you must carefully evaluate the implications of adding a unique index column to the source schema. Only after confirmation that adding a unique index column won't affect the application should you go ahead with the changes.

+- If the table doesn't have a simple primary key or unique index of type `int` or `significant int` but has a column that meets the data type criteria, the column can be converted into a unique index by using the following command. This command doesn't require a lock on the table.

+- If the table doesn't have a `simple int`/`big int` primary key or unique index or any column that meets the data type criteria, you can add such a column by using [ALTER](https://www.postgresql.org/docs/current/sql-altertable.html) and drop it post-migration. Running the `ALTER` command requires a lock on the table.

+If any of the preceding conditions are satisfied, the table is migrated in multiple partitions in parallel, which should provide an increase in the migration speed.

+- The migration service looks up the maximum and minimum integer value of the table's primary key/unique index that must be split up and migrated in parallel.

+- If the difference between the minimum and maximum value is more than 10,000,000 (10 m), the table is split into multiple parts and each part is migrated in parallel.

+- The table has at least 10,000,000 (10 m) rows so that the difference between the minimum and maximum value of the primary key is more than 10,000,000 (10 m).

+Over time, as data is added, updated, and deleted, PostgreSQL might accumulate dead rows and wasted storage space. This bloat can lead to increased storage requirements and decreased query performance. Vacuuming is a crucial maintenance task that helps reclaim this wasted space and ensures the database operates efficiently. Vacuuming addresses issues such as dead rows and table bloat to ensure efficient use of storage. It also helps to ensure quicker migration because the migration time is a function of the database size.

+PostgreSQL provides the `VACUUM` command to reclaim storage occupied by dead rows. The `ANALYZE` option also gathers statistics to further optimize query planning. For tables with heavy write activity, the `VACUUM` process can be more aggressive by using `VACUUM FULL`, but it requires more time to run.

+- Standard vacuum

+ ```sql

+ VACUUM your_table;

+ ```

+- Vacuum with analyze

+ ```sql

+ VACUUM ANALYZE your_table;

+ ```

+- Aggressive vacuum for heavy write tables

+ ```sql

+ VACUUM FULL your_table;

+ ```

+In this example, replace your_table with the actual table name. The `VACUUM` command without `FULL` reclaims space efficiently, whereas `VACUUM ANALYZE` optimizes query planning. The `VACUUM FULL` option should be used judiciously because of its heavier performance impact.

+Some databases store large objects, such as images or documents, that can contribute to database bloat over time. The `VACUUMLO` command is designed for large objects in PostgreSQL.

+- Vacuum large objects

+ ```sql

+ VACUUMLO;

+ ```

+There are special conditions that typically refer to unique circumstances, configurations, or prerequisites that you need to be aware of before you proceed with a tutorial or module. These conditions could include specific software versions, hardware requirements, or other tools that are necessary for successful completion of the learning content.

+Online migration makes use of [pgcopydb follow](https://pgcopydb.readthedocs.io/en/latest/ref/pgcopydb_follow.html), and some of the [logical decoding restrictions](https://pgcopydb.readthedocs.io/en/latest/ref/pgcopydb_follow.html#pgcopydb-follow) apply. We also recommend that you have a primary key in all the tables of a database that's undergoing online migration. If a primary key is absent, the deficiency results in only `insert` operations being reflected during migration, excluding updates or deletes. Add a temporary primary key to the relevant tables before you proceed with the online migration.

+> In the case of online migration of tables without a primary key, only `insert` operations are replayed on the target. This can potentially introduce inconsistency in the database if records that are updated or deleted on the source don't reflect on the target.

+An alternative is to use the `ALTER TABLE` command where the action is [REPLICA IDENTIY](https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY) with the `FULL` option. The `FULL` option records the old values of all columns in the row so that even in the absence of a primary key, all CRUD operations are reflected on the target during the online migration. If none of these options work, perform an offline migration as an alternative.

+The [postgres_fdw module](https://www.postgresql.org/docs/current/postgres-fdw.html) provides the foreign data wrapper postgres_fdw, which can be used to access data stored in external PostgreSQL servers. If your database uses this extension, the following steps must be performed to ensure a successful migration.

+1. Temporarily remove (unlink) the foreign data wrapper on the source instance.

+1. Perform data migration of the rest by using the migration service.

+1. Restore the foreign data wrapper roles, user, and links to the target after migration.

+The postGIS extension has breaking changes/compact issues between different versions. If you migrate to a flexible server, the application should be checked against the newer postGIS version to ensure that the application isn't affected or that the necessary changes must be made. The [postGIS news](https://postgis.net/news/) and [release notes](https://postgis.net/docs/release_notes.html#idm45191) are a good starting point to understand the breaking changes across versions.

+Sometimes, you might encounter this error when you start a migration:

+In this scenario, you can grant the `migration user` permission to close all active connections to the database or close the connections manually before you retry the migration.

+ Title: "Migration service - Known issues and limitations"

+description: This article provides the limitations and known issues of the migration service in Azure Database for PostgreSQL.

+- You can have only one active migration or validation to your flexible server.

+- The migration service only supports migration for users and roles when the source is Azure Database for PostgreSQL - Single Server.

+- The migration service migrates only user databases, not system databases, such as template_0 and template_1.

+- The migration service doesn't support moving TIMESCALEDB, POSTGIS_TOPOLOGY, POSTGIS_TIGER_GEOCODER, or PG_PARTMAN extensions from source to target.

+- You can't move extensions not supported by Azure Database for PostgreSQL - Flexible Server. The supported extensions are listed in [Extensions - Azure Database for PostgreSQL](/azure/postgresql/flexible-server/concepts-extensions).

+- User-defined collations can't be migrated into Azure Database for PostgreSQL - Flexible Server.

+- You can't migrate to an older version. For instance, you can't migrate from Azure Database for PostgreSQL version 15 to version 14.

+- Azure Database for PostgreSQL - Flexible Server doesn't support the creation of custom tablespaces because of superuser privilege restrictions. During migration, data from custom tablespaces in the source PostgreSQL instance is migrated into the default tablespaces of the target Azure Database for PostgreSQL - Flexible Server instance.

+- The migration service is unable to perform migration when the source database is Azure Database for PostgreSQL - Single Server with no public access or is an on-premises/AWS using a private IP, and the target Azure Database for PostgreSQL - Flexible Server instance is accessible only through a private endpoint.

+- Migration to burstable SKUs isn't supported. Databases must first be migrated to a nonburstable SKU and then scaled down if needed.

+- The Migration Runtime Server is designed to operate with the default DNS servers/private DNS zones, for example, `privatelink.postgres.database.azure.com`. Custom DNS names/DNS servers aren't supported by the migration service when you use the Migration Runtime Server feature. When you're configuring private endpoints for both the source and target databases, it's imperative to use the default private DNS zone provided by Azure for the private link service. The use of custom DNS configurations isn't yet supported and might lead to connectivity issues during the migration process.

+## Limitations migrating from Azure Database for PostgreSQL - Single Server

+- Microsoft Entra ID users present on your source server aren't migrated to the target server. To mitigate this limitation, see [Manage Microsoft Entra roles](../../flexible-server/how-to-manage-azure-ad-users.md) to manually create all Microsoft Entra users on your target server before you trigger a migration. If Microsoft Entra users aren't created on the target server, migration fails.

+- If the target flexible server uses the SCRAM-SHA-256 password encryption method, connection to a flexible server using the users/roles on a single server fails because the passwords are encrypted by using the md5 algorithm. To mitigate this limitation, choose the option `MD5` for the `password_encryption` server parameter on your flexible server.

+- Online migration makes use of [pgcopydb follow](https://pgcopydb.readthedocs.io/en/latest/ref/pgcopydb_follow.html), and some of the [logical decoding restrictions](https://pgcopydb.readthedocs.io/en/latest/ref/pgcopydb_follow.html#pgcopydb-follow) apply.

+ Title: "Migration Runtime Server in Azure Database for PostgreSQL"

+description: "This article discusses concepts about Migration Runtime Server with the migration service in Azure Database for PostgreSQL."

+# Migration Runtime Server with the migration service in Azure Database for PostgreSQL preview

+Migration Runtime Server is a specialized feature in the [migration service in Azure Database for PostgreSQL](concepts-migration-service-postgresql.md) that acts as an intermediary server during migration. It's a separate Azure Database for PostgreSQL - Flexible Server instance that isn't the target server. It's used to facilitate the migration of databases from a source environment that's only accessible via a private network.

+Migration Runtime Server is helpful in scenarios where both the source PostgreSQL instances and the target Azure Database for PostgreSQL - Flexible Server instance are configured to communicate over private endpoints or private IPs. This arrangement ensures that the migration occurs within a secure and isolated network space. Migration Runtime Server handles the data transfer. It connects to the source PostgreSQL instance to retrieve data and then push it to the target server.

+Migration Runtime Server is distinct from the target server and is configured to handle the data transfer process, ensuring a secure and efficient migration path.

+Migration Runtime Server is essential for transferring data between different source PostgreSQL instances and the Azure Database for PostgreSQL - Flexible Server instance. It's necessary in the following scenarios:

+- When the source is an Azure Database for PostgreSQL - Single Server configured with a private endpoint and the target is an Azure Database for PostgreSQL - Flexible Server with a private endpoint.

+- For sources such as on-premises databases, Azure virtual machines, or AWS instances that are only accessible via private networks and the target Azure Database for PostgreSQL - Flexible Server instance with a private endpoint.

+To use the Migration Runtime Server feature within the migration service in Azure Database for PostgreSQL, you have two migration options:

+- Use the Azure portal during setup.

+- Specify the `migrationRuntimeResourceId` parameter in the JSON properties file during the migration create command in the Azure CLI.

+Here's how to do it in both methods.

+1. Sign in to the Azure portal and access the migration service (from the target server) in the Azure Database for PostgreSQL instance.

+1. Begin a new migration workflow within the service.

+1. When you reach the **Select runtime server** tab, select **Yes** to use Migration Runtime Server.

+1. Select your Azure subscription and resource group. Select the location of the virtual network-integrated Azure Database for PostgreSQL - Flexible Server instance.

+1. Select the appropriate Azure Database for PostgreSQL - Flexible Server instance to serve as your Migration Runtime Server instance.

+ :::image type="content" source="media/concepts-migration-service-runtime-server/select-runtime-server.png" alt-text="Screenshot that shows selecting Migration Runtime Server.":::

+### Use the Azure CLI

+1. Open your command-line interface.

+1. Ensure that you have the Azure CLI installed and that you're signed in to your Azure account by using `az sign-in`.

+1. The version should be at least 2.62.0 or above to use the Migration Runtime Server option.

+1. The `az postgres flexible-server migration create` command requires a JSON file path as part of the `--properties` parameter, which contains configuration details for the migration. Provide the `migrationRuntimeResourceId` parameter in the JSON properties file.

+- **Minimal configuration**: Despite being created from Azure Database for PostgreSQL - Flexible Server, Migration Runtime Server solely facilitates migration without the need for high availability, backups, version specificity, or advanced storage features.

+- **Performance and sizing**: Migration Runtime Server must be appropriately scaled to manage the workload. We recommend that you select an SKU equivalent to or greater than that of the target server.

+- **Networking**: Ensure that Migration Runtime Server is appropriately integrated into the virtual network and that network security allows for secure communication with both the source and target servers. For more information, see [Network guide for migration service](how-to-network-setup-migration-service.md).

+- **Post-migration cleanup**: After the migration is finished, Migration Runtime Server should be decommissioned to avoid unnecessary costs. Before deletion, ensure that all data was successfully migrated and that the server is no longer needed.

+ Title: "Migration service - Premigration validations"

+description: Learn about premigration validations to identify issues before you run a migration to Azure Database for PostgreSQL.

+To use premigration validation when you migrate to Azure Database for PostgreSQL - Flexible Server, you have two migration options:

+- Use the Azure portal during setup.

+- Specify the `--migration-option` parameter in the Azure CLI when you create a migration.

+Here's how to do it in both methods.

+### Use the Azure portal

+1. Go to the migration tab in Azure Database for PostgreSQL.

+1. Select **Create**.

+1. On the **Setup** page, choose the migration option that includes validation. Select **Validate** or **Validate and Migrate**.

+ :::image type="content" source="media\concepts-premigration-migration-service\premigration-option.png" alt-text="Screenshot that shows the premigration option to start migration." lightbox="media\concepts-premigration-migration-service\premigration-option.png":::

+### Use the Azure CLI

+1. Open your command-line interface.

+1. Ensure that you have the Azure CLI installed and that you're signed in to your Azure account by using `az sign-in`.

+ The version should be at least 2.56.0 or above to use the migration option.

+1. Construct your migration task creation command with the Azure CLI.

+ ```bash

+ az postgres flexible-server migration create --subscription <subscription ID> --resource-group <Resource group Name> --name <Flexible server Name> --migration-name <Unique migration ID> --migration-option ValidateAndMigrate --properties "Path of the JSON File" --migration-mode offline

+ ```

+1. Include the `--migration-option` parameter followed by the `Validate` option to perform only the premigration. Use `Validate`, `Migrate`, or `ValidateAndMigrate` to perform validation. If the validation is successful, continue with the migration.

+## Premigration validation options

+You can choose any of the following options:

+- **Validate**: Use this option to check your server and database readiness for migration to the target. *This option won't start data migration and won't require any server downtime.*

+ - Plan your migrations better by performing premigration validations in advance to know the potential issues you might encounter while you perform migrations.

+- **Migrate**: Use this option to kickstart the migration without going through a validation process. Perform validation before you trigger a migration to increase the chances of success. After validation is finished, you can use this option to start the migration process.

+- **Validate and Migrate**: This option performs validations, and migration gets triggered if all checks are in the **Succeeded** or **Warning** state. Validation failures don't start the migration between source and target servers.

+We recommend that you use premigration validations to identify issues before you run migrations. This technique helps you to plan your migrations better and avoid any surprises during the migration process.

+1. Rerun step 1 until the validation is successful.

+1. Start the migration by using the **Validate and Migrate** option on the planned date and time.

+## Validation states

+After you run the **Validate** option, you see one of the following options:

+- **Succeeded**: No issues were found and you can plan for the migration.

+- **Failed**: Errors were found during validation, which can cause the migration to fail. Review the list of errors and their suggested workarounds. Take corrective measures before you plan the migration.

+- **Warning**: Warnings are informative messages you must remember while you plan the migration.

+description: Learn about the migration of user roles, ownerships, and privileges along with schema and data for the migration service in Azure Database for PostgreSQL.

+# Migration of user roles, ownerships, and privileges for the migration service in Azure Database for PostgreSQL

+> The migration of user roles, ownerships, and privileges feature is available only for the Azure Database for PostgreSQL - Single Server instance as the source. This feature is currently disabled for PostgreSQL version 16 servers.

+The migration service automatically provides the following built-in capabilities for Azure Database for PostgreSQL - Single Server as the source and data migration:

+- Migration of permissions of database objects on your source server, such as `GRANT`/`REVOKE`, to the target server.

+## Permission differences between Azure Database for PostgreSQL - Single Server and Flexible Server

+Unlike user-created schemas, which organize database objects into logical groups, pg_catalog is a system schema. It houses crucial system-level information, such as details about tables, columns, and other internal bookkeeping data. It's where PostgreSQL stores important metadata.

+- In a single server environment, a user belonging to the azure_pg_admin role is granted select privileges for all pg_catalog tables and views.

+- In a flexible server environment, privileges are restricted for certain tables and views so that only superusers are allowed to query them.

+We removed all privileges for non-superusers on the following pg_catalog tables:

+- pg_authid

+- pg_largeobject

+- pg_subscription

+- pg_user_mapping

+We removed all privileges for non-superusers on the following pg_catalog views:

+- pg_config

+- pg_file_settings

+- pg_hba_file_rules

+- pg_replication_origin_status

+- pg_shadow

+Allowing unrestricted access to these system tables and views could lead to unauthorized modifications, accidental deletions, or even security breaches. Restricted access reduces the risk of unintended changes or data exposure.

+Another important consideration is the deprecation of the **pg_pltemplate** system table within the pg_catalog schema by the PostgreSQL community *starting from version 13*. If you're migrating to Flexible Server versions 13 and above and have granted permissions to users on the pg_pltemplate table on your single server, you must revoke these permissions before you initiate a new migration.

+- If your application is designed to directly query the affected tables and views, it encounters issues upon migrating to the flexible server. We strongly advise you to refactor your application to avoid direct queries to these system tables.

+- If you've granted or revoked privileges to any users or roles for the affected pg_catalog tables and views, you encounter an error during the migration process. You can identify this error by the following pattern:

+ ```sql

+ pg_restore error: could not execute query <GRANT/REVOKE> <PRIVILEGES> on <affected TABLE/VIEWS> to <user>.

+ ```

+To resolve this error, it's necessary to undo the privileges granted to users and roles on the affected pg_catalog tables and views. You can accomplish this task by taking the following steps.

+**Step 1: Identify privileges**

+**Step 2: Review the output**

+To undo the privileges, run `REVOKE` statements for each privilege on the relation from the grantee. In this example, you would run:

+**Step 4: Final verification**

+Run the query from step 1 again to ensure that the resulting output set is empty.

+> Make sure you perform the preceding steps for all the databases included in the migration to avoid any permission-related issues during the migration.

+After you finish these steps, you can proceed to initiate a new migration from the single server to the flexible server by using the migration service. You shouldn't encounter permission-related issues during this process.

+When configuring Azure Private Link service, the explicit setting `privateLinkServiceNetworkPolicies` must be disabled on the subnet. This setting only affects the Private Link service. For other resources in the subnet, access is controlled based on the network security group security rules definition.

+| **Changes to existing incident names** | In the unified SOC operations platform, the Defender portal uses a unique engine to correlate incidents and alerts. When onboarding your workspace to the unified SOC operations platform, existing incident names might be changed if the correlation is applied. To ensure that your automation rules always run correctly, we therefore recommend that you avoid using incident titles as condition criteria in your automation rules, and suggest instead to use the name of the analytics rule that created the incident, and tags if more specificity is required. |

+ | **Status** | **Enabled**: The rule runs immediately upon creation, or at the [specific date and time you choose to schedule it (currently in PREVIEW)](#schedule-and-scope-the-query).<br>**Disabled**: The rule is created but doesn't run. Enable it later from your **Active rules** tab when you need it. |

+Also, learn from an example of using custom analytics rules when [monitoring Zoom](https://techcommunity.microsoft.com/t5/azure-sentinel/monitoring-zoom-with-azure-sentinel/ba-p/1341516) with a [custom connector](create-custom-connector.md).

+**Status:** When you create the rule, its **Status** is **Enabled** by default, which means it runs immediately after you finish creating it. If you donΓÇÖt want it to run immediately, you have two options:

+- Select **Disabled**, and the rule is created without running. When you want the rule to run, find it in your **Active rules** tab, and enable it from there.

+ var serviceBusNamespace = $"{namespace}.servicebus.windows.net";

+ "%s.servicebus.windows.net",

+ namespace + ".servicebus.windows.net";

+ namespace + ".servicebus.windows.net";

+ namespace + ".servicebus.windows.net";

+ const serviceBusNamespace = `${namespace}.servicebus.windows.net`;

+ service_bus_namespace = "%s.servicebus.windows.net" % namespace

+The maximum prefetch count and the lock duration configured on the queue or subscription need to be balanced such that the lock timeout at least exceeds the cumulative expected message processing time for the maximum size of the prefetch buffer, plus one message. At the same time, the lock duration shouldn't be so long that messages can exceed their maximum time to live while being locked, as this would mean they get removed if they could not be completed when they were prefetched.

+> [!NOTE]

+> Automatic OS image upgrade is supported for both platform and gallery based OS images.

+To adjust the placement properties for a node type using an ARM Template, adjust the `placementProperties` property with one or more new values and do a cluster deployment for the setting to take effect. The following sample shows three values being set for a node type.

+## Verify account permissions

+If you just created your free Azure account, you're the administrator of your subscription and you have the permissions you need. If you're not the subscription administrator, work with the administrator to assign the permissions you need. To enable replication for a new virtual machine, you must have permission to:

+- Create a virtual machine in the selected resource group.

+- Create a virtual machine in the selected virtual network.

+- Write to an Azure storage account.

+- Write to an Azure managed disk.

+To complete these tasks your account should be assigned the Virtual Machine Contributor built-in role. In addition, to manage Site Recovery operations in a vault, your account should be assigned the Site Recovery Contributor built-in role.

+## VMware account permissions

+**Task** | **Role/Permissions** | **Details**

+ | |

+**VM discovery** | At least a read-only user<br/><br/> Data Center object ΓÇô> Propagate to Child Object, role=Read-only | User assigned at datacenter level, and has access to all the objects in the datacenter.<br/><br/> To restrict access, assign the **No access** role with the **Propagate to child** object, to the child objects (vSphere hosts, datastores, VMs and networks).

+**Full replication, failover, failback** | Create a role (Azure_Site_Recovery) with the required permissions, and then assign the role to a VMware user or group<br/><br/> Data Center object ΓÇô> Propagate to Child Object, role=Azure_Site_Recovery<br/><br/> Datastore -> Allocate space, browse datastore, low-level file operations, remove file, update virtual machine files<br/><br/> Network -> Network assign<br/><br/> Resource -> Assign VM to resource pool, migrate powered off VM, migrate powered on VM<br/><br/> Tasks -> Create task, update task<br/><br/> Virtual machine -> Configuration<br/><br/> Virtual machine -> Interact -> answer question, device connection, configure CD media, configure floppy media, power off, power on, VMware tools install<br/><br/> Virtual machine -> Inventory -> Create, register, unregister<br/><br/> Virtual machine -> Provisioning -> Allow virtual machine download, allow virtual machine files upload<br/><br/> Virtual machine -> Snapshots -> Remove snapshots, Create snapshots | User assigned at datacenter level, and has access to all the objects in the datacenter.<br/><br/> To restrict access, assign the **No access** role with the **Propagate to child** object, to the child objects (vSphere hosts, datastores, VMs and networks).

+## Delete an unhealthy appliance

+[Rollup 74](https://support.microsoft.com/topic/update-rollup-74-for-azure-site-recovery-584e3586-4c55-4cc2-8b1c-63038b6b4464) | 9.62.7096.1 | 9.62.7096.1 | 9.62.7096.1 | 5.24.0614.1 | 2.0.9919.0

+## Updates (July 2024)

+### Update Rollup 74

+[Update rollup 74](https://support.microsoft.com/topic/update-rollup-74-for-azure-site-recovery-584e3586-4c55-4cc2-8b1c-63038b6b4464) provides the following updates:

+**Update** | **Details**

+ |

+**Providers and agents** | Updates to Site Recovery agents and providers as detailed in the rollup KB article.

+**Issue fixes/improvements** | Many fixes and improvement as detailed in the rollup KB article.

+**Azure VM disaster recovery** | Added support for Debian 11, SLES 12, SLES 15, and RHEL 9 Linux distros. <br><br/> Added capacity reservation support for Virtual Machine Scale Sets Flex machines protected using Site Recovery.

+**VMware VM/physical disaster recovery to Azure** | Added support for Debian 11, SLES 12, SLES 15, and RHEL 9 Linux distros. <br><br/> Added capacity reservation support for Virtual Machine Scale Sets Flex machines protected using Site Recovery. <br><br/> Added support to enable replication for newly added data disks that are added to a VMware virtual machine, which already has disaster recovery enabled. [Learn more](./vmware-azure-enable-replication-added-disk.md)

+[Update rollup 73](https://support.microsoft.com/topic/update-rollup-73-for-azure-site-recovery-d3845f1e-2454-4ae8-b058-c1fec6206698) provides the following updates:

+> [!CAUTION]

+> * Effective July 23, 2024, **disablement** of jobs running on Azure Synapse Runtime for Apache Spark 2.4 will be executed. **Immediately** migrate to higher runtime versions otherwise your jobs will stop executing.

+> * **All Spark jobs running on Azure Synapse Runtime for Apache Spark 2.4 will be disabled as of July 23, 2024.**

+> * End of Support announced for Azure Synapse Runtime for Apache Spark 3.2 July 8, 2023.

+> * Effective July 8, 2024, Azure Synapse will discontinue official support for Spark 3.2 Runtimes.

+> * In accordance with the Synapse runtime for Apache Spark lifecycle policy, Azure Synapse runtime for Apache Spark 3.2 will be retired as of July 8, 2024. Existing workflows will continue to run but security updates and bug fixes will no longer be available. Metadata will temporarily remain in the Synapse workspace.

+> * **We strongly recommend that you upgrade your Apache Spark 3.2 workloads to [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md) before July 8, 2024.**

+There are several types of encryption available for your managed disks, including Azure Disk Encryption (ADE), Server-Side Encryption (SSE), and encryption at host.

+- **Confidential disk encryption** binds disk encryption keys to the virtual machine's TPM and makes the protected disk content accessible only to the VM. The TPM and VM guest state is always encrypted in attested code using keys released by a secure protocol that bypasses the hypervisor and host operating system. Currently only available for the OS disk; [temp disk support is in preview](https://techcommunity.microsoft.com/t5/azure-confidential-computing/confidential-temp-disk-encryption-for-confidential-vms-in-public/ba-p/3971393). Encryption at host may be used for other disks on a Confidential VM in addition to Confidential Disk Encryption. For full details, see [DCasv5 and ECasv5 series confidential VMs](../confidential-computing/confidential-vm-overview.md#confidential-os-disk-encryption).

+| Temp disk encryption | &#10060; | &#x2705; Only supported with platform managed key | &#x2705; | &#x2705; [In Preview](https://techcommunity.microsoft.com/t5/azure-confidential-computing/confidential-temp-disk-encryption-for-confidential-vms-in-public/ba-p/3971393)|

+* [Virtual machines and virtual machine scale sets should have encryption at host enabled](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Ffc4d8e41-e223-45ea-9bf5-eada37891d87) (Only detects Encryption at Host)

+* [Windows virtual machines should enable Azure Disk Encryption or EncryptionAtHost](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2fproviders%2fMicrosoft.Authorization%2fpolicyDefinitions%2f3dc5edcd-002d-444c-b216-e123bbfa37c0) (Detects both Azure Disk Encryption and EncryptionAtHost)

+* [Linux virtual machines should enable Azure Disk Encryption or EncryptionAtHost](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2fproviders%2fMicrosoft.Authorization%2fpolicyDefinitions%2fca88aadc-6e2b-416c-9de2-5a0f01d1693f) (Detects both Azure Disk Encryption and EncryptionAtHost)

+ Title: Dlsv6 and Dldsv6-series

+description: Specifications for the Dlsv6 and Dldsv6-series VMs

+# Dlsv6 and Dldsv6-series (Preview)

+Applies to ✔️ Linux VMs ✔️ Windows VMs ✔️ Flexible scale sets ✔️ Uniform scale sets

+> [!NOTE]

+> Azure Virtual Machine Series Dsv6 and Ddsv6 are currently in **Preview**. See the [Preview Terms Of Use | Microsoft Azure](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+The Dlsv6 and Dldsv6-series Virtual Machines runs on Intel® Xeon® Platinum 8473C (Emerald Rapids) processor in a [hyper threaded](https://www.intel.com/content/www/us/en/architecture-and-technology/hyper-threading/hyper-threading-technology.html) configuration. This new processor features an all core turbo clock speed of 3.0 GHz with [Intel® Turbo Boost Technology](https://www.intel.com/content/www/us/en/architecture-and-technology/turbo-boost/turbo-boost-technology.html), [Intel® Advanced-Vector Extensions 512 (Intel® AVX-512)](https://www.intel.com/content/www/us/en/architecture-and-technology/avx-512-overview.html) and [Intel® Deep Learning Boost](https://software.intel.com/content/www/us/en/develop/topics/ai/deep-learning-boost.html). The Dlsv6 and Dldsv6 VM series provides 2GiBs of RAM per vCPU and optimized for workloads that require less RAM per vCPU than standard VM sizes. Target workloads include web servers, gaming, video encoding, AI/ML, and batch processing.

+These new Intel based VMs have two variants: Dlsv6 without local SSD and Dldsv6 with local SSD.

+## Dlsv6-series

+Dlsv6-series virtual machines run on 5^th Generation Intel® Xeon® Platinum 8473C (Emerald Rapids) CPU processor reaching an all- core turbo clock speed of up to 3.0 GHz. These virtual machines offer up to 128 vCPU and 256 GiB of RAM. These VM sizes can reduce cost when running non-memory intensive applications.

+Dlsv6-series virtual machines do not have any temporary storage thus lowering the price of entry. You can attach Standard SSD, Standard HDD, and Premium SSD disk types. You can also attach Ultra Disk storage based on its regional availability. Disk storage is billed separately from virtual machines. [See pricing for disks](https://azure.microsoft.com/pricing/details/managed-disks/).

+[Premium Storage](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported <br>[Premium Storage caching](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported <br>[Live Migration](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Not Supported for Preview <br>[Memory Preserving Updates](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported <br>[VM Generation Support](https://learn.microsoft.com/azure/virtual-machines/generation-2): Generation 2<br>[Accelerated Networking](https://learn.microsoft.com/azure/virtual-network/create-vm-accelerated-networking-cli): Supported <br>[Ephemeral OS Disks](https://learn.microsoft.com/azure/virtual-machines/ephemeral-os-disks): Not Supported for Preview<br>[Nested Virtualization](https://learn.microsoft.com/virtualization/hyper-v-on-windows/user-guide/nested-virtualization): Supported

+| **Size** | **vCPU** | **Memory: GiB** | **Temp storage (SSD) GiB** | **Max data disks** | **Max temp storage throughput: IOPS/MBPS (RR)** | **Max temp storage throughput: IOPS/MBPS (RW)** | **Max** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max burst** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max burst** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max NICs** | **Network bandwidth (Mbps)** |

+||||||||||||||

+| **Standard_D2ls_v6** | 2 | 4 | 0 | 8 | NA | NA | 3750/106 | 40000/1250 | 4167/124 | 44444/1463 | 2 | 12500 |

+| **Standard_D4ls_v6** | 4 | 8 | 0 | 12 | NA | NA | 6400/212 | 40000/1250 | 8333/248 | 52083/1463 | 2 | 12500 |

+| **Standard_D8ls_v6** | 8 | 16 | 0 | 24 | NA | NA | 12800/424 | 40000/1250 | 16667/496 | 52083/1463 | 4 | 12500 |

+| **Standard_D16ls_v6** | 16 | 32 | 0 | 48 | NA | NA | 25600/848 | 40000/1250 | 33333/992 | 52083/1463 | 8 | 12500 |

+| **Standard_D32ls_v6** | 32 | 64 | 0 | 64 | NA | NA | 51200/1696 | 80000/1696 | 66667/1984 | 104167/1984 | 8 | 16000 |

+| **Standard_D48ls_v6** | 48 | 96 | 0 | 64 | NA | NA | 76800/2544 | 80000/2544 | 100000/2976 | 104167/2976 | 8 | 24000 |

+| **Standard_D64ls_v6** | 64 | 128 | 0 | 64 | NA | NA | 102400/3392 | 102400/3392 | 133333/3969 | 133333/3969 | 8 | 30000 |

+| **Standard_D96ls_v6** | 96 | 192 | 0 | 64 | NA | NA | 153600/5088 | 153600/5088 | 200000/5953 | 200000/5953 | 8 | 41000 |

+| **Standard_D128ls_v6** | 128 | 256 | 0 | 64 | NA | NA | 204800/6782 | 204800/6782 | 266667/7935 | 266667/7935 | 8 | 54000 |

+## Dldsv6-series

+Dldsv6-series virtual machines run on the 5th Generation Intel® Xeon® Platinum 8473C (Emerald Rapids) processor reaching an all- core turbo clock speed of up to 3.0 GHz. These virtual machines offer up to 128 vCPU and 256 GiB of RAM as well as fast, local SSD storage up to 4x1760 GiB. These VM sizes can reduce cost when running non-memory intensive applications.

+Dldsv5-series virtual machines support Standard SSD, Standard HDD, and Premium SSD disk types. You can also attach Ultra Disk storage based on its regional availability. Disk storage is billed separately from virtual machines. [See pricing for disks](https://azure.microsoft.com/pricing/details/managed-disks/).

+[Premium Storage](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported <br>[Premium Storage caching](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported <br>[Live Migration](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Not Supported for Preview <br>[Memory Preserving Updates](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported <br>[VM Generation Support](https://learn.microsoft.com/azure/virtual-machines/generation-2): Generation 2<br>[Accelerated Networking](https://learn.microsoft.com/azure/virtual-network/create-vm-accelerated-networking-cli): Supported <br>[Ephemeral OS Disks](https://learn.microsoft.com/azure/virtual-machines/ephemeral-os-disks): Not Supported for Preview <br>[Nested Virtualization](https://learn.microsoft.com/virtualization/hyper-v-on-windows/user-guide/nested-virtualization): Supported

+| **Size** | **vCPU** | **Memory: GiB** | **Temp storage (SSD) GiB** | **Max data disks** | **Max temp storage throughput: IOPS/MBPS (RR)** | **Max temp storage throughput: IOPS/MBPS (RW)** | **Max** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max burst** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max burst** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max NICs** | **Network bandwidth** |

+||||||||||||||

+| **Standard_D2lds_v6** | 2 | 4 | 1x110 | 8 | 37500/180 | 15000/90 | 3750/106 | 40000/1250 | 4167/124 | 44444/1463 | 2 | 12500 |

+| **Standard_D4lds_v6** | 4 | 8 | 1x220 | 12 | 75000/360 | 30000/180 | 6400/212 | 40000/1250 | 8333/248 | 52083/1463 | 2 | 12500 |

+| **Standard_D8lds_v6** | 8 | 16 | 1x440 | 24 | 150000/720 | 60000/360 | 12800/424 | 40000/1250 | 16667/496 | 52083/1463 | 4 | 12500 |

+| **Standard_D16lds_v6** | 16 | 32 | 2x440 | 48 | 300000/1440 | 120000/720 | 25600/848 | 40000/1250 | 33333/992 | 52083/1463 | 8 | 12500 |

+| **Standard_D32lds_v6** | 32 | 64 | 4x440 | 64 | 600000/2880 | 240000/1440 | 51200/1696 | 80000/1696 | 66667/1984 | 104167/1984 | 8 | 16000 |

+| **Standard_D48lds_v6** | 48 | 96 | 6x440 | 64 | 900000/4320 | 360000/2160 | 76800/2544 | 80000/2544 | 100000/2976 | 104167/2976 | 8 | 24000 |

+| **Standard_D64lds_v6** | 64 | 128 | 4x880 | 64 | 1200000/5760 | 480000/2880 | 102400/3392 | 102400/3392 | 133333/3969 | 133333/3969 | 8 | 30000 |

+| **Standard_D96lds_v6** | 96 | 192 | 6x880 | 64 | 1800000/8640 | 720000/4320 | 153600/5088 | 153600/5088 | 200000/5953 | 200000/5953 | 8 | 41000 |

+| **Standard_D128lds_v6** | 128 | 256 | 4x1760 | 64 | 2400000/11520 | 960000/5760 | 204800/6782 | 204800/6782 | 266667/7935 | 266667/7935 | 8 | 54000 |

+## Size table definitions

+Storage capacity is shown in units of GiB or 1024^3 bytes. When you compare disks measured in GB (1000^3 bytes) to disks measured in GiB (1024^3) remember that capacity numbers given in GiB may appear smaller. For example, 1023 GiB = 1098.4 GB.

+Disk throughput is measured in input/output operations per second (IOPS) and MBps where MBps = 10^6 bytes/sec.

+Data disks can operate in cached or uncached modes. For cached data disk operation, the host cache mode is set to **ReadOnly** or **ReadWrite**. For uncached data disk operation, the host cache mode is set to **None**.

+To learn how to get the best storage performance for your VMs, see [Virtual machine and disk performance](https://learn.microsoft.com/azure/virtual-machines/disks-performance).

+**Expected network bandwidth** is the maximum aggregated bandwidth allocated per VM type across all NICs, for all destinations. For more information, see [Virtual machine network bandwidth](../virtual-network/virtual-machine-network-throughput.md).

+Upper limits aren't guaranteed. Limits offer guidance for selecting the right VM type for the intended application. Actual network performance will depend on several factors including network congestion, application loads, and network settings. For information on optimizing network throughput, see [Optimize network throughput for Azure virtual machines](https://learn.microsoft.com/azure/virtual-network/virtual-network-optimize-network-bandwidth). To achieve the expected network performance on Linux or Windows, you may need to select a specific version or optimize your VM. For more information, see [Bandwidth/Throughput testing (NTTTCP)](https://learn.microsoft.com/azure/virtual-network/virtual-network-bandwidth-testing)

+ Title: Esv6 and Edsv6-series

+description: Specifications for Esv6 and Edsv6-series

+# Esv6-series and Edsv6-series (Preview)

+Applies to ✔️ Linux VMs ✔️ Windows VMs ✔️ Flexible scale sets ✔️ Uniform scale sets

+>[!NOTE]

+>Azure Virtual Machine Series Esv6 and Edsv6 are currently in **Preview**. See the [Preview Terms Of Use | Microsoft Azure](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+The Esv6 and Edsv6-series Virtual Machine (VM) run on the 5th Generation Intel® Xeon® Platinum 8473C (Emerald Rapids) processor in a [hyper threaded](https://www.intel.com/content/www/us/en/architecture-and-technology/hyper-threading/hyper-threading-technology.html) configuration, providing a better value proposition for most general-purpose workloads. This new processor features an all- core turbo clock speed of 3.0 GHz with [Intel® Turbo Boost Technology](https://www.intel.com/content/www/us/en/architecture-and-technology/turbo-boost/turbo-boost-technology.html), [Intel® Advanced-Vector Extensions 512 (Intel® AVX-512)](https://www.intel.com/content/www/us/en/architecture-and-technology/avx-512-overview.html) and [Intel® Deep Learning Boost](https://software.intel.com/content/www/us/en/develop/topics/ai/deep-learning-boost.html). The Esv6 and Edsv6-series feature up to 1024 GiB of RAM. These virtual machines are ideal for memory-intensive enterprise applications, relational database servers, and in-memory analytics workloads.

+These new Intel based VMs have two variants: Esv6 without local SSD and Edsv6 with local SSD.

+## Esv6-series

+Esv6-series virtual machines run on the 5th Generation Intel® Xeon® Platinum 8473C (Emerald Rapids) processor reaching an all- core turbo clock speed of up to 3.0 GHz. These virtual machines offer up to 128 vCPU and 1024 GiB of RAM. Esv6-series virtual machines are ideal for memory-intensive enterprise applications and applications that benefit from low latency, high-speed local storage.

+[Premium Storage](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Not Supported<br>[Premium Storage caching](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Not Supported<br>[Live Migration](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported<br>[Memory Preserving Updates](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported<br>[VM Generation Support](https://learn.microsoft.com/azure/virtual-machines/generation-2): Generation 2<br>[Accelerated Networking](https://learn.microsoft.com/azure/virtual-network/create-vm-accelerated-networking-cli)¹: Required<br>[Ephemeral OS Disks](https://learn.microsoft.com/azure/virtual-machines/ephemeral-os-disks): Not Supported for Preview<br>[Nested Virtualization](https://learn.microsoft.com/virtualization/hyper-v-on-windows/user-guide/nested-virtualization): Supported

+| **Size** | **vCPU** | **Memory: GiB** | **Temp storage (SSD) GiB** | **Max data disks** | **Max temp storage throughput: IOPS/MBPS (RR)** | **Max temp storage throughput: IOPS/MBPS (RW)** | **Max** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max burst** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max burst** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max NICs** | **Network bandwidth** |

+||||||||||||||

+| **Standard_E2s_v6** | 2 | 16 | 0 | 8 | NA | NA | 3750/106 | 40000/1250 | 4167/124 | 44444/1463 | 2 | 12500 |

+| **Standard_E4s_v6** | 4 | 32 | 0 | 12 | NA | NA | 6400/212 | 40000/1250 | 8333/248 | 52083/1463 | 2 | 12500 |

+| **Standard_E8s_v6** | 8 | 64 | 0 | 24 | NA | NA | 12800/424 | 40000/1250 | 16667/496 | 52083/1463 | 4 | 12500 |

+| **Standard_E16s_v6** | 16 | 128 | 0 | 48 | NA | NA | 25600/848 | 40000/1250 | 33333/992 | 52083/1463 | 8 | 12500 |

+| **Standard_E20s_v6** | 20 | 160 | 0 | 48 | NA | NA | 32000/1060 | 64000/1600 | 41667/1240 | 83333/1872 | 8 | 12500 |

+| **Standard_E32s_v6** | 32 | 256 | 0 | 64 | NA | NA | 51200/1696 | 80000/1696 | 66667/1984 | 104167/1984 | 8 | 16000 |

+| **Standard_E48s_v6** | 48 | 384 | 0 | 64 | NA | NA | 76800/2544 | 80000/2544 | 100000/2976 | 104167/2976 | 8 | 24000 |

+| **Standard_E64s_v6** | 64 | 512 | 0 | 64 | NA | NA | 102400/3392 | 102400/3392 | 133333/3969 | 133333/3969 | 8 | 30000 |

+| **Standard_E96s_v6** | 96 | 768 | 0 | 64 | NA | NA | 153600/5088 | 153600/5088 | 200000/5953 | 200000/5953 | 8 | 41000 |

+| **Standard_E128s_v6** | 128 | 1024 | 0 | 64 | NA | NA | 204800/6782 | 204800/6782 | 266667/7935 | 266667/7935 | 8 | 54000 |

+## Edsv6-series

+Edsv6-series virtual machines run on the 5th Generation Intel® Xeon® Platinum 8473C (Emerald Rapids) processor reaching an all core turbo clock speed of up to 3.0 GHz. These virtual machines offer up to 192 vCPU and 1832 GiB of RAM and fast, local SSD storage up to 6x1760 GiB. Edsv6-series virtual machines are ideal for memory-intensive enterprise applications and applications that benefit from low latency, high-speed local storage.

+Edsv6-series virtual machines support Standard SSD and Standard HDD disk types. To use Premium SSD or Ultra Disk storage, select Edsv6-series virtual machines. Disk storage is billed separately from virtual machines. [See pricing for disks](https://azure.microsoft.com/pricing/details/managed-disks/).

+[Premium Storage](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported<br>[Premium Storage caching](https://learn.microsoft.com/azure/virtual-machines/premium-storage-performance): Supported<br>[Live Migration](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported<br>[Memory Preserving Updates](https://learn.microsoft.com/azure/virtual-machines/maintenance-and-updates): Supported<br>[VM Generation Support](https://learn.microsoft.com/azure/virtual-machines/generation-2): Generation 2<br>[Accelerated Networking](https://learn.microsoft.com/azure/virtual-network/create-vm-accelerated-networking-cli)¹: Required<br>[Ephemeral OS Disks](https://learn.microsoft.com/azure/virtual-machines/ephemeral-os-disks): Not Supported for Preview<br>[Nested Virtualization](https://learn.microsoft.com/virtualization/hyper-v-on-windows/user-guide/nested-virtualization): Supported

+| **Size** | **vCPU** | **Memory: GiB** | **Temp storage (SSD) GiB** | **Max data disks** | **Max temp storage throughput: IOPS/MBPS (RR)** | **Max temp storage throughput: IOPS/MBPS (RW)** | **Max** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max burst** **uncached Premium SSD and Standard SSD/HDD disk throughput: IOPS/MBps** | **Max** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max burst** **uncached Ultra Disk and Premium SSD V2 disk throughput: IOPS/MBps** | **Max NICs** | **Network bandwidth** |

+||||||||||||||

+| **Standard_E2ds_v6** | 2 | 16 | 1x110 | 8 | 37500/180 | 15000/90 | 3750/106 | 40000/1250 | 4167/124 | 44444/1463 | 2 | 12500 |

+| **Standard_E4ds_v6** | 4 | 32 | 1x220 | 12 | 75000/360 | 30000/180 | 6400/212 | 40000/1250 | 8333/248 | 52083/1463 | 2 | 12500 |

+| **Standard_E8ds_v6** | 8 | 64 | 1x440 | 24 | 150000/720 | 60000/360 | 12800/424 | 40000/1250 | 16667/496 | 52083/1463 | 4 | 12500 |

+| **Standard_E16ds_v6** | 16 | 128 | 2x440 | 48 | 300000/1440 | 120000/720 | 25600/848 | 40000/1250 | 33333/992 | 52083/1463 | 8 | 12500 |

+| **Standard_E20ds_v6** | 20 | 160 | 2x550 | 48 | 375000/1800 | 150000/900 | 32000/1060 | 64000/1600 | 41667/1240 | 83333/1872 | 8 | 12500 |

+| **Standard_E32ds_v6** | 32 | 256 | 4x440 | 64 | 600000/2880 | 240000/1440 | 51200/1696 | 80000/1696 | 66667/1984 | 104167/1984 | 8 | 16000 |

+| **Standard_E48ds_v6** | 48 | 384 | 6x440 | 64 | 900000/4320 | 360000/2160 | 76800/2544 | 80000/2544 | 100000/2976 | 104167/2976 | 8 | 24000 |

+| **Standard_E64ds_v6** | 64 | 512 | 4x880 | 64 | 1200000/5760 | 480000/2880 | 102400/3392 | 102400/3392 | 133333/3969 | 133333/3969 | 8 | 30000 |

+| **Standard_E96ds_v6** | 96 | 768 | 6x880 | 64 | 1800000/8640 | 720000/4320 | 153600/5088 | 153600/5088 | 200000/5953 | 200000/5953 | 8 | 41000 |

+| **Standard_E128ds_v6** | 128 | 1024 | 4x1760 | 64 | 2400000/11520 | 960000/5760 | 204800/6782 | 204800/6782 | 266667/7935 | 266667/7935 | 8 | 54000 |

+## Size table definitions

+Storage capacity is shown in units of GiB or 1024^3 bytes. When you compare disks measured in GB (1000^3 bytes) to disks measured in GiB (1024^3) remember that capacity numbers given in GiB may appear smaller. For example, 1023 GiB = 1098.4 GB.

+Disk throughput is measured in input/output operations per second (IOPS) and MBps where MBps = 10^6 bytes/sec.

+Data disks can operate in cached or uncached modes. For cached data disk operation, the host cache mode is set to **ReadOnly** or **ReadWrite**. For uncached data disk operation, the host cache mode is set to **None**.

+To learn how to get the best storage performance for your VMs, see [Virtual machine and disk performance](https://learn.microsoft.com/azure/virtual-machines/disks-performance).

+**Expected network bandwidth** is the maximum aggregated bandwidth allocated per VM type across all NICs, for all destinations. For more information, see [Virtual machine network bandwidth](https://learn.microsoft.com/azure/virtual-network/virtual-machine-network-throughput).

+Upper limits aren't guaranteed. Limits offer guidance for selecting the right VM type for the intended application. Actual network performance will depend on several factors including network congestion, application loads, and network settings. For information on optimizing network throughput, see [Optimize network throughput for Azure virtual machines](https://learn.microsoft.com/azure/virtual-network/virtual-network-optimize-network-bandwidth). To achieve the expected network performance on Linux or Windows, you may need to select a specific version or optimize your VM. For more information, see [Bandwidth/Throughput testing (NTTTCP)](https://learn.microsoft.com/azure/virtual-network/virtual-network-bandwidth-testing).

+- A Java Development Kit (JDK), version 17. In this guide, we recommend the [Red Hat Build of OpenJDK](https://developers.redhat.com/products/openjdk/download). Ensure that your `JAVA_HOME` environment variable is set correctly in the shells in which you run the commands.

+Ethernet Microsoft Hyper-V Network Adapter 13 Up 00-0D-3A-AA-00-AA 200 Gbps

+Ethernet 3 Microsoft Azure Network Adapter #2 8 Up 00-0D-3A-AA-00-AA 200 Gbps