+ language jsonresp [documents]{0}[detectedLanguage] [Name] in language

+Requires Android 7.0 (API level 24) or higher on Arm64 (`arm64-v8a`) or Arm32 (`armeabi-v7a`) hardware.

+Embedded TTS with neural voices is only supported on Arm64.

+Requires Linux on x64, Arm64, or Arm32 hardware with [supported Linux distributions](quickstarts/setup-platform.md?tabs=linux).

+Embedded TTS with neural voices isn't supported on Arm32.

+Requires 10.14 or newer on x64 or Arm64 hardware.

+Requires Windows 10 or newer on x64 or Arm64 hardware.

+The Speech SDK for Java doesn't support Windows on Arm64.

+ AKV_SCOPE=$(az keyvault show --name <AZURE_AKV_RESOURCE_NAME> --query id --output tsv)

+ kubectl apply -f consumer.yaml

+ kubectl apply -f producer.yaml

+[az-attestation-show]: /cli/azure/attestation#az-attestation-show

+> For more information on configuring and using premium cache features, see [How to configure persistence](cache-how-to-premium-persistence.md) and [How to configure Virtual Network support](cache-how-to-premium-vnet.md).

+ - All premium caches with Redis cluster enabled - Redis cluster only supports use of database 0 so the `databases` limit for any premium cache with Redis cluster enabled is effectively 1 and the [Select](https://redis.io/commands/select) command isn't allowed.

+Each metric includes two versions: One metric measures performance for the entire cache, and for caches that use clustering. A second version of the metric, which includes `(Shard 0-9)` in the name, measures performance for a single shard in a cache. For example if a cache has four shards, `Cache Hits` is the total number of hits for the entire cache, and `Cache Hits (Shard 3)` measures just the hits for that shard of the cache.

+ :::image type="content" source="media/cache-how-to-scale/redis-cache-clustering.png" alt-text="Screenshot showing the clustering toggle.":::

+ :::image type="content" source="media/cache-how-to-scale/redis-cache-clustering-selected.png" alt-text="Screenshot showing the clustering toggle selected.":::

+- If you enabled clustering when you created your **Premium** cache, you can change the cluster size. If your cache was created without clustering enabled, you can configure clustering at a later time.

+- You can't change the number of shards if the cache has clustering enabled.

+Azure Cache for Redis supports OSS Redis version 4.0.x and 6.0.x. We've made the decision to skip Redis 5.0 to bring you the latest version. Previously, Azure Cache for Redis maintained a single Redis version. In the future, you can choose from a newer major release upgrade and at least one older stable version. You can choose the version that works the best for your application.

+| OSS clustering |-|-|Γ£ö|Γ£ö|Γ£ö|

+- **Memory**: The Basic and Standard tiers offer 250 MB ΓÇô 53 GB; the Premium tier 6 GB - 1.2 TB; the Enterprise tier 4 GB - 2 TB, and the Enterprise Flash tier 300 GB - 4.5 TB. To create a Premium tier cache larger than 120 GB, you can use Redis OSS clustering. For more information, see [Azure Cache for Redis Pricing](https://azure.microsoft.com/pricing/details/cache/).

+Get started with Azure Cache for Redis 6.0, today, and select Redis 6.0 during cache creation. Also, you can upgrade your existing Redis 4.0 cache instances.

+## Methods recognized as functions

+A function method is a public method of a public class with a `Function` attribute applied to the method and a trigger attribute applied to an input parameter, as shown in the following example:

+The `Function` attribute marks the method as a function entry point. The name must be unique within a project, start with a letter and only contain letters, numbers, `_`, and `-`, up to 127 characters in length. Project templates often create a method named `Run`, but the method name can be any valid C# method name. The method must be a public member of a public class. It should generally be an instance method so that services can be passed in via [dependency injection](#dependency-injection).

+## Execution context

+.NET isolated passes a [FunctionContext] object to your function methods. This object lets you get an [`ILogger`][ILogger] instance to write to the logs by calling the [GetLogger] method and supplying a `categoryName` string. To learn more, see [Logging](#logging).

+## Bindings

+Bindings are defined by using attributes on methods, parameters, and return types. Bindings can provide data as strings, arrays, and serializable types, such as plain old class objects (POCOs). You can also bind to [types from some service SDKs](#sdk-types). For HTTP triggers, see the [HTTP trigger](#http-trigger) section below.

+ Title: Configure Azure AI Health Insights containers

+description: Azure AI Health Insights containers use a common configuration framework, so that you can easily configure and manage storage, logging and telemetry, and security settings for your containers.

+# Configure Azure AI Health Insights docker containers

+Azure AI Health Insights service provides each container with a common configuration framework, so that you can easily configure and manage storage, logging and telemetry, and security settings for your containers. Several [example docker run commands](use-containers.md#run-the-container-with-docker-run) are also available.

+The `ApplicationInsights` setting allows you to add [Azure Application Insights](/azure/application-insights) telemetry support to your container. Application Insights service provides in-depth monitoring of your container. You can easily monitor your container for availability, performance, and usage. You can also quickly identify and diagnose errors in your container.

+Azure AI Health Insights containers are licensed under [your agreement](https://go.microsoft.com/fwlink/?linkid=2018657) governing your use of Azure. If you don't have an existing agreement governing your use of Azure, you agree that your agreement use of Azure is the [Microsoft Online Subscription Agreement](https://go.microsoft.com/fwlink/?linkid=2018755), which incorporates the [Online Services Terms](https://go.microsoft.com/fwlink/?linkid=2018760). For previews, you also agree to the [Supplemental Terms of Use for Microsoft Azure Previews](https://go.microsoft.com/fwlink/?linkid=2018815). By using the container, you agree to these terms.

+ Title: Deploy Azure AI Health Insights using the Azure portal

+description: This article describes how to deploy Azure AI Health Insights in the Azure portal.

+# Quickstart: Deploy Azure AI Health Insights using the Azure portal

+In this quickstart, you learn how to deploy Azure AI Health Insights using the Azure portal.

+Once deployment is complete, you can use the Azure portal to navigate to the newly created Azure AI Health Insights, and retrieve the needed details such your service URL, keys and manage your access controls.

+## Deploy Azure AI Health Insights

+ 

+To get started using Azure AI Health Insights, get started with one of the following models:

+> [Onco-Phenotype](oncophenotype/index.yml)

+ Title: Onco-Phenotype frequently asked questions

+description: Onco-Phenotype frequently asked questions

+# Onco-Phenotype Frequently Asked Questions

+ It is populated on ICD-O-3 SEER Site/Histology Validation List [here](https://seer.cancer.gov/icd-o-3/).

+ `N+` means there's involvement of regional lymph nodes without explicitly mentioning the extent of spread. Microsoft trained the models to classify whether or not there's regional lymph node involvement but not the extent of spread and hence `N1, N2, N3` inference values aren't supported.

+ No, subcategories or isolated tumor cell modifiers aren't supported. For instance, 'T3 a' would be predicted as T3, and N0(i+) would be predicted as N0.

+ Title: Use Onco-Phenotype

+description: This article describes how to use the Onco-Phenotype

+# Quickstart: Use the Onco-Phenotype model

+This quickstart provides an overview on how to use the Onco-Phenotype.

+To use the Onco-Phenotype model, you must have an Azure AI services account created. If you haven't already created an Azure AI services account, see [Deploy Azure AI Health Insights using the Azure portal.](../deploy-portal.md)

+

+You can use the data from this example, to test your first request to the Onco-Phenotype model.

+You get the status of the job by sending a request to the Onco-Phenotype model and adding the job ID from the initial request in the URL, as seen in the code snippet:

+Every request has required and optional fields that should be provided to the Onco-Phenotype model.

+ Title: Onco-Phenotype inference information

+description: This article provides Onco-Phenotype inference information.

+# Onco-Phenotype inference information

+Azure AI Health Insights Onco-Phenotype model was trained with labels that conform to the following standards.

+ Title: Onco-Phenotype model configuration

+description: This article provides Onco-Phenotype model configuration information.

+# Onco-Phenotype model configuration

+To interact with the Onco-Phenotype model, you can provide several model configurations parameters that modify the outcome of the responses.

+The Onco-Phenotype model configuration helps you find if any cancer cases exist. The API allows you to explicitly check if a cancer case exists in the provided clinical documents.

+ Title: What is Onco-Phenotype (Preview)

+# What is Onco-Phenotype (Preview)?

+Onco-Phenotype is an AI model thatΓÇÖs offered within the context of the broader Azure AI Health Insights. It augments traditional clinical natural language processing tools by enabling healthcare organizations to rapidly identify key cancer attributes within their patient populations.

+> The Onco-Phenotype model is a capability provided ΓÇ£AS ISΓÇ¥ and ΓÇ£WITH ALL FAULTS.ΓÇ¥ The Onco-Phenotype model isn't intended or made available for use as a medical device, clinical support, diagnostic tool, or other technology intended to be used in the diagnosis, cure, mitigation, treatment, or prevention of disease or other conditions, and no license or right is granted by Microsoft to use this capability for such purposes. This capability isn't designed or intended to be implemented or deployed as a substitute for professional medical advice or healthcare opinion, diagnosis, treatment, or the clinical judgment of a healthcare professional, and should not be used as such. The customer is solely responsible for any use of the Onco-Phenotype model. The customer is responsible for ensuring compliance with those license terms, including any geographic or other applicable restrictions.

+## Onco-Phenotype features

+The Onco-Phenotype model, available in the Azure AI Health Insights cognitive service as an API, augments traditional clinical natural language processing (NLP) tools by helping healthcare providers rapidly identify key attributes of a cancer within their patient populations with an existing cancer diagnosis. You can use this model to infer tumor site; histology; clinical stage tumor (T), node (N), and metastasis (M) categories; and pathologic stage TNM categories from unstructured clinical documents, along with confidence scores and relevant evidence.

+Cancer staging describes the severity of an individual's cancer based on the magnitude of the original tumor, as well as on the extent cancer has spread in the body. The Onco-Phenotype model supports inferring two types of staging from the clinical documents - clinical staging and pathologic staging. TheyΓÇÖre both expressed in the form of TNM categories, where TNM indicates the extent of the tumor (T), the extent of spread to the lymph nodes (N), and the presence of metastasis (M).

+The Onco-Phenotype model enables cancer registrars to efficiently abstract cancer patients as it infers the above-mentioned key cancer attributes from unstructured clinical documents along with evidence that are relevant to those attributes. Leveraging this API can reduce the manual time spent combing through large amounts of patient documentation by focusing on the most relevant content in support of a clinician.

+Get started using the Onco-Phenotype model:

+ Title: Onco-Phenotype patient info

+description: This article describes how and which patient information can be sent to the Onco-Phenotype model

+# Onco-Phenotype patient info

+The Onco-Phenotype currently can receive patient information in the form of unstructured clinical notes.

+In this example, the Onco-Phenotype model receives patient information in the form of unstructured clinical notes.

+To get started using the Onco-Phenotype model:

+ Title: Onco-Phenotype support and help options

+description: How to obtain help and support for questions and problems when you create applications that use with Onco-Phenotype model

+# Onco-Phenotype model support and help options

+Are you just starting to explore the functionality of the Onco-Phenotype model? Perhaps you're implementing a new feature in your application. Or after using the service, do you have suggestions on how to improve it? Here are options for where you can get support, stay up-to-date, give feedback, and report bugs for Azure AI Health Insights.

+ Title: Transparency Note for Onco-Phenotype

+description: Transparency Note for Onco-Phenotype

+# Transparency Note for Onco-Phenotype

+## The basics of Onco-Phenotype

+The Onco-Phenotype model, available in the Azure AI Health Insights cognitive service as an API, augments traditional clinical natural language processing (NLP) tools by helping healthcare providers rapidly identify key cancer attributes of a cancer within their patient populations with an existing cencer diagnosis. You can use this model to infer tumor site; histology; clinical stage tumor (T), lymph node (N), and metastasis (M) categories; and pathologic stage TNM categories from unstructured clinical documents, along with confidence scores and relevant evidence.

+The Onco-Phenotype model, available in the Azure AI Health Insights cognitive service as an API, takes in unstructured clinical documents as input and returns inferences for cancer attributes along with confidence scores as output. Through the model configuration as part of the API request, it also allows the user to seek evidence with the inference values and to explicitly check for the existence of a cancer case before generating the inferences for cancer attributes.

+The Onco-Phenotype model can be used in the following scenario. The systemΓÇÖs intended uses include:

+We encourage customers to use the Onco-Phenotype model in their innovative solutions or applications. However, here are some considerations when choosing a use case:

+Specific characteristics and limitations of the Onco-Phenotype model include:

+For each inference, the Onco-Phenotype model returns a confidence score that expresses how confident the model is with the response. Confidence scores range from 0 to 1. The higher the confidence score, the more certain the model is about the inference value it provided. However, the system isn't designed for workflows or scenarios without a human in the loop. Also, inference values can't be consumed without human review, irrespective of the confidence score. You can choose to completely discard an inference value if its confidence score is below a confidence score threshold that best suits the scenario.

+## Evaluation of Onco-Phenotype

+The Onco-Phenotype model was evaluated on a held-out dataset that shares the same characteristics as the training dataset. The training and held-out datasets consist of patients located only in the United States. The patient races include White or Caucasian, Black or African American, Asian, Native Hawaiian or Pacific Islander, American Indian or Alaska native, and Other. During model development and training, a separate development dataset was used for error analysis and model improvement.

+Although the Onco-Phenotype model makes mistakes on the held-out dataset, it was observed that the inferences, and the evidence spans identified by the model are helpful in speeding up manual curation effort.

+The evaluation performance of the Onco-Phenotype model was stratified by race to ensure minimal performance discrepancy between different patient racial groups. The lowest performance by racial group is well within 80% of the highest performance by racial group. When the evaluation performance was stratified by gender, there was no significant difference.

+## Evaluating and integrating Onco-Phenotype for your use

+As Microsoft works to help customers safely develop and deploy solutions that use the Onco-Phenotype model, we offer guidance for considering the AI systems' fairness, reliability & safety, privacy &security, inclusiveness, transparency, and human accountability. These considerations are in line with our commitment to developing responsible AI.

+- **Understand what it can do:** Fully vet and review the capabilities of Onco-Phenotype to understand its capabilities and limitations.

+- **Test with real, diverse data:** Understand how Onco-Phenotype will perform in your scenario by thoroughly testing it by using real-life conditions and data that reflects the diversity in your users, geography, and deployment contexts. Small datasets, synthetic data, and tests that don't reflect your end-to-end scenario are unlikely to sufficiently represent your production performance.

+## Learn more about Onco-Phenotype

+[Overview of Onco-Phenotype](overview.md)

+ Title: What is Azure AI Health Insights (Preview)

+# What is Azure AI Health Insights (Preview)?

+Azure AI Health Insights is a Cognitive Service providing an API that serves insight models, which perform analysis and provide inferences to be used by a human. The models can receive input in different modalities, and return insight inferences including evidence as a result, for key high value scenarios in the health domain

+> Azure AI Health Insights is a capability provided ΓÇ£AS ISΓÇ¥ and ΓÇ£WITH ALL FAULTS.ΓÇ¥ Azure AI Health Insights isn't intended or made available for use as a medical device, clinical support, diagnostic tool, or other technology intended to be used in the diagnosis, cure, mitigation, treatment, or prevention of disease or other conditions, and no license or right is granted by Microsoft to use this capability for such purposes. This capability isn't designed or intended to be implemented or deployed as a substitute for professional medical advice or healthcare opinion, diagnosis, treatment, or the clinical judgment of a healthcare professional, and should not be used as such. The customer is solely responsible for any use of Azure AI Health Insights.

+## Why use Azure AI Health Insights?

+Azure AI Health Insights is a Cognitive Service that provides prebuilt models that assist with solving those business problems.

+There are currently two models available in Azure AI Health Insights:

+The [Onco-Phenotype](./oncophenotype/overview.md) receives clinical records of oncology patients and outputs cancer staging, such as **clinical stage TNM** categories and **pathologic stage TNM categories** as well as **tumor site** and **histology**.

+

+Azure AI Health Insights service receives patient data through multiple input channels. This can be unstructured healthcare data, FHIR resources or specific JSON format data. This in combination with the correct model configuration, such as ```includeEvidence```.

+With these input channels and configuration, the service can run the data through several health insights AI models, such as Trial Matcher or Onco-Phenotype.

+Review the following information to learn how to deploy Azure AI Health Insights and to learn additional information about each of the models:

+> [Deploy Azure AI Health Insights](deploy-portal.md)

+> [Onco-Phenotype](oncophenotype/overview.md)

+ Title: Azure AI Health Insights request info

+description: this article describes the required properties to interact with Azure AI Health Insights

+# Azure AI Health Insights request info

+This page describes the request models and parameters that are used to interact with Azure AI Health Insights service.

+The generic part of Azure AI Health Insights request, common to all models.

+ Title: Azure AI Health Insights response info

+# Azure AI Health Insights response info

+This page describes the response models and parameters that are returned by Azure AI Health Insights service.

+The generic part of Azure AI Health Insights response, common to all models.

+ Title: Data, privacy, and security for Azure AI Health Insights

+description: details regarding how Azure AI Health Insights service processes your data.

+# Data, privacy, and security for Azure AI Health Insights

+This article provides high level details regarding how Azure AI Health Insights service processes data provided by customers. As an important reminder, you're responsible for the implementation of your use case and are required to obtain all necessary permissions or other proprietary rights required to process the data you send to the system. It's your responsibility to comply with all applicable laws and regulations in your jurisdiction.

+Azure AI Health Insights:

+- The input data sent to Azure AI Health Insights is temporarily stored for up to 24 hours and is purged thereafter.

+- Azure AI Health Insights response data is temporarily stored for 24 hours and is purged thereafter.

+YouΓÇÖll find answers to commonly asked questions about Trial Matcher, part of Azure AI Health Insights service, in this article

+To use Trial Matcher, you must have an Azure AI services account created. If you haven't already created an Azure AI services account, see [Deploy Azure AI Health Insights using the Azure portal.](../deploy-portal.md)

+The Trial Matcher is an AI model, offered within the context of the broader Azure AI Health Insights. Trial Matcher is designed to match patients to potentially suitable clinical trials or find a group of potentially eligible patients to a list of clinical trials.

+ Title: How to use Azure AI Health Insights containers

+# Use Azure AI Health Insights containers

+These services enable you to host Azure AI Health Insights API on your own infrastructure. If you have security or data governance requirements that can't be fulfilled by calling Azure AI Health Insights remotely, then on-premises Azure AI Health Insights services might be a good solution.

+You must meet the following prerequisites before using Azure AI Health Insights containers.

+Azure AI Health Insights container images can be found on the `mcr.microsoft.com` container registry syndicate. They reside within the `azure-cognitive-services/health-insights/` repository and can be found by their model name.

+- Use the [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/) command to download this container image from the Microsoft public container registry. You can find the featured tags on the [docker hub clinical matching page](https://hub.docker.com/_/microsoft-azure-cognitive-services-health-insights-clinical-matching) and [docker hub cancer profiling page](https://hub.docker.com/_/microsoft-azure-cognitive-services-health-insights-cancer-profiling)

+- For Clinical Trial Matcher, use the [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/) command to download textanalytics healthcare container image from the Microsoft public container registry. You can find the featured tags on the [docker hub](https://hub.docker.com/_/microsoft-azure-cognitive-services-textanalytics-healthcare)

+There are multiple ways you can install and run Azure AI Health Insights containers.

+- Use the Azure portal to create an Azure AI Health Insights resource, and use Docker to get your container.

+When you use Azure AI Health Insights container, the data contained in your API requests and responses isn't visible to Microsoft, and is not used for training the model applied to your data.

+- Runs Azure AI Health Insights container from the container image

+You can have this container and a different Azure AI Health Insights container running on the HOST together. You also can have multiple containers of the same Azure AI Health Insights container running.

+Azure AI Health Insights containers send billing information to Azure, using a *Language* resource on your Azure account.

+Azure AI Health Insights containers aren't licensed to run without being connected to the metering or billing endpoint. You must enable the containers to communicate billing information with the billing endpoint always. Azure AI Health Insights containers don't send customer data, such as the image or text that's being analyzed, to Microsoft.

+| `ApiKey` | The API key of Azure AI Health Insights resource that's used to track billing information.<br/>The value of this option must be set to an API key for the provisioned resource that's specified in `Billing`. |

+| `Billing` | The endpoint of Azure AI Health Insights resource that's used to track billing information.<br/>The value of this option must be set to the endpoint URI of a provisioned Azure resource.|

+In this article, you learned concepts and workflow for downloading, installing, and running Azure AI Health Insights containers. In summary:

+* Azure AI Health Insights service provides a Linux container for Docker

+* You can use either the REST API or SDK to call operations in Azure AI Health Insights containers by specifying the host URI of the container.

+> Azure AI Health Insights containers are not licensed to run without being connected to Azure for metering. Customers need to enable the containers to communicate billing information with the metering service at all times. Azure AI Health Insights containers do not send customer data (e.g. text that is being analyzed) to Microsoft.

+1. In the **Work Item** type field, select **Incident**.

+ > As of September 2022, we are starting the 3-year process of deprecating support for using ITSM actions to send alerts and events to ServiceNow. For information on the deprecated behavior, see [Use Azure alerts to create a ServiceNow alert or event work item](https://learn.microsoft.com/previous-versions/azure/azure-monitor/alerts/alerts-create-itsm-work-items).

+1. You can configure predefined fields to contain constant values as a part of the payload. Three options can be used as a part of the payload:

+A complete sample is given in [chat-with-auth-passport](https://github.com/Azure/azure-webpubsub/blob/main/sdk/webpubsub-socketio-extension/examples/chat-with-auth-passport).

+>[!IMPORTANT]

+> The wrong order of using middlewares could make the authentication workflow fail. Please follow the order given in our sample, unless you know the mechanism of these middlewares.

+### Installed multiple versions for the same package

+#### Possible error

+Server throws error:

+```

+ const io = await require('socket.io')(server).useAzureSocketIO(wpsOptions);

+ ^

+TypeError: require(...)(...).useAzureSocketIO is not a function

+```

+#### Root cause

+An `socket.io` or `engine.io` package is added to `package.json` under the dependencies field by user, while the SDK package `@azure/web-pubsub-socket.io` specifies a different version internally. For example:

+```json

+"dependencies": {

+ "@azure/web-pubsub-socket.io": "1.0.1-beta.6",

+ "socket.io": "4.6.1"

+},

+```

+After `yarn install`, both of two different versions are installed. You could verify by running `npm list socket.io`.

+This command should show two versions of `socket.io` packages:

+```bash

+demo@0.0.0 G:\demo

+Γö£ΓöÇΓö¼ @azure/web-pubsub-socket.io@1.0.0-beta.6

+Γöé ΓööΓöÇΓöÇ socket.io@4.7.1

+ΓööΓöÇΓöÇ socket.io@4.6.1

+```

+#### Solution

+The solution depends on whether a customized version of `socket.io` or `engine.io` package is necessary for your need or not.

+- Customized version of `socket.io`/`engine.io` package is NOT necessary

+Simply removing `socket.io`/`engine.io` in `package.json` dependencies works. For example:

+```json

+"dependencies": {

+ "@azure/web-pubsub-socket.io": "1.0.1-beta.6",

+},

+```

+- Customized version of `socket.io`/`engine.io` package is necessary

+In this case, `package.json` could be:

+```json

+"dependencies": {

+ "@azure/web-pubsub-socket.io": "1.0.1-beta.6",

+ "socket.io": "4.6.1"

+},

+```

+Then you should run `yarn install --flat`. It installs all the dependencies, but only allow one version for each package. On the first run, it prompts you to choose a single version for each package that is depended on at multiple version ranges.

+For our case, it could prompt you to choose versions of `socket.io`, `enigne.io`, `engine.io-parser` and maybe more. Make sure their versions are matched with each other according to [the native implementation of `socket.io` package](https://github.com/socketio/socket.io/) and [`engine.io` package](https://github.com/socketio/engine.io/).

+The final versions are added to your `package.json`` under a resolutions field.

+```json

+"resolutions": {

+ "package-a": "a.b.c",

+ "package-b": "d.e.f",

+ "package-c": "g.h.i"

+}

+```

+>[!Important]

+>We're temporarily pausing the release of Update Rollup 1 for Microsoft Azure Backup Server V4 due to the known issue - **Hyper-V scheduled backups take a long time to complete because each backup job triggers a consistency check.**

+>

+>Error message: The replica of Microsoft Hyper-V RCT on `<Machine Name>` is not consistent with the protected data source. DPM has detected changes in file locations or volume configurations of protected objects since the data source was configured for protection. (ID 30135).

+>

+>Resolution: This will be resolved in the new version planned to be released soon.

+>

+>- If you haven't installed UR1 for MABS V4 already, wait for the new release.

+>- If you have already installed UR1 for MABS V4, this new build will install on top of UR1 for MABS V4 to fix the known issues.

+>

+>For additional information, reach out to Microsoft Support.

+We strongly recommend that you have a support plan that includes technical support, such as [Microsoft Unified Support](https://www.microsoft.com/en-us/unifiedsupport/overview) or Premier Support.

+We strongly recommend a Microsoft support plan that includes technical support, such as [Microsoft Unified Support](https://www.microsoft.com/en-us/unifiedsupport/overview) or Premier Support.

+Before creating your request, review the details and diagnostic files that you're providing. If you want to change your request or the files that you're uploading, select **Previous** to return to any tab. When you're happy with your request, select **Create**.

+- [`2023-05-01`](/rest/api/containerapps/stable/container-apps?view=rest-containerapps-2023-05-01&preserve-view=true) (stable)

+- [`2023-08-01-preview`](/rest/api/containerapps/container-apps?view=rest-containerapps-2023-08-01-preview&preserve-view=true) (preview)

+ Title: Alerts and incidents in Microsoft 365 Defender (Preview)

+# Alerts and incidents in Microsoft 365 Defender (Preview)

+## Integrate with Microsoft 365 Defender (Preview)

+## Integrate with Microsoft 365 Defender (Preview)

+## Integrate with Microsoft 365 Defender (Preview)

+## Integrate with Microsoft 365 Defender (Preview)

+| November 15 | [Defender for Cloud is now integrated with Microsoft 365 Defender (Preview)](#defender-for-cloud-is-now-integrated-with-microsoft-365-defender-preview) |

+### Defender for Cloud is now integrated with Microsoft 365 Defender (Preview)

+The integration also brings competitive cloud protection capabilities into the Security Operations Center (SOC) day-to-day. With Microsoft 365 Defender, SOC teams can easily discover attacks that combine detections from multiple pillars, including Cloud, Endpoint, Identity, Office 365, and more.

+| [Taiwanese identification number](/purview/sit-defn-taiwan-national-identification-number) | NO |

+1. Sign in to the sensor as the `support` user, access the system shell and copy the update file to a location accessible for the update process. For example:

+ curl -H "X-Auth-Token: $(python3 -c 'from cyberx.credentials.credentials_wrapper import CredentialsWrapper;creds_wrapper = CredentialsWrapper();print(creds_wrapper.get("api.token"))')" -X POST http://127.0.0.1:9090/core/api/v1/configuration/agent

+To create an alias record set in your DNS zone using the Azure portal, add a record set and choose **Yes** under **Alias record set**. You must also specify the **Alias type** as either an **Azure resource** or **Zone record set**. If the record set is for an Azure resource, also **Choose a subscription** and then choose the **Azure resource**.

+In the following example, an alias named **vm1** is added that points to the public IP address of a virtual machine:

+ <br><img src="./media/dns-alias/add-record-set.png" alt="A screenshot showing how to add an alias record set." width="50%">

+- **Point to an Azure Content Delivery Network (CDN) endpoint** - This alias type is useful when you create static websites using Azure storage and Azure CDN.

+- **Point to another DNS record set within the same zone** - Alias records can reference other record sets of the same type. For example, a DNS CNAME record set can be an alias to another CNAME record set. This arrangement is useful if you want some but not all record sets to be aliases.

+To resolve this issue, you can use alias records. Unlike CNAME records, alias records are created at the zone apex. Application owners can use it to point their zone apex record to a Traffic Manager profile that has external endpoints. Application owners point to the same Traffic Manager profile used for any other domain within their DNS zone.

+Just like a Traffic Manager profile, you can also use alias records to point your DNS zone apex to Azure CDN endpoints. This alias is useful when you create static websites using Azure storage and Azure CDN. You can then access the website without prepending "www" to your DNS name.

+In this document, you learn how to execute a Spark job in a secure Spark cluster that reads from a topic in secure Kafka cluster, provided the virtual networks are same/peered.

+1. Execute the following commands based on your parameter values

+1. Create a `bobadmin_jaas.conf` as shown in following sample

+1. Create an `alicetest_jaas.conf` as shown in following sample

+ If you see the following error, which denotes the DNS (Domain Name Server) issue. Make sure to check Kafka worker nodes entry in `/etc/hosts` file in Spark cluster.

+1. Following are the detailed steps on how to check the application output from YARN UI.

+ 1. Go to YARN UI and open your application. Wait for the job to go to RUNNING state. You'll see the following application details.

+ 1. Click Logs. You'll see the following list of logs.

+ 1. Click 'stdout'. You'll see the following output with the count of words from your Kafka topic.

+If you're using multiple clusters together, you can create a virtual network, and if you're using a Spark cluster you can use the Hive Warehouse Connector. For more information, see [Plan a virtual network for Azure HDInsight](../hdinsight-plan-virtual-network-deployment.md) and [Integrate Apache Spark and Apache Hive with the Hive Warehouse Connector](../interactive-query/apache-hive-warehouse-connector.md).

+2. Set environment variables. The use of variables in this quickstart is based on Bash. Slight variations are needed for other environments. Replace RESOURCEGROUPNAME, LOCATION, CLUSTERNAME, STORAGEACCOUNTNAME, and PASSWORD in the following code snippet with the desired values. Then enter the CLI commands to set the environment variables.

+3. Create the resource group by entering the following command:

+4. Create an Azure storage account by entering the following command:

+5. Extract the primary key from the Azure storage account and store it in a variable by entering the following command:

+6. Create an Azure storage container by entering the following command:

+This error can be caused by specifying a version of the `spark-streaming-kafka` jar file that is different than the version of the Kafka cluster you're running.

+For example, if you're running a Kafka cluster version 0.10.1, the following command results in an error:

+Use the `Spark-submit` command with the `ΓÇôpackages` option, and ensure that the version of the spark-streaming-kafka jar file is the same as the version of the Kafka cluster that you are running.

+In this document we go over the steps to configure settings on the FHIR service for $import operation. To learn about import capabilties FHIR service offers, see [$import operation](import-data.md).

+To configure settings you will need to -

+Import operation enables loading Fast Healthcare Interoperability Resources (FHIR&#174;) data to the FHIR server at high throughput. Import supports both initial and incremental data load into FHIR server.

+There are two modes of $import supported today-

+* Initial mode

+1. Initial mode is intended to load FHIR resources into an empty FHIR server.

+1. Initial mode only supports CREATE operations and, when enabled, blocks API writes to the FHIR server.

+* Incremental mode

+1. Incremental mode is optimized to load data into FHIR server periodically and doesn't block writes via API.

+1. Incremental mode allows you to load lastUpdated and versionId from resource Meta (if present in resource JSON).

+ Incase

+ * Import files don't have `version` and `lastUpdated` field values specified, there's no guarantee of importing resources in FHIR service.

+ * Import files have resources with duplicate `version` and `lastUpdated` field values, then only one resource is ingested in the FHIR service.

+1. Incremental mode allows you to ingest soft deleted resources. This capability is beneficial in case you would like to migrate from Azure API for FHIR to Azure Health Data Services, FHIR service.

+Note:

+* Import operation does not support conditional references in resources.

+* During import operation, If multiple resources share the same resource ID, then only one of those resources is imported at random. There is an error logged for the resources sharing the same resource ID.

+| mode | Import mode value | 1..1 | For initial mode import, use ```InitialLoad``` mode value. For incremental mode import, use ```IncrementalLoad``` mode value. If no mode value is provided, IncrementalLoad mode value is considered by default. |

+**Sample body for import:**

+ "valueString": "<Use "InitialLoad" for initial mode import / Use "IncrementalLoad" for incremental mode import>",

+### Ingestion of soft deleted resources

+Incremental mode import supports ingestion of soft deleted resources. You need to use the extension to ingest soft deleted resources in FHIR service.

+**Sample import file with soft deleted resources:**

+Extension needs to be added to the resource to inform FHIR service, if it is soft deleted. Below is an example of the extension.

+```ndjson

+{"resourceType": "Patient", "id": "example10", "meta": { "lastUpdated": "2023-10-27T04:00:00.000Z", "versionId": 4, "extension": [ { "url": "http://azurehealthcareapis.com/data-extensions/deleted-state", "valueString": "soft-deleted" } ] } }

+```

+**Validate ingestion of soft deleted resources:**

+After import operation is successfully completed, to validate soft deleted resources in FHIR service, you need to perform history search on the resource.

+If the ID of the resource that was deleted is known, use the following URL pattern:

+```json

+<FHIR_URL>/<resource-type>/<resource-id>/_history

+```

+Incase the ID of the resource isn't known, do a history search on the entire resource type:

+```json

+<FHIR_URL>/<resource-type>/_history

+```

+1. Implement your device software or firmware such that your telemetry, properties, and commands follow the [IoT Plug and Play conventions](concepts-convention.md).

+ The following snippet shows the YAML file that you applied:

+ :::code language="yaml" source="~/azure-iot-operations-samples/samples/quickstarts/opc-ua-connector-0.yaml":::

+ The following snippet shows the YAML file that you applied:

+ :::code language="yaml" source="~/azure-iot-operations-samples/samples/quickstarts/mqtt-client.yaml":::

+The following snippet shows the YAML file that you applied:

+* [Azure Monitor workspace](how-to-connect-azure-monitor-workspace.md) in Azure Managed Grafana. This feature in preview allows you to add Azure Monitor workspaces to an Azure Managed Grafana workspace, in the Azure portal. This integration simplifies the process of collecting Prometheus data.

+**Disk size** | up to 2 TB OS disk for gen 1 VM; up to 4 TB OS disk for gen 2 VM; 32 TB for data disks.

+**Multipath IO** | Supported for Windows servers with Microsoft or vendor-specific Device Specific Module (DSM) installed.

+**Supported operating systems** | Windows Server 2003 and later versions. [Learn more](https://learn.microsoft.com/troubleshoot/azure/virtual-machines/server-software-support). <br/><br/> You can migrate all the Linux operating systems supported by Azure listed [here](https://learn.microsoft.com/troubleshoot/azure/cloud-services/support-linux-open-source-technology).

+**Automatic installation of Azure VM agent (Windows and Linux Agent)** | Windows: <br/>Supported for Windows Server 2008 R2 onwards. <br/><br/>Linux: <br/>- Red Hat Enterprise Linux 9.x, 8.x, 7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.0, 6.x<br/>- CentOS 9.x (Release and Stream), 8.x (Release and Stream), 7.9, 7.7, 7.6, 7.5, 7.4, 6.x<br/>- SUSE Linux Enterprise Server 15 SP4, 15 SP3, 15 SP2, 15 SP1, 15 SP0, 12, 11 SP4, 11 SP3<br/>- Ubuntu 22.04, 21.04, 20.04, 19.04, 19.10, 18.04LTS, 16.04LTS, 14.04LTS<br/>- Debian 11, 10, 9, 8, 7<br/>- Oracle Linux 9, 8, 7.7-CI, 7.7, 6<br/>- Kali Linux (2016, 2017, 2018, 2019, 2020, 2021, 2022)<br/>

+- Australia East

+- China North 3

+- East Asia

+- France Central

+- North Europe

+- Norway East

+- South Africa North

+- Sweden Central

+- Switzerland North

+- UAE North

+- UK South

+- US Gov Virginia

+- West Europe

+> [!IMPORTANT]

+> The accelerated logs feature is only available for servers based on the Business Critical service tier. It is recommended to disable the feature when scaling down to any other service tier.

+The stored procedure for replication using GTID is available on all HA-enabled servers by the name `mysql.az_replication_change_master_with_gtid`.

+- **Accelerated logs in Azure Database for MySQL - Flexible Server (Preview)**

+We're excited to announce preview of the accelerated logs feature for Azure Database for MySQL ΓÇô Flexible Server. This feature is available within the Business Critical service tier. Accelerated logs significantly enhances the performance of MySQL flexible servers, offering a dynamic solution that is designed for high throughput needs that also reduces latency and optimizes cost efficiency.[Learn more](./concepts-accelerated-logs.md).

+|Slave_SQL_Running_State| Indicates the current SQL thread status. In this state, `System lock` is normal. It's also normal to see a status of `Waiting for dependent transaction to commit`. This status indicates that the replica is waiting for other SQL worker threads to update committed transactions.|

+ **ARR -> AccessControl (IAM)**

+With the Azure side of things in place, we now need to modify how your code connects to the ARR service. We do that by implementing an instance of **BaseARRAuthentication**, which returns a new **SessionConfiguration** object. In this case, the account info is configured with the Azure Access Token.

+ RemoteRenderingCoordinator.ARRCredentialGetter = GetARRCredentials;

+ public async override Task<SessionConfiguration> GetARRCredentials()

+Oracle Linux | 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, 6.10, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, [7.7](https://support.microsoft.com/help/4531426/update-rollup-42-for-azure-site-recovery), [7.8](https://support.microsoft.com/help/4573888/), [7.9](https://support.microsoft.com/help/4597409), [8.0](https://support.microsoft.com/help/4573888/), [8.1](https://support.microsoft.com/help/4573888/), [8.2](https://support.microsoft.com/topic/update-rollup-55-for-azure-site-recovery-kb5003408-b19c8190-5f88-43ea-85b1-d9e0cc5ca7e8), [8.3](https://support.microsoft.com/topic/update-rollup-55-for-azure-site-recovery-kb5003408-b19c8190-5f88-43ea-85b1-d9e0cc5ca7e8) (running the Red Hat compatible kernel or Unbreakable Enterprise Kernel Release 3, 4, 5, and 6 (UEK3, UEK4, UEK5, UEK6), [8.4](https://support.microsoft.com/topic/update-rollup-59-for-azure-site-recovery-kb5008707-66a65377-862b-4a4c-9882-fd74bdc7a81e), 8.5, 8.6, 8.7, 8.8 <br> **Note:** Support for Oracle Linux 9.1 is removed from support matrix as issues were observed while using Azure Site Recovery with Oracle Linux 9.1. <br/><br/>8.1 (running on all UEK kernels and RedHat kernel <= 3.10.0-1062.* are supported in [9.35](https://support.microsoft.com/help/4573888/) Support for rest of the RedHat kernels is available in [9.36](https://support.microsoft.com/help/4578241/)).

+Oracle Linux | 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, 6.10, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, [7.7](https://support.microsoft.com/help/4531426/update-rollup-42-for-azure-site-recovery), [7.8](https://support.microsoft.com/help/4573888/), [7.9](https://support.microsoft.com/help/4597409/), [8.0](https://support.microsoft.com/help/4573888/), [8.1](https://support.microsoft.com/help/4573888/), [8.2](https://support.microsoft.com/topic/b19c8190-5f88-43ea-85b1-d9e0cc5ca7e8), [8.3](https://support.microsoft.com/topic/b19c8190-5f88-43ea-85b1-d9e0cc5ca7e8), [8.4](https://support.microsoft.com/topic/update-rollup-59-for-azure-site-recovery-kb5008707-66a65377-862b-4a4c-9882-fd74bdc7a81e), 8.5, 8.6, 8.7, 8.8 <br/><br/> **Note:** Support for Oracle Linux `9.0` and `9.1` is removed from support matrix, as issues were observed using Azure Site Recovery with Oracle Linux 9.0 and 9.1. <br><br> Running the Red Hat compatible kernel or Unbreakable Enterprise Kernel Release 3, 4 & 5 (UEK3, UEK4, UEK5)<br/><br/>8.1<br/>Running on all UEK kernels and RedHat kernel <= 3.10.0-1062.* are supported in [9.35](https://support.microsoft.com/help/4573888/) Support for rest of the RedHat kernels is available in [9.36](https://support.microsoft.com/help/4578241/).

+## Migration between Apache Spark versions - support

+For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4 refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md).

+## Migration between Apache Spark versions - support

+For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4 please refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md).

+## Migration between Apache Spark versions - support

+For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4 please refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md).

+ Title: Azure Synapse Runtime for Apache Spark 3.4

+description: New runtime is in Public Preview. Try it and use Spark 3.4.1, Python 3.10, Delta Lake 2.4.

+# Azure Synapse Runtime for Apache Spark 3.4 (Public Preview)

+Azure Synapse Analytics supports multiple runtimes for Apache Spark. This document covers the runtime components and versions for the Azure Synapse Runtime for Apache Spark 3.4.

+## Component versions

+| Component | Version |

+| -- |--|

+| Apache Spark | 3.4.1 |

+| Operating System | Mariner 2.0 |

+| Java | 11 |

+| Scala | 2.12.17 |

+| Delta Lake | 2.4.0 |

+| Python | 3.10 |

+| R | 4.2.2 |

+As of now, creation of Spark 3.4 pools will be available only thru Azure Synapse Studio. In the upcoming weeks we will add the Azure Portal and ARM support.

+## Libraries

+The following sections present the libraries included in Azure Synapse Runtime for Apache Spark 3.4 (Public Preview).

+### Scala and Java default libraries

+The following table lists all the default level packages for Java/Scala and their respective versions.

+| GroupID | ArtifactID | Version |

+|-||--|

+| com.aliyun | aliyun-java-sdk-core | 4.5.10 |

+| com.aliyun | aliyun-java-sdk-kms | 2.11.0 |

+| com.aliyun | aliyun-java-sdk-ram | 3.1.0 |

+| com.aliyun | aliyun-sdk-oss | 3.13.0 |

+| com.amazonaws | aws-java-sdk-bundle | 1.12.1026 |

+| com.chuusai | shapeless_2.12 | 2.3.7 |

+| com.clearspring.analytics | stream | 2.9.6 |

+| com.esotericsoftware | kryo-shaded | 4.0.2 |

+| com.esotericsoftware | minlog | 1.3.0 |

+| com.fasterxml.jackson | jackson-annotations | 2.13.4 |

+| com.fasterxml.jackson | jackson-core | 2.13.4 |

+| com.fasterxml.jackson | jackson-core-asl | 1.9.13 |

+| com.fasterxml.jackson | jackson-databind | 2.13.4.1 |

+| com.fasterxml.jackson | jackson-dataformat-cbor | 2.13.4 |

+| com.fasterxml.jackson | jackson-mapper-asl | 1.9.13 |

+| com.fasterxml.jackson | jackson-module-scala_2.12 | 2.13.4 |

+| com.github.joshelser | dropwizard-metrics-hadoop-metrics2-reporter | 0.1.2 |

+| com.github.luben | zstd-jni | 1.5.2-1 |

+| com.github.luben | zstd-jni | 1.5.2-1 |

+| com.github.vowpalwabbit | vw-jni | 9.3.0 |

+| com.github.wendykierp | JTransforms | 3.1 |

+| com.google.code.findbugs | jsr305 | 3.0.0 |

+| com.google.code.gson | gson | 2.8.6 |

+| com.google.crypto.tink | tink | 1.6.1 |

+| com.google.flatbuffers | flatbuffers-java | 1.12.0 |

+| com.google.guava | guava | 14.0.1 |

+| com.google.protobuf | protobuf-java | 2.5.0 |

+| com.googlecode.json-simple | json-simple | 1.1.1 |

+| com.jcraft | jsch | 0.1.54 |

+| com.jolbox | bonecp | 0.8.0.RELEASE |

+| com.linkedin.isolation-forest | isolation-forest_3.2.0_2.12 | 2.0.8 |

+| com.microsoft.azure | azure-data-lake-store-sdk | 2.3.9 |

+| com.microsoft.azure | azure-eventhubs | 3.3.0 |

+| com.microsoft.azure | azure-eventhubs-spark_2.12 | 2.3.22 |

+| com.microsoft.azure | azure-keyvault-core | 1.0.0 |

+| com.microsoft.azure | azure-storage | 7.0.1 |

+| com.microsoft.azure | cosmos-analytics-spark-3.4.1-connector_2.12 | 1.8.10 |

+| com.microsoft.azure | qpid-proton-j-extensions | 1.2.4 |

+| com.microsoft.azure | synapseml_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-cognitive_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-core_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-deep-learning_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-internal_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-lightgbm_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-opencv_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure | synapseml-vw_2.12 | 0.11.3-spark3.3 |

+| com.microsoft.azure.kusto | kusto-data | 3.2.1 |

+| com.microsoft.azure.kusto | kusto-ingest | 3.2.1 |

+| com.microsoft.azure.kusto | kusto-spark_3.0_2.12 | 3.1.16 |

+| com.microsoft.azure.kusto | spark-kusto-synapse-connector_3.1_2.12 | 1.3.3 |

+| com.microsoft.cognitiveservices.speech | client-jar-sdk | 1.14.0 |

+| com.microsoft.sqlserver | msslq-jdbc | 8.4.1.jre8 |

+| com.ning | compress-lzf | 1.1 |

+| com.sun.istack | istack-commons-runtime | 3.0.8 |

+| com.tdunning | json | 1.8 |

+| com.thoughtworks.paranamer | paranamer | 2.8 |

+| com.twitter | chill-java | 0.10.0 |

+| com.twitter | chill_2.12 | 0.10.0 |

+| com.typesafe | config | 1.3.4 |

+| com.univocity | univocity-parsers | 2.9.1 |

+| com.zaxxer | HikariCP | 2.5.1 |

+| commons-cli | commons-cli | 1.5.0 |

+| commons-codec | commons-codec | 1.15 |

+| commons-collections | commons-collections | 3.2.2 |

+| commons-dbcp | commons-dbcp | 1.4 |

+| commons-io | commons-io | 2.11.0 |

+| commons-lang | commons-lang | 2.6 |

+| commons-logging | commons-logging | 1.1.3 |

+| commons-pool | commons-pool | 1.5.4 |

+| dev.ludovic.netlib | arpack | 2.2.1 |

+| dev.ludovic.netlib | blas | 2.2.1 |

+| dev.ludovic.netlib | lapack | 2.2.1 |

+| io.airlift | aircompressor | 0.21 |

+| io.delta | delta-core_2.12 | 2.2.0.9 |

+| io.delta | delta-storage | 2.2.0.9 |

+| io.dropwizard.metrics | metrics-core | 4.2.7 |

+| io.dropwizard.metrics | metrics-graphite | 4.2.7 |

+| io.dropwizard.metrics | metrics-jmx | 4.2.7 |

+| io.dropwizard.metrics | metrics-json | 4.2.7 |

+| io.dropwizard.metrics | metrics-jvm | 4.2.7 |

+| io.github.resilience4j | resilience4j-core | 1.7.1 |

+| io.github.resilience4j | resilience4j-retry | 1.7.1 |

+| io.netty | netty-all | 4.1.74.Final |

+| io.netty | netty-buffer | 4.1.74.Final |

+| io.netty | netty-codec | 4.1.74.Final |

+| io.netty | netty-codec-http2 | 4.1.74.Final |

+| io.netty | netty-codec-http-4 | 4.1.74.Final |

+| io.netty | netty-codec-socks | 4.1.74.Final |

+| io.netty | netty-common | 4.1.74.Final |

+| io.netty | netty-handler | 4.1.74.Final |

+| io.netty | netty-resolver | 4.1.74.Final |

+| io.netty | netty-tcnative-classes | 2.0.48 |

+| io.netty | netty-transport | 4.1.74.Final |

+| io.netty | netty-transport-classes-epoll | 4.1.87.Final |

+| io.netty | netty-transport-classes-kqueue | 4.1.87.Final |

+| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-aarch_64 |

+| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-x86_64 |

+| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-aarch_64 |

+| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-x86_64 |

+| io.netty | netty-transport-native-unix-common | 4.1.87.Final |

+| io.opentracing | opentracing-api | 0.33.0 |

+| io.opentracing | opentracing-noop | 0.33.0 |

+| io.opentracing | opentracing-util | 0.33.0 |

+| io.spray | spray-json_2.12 | 1.3.5 |

+| io.vavr | vavr | 0.10.4 |

+| io.vavr | vavr-match | 0.10.4 |

+| jakarta.annotation | jakarta.annotation-api | 1.3.5 |

+| jakarta.inject | jakarta.inject | 2.6.1 |

+| jakarta.servlet | jakarta.servlet-api | 4.0.3 |

+| jakarta.validation-api | | 2.0.2 |

+| jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |

+| jakarta.xml.bind | jakarta.xml.bind-api | 2.3.2 |

+| javax.activation | activation | 1.1.1 |

+| javax.jdo | jdo-api | 3.0.1 |

+| javax.transaction | jta | 1.1 |

+| javax.transaction | transaction-api | 1.1 |

+| javax.xml.bind | jaxb-api | 2.2.11 |

+| javolution | javolution | 5.5.1 |

+| jline | jline | 2.14.6 |

+| joda-time | joda-time | 2.10.13 |

+| mysql | mysql-connector-java | 8.0.18 |

+| net.razorvine | pickle | 1.2 |

+| net.sf.jpam | jpam | 1.1 |

+| net.sf.opencsv | opencsv | 2.3 |

+| net.sf.py4j | py4j | 0.10.9.5 |

+| net.sf.supercsv | super-csv | 2.2.0 |

+| net.sourceforge.f2j | arpack_combined_all | 0.1 |

+| org.antlr | ST4 | 4.0.4 |

+| org.antlr | antlr-runtime | 3.5.2 |

+| org.antlr | antlr4-runtime | 4.8 |

+| org.apache.arrow | arrow-format | 7.0.0 |

+| org.apache.arrow | arrow-memory-core | 7.0.0 |

+| org.apache.arrow | arrow-memory-netty | 7.0.0 |

+| org.apache.arrow | arrow-vector | 7.0.0 |

+| org.apache.avro | avro | 1.11.0 |

+| org.apache.avro | avro-ipc | 1.11.0 |

+| org.apache.avro | avro-mapred | 1.11.0 |

+| org.apache.commons | commons-collections4 | 4.4 |

+| org.apache.commons | commons-compress | 1.21 |

+| org.apache.commons | commons-crypto | 1.1.0 |

+| org.apache.commons | commons-lang3 | 3.12.0 |

+| org.apache.commons | commons-math3 | 3.6.1 |

+| org.apache.commons | commons-pool2 | 2.11.1 |

+| org.apache.commons | commons-text | 1.10.0 |

+| org.apache.curator | curator-client | 2.13.0 |

+| org.apache.curator | curator-framework | 2.13.0 |

+| org.apache.curator | curator-recipes | 2.13.0 |

+| org.apache.derby | derby | 10.14.2.0 |

+| org.apache.hadoop | hadoop-aliyun | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-annotations | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-aws | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-azure | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-azure-datalake | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-client-api | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-client-runtime | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-cloud-storage | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-openstack | 3.3.3.5.2-106693326 |

+| org.apache.hadoop | hadoop-shaded-guava | 1.1.1 |

+| org.apache.hadoop | hadoop-yarn-server-web-proxy | 3.3.3.5.2-106693326 |

+| org.apache.hive | hive-beeline | 2.3.9 |

+| org.apache.hive | hive-cli | 2.3.9 |

+| org.apache.hive | hive-common | 2.3.9 |

+| org.apache.hive | hive-exec | 2.3.9 |

+| org.apache.hive | hive-jdbc | 2.3.9 |

+| org.apache.hive | hive-llap-common | 2.3.9 |

+| org.apache.hive | hive-metastore | 2.3.9 |

+| org.apache.hive | hive-serde | 2.3.9 |

+| org.apache.hive | hive-service-rpc | 2.3.9 |

+| org.apache.hive | hive-shims-0.23 | 2.3.9 |

+| org.apache.hive | hive-shims | 2.3.9 |

+| org.apache.hive | hive-shims-common | 2.3.9 |

+| org.apache.hive | hive-shims-scheduler | 2.3.9 |

+| org.apache.hive | hive-storage-api | 2.7.2 |

+| org.apache.httpcomponents | httpclient | 4.5.13 |

+| org.apache.httpcomponents | httpcore | 4.4.14 |

+| org.apache.httpcomponents | httpmime | 4.5.13 |

+| org.apache.httpcomponents.client5 | httpclient5 | 5.1.3 |

+| org.apache.iceberg | delta-iceberg | 2.2.0.9 |

+| org.apache.ivy | ivy | 2.5.1 |

+| org.apache.kafka | kafka-clients | 2.8.1 |

+| org.apache.logging.log4j | log4j-1.2-api | 2.17.2 |

+| org.apache.logging.log4j | log4j-api | 2.17.2 |

+| org.apache.logging.log4j | log4j-core | 2.17.2 |

+| org.apache.logging.log4j | log4j-slf4j-impl | 2.17.2 |

+| org.apache.orc | orc-core | 1.7.6 |

+| org.apache.orc | orc-mapreduce | 1.7.6 |

+| org.apache.orc | orc-shims | 1.7.6 |

+| org.apache.parquet | parquet-column | 1.12.3 |

+| org.apache.parquet | parquet-common | 1.12.3 |

+| org.apache.parquet | parquet-encoding | 1.12.3 |

+| org.apache.parquet | parquet-format-structures | 1.12.3 |

+| org.apache.parquet | parquet-hadoop | 1.12.3 |

+| org.apache.parquet | parquet-jackson | 1.12.3 |

+| org.apache.qpid | proton-j | 0.33.8 |

+| org.apache.spark | spark-avro_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-catalyst_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-core_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-graphx_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-hadoop-cloud_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-hive_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-kvstore_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-launcher_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-mllib_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-mllib-local_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-network-common_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-network-shuffle_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-repl_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-sketch_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-sql_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-sql-kafka-0-10_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-streaming_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-streaming-kafka-0-10-assembly_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-tags_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-token-provider-kafka-0-10_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-unsafe_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.spark | spark-yarn_2.12 | 3.3.1.5.2-106693326 |

+| org.apache.thrift | libfb303 | 0.9.3 |

+| org.apache.thrift | libthrift | 0.12.0 |

+| org.apache.velocity | velocity | 1.5 |

+| org.apache.xbean | xbean-asm9-shaded | 4.2 |

+| org.apache.yetus | audience-annotations | 0.5.0 |

+| org.apache.zookeeper | zookeeper | 3.6.2.5.2-106693326 |

+| org.apache.zookeeper | zookeeper-jute | 3.6.2.5.2-106693326 |

+| org.apache.zookeeper | zookeeper | 3.6.2.5.2-106693326 |

+| org.apache.zookeeper | zookeeper-jute | 3.6.2.5.2-106693326 |

+| org.apiguardian | apiguardian-api | 1.1.0 |

+| org.codehaus.janino | commons-compiler | 3.0.16 |

+| org.codehaus.janino | janino | 3.0.16 |

+| org.codehaus.jettison | jettison | 1.1 |

+| org.datanucleus | datanucleus-api-jdo | 4.2.4 |

+| org.datanucleus | datanucleus-core | 4.1.17 |

+| org.datanucleus | datanucleus-rdbms | 4.1.19 |

+| org.datanucleusjavax.jdo | | 3.2.0-m3 |

+| org.eclipse.jetty | jetty-util | 9.4.48.v20220622 |

+| org.eclipse.jetty | jetty-util-ajax | 9.4.48.v20220622 |

+| org.fusesource.leveldbjni | leveldbjni-all | 1.8 |

+| org.glassfish.hk2 | hk2-api | 2.6.1 |

+| org.glassfish.hk2 | hk2-locator | 2.6.1 |

+| org.glassfish.hk2 | hk2-utils | 2.6.1 |

+| org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |

+| org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |

+| org.glassfish.jaxb | jaxb-runtime | 2.3.2 |

+| org.glassfish.jersey.containers | jersey-container-servlet | 2.36 |

+| org.glassfish.jersey.containers | jersey-container-servlet-core | 2.36 |

+| org.glassfish.jersey.core | jersey-client | 2.36 |

+| org.glassfish.jersey.core | jersey-common | 2.36 |

+| org.glassfish.jersey.core | jersey-server | 2.36 |

+| org.glassfish.jersey.inject | jersey-hk2 | 2.36 |

+| org.ini4j | ini4j | 0.5.4 |

+| org.javassist | javassist | 3.25.0-GA |

+| org.javatuples | javatuples | 1.2 |

+| org.jdom | jdom2 | 2.0.6 |

+| org.jetbrains | annotations | 17.0.0 |

+| org.jodd | jodd-core | 3.5.2 |

+| org.json | json | 20210307 |

+| org.json4s | json4s-ast_2.12 | 3.7.0-M11 |

+| org.json4s | json4s-core_2.12 | 3.7.0-M11 |

+| org.json4s | json4s-jackson_2.12 | 3.7.0-M11 |

+| org.json4s | json4s-scalap_2.12 | 3.7.0-M11 |

+| org.junit.jupiter | junit-jupiter | 5.5.2 |

+| org.junit.jupiter | junit-jupiter-api | 5.5.2 |

+| org.junit.jupiter | junit-jupiter-engine | 5.5.2 |

+| org.junit.jupiter | junit-jupiter-params | 5.5.2 |

+| org.junit.platform | junit-platform-commons | 1.5.2 |

+| org.junit.platform | junit-platform-engine | 1.5.2 |

+| org.lz4 | lz4-java | 1.8.0 |

+| org.mlflow | mlfow-spark | 2.1.1 |

+| org.objenesis | objenesis | 3.2 |

+| org.openpnp | opencv | 3.2.0-1 |

+| org.opentest4j | opentest4j | 1.2.0 |

+| org.postgresql | postgresql | 42.2.9 |

+| org.roaringbitmap | RoaringBitmap | 0.9.25 |

+| org.roaringbitmap | shims | 0.9.25 |

+| org.rocksdb | rocksdbjni | 6.20.3 |

+| org.scalactic | scalactic_2.12 | 3.2.14 |

+| org.scala-lang | scala-compiler | 2.12.15 |

+| org.scala-lang | scala-library | 2.12.15 |

+| org.scala-lang | scala-reflect | 2.12.15 |

+| org.scala-lang.modules | scala-collection-compat_2.12 | 2.1.1 |

+| org.scala-lang.modules | scala-java8-compat_2.12 | 0.9.0 |

+| org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 |

+| org.scala-lang.modules | scala-xml_2.12 | 1.2.0 |

+| org.scalanlp | breeze-macros_2.12 | 1.2 |

+| org.scalanlp | breeze_2.12 | 1.2 |

+| org.slf4j | jcl-over-slf4j | 1.7.32 |

+| org.slf4j | jul-to-slf4j | 1.7.32 |

+| org.slf4j | slf4j-api | 1.7.32 |

+| org.threeten | threeten-extra | 1.5.0 |

+| org.tukaani | xz | 1.8 |

+| org.typelevel | algebra_2.12 | 2.0.1 |

+| org.typelevel | cats-kernel_2.12 | 2.1.1 |

+| org.typelevel | spire_2.12 | 0.17.0 |

+| org.typelevel | spire-macros_2.12 | 0.17.0 |

+| org.typelevel | spire-platform_2.12 | 0.17.0 |

+| org.typelevel | spire-util_2.12 | 0.17.0 |

+| org.wildfly.openssl | wildfly-openssl | 1.0.7.Final |

+| org.xerial.snappy | snappy-java | 1.1.8.4 |

+| oro | oro | 2.0.8 |

+| pl.edu.icm | JLargeArrays | 1.5 |

+| stax | stax-api | 1.0.1 |

+### Python libraries

+The Azure Synapse Runtime for Apache Spark 3.4 is currently in Public Preview. During this phase, the Python libraries will experience significant updates. Additionally, please note that some machine learning capabilities are not yet supported, such as the PREDICT method and Synapse ML.

+### R libraries

+The following table lists all the default level packages for R and their respective versions.

+| Library | Version | Library | Version | Library | Version |

+||--|--||||

+| _libgcc_mutex | 0.1 | r-caret | 6.0_94 | r-praise | 1.0.0 |

+| _openmp_mutex | 4.5 | r-cellranger | 1.1.0 | r-prettyunits | 1.2.0 |

+| _r-mutex | 1.0.1 | r-class | 7.3_22 | r-proc | 1.18.4 |

+| _r-xgboost-mutex | 2 | r-cli | 3.6.1 | r-processx | 3.8.2 |

+| aws-c-auth | 0.7.0 | r-clipr | 0.8.0 | r-prodlim | 2023.08.28 |

+| aws-c-cal | 0.6.0 | r-clock | 0.7.0 | r-profvis | 0.3.8 |

+| aws-c-common | 0.8.23 | r-codetools | 0.2_19 | r-progress | 1.2.2 |

+| aws-c-compression | 0.2.17 | r-collections | 0.3.7 | r-progressr | 0.14.0 |

+| aws-c-event-stream | 0.3.1 | r-colorspace | 2.1_0 | r-promises | 1.2.1 |

+| aws-c-http | 0.7.10 | r-commonmark | 1.9.0 | r-proxy | 0.4_27 |

+| aws-c-io | 0.13.27 | r-config | 0.3.2 | r-pryr | 0.1.6 |

+| aws-c-mqtt | 0.8.13 | r-conflicted | 1.2.0 | r-ps | 1.7.5 |

+| aws-c-s3 | 0.3.12 | r-coro | 1.0.3 | r-purrr | 1.0.2 |

+| aws-c-sdkutils | 0.1.11 | r-cpp11 | 0.4.6 | r-quantmod | 0.4.25 |

+| aws-checksums | 0.1.16 | r-crayon | 1.5.2 | r-r2d3 | 0.2.6 |

+| aws-crt-cpp | 0.20.2 | r-credentials | 2.0.1 | r-r6 | 2.5.1 |

+| aws-sdk-cpp | 1.10.57 | r-crosstalk | 1.2.0 | r-r6p | 0.3.0 |

+| binutils_impl_linux-64 | 2.4 | r-crul | 1.4.0 | r-ragg | 1.2.6 |

+| bwidget | 1.9.14 | r-curl | 5.1.0 | r-rappdirs | 0.3.3 |

+| bzip2 | 1.0.8 | r-data.table | 1.14.8 | r-rbokeh | 0.5.2 |

+| c-ares | 1.20.1 | r-dbi | 1.1.3 | r-rcmdcheck | 1.4.0 |

+| ca-certificates | 2023.7.22 | r-dbplyr | 2.3.4 | r-rcolorbrewer | 1.1_3 |

+| cairo | 1.18.0 | r-desc | 1.4.2 | r-rcpp | 1.0.11 |

+| cmake | 3.27.6 | r-devtools | 2.4.5 | r-reactable | 0.4.4 |

+| curl | 8.4.0 | r-diagram | 1.6.5 | r-reactr | 0.5.0 |

+| expat | 2.5.0 | r-dials | 1.2.0 | r-readr | 2.1.4 |

+| font-ttf-dejavu-sans-mono | 2.37 | r-dicedesign | 1.9 | r-readxl | 1.4.3 |

+| font-ttf-inconsolata | 3 | r-diffobj | 0.3.5 | r-recipes | 1.0.8 |

+| font-ttf-source-code-pro | 2.038 | r-digest | 0.6.33 | r-rematch | 2.0.0 |

+| font-ttf-ubuntu | 0.83 | r-downlit | 0.4.3 | r-rematch2 | 2.1.2 |

+| fontconfig | 2.14.2 | r-dplyr | 1.1.3 | r-remotes | 2.4.2.1 |

+| fonts-conda-ecosystem | 1 | r-dtplyr | 1.3.1 | r-reprex | 2.0.2 |

+| fonts-conda-forge | 1 | r-e1071 | 1.7_13 | r-reshape2 | 1.4.4 |

+| freetype | 2.12.1 | r-ellipsis | 0.3.2 | r-rjson | 0.2.21 |

+| fribidi | 1.0.10 | r-evaluate | 0.23 | r-rlang | 1.1.1 |

+| gcc_impl_linux-64 | 13.2.0 | r-fansi | 1.0.5 | r-rlist | 0.4.6.2 |

+| gettext | 0.21.1 | r-farver | 2.1.1 | r-rmarkdown | 2.22 |

+| gflags | 2.2.2 | r-fastmap | 1.1.1 | r-rodbc | 1.3_20 |

+| gfortran_impl_linux-64 | 13.2.0 | r-fontawesome | 0.5.2 | r-roxygen2 | 7.2.3 |

+| glog | 0.6.0 | r-forcats | 1.0.0 | r-rpart | 4.1.21 |

+| glpk | 5 | r-foreach | 1.5.2 | r-rprojroot | 2.0.3 |

+| gmp | 6.2.1 | r-forge | 0.2.0 | r-rsample | 1.2.0 |

+| graphite2 | 1.3.13 | r-fs | 1.6.3 | r-rstudioapi | 0.15.0 |

+| gsl | 2.7 | r-furrr | 0.3.1 | r-rversions | 2.1.2 |

+| gxx_impl_linux-64 | 13.2.0 | r-future | 1.33.0 | r-rvest | 1.0.3 |

+| harfbuzz | 8.2.1 | r-future.apply | 1.11.0 | r-sass | 0.4.7 |

+| icu | 73.2 | r-gargle | 1.5.2 | r-scales | 1.2.1 |

+| kernel-headers_linux-64 | 2.6.32 | r-generics | 0.1.3 | r-selectr | 0.4_2 |

+| keyutils | 1.6.1 | r-gert | 2.0.0 | r-sessioninfo | 1.2.2 |

+| krb5 | 1.21.2 | r-ggplot2 | 3.4.2 | r-shape | 1.4.6 |

+| ld_impl_linux-64 | 2.4 | r-gh | 1.4.0 | r-shiny | 1.7.5.1 |

+| lerc | 4.0.0 | r-gistr | 0.9.0 | r-slider | 0.3.1 |

+| libabseil | 20230125 | r-gitcreds | 0.1.2 | r-sourcetools | 0.1.7_1 |

+| libarrow | 12.0.0 | r-globals | 0.16.2 | r-sparklyr | 1.8.2 |

+| libblas | 3.9.0 | r-glue | 1.6.2 | r-squarem | 2021.1 |

+| libbrotlicommon | 1.0.9 | r-googledrive | 2.1.1 | r-stringi | 1.7.12 |

+| libbrotlidec | 1.0.9 | r-googlesheets4 | 1.1.1 | r-stringr | 1.5.0 |

+| libbrotlienc | 1.0.9 | r-gower | 1.0.1 | r-survival | 3.5_7 |

+| libcblas | 3.9.0 | r-gpfit | 1.0_8 | r-sys | 3.4.2 |

+| libcrc32c | 1.1.2 | r-gt | 0.9.0 | r-systemfonts | 1.0.5 |

+| libcurl | 8.4.0 | r-gtable | 0.3.4 | r-testthat | 3.2.0 |

+| libdeflate | 1.19 | r-gtsummary | 1.7.2 | r-textshaping | 0.3.7 |

+| libedit | 3.1.20191231 | r-hardhat | 1.3.0 | r-tibble | 3.2.1 |

+| libev | 4.33 | r-haven | 2.5.3 | r-tidymodels | 1.1.0 |

+| libevent | 2.1.12 | r-hexbin | 1.28.3 | r-tidyr | 1.3.0 |

+| libexpat | 2.5.0 | r-highcharter | 0.9.4 | r-tidyselect | 1.2.0 |

+| libffi | 3.4.2 | r-highr | 0.1 | r-tidyverse | 2.0.0 |

+| libgcc-devel_linux-64 | 13.2.0 | r-hms | 1.1.3 | r-timechange | 0.2.0 |

+| libgcc-ng | 13.2.0 | r-htmltools | 0.5.6.1 | r-timedate | 4022.108 |

+| libgfortran-ng | 13.2.0 | r-htmlwidgets | 1.6.2 | r-tinytex | 0.48 |

+| libgfortran5 | 13.2.0 | r-httpcode | 0.3.0 | r-torch | 0.11.0 |

+| libgit2 | 1.7.1 | r-httpuv | 1.6.12 | r-triebeard | 0.4.1 |

+| libglib | 2.78.0 | r-httr | 1.4.7 | r-ttr | 0.24.3 |

+| libgomp | 13.2.0 | r-httr2 | 0.2.3 | r-tune | 1.1.2 |

+| libgoogle-cloud | 2.12.0 | r-ids | 1.0.1 | r-tzdb | 0.4.0 |

+| libgrpc | 1.55.1 | r-igraph | 1.5.1 | r-urlchecker | 1.0.1 |

+| libiconv | 1.17 | r-infer | 1.0.5 | r-urltools | 1.7.3 |

+| libjpeg-turbo | 3.0.0 | r-ini | 0.3.1 | r-usethis | 2.2.2 |

+| liblapack | 3.9.0 | r-ipred | 0.9_14 | r-utf8 | 1.2.4 |

+| libnghttp2 | 1.55.1 | r-isoband | 0.2.7 | r-uuid | 1.1_1 |

+| libnuma | 2.0.16 | r-iterators | 1.0.14 | r-v8 | 4.4.0 |

+| libopenblas | 0.3.24 | r-jose | 1.2.0 | r-vctrs | 0.6.4 |

+| libpng | 1.6.39 | r-jquerylib | 0.1.4 | r-viridislite | 0.4.2 |

+| libprotobuf | 4.23.2 | r-jsonlite | 1.8.7 | r-vroom | 1.6.4 |

+| libsanitizer | 13.2.0 | r-juicyjuice | 0.1.0 | r-waldo | 0.5.1 |

+| libssh2 | 1.11.0 | r-kernsmooth | 2.23_22 | r-warp | 0.2.0 |

+| libstdcxx-devel_linux-64 | 13.2.0 | r-knitr | 1.45 | r-whisker | 0.4.1 |

+| libstdcxx-ng | 13.2.0 | r-labeling | 0.4.3 | r-withr | 2.5.2 |

+| libthrift | 0.18.1 | r-labelled | 2.12.0 | r-workflows | 1.1.3 |

+| libtiff | 4.6.0 | r-later | 1.3.1 | r-workflowsets | 1.0.1 |

+| libutf8proc | 2.8.0 | r-lattice | 0.22_5 | r-xfun | 0.41 |

+| libuuid | 2.38.1 | r-lava | 1.7.2.1 | r-xgboost | 1.7.4 |

+| libuv | 1.46.0 | r-lazyeval | 0.2.2 | r-xml | 3.99_0.14 |

+| libv8 | 8.9.83 | r-lhs | 1.1.6 | r-xml2 | 1.3.5 |

+| libwebp-base | 1.3.2 | r-lifecycle | 1.0.3 | r-xopen | 1.0.0 |

+| libxcb | 1.15 | r-lightgbm | 3.3.5 | r-xtable | 1.8_4 |

+| libxgboost | 1.7.4 | r-listenv | 0.9.0 | r-xts | 0.13.1 |

+| libxml2 | 2.11.5 | r-lobstr | 1.1.2 | r-yaml | 2.3.7 |

+| libzlib | 1.2.13 | r-lubridate | 1.9.3 | r-yardstick | 1.2.0 |

+| lz4-c | 1.9.4 | r-magrittr | 2.0.3 | r-zip | 2.3.0 |

+| make | 4.3 | r-maps | 3.4.1 | r-zoo | 1.8_12 |

+| ncurses | 6.4 | r-markdown | 1.11 | rdma-core | 28.9 |

+| openssl | 3.1.4 | r-mass | 7.3_60 | re2 | 2023.03.02 |

+| orc | 1.8.4 | r-matrix | 1.6_1.1 | readline | 8.2 |

+| pandoc | 2.19.2 | r-memoise | 2.0.1 | rhash | 1.4.4 |

+| pango | 1.50.14 | r-mgcv | 1.9_0 | s2n | 1.3.46 |

+| pcre2 | 10.4 | r-mime | 0.12 | sed | 4.8 |

+| pixman | 0.42.2 | r-miniui | 0.1.1.1 | snappy | 1.1.10 |

+| pthread-stubs | 0.4 | r-modeldata | 1.2.0 | sysroot_linux-64 | 2.12 |

+| r-arrow | 12.0.0 | r-modelenv | 0.1.1 | tk | 8.6.13 |

+| r-askpass | 1.2.0 | r-modelmetrics | 1.2.2.2 | tktable | 2.1 |

+| r-assertthat | 0.2.1 | r-modelr | 0.1.11 | ucx | 1.14.1 |

+| r-backports | 1.4.1 | r-munsell | 0.5.0 | unixodbc | 2.3.12 |

+| r-base | 4.2.3 | r-nlme | 3.1_163 | xorg-kbproto | 1.0.7 |

+| r-base64enc | 0.1_3 | r-nnet | 7.3_19 | xorg-libice | 1.1.1 |

+| r-bigd | 0.2.0 | r-numderiv | 2016.8_1.1 | xorg-libsm | 1.2.4 |

+| r-bit | 4.0.5 | r-openssl | 2.1.1 | xorg-libx11 | 1.8.7 |

+| r-bit64 | 4.0.5 | r-parallelly | 1.36.0 | xorg-libxau | 1.0.11 |

+| r-bitops | 1.0_7 | r-parsnip | 1.1.1 | xorg-libxdmcp | 1.1.3 |

+| r-blob | 1.2.4 | r-patchwork | 1.1.3 | xorg-libxext | 1.3.4 |

+| r-brew | 1.0_8 | r-pillar | 1.9.0 | xorg-libxrender | 0.9.11 |

+| r-brio | 1.1.3 | r-pkgbuild | 1.4.2 | xorg-libxt | 1.3.0 |

+| r-broom | 1.0.5 | r-pkgconfig | 2.0.3 | xorg-renderproto | 0.11.1 |

+| r-broom.helpers | 1.14.0 | r-pkgdown | 2.0.7 | xorg-xextproto | 7.3.0 |

+| r-bslib | 0.5.1 | r-pkgload | 1.3.3 | xorg-xproto | 7.0.31 |

+| r-cachem | 1.0.8 | r-plotly | 4.10.2 | xz | 5.2.6 |

+| r-callr | 3.7.3 | r-plyr | 1.8.9 | zlib | 1.2.13 |

+| | | | | zstd | 1.5.5 |

+## Migration between Apache Spark versions - support

+For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.4 refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md).

+| [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) | Nov 21, 2023 | Public Preview (GA expected in Q1 2024) |

+| [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) | Nov 17, 2022 | GA (as of Feb 23, 2023) | Q1/Q2 2024 | Q1 2025 |

+## Migration between Apache Spark versions - support

+General Upgrade guidelines/ FAQ's:

+Question: If a customer is seeking advice on how to migrate from 2.4 to 3.X, what steps should be taken?

+Answer: Refer to the following migration guide: https://spark.apache.org/docs/latest/sql-migration-guide.html

+Question: I get an error when I try to upgrade Spark pool runtime using PowerShell commandlet when they have attached libraries

+Answer: Do not use PowerShell Commandlet if you have custom libraries installed in your synapse workspace. Instead follow these steps:

+ -Recreate Spark Pool 3.3 from the ground up.

+ -Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3

+ | Hibernate (preview) | Check the box to enable hibernate. Hibernate is only available for personal host pools. You will need to [self-register your subscription](../virtual-machines/hibernate-resume.md) to use the hibernation feature. For more information, see [Hibernation in virtual machines](/azure/virtual-machines/hibernate-resume). <br /><br />Note: We recommend users using Teams media optimizations to upgrade their host pools to WebRTC redirector service 1.45.2310.13001, learn more [here](whats-new-webrtc.md).|

+ | Hibernate (preview) | Check the box to enable hibernate. Hibernate is only available for personal host pools. You will need to [self-register your subscription](../virtual-machines/hibernate-resume.md) to use the hibernation feature. For more information, see [Hibernation in virtual machines](/azure/virtual-machines/hibernate-resume). <br /><br />Note: We recommend users using Teams media optimizations to upgrade their host pools to WebRTC redirector service 1.45.2310.13001, learn more [here](whats-new-webrtc.md).|

+## Updates for version 1.45.2310.13001

+## Updates for version 1.43.2306.30001

+## Updates for version 1.33.2302.07001

+## Updates for version 1.31.2211.15001

+## Updates for version 1.17.2205.23001

+## Updates for version 1.4.2111.18001

+## Updates for version 1.1.2110.16001

+Learn about known issues, limitations, and how to log issues at [Troubleshoot Teams on Azure Virtual Desktop](troubleshoot-teams.md).

+In rare cases if platform catchup host update window happens to coincide with the guest (VM) patching window and if the guest patching window don't get sufficient time to execute after host update then the system would show **Schedule timeout, waiting for an ongoing update to complete the resource** error since only a single update is allowed by the platform at a time.

+> 1. The count of characters of Resource Group name along with Maintenance Configuration name should be less than 128 characters

+> 2. If you move a VM to a different resource group or subscription, the scheduled patching for the VM stops working as this scenario is currently unsupported by the system. You can delete the older association of the moved VM and create the new association to include the moved VMs in a maintenance configuration.

+> 3. Schedules triggered on machines deleted and recreated with the same resource ID within 8 hours may fail with ShutdownOrUnresponsive error due to a known limitation. It will be resolved by December, 2023.

+> Pre/Post **tasks** property is currently exposed in the API but it is not supported at this time.

+ Title: 'Azure premium storage: Design for high performance'

+description: Design high-performance apps by using Azure premium SSD managed disks. Azure premium storage offers high-performance, low-latency disk support for I/O-intensive workloads running on Azure VMs.

+# Azure premium storage: Design for high performance

+This article provides guidelines for building high-performance applications by using Azure premium storage. You can use the instructions provided in this document combined with performance best practices applicable to technologies used by your application. To illustrate the guidelines, we use SQL Server running on premium storage as an example throughout this document.

+While we address performance scenarios for the storage layer in this article, you need to optimize the application layer. For example, if you're hosting a SharePoint Farm on premium storage, you can use the SQL Server examples from this article to optimize the database server. You can also optimize the SharePoint Farm's web server and application server to get the most performance.

+This article helps to answer the following common questions about optimizing application performance on premium storage:

+* How can you measure your application performance?

+* Why aren't you seeing expected high performance?

+* Which factors influence your application performance on premium storage?

+* How do these factors influence performance of your application on premium storage?

+* How can you optimize for input/output operations per second (IOPS), bandwidth, and latency?

+We provide these guidelines specifically for premium storage because workloads running on premium storage are highly performance sensitive. We provide examples where appropriate. You can also apply some of these guidelines to applications running on infrastructure as a service (IaaS) VMs with standard storage disks.

+> Sometimes what appears to be a disk performance issue is actually a network bottleneck. In these situations, you should optimize your [network performance](../virtual-network/virtual-network-optimize-network-bandwidth.md).

+> If you're looking to benchmark your disk, see the following articles:

+> * For Windows: [Benchmark a disk](./disks-benchmarks.md)

+> If your VM supports accelerated networking, make sure it's enabled. If it's not enabled, you can enable it on already deployed VMs on both [Windows](../virtual-network/create-vm-accelerated-networking-powershell.md#enable-accelerated-networking-on-existing-vms) and [Linux](../virtual-network/create-vm-accelerated-networking-cli.md#enable-accelerated-networking-on-existing-vms).

+Before you begin, if you're new to premium storage, first read [Select an Azure disk type for IaaS VMs](disks-types.md) and [Scalability targets for premium page blob storage accounts](../storage/blobs/scalability-targets-premium-page-blobs.md).

+We assess whether an application is performing well or not by using performance indicators like:

+* How fast an application is processing a user request.

+* How much data an application is processing per request.

+* How many requests an application is processing in a specific period of time.

+* How long a user has to wait to get a response after submitting their request.

+The technical terms for these performance indicators are IOPS, throughput or bandwidth, and latency.

+In this section, we discuss the common performance indicators in the context of premium storage. In the section [Performance application checklist for disks](#performance-application-checklist-for-disks), you learn how to measure these performance indicators for your application. Later in [Optimize application performance](#optimize-application-performance), you learn about the factors that affect these performance indicators and recommendations to optimize them.

+IOPS is the number of requests that your application is sending to storage disks in one second. An input/output operation could be read or write, sequential, or random. Online transaction processing (OLTP) applications like an online retail website need to process many concurrent user requests immediately. The user requests are insert- and update-intensive database transactions, which the application must process quickly. For this reason, OLTP applications require very high IOPS.

+OLTP applications handle millions of small and random I/O requests. If you have such an application, you must design the application infrastructure to optimize for IOPS. For more information on all the factors to consider to get high IOPS, see [Optimize application performance](#optimize-application-performance).

+When you attach a premium storage disk to your high-scale VM, Azure provisions for you a guaranteed number of IOPS according to the disk specification. For example, a P50 disk provisions 7,500 IOPS. Each high-scale VM size also has a specific IOPS limit that it can sustain. For example, a Standard GS5 VM has an 80,000 IOPS limit.

+Throughput, or bandwidth, is the amount of data that your application is sending to the storage disks in a specified interval. If your application is performing input/output operations with large I/O unit sizes, it requires high throughput. Data warehouse applications tend to issue scan-intensive operations that access large portions of data at a time and commonly perform bulk operations. In other words, such applications require higher throughput. If you have such an application, you must design its infrastructure to optimize for throughput. In the next section, we discuss the factors you must tune to achieve this optimization.

+When you attach a premium storage disk to a high-scale VM, Azure provisions throughput according to that disk specification. For example, a P50 disk provisions 250 MB/sec disk throughput. Each high-scale VM size also has a specific throughput limit that it can sustain. For example, Standard GS5 VM has a maximum throughput of 2,000 MB/sec.

+There's a relation between throughput and IOPS, as shown in the following formula.

+

+It's important to determine the optimal throughput and IOPS values that your application requires. As you try to optimize one, the other is also affected. For more information about optimizing IOPS and throughput, see [Optimize application performance](#optimize-application-performance).

+Latency is the time it takes an application to receive a single request, send it to storage disks, and send the response to the client. Latency is a critical measure of an application's performance in addition to IOPS and throughput. The latency of a premium storage disk is the time it takes to retrieve the information for a request and communicate it back to your application. Premium storage provides consistently low latencies. Premium disks are designed to provide single-digit millisecond latencies for most I/O operations. If you enable **ReadOnly** host caching on premium storage disks, you can get much lower read latency. For more information on disk caching, see [Disk caching](#disk-caching).

+When you optimize your application to get higher IOPS and throughput, it affects the latency of your application. After you tune the application performance, always evaluate the latency of the application to avoid unexpected high latency behavior.

+The following control plane operations on managed disks might involve movement of the disk from one storage location to another. This movement is orchestrated via the background copy of data, which can take several hours to complete. Typically, the time is less than 24 hours depending on the amount of data in the disks. During that time, your application can experience higher than usual read latency because some reads can get redirected to the original location and take longer to complete.

+There's no effect on write latency during this period. For Premium SSD v2 and Ultra Disks, if the disk has a 4K sector size, it experiences higher read latency. If the disk has a 512e sector size, it experiences both higher read and write latency.

+Control plane operations are used to:

+## Performance application checklist for disks

+The first step in designing high-performance applications running on premium storage is understanding the performance requirements of your application. After you gather performance requirements, you can optimize your application to achieve the most optimal performance.

+In the previous section, we explained the common performance indicators: IOPS, throughput, and latency. You must identify which of these performance indicators are critical to your application to deliver the desired user experience. For example, high IOPS matters most to OLTP applications processing millions of transactions in a second. High throughput is critical for data warehouse applications processing large amounts of data in a second. Extremely low latency is crucial for real-time applications like live video-streaming websites.

+Next, measure the maximum performance requirements of your application throughout its lifetime. Use the following sample checklist as a start. Record the maximum performance requirements during normal, peak, and off-hour workload periods. By identifying requirements for all workload levels, you can determine the overall performance requirement of your application.

+For example, the normal workload of an e-commerce website is the transactions it serves during most days in a year. The peak workload of the website is the transactions it serves during holiday seasons or special sale events. The peak workload is typically experienced for a limited period but can require your application to scale two or more times its normal operation. Find out the 50 percentile, 90 percentile, and 99 percentile requirements. This information helps filter out any outliers in the performance requirements, and you can focus your efforts on optimizing for the right values.

+| Performance requirements | 50 percentile | 90 percentile | 99 percentile |

+| Maximum transactions per second | | | |

+| I/O request size | | | |

+| Average throughput | | | |

+| Maximum throughput | | | |

+| Minimum latency | | | |

+| Average latency | | | |

+| Maximum CPU | | | |

+| Maximum memory | | | |

+| Average memory | | | |

+| Queue depth | | | |

+> Consider scaling these numbers based on expected future growth of your application. It's a good idea to plan for growth ahead of time because it could be harder to change the infrastructure for improving performance later.

+If you have an existing application and want to move to premium storage, first build the preceding checklist for the existing application. Then, build a prototype of your application on premium storage and design the application based on guidelines described in [Optimize application performance](#optimize-application-performance). The next article describes the tools you can use to gather the performance measurements.

+The best way to measure performance requirements of your application is to use `PerfMon`-monitoring tools provided by the operating system of the server. You can use `PerfMon` for Windows and `iostat` for Linux. These tools capture counters corresponding to each measure explained in the preceding section. You must capture the values of these counters when your application is running its normal, peak, and off-hour workloads.

+The `PerfMon` counters are available for processor, memory, and each logical disk and physical disk of your server. When you use premium storage disks with a VM, the physical disk counters are for each premium storage disk, and logical disk counters are for each volume created on the premium storage disks. You must capture the values for the disks that host your application workload. If there's a one-to-one mapping between logical and physical disks, you can refer to physical disk counters. Otherwise, refer to the logical disk counters.

+On Linux, the `iostat` command generates a CPU and disk utilization report. The disk utilization report provides statistics per physical device or partition. If you have a database server with its data and logs on separate disks, collect this data for both disks. The following table describes counters for disks, processors, and memory.

+| IOPS or transactions/sec |Number of I/O requests issued to the storage disk/sec |Disk reads/sec <br> Disk writes/sec |tps <br> r/s <br> w/s |

+| Disk reads and writes |% of read and write operations performed on the disk |% Disk read time <br> % Disk write time |r/s <br> w/s |

+| Throughput |Amount of data read from or written to the disk/sec |Disk read bytes/sec <br> Disk write bytes/sec |kB_read/s <br> kB_wrtn/s |

+| Latency |Total time to complete a disk I/O request |Average disk sec/read <br> Average disk sec/write |await <br> svctm |

+| I/O size |The size of I/O request issues to the storage disks |Average disk bytes/read <br> Average disk bytes/write |avgrq-sz |

+| Queue depth |Number of outstanding I/O requests waiting to be read from or written to the storage disk |Current disk queue length |avgqu-sz |

+| Maximum memory |Amount of memory required to run the application smoothly |% Committed bytes in use |Use vmstat |

+| Maximum CPU |Amount of CPU required to run the application smoothly |% Processor time |%util |

+## Optimize application performance

+The main factors that influence performance of an application running on premium storage are the nature of I/O requests, VM size, disk size, number of disks, disk caching, multithreading, and queue depth. You can control some of these factors with knobs provided by the system.

+Most applications might not give you an option to alter the I/O size and queue depth directly. For example, if you're using SQL Server, you can't choose the I/O size and queue depth. SQL Server chooses the optimal I/O size and queue depth values to get the most performance. It's important to understand the effects of both types of factors on your application performance so that you can provision appropriate resources to meet performance needs.

+Throughout this section, refer to the application requirements checklist that you created to identify how much you need to optimize your application performance. Based on the checklist, you can determine which factors from this section you need to tune.

+To witness the effects of each factor on your application performance, run benchmarking tools on your application setup. For steps to run common benchmarking tools on Windows and Linux VMs, see the benchmarking articles at the end of this document.

+The following table summarizes performance factors and the steps necessary to optimize IOPS, throughput, and latency. The sections following this summary describe each factor in more depth.

+| Performance factors | IOPS | Throughput | Latency |

+| Example scenario |Enterprise OLTP application requiring very high transactions per second rate. |Enterprise Data warehousing application processing large amounts of data. |Near real-time applications requiring instant responses to user requests, like online gaming. |

+| Performance factors | &nbsp; | &nbsp; | &nbsp; |

+| I/O size |Smaller I/O size yields higher IOPS. |Larger I/O size yields higher throughput. | &nbsp;|

+| VM size |Use a VM size that offers IOPS greater than your application requirement. |Use a VM size with a throughput limit greater than your application requirement. |Use a VM size that offers scale limits greater than your application requirement. |

+| Disk size |Use a disk size that offers IOPS greater than your application requirement. |Use a disk size with a throughput limit greater than your application requirement. |Use a disk size that offers scale limits greater than your application requirement. |

+| VM and disk scale limits |IOPS limit of the VM size chosen should be greater than the total IOPS driven by the storage disks attached to it. |Throughput limit of the VM size chosen should be greater than the total throughput driven by the premium storage disks attached to it. |Scale limits of the VM size chosen must be greater than the total scale limits of the attached premium storage disks. |

+| Disk caching |Enable **ReadOnly** cache on premium storage disks with read-heavy operations to get higher read IOPS. | &nbsp; |Enable **ReadOnly** cache on premium storage disks with read-heavy operations to get very low read latencies. |

+| Disk striping |Use multiple disks and stripe them together to get a combined higher IOPS and throughput limit. The combined limit per VM should be higher than the combined limits of attached premium disks. | &nbsp; | &nbsp; |

+| Stripe size |Smaller stripe size for random small I/O pattern seen in OLTP applications. For example, use a 64-KB stripe size for a SQL Server OLTP application. |Larger stripe size for sequential large I/O pattern seen in data warehouse applications. For example, use a 256-KB stripe size for a SQL Server data warehouse application. | &nbsp; |

+| Multithreading |Use multithreading to push a higher number of requests to premium storage to lead to higher IOPS and throughput. For example, on SQL Server, set a high MAXDOP value to allocate more CPUs to SQL Server. | &nbsp; | &nbsp; |

+| Queue depth |Larger queue depth yields higher IOPS. |Larger queue depth yields higher throughput. |Smaller queue depth yields lower latencies. |

+## Nature of I/O requests

+An I/O request is a unit of input/output operation that your application is performing. Identifying the nature of I/O requests, random or sequential, read or write, small or large, helps you determine the performance requirements of your application. It's important to understand the nature of I/O requests to make the right decisions when you design your application infrastructure. I/Os must be distributed evenly to achieve the best performance possible.

+I/O size is one of the more important factors. The I/O size is the size of the input/output operation request generated by your application. The I/O size affects performance significantly, especially on the IOPS and bandwidth that the application can achieve. The following formula shows the relationship between IOPS, I/O size, and bandwidth/throughput.

+

+Some applications allow you to alter their I/O size, while some applications don't. For example, SQL Server determines the optimal I/O size itself and doesn't provide users with any knobs to change it. On the other hand, Oracle provides a parameter called [DB\_BLOCK\_SIZE](https://docs.oracle.com/cd/B19306_01/server.102/b14211/iodesign.htm#i28815), which you can use to configure the I/O request size of the database.

+If you're using an application, which doesn't allow you to change the I/O size, use the guidelines in this article to optimize the performance KPI that's most relevant to your application. For example:

+* An OLTP application generates millions of small and random I/O requests. To handle these types of I/O requests, you must design your application infrastructure to get higher IOPS.

+* A data warehousing application generates large and sequential I/O requests. To handle these types of I/O requests, you must design your application infrastructure to get higher bandwidth or throughput.

+If you're using an application that allows you to change the I/O size, use this rule of thumb for the I/O size in addition to other performance guidelines:

+* Smaller I/O size to get higher IOPS. For example, 8 KB for an OLTP application.

+* Larger I/O size to get higher bandwidth/throughput. For example, 1,024 KB for a data warehouse application.

+Here's an example of how you can calculate the IOPS and throughput/bandwidth for your application.

+Consider an application that uses a P30 disk. The maximum IOPS and throughput/bandwidth a P30 disk can achieve is 5,000 IOPS and 200 MB/sec, respectively. If your application requires the maximum IOPS from the P30 disk and you use a smaller I/O size, like 8 KB, the resulting bandwidth you can get is 40 MB/sec. If your application requires the maximum throughput/bandwidth from a P30 disk and you use a larger I/O size, like 1,024 KB, the resulting IOPS is less, such as 200 IOPS.

+Tune the I/O size so that it meets both your application's IOPS and throughput/bandwidth requirement. The following table summarizes the different I/O sizes and their corresponding IOPS and throughput for a P30 disk.

+| Application requirement | I/O size | IOPS | Throughput/Bandwidth |

+| Maximum IOPS |8 KB |5,000 |40 MB/sec |

+| Maximum throughput |1,024 KB |200 |200 MB/sec |

+| Maximum throughput + high IOPS |64 KB |3,200 |200 MB/sec |

+| Maximum IOPS + high throughput |32 KB |5,000 |160 MB/sec |

+To get IOPS and bandwidth higher than the maximum value of a single premium storage disk, use multiple premium disks striped together. For example, stripe two P30 disks to get a combined IOPS of 10,000 IOPS or a combined throughput of 400 MB/sec. As explained in the next section, you must use a VM size that supports the combined disk IOPS and throughput.

+> As you increase either IOPS or throughput, the other also increases. Make sure you don't hit throughput or IOPS limits of the disk or VM when you increase either one.

+To witness the effects of I/O size on application performance, you can run benchmarking tools on your VM and disks. Create multiple test runs and use different I/O size for each run to see the effect. For more information, see the benchmarking articles at the end of this document.

+## High-scale VM sizes

+When you start designing an application, one of the first things to do is choose a VM to host your application. Premium storage comes with high-scale VM sizes that can run applications requiring higher compute power and a high local disk I/O performance. These VMs provide faster processors, a higher memory-to-core ratio, and a solid-state drive (SSD) for the local disk. Examples of high-scale VMs supporting premium storage are the DS and GS series VMs.

+High-scale VMs are available in different sizes with a different number of CPU cores, memory, OS, and temporary disk size. Each VM size also has a maximum number of data disks that you can attach to the VM. The chosen VM size affects how much processing, memory, and storage capacity are available for your application. It also affects the compute and storage cost. For example, the following specifications are for the largest VM size in a DS series and a GS series.

+| VM size | CPU cores | Memory | VM disk sizes | Maximum data disks | Cache size | IOPS | Bandwidth cache I/O limits |

+| Standard_DS14 |16 |112 GB |OS = 1,023 GB <br> Local SSD = 224 GB |32 |576 GB |50,000 IOPS <br> 512 MB/sec |4,000 IOPS and 33 MB/sec |

+| Standard_GS5 |32 |448 GB |OS = 1,023 GB <br> Local SSD = 896 GB |64 |4224 GB |80,000 IOPS <br> 2,000 MB/sec |5,000 IOPS and 50 MB/sec |

+To view a complete list of all available Azure VM sizes, see [Sizes for virtual machines in Azure](sizes.md). Choose a VM size that can meet and scale to your desired application performance requirements. Also take into account the following important considerations when you choose VM sizes.

+### Scale limits

+The maximum IOPS limits per VM and per disk are different and independent of each other. Make sure that the application is driving IOPS within the limits of the VM and the premium disks attached to it. Otherwise, application performance experiences throttling.

+As an example, suppose an application requirement is a maximum of 4,000 IOPS. To achieve this level, you provision a P30 disk on a DS1 VM. The P30 disk can deliver up to 5,000 IOPS. However, the DS1 VM is limited to 3,200 IOPS. So, the application performance is constrained by the VM limit at 3,200 IOPS and performance is degraded. To prevent this situation, choose a VM and disk size that both meet application requirements.

+### Cost of operation

+In many cases, it's possible that your overall cost of operation using premium storage is lower than using standard storage.

+For example, consider an application requiring 16,000 IOPS. To achieve this performance, you need a Standard\_D14 Azure IaaS VM, which can give a maximum IOPS of 16,000 by using 32 standard storage 1-TB disks. Each 1-TB standard storage disk can achieve a maximum of 500 IOPS.

+- The estimated cost of this VM per month is $1,570.

+- The monthly cost of 32 standard storage disks is $1,638.

+- The estimated total monthly cost is $3,208.

+If you hosted the same application on premium storage, you need a smaller VM size and fewer premium storage disks, reducing the overall cost. A Standard\_DS13 VM can meet the 16,000 IOPS requirement by using four P30 disks. The DS13 VM has a maximum IOPS of 25,600, and each P30 disk has a maximum IOPS of 5,000. Overall, this configuration can achieve 5,000 x 4 = 20,000 IOPS.

+- The estimated cost of this VM per month is $1,003.

+- The monthly cost of four P30 premium storage disks is $544.34.

+- The estimated total monthly cost is $1,544.

+The following table summarizes the cost breakdown of this scenario for standard and premium storage.

+| Monthly cost | Standard | Premium |

+| Cost of VM per month |$1,570.58 (Standard\_D14) |$1,003.66 (Standard\_DS13) |

+| Cost of disks per month |$1,638.40 (32 x 1-TB disks) |$544.34 (4 x P30 disks) |

+| Overall cost per month |$3,208.98 |$1,544.34 |

+### Linux distros

+With premium storage, you get the same level of performance for VMs running Windows and Linux. We support many flavors of Linux distros. For more information, see [Linux distributions endorsed on Azure](linux/endorsed-distros.md).

+Different distros are better suited for different types of workloads. You see different levels of performance depending on the distro on which your workload is running. Test the Linux distros with your application and choose the one that works best.

+When you run Linux with premium storage, check the latest updates about required drivers to ensure high performance.

+Premium storage offers various sizes so you can choose one that best suits your needs. Each disk size has a different scale limit for IOPS, bandwidth, and storage. Choose the right premium storage disk size depending on the application requirements and the high-scale VM size. The following table shows the disks sizes and their capabilities. P4, P6, P15, P60, P70, and P80 sizes are currently only supported for managed disks.

+How many disks you choose depends on the disk size chosen. You could use a single P50 disk or multiple P10 disks to meet your application requirement. Take into account considerations listed here when you're making the choice.

+### Scale limits (IOPS and throughput)

+The IOPS and throughput limits of each premium disk size is different and independent from the VM scale limits. Make sure that the total IOPS and throughput from the disks are within scale limits of the chosen VM size.

+For example, if an application requirement is a maximum of 250 MB/sec throughput and you're using a DS4 VM with a single P30 disk, the DS4 VM can give up to 256 MB/sec throughput. However, a single P30 disk has a throughput limit of 200 MB/sec. So, the application is constrained at 200 MB/sec because of the disk limit. To overcome this limit, provision more than one data disk to the VM or resize your disks to P40 or P50.

+> Reads served by the cache aren't included in the disk IOPS and throughput, so they aren't subject to disk limits. Cache has its separate IOPS and throughput limit per VM.

+> For example, initially your reads and writes are 60 MB/sec and 40 MB/sec, respectively. Over time, the cache warms up and serves more and more of the reads from the cache. Then, you can get higher write throughput from the disk.

+### Number of disks

+Determine the number of disks you need by assessing application requirements. Each VM size also has a limit on the number of disks that you can attach to the VM. Typically, this amount is twice the number of cores. Ensure that the VM size you choose can support the number of disks needed.

+Remember, the premium storage disks have higher performance capabilities compared to standard storage disks. If you're migrating your application from an Azure IaaS VM using standard storage to premium storage, you likely need fewer premium disks to achieve the same or higher performance for your application.

+High-scale VMs that use premium storage have a multitier caching technology called **BlobCache**. **BlobCache** uses a combination of the host RAM and local SSD for caching. This cache is available for the premium storage persistent disks and the VM local disks. By default, this cache setting is set to **ReadWrite** for OS disks and **ReadOnly** for data disks hosted on premium storage. With disk caching enabled on the premium storage disks, the high-scale VMs can achieve extremely high levels of performance that exceed the underlying disk performance.

+> Disk caching isn't supported for disks 4 TiB and larger. If multiple disks are attached to your VM, each disk that's smaller than 4 TiB supports caching.

+> Changing the cache setting of an Azure disk detaches and reattaches the target disk. If it's the operating system disk, the VM is restarted. Stop all applications and services that might be affected by this disruption before you change the disk cache setting. Not following those recommendations could lead to data corruption.

+To learn more about how **BlobCache** works, see the Inside [Azure premium storage](https://azure.microsoft.com/blog/azure-premium-storage-now-generally-available-2/) blog post.

+It's important to enable caching on the right set of disks. Whether you should enable disk caching on a premium disk or not depends on the workload pattern that disk is handling. The following table shows the default cache settings for OS and data disks.

+| Disk type | Default cache setting |

+We recommend the following disk cache settings for data disks.

+| Disk caching setting | Recommendation for when to use this setting |

+| None |Configure host-cache as **None** for write-only and write-heavy disks. |

+| ReadOnly |Configure host-cache as **ReadOnly** for read-only and read-write disks. |

+| ReadWrite |Configure host-cache as **ReadWrite** only if your application properly handles writing cached data to persistent disks when needed. |

+### ReadOnly

+By configuring **ReadOnly** caching on premium storage data disks, you can achieve low read latency and get very high read IOPS and throughput for your application for two reasons:

+1. Reads performed from cache, which is on the VM memory and local SSD, are faster than reads from the data disk, which is on Azure Blob Storage.

+1. Premium storage doesn't count the reads served from the cache toward the disk IOPS and throughput. For this reason, your application can achieve higher total IOPS and throughput.

+### ReadWrite

+By default, the OS disks have **ReadWrite** caching enabled. We recently added support for **ReadWrite** caching on data disks too. If you're using **ReadWrite** caching, you must have a proper way to write the data from cache to persistent disks. For example, SQL Server handles writing cached data to the persistent storage disks on its own. Using **ReadWrite** cache with an application that doesn't handle persisting the required data can lead to data loss, if the VM crashes.

+### None

+Currently, **None** is only supported on data disks. It isn't supported on OS disks. If you set **None** on an OS disk, it overrides this setting internally and sets it to **ReadOnly**.

+As an example, you can apply these guidelines to SQL Server running on premium storage by following these steps:

+1. Configure the **ReadOnly** cache on premium storage disks hosting data files.

+ 1. The fast reads from cache lower the SQL Server query time because data pages are retrieved faster from the cache compared to directly from the data disks.

+ 1. Serving reads from cache means there's more throughput available from premium data disks. SQL Server can use this extra throughput toward retrieving more data pages and other operations like backup/restore, batch loads, and index rebuilds.

+1. Configure the **None** cache on premium storage disks hosting the log files.

+ 1. Log files have primarily write-heavy operations, so they don't benefit from the **ReadOnly** cache.

+For all Premium SSDs or Ultra Disks, you might be able to disable *barriers* for file systems on the disk to improve performance when it's known that there are no caches that could lose data. If Azure disk caching is set to **ReadOnly** or **None**, you can disable barriers. But if caching is set to **ReadWrite**, barriers should remain enabled to ensure write durability. Barriers are typically enabled by default, but you can disable barriers by using one of the following methods depending on the file system type:

+* **reiserFS**: Use the **barrier=none** mount option to disable barriers. To explicitly enable barriers, use **barrier=flush**.

+* **ext3/ext4**: Use the **barrier=0** mount option to disable barriers. To explicitly enable barriers, use **barrier=1**.

+* **XFS**: Use the **nobarrier** mount option to disable barriers. To explicitly enable barriers, use **barrier**. As of version 4.10 of the mainline Linux kernel, the design of the XFS file system always ensures durability. Disabling barriers has no effect and the **nobarrier** option is deprecated. However, some Linux distributions might have backported the changes to a distribution release with an earlier kernel version. Check with your distribution vendor for the status in the distribution and version that you're running.

+When a high-scale VM is attached with several premium storage persistent disks, the disks can be striped together to aggregate their IOPs, bandwidth, and storage capacity.

+On Windows, you can use Storage Spaces to stripe disks together. You must configure one column for each disk in a pool. Otherwise, the overall performance of striped volume can be lower than expected because of uneven distribution of traffic across the disks.

+By using the Server Manager UI, you can set the total number of columns up to `8` for a striped volume. When you're attaching more than eight disks, use PowerShell to create the volume. By using PowerShell, you can set the number of columns equal to the number of disks. For example, if there are 16 disks in a single stripe set, specify `16` columns in the `NumberOfColumns` parameter of the `New-VirtualDisk` PowerShell cmdlet.

+On Linux, use the MDADM utility to stripe disks together. For steps on how to stripe disks on Linux, see [Configure Software RAID on Linux](/previous-versions/azure/virtual-machines/linux/configure-raid).

+### Stripe size

+An important configuration in disk striping is the stripe size. The stripe size or block size is the smallest chunk of data that an application can address on a striped volume. The stripe size you configure depends on the type of application and its request pattern. If you choose the wrong stripe size, it could lead to I/O misalignment, which leads to degraded performance of your application.

+For example, if an I/O request generated by your application is bigger than the disk stripe size, the storage system writes it across stripe unit boundaries on more than one disk. When it's time to access that data, it has to seek across more than one stripe unit to complete the request. The cumulative effect of such behavior can lead to substantial performance degradation. On the other hand, if the I/O request size is smaller than the stripe size, and if it's random in nature, the I/O requests might add up on the same disk, causing a bottleneck and ultimately degrading the I/O performance.

+Depending on the type of workload your application is running, choose an appropriate stripe size. For random small I/O requests, use a smaller stripe size. For large sequential I/O requests, use a larger stripe size. Find out the stripe size recommendations for the application you'll be running on premium storage. For SQL Server, configure a stripe size of 64 KB for OLTP workloads and 256 KB for data warehousing workloads. For more information, see [Performance best practices for SQL Server on Azure VMs](/azure/azure-sql/virtual-machines/windows/performance-guidelines-best-practices-checklist).

+## Multithreading

+Azure designed the premium storage platform to be massively parallel. For this reason, a multithreaded application achieves higher performance than a single-threaded application. A multithreaded application splits up its tasks across multiple threads and increases efficiency of its execution by utilizing the VM and disk resources to the maximum.

+For example, if your application is running on a single core VM using two threads, the CPU can switch between the two threads to achieve efficiency. While one thread is waiting on a disk I/O to complete, the CPU can switch to the other thread. In this way, two threads can accomplish more than a single thread would. If the VM has more than one core, it further decreases running time because each core can run tasks in parallel.

+You might not be able to change the way an off-the-shelf application implements single threading or multithreading. For example, SQL Server is capable of handling multi-CPU and multicore. However, SQL Server decides under what conditions it uses one or more threads to process a query. It can run queries and build indexes by using multithreading. For a query that involves joining large tables and sorting data before returning to the user, SQL Server likely uses multiple threads. A user can't control whether SQL Server runs a query by using a single thread or multiple threads.

+There are configuration settings that you can alter to influence the multithreading or parallel processing of an application. For example, for SQL Server it's the `max degree of parallelism` configuration. This setting called MAXDOP allows you to configure the maximum number of processors SQL Server can use when parallel processing. You can configure MAXDOP for individual queries or index operations. This capability is beneficial when you want to balance resources of your system for a performance critical application.

+For example, say your application that's using SQL Server is running a large query and an index operation at the same time. Let's assume that you wanted the index operation to be more performant compared to the large query. In such a case, you can set the MAXDOP value of the index operation to be higher than the MAXDOP value for the query. This way, SQL Server has more processors than it can use for the index operation compared to the number of processors it can dedicate to the large query. Remember, you don't control the number of threads that SQL Server uses for each operation. You can control the maximum number of processors being dedicated for multithreading.

+Learn more about [degrees of parallelism](/previous-versions/sql/sql-server-2008-r2/ms188611(v=sql.105)) in SQL Server. Find out how such settings influence multithreading in your application and their configurations to optimize performance.

+The queue depth or queue length or queue size is the number of pending I/O requests in the system. The value of queue depth determines how many I/O operations your application can line up, which the storage disks process. It affects all three application performance indicators discussed in this article: IOPS, throughput, and latency.

+Queue depth and multithreading are closely related. The queue depth value indicates how much multithreading can be achieved by the application. If the queue depth is large, the application can run more operations concurrently, in other words, more multithreading. If the queue depth is small, even though the application is multithreaded, it won't have enough requests lined up for concurrent execution.

+Typically, off-the-shelf applications don't allow you to change queue depth, because if it's set incorrectly, it does more harm than good. Applications set the right value of queue depth to get the optimal performance. It's important to understand this concept so that you can troubleshoot performance issues with your application. You can also observe the effects of queue depth by running benchmarking tools on your system.

+Some applications provide settings to influence the queue depth. For example, the MAXDOP setting in SQL Server explained in the previous section. MAXDOP is a way to influence queue depth and multithreading, although it doesn't directly change the queue depth value of SQL Server.

+### High queue depth

+A high queue depth lines up more operations on the disk. The disk knows the next request in its queue ahead of time. So, the disk can schedule operations ahead of time and process them in an optimal sequence. Because the application is sending more requests to the disk, the disk can process more parallel I/Os. Ultimately, the application can achieve higher IOPS. Because the application is processing more requests, the total throughput of the application also increases.

+Typically, an application can achieve maximum throughput with 8 to 16+ outstanding I/Os per attached disk. If a queue depth is one, the application isn't pushing enough I/Os to the system, and it processes a smaller amount in a given period. In other words, less throughput.

+For example, in SQL Server, setting the MAXDOP value for a query to `4` informs SQL Server that it can use up to four cores to run the query. SQL Server determines the best queue depth value and the number of cores for the query execution.

+### Optimal queue depth

+A very high queue depth value also has its drawbacks. If the queue depth value is too high, the application tries to drive very high IOPS. Unless the application has persistent disks with sufficient provisioned IOPS, a very high queue depth value can negatively affect application latencies. The following formula shows the relationship between IOPS, latency, and queue depth.

+

+You shouldn't configure queue depth to any high value, but to an optimal value, which can deliver enough IOPS for the application without affecting latencies. For example, if the application latency needs to be 1 millisecond, the queue depth required to achieve 5,000 IOPS is QD = 5,000 x 0.001 = 5.

+### Queue depth for striped volume

+For a striped volume, maintain a high-enough queue depth so that every disk has a peak queue depth individually. For example, consider an application that pushes a queue depth of `2` and there are four disks in the stripe. The two I/O requests go to two disks and the remaining two disks are idle. Therefore, configure the queue depth so that all the disks can be busy. The following formula shows how to determine the queue depth of striped volumes.

+

+Premium storage provisions a specified number of IOPS and throughput depending on the VM sizes and disk sizes you choose. Anytime your application tries to drive IOPS or throughput above these limits of what the VM or disk can handle, premium storage throttles it. The result is degraded performance in your application, which can mean higher latency, lower throughput, or lower IOPS.

+If premium storage doesn't throttle, your application could completely fail by exceeding what its resources are capable of achieving. To avoid performance issues because of throttling, always provision sufficient resources for your application. Take into consideration what we discussed in the previous VM sizes and disk sizes sections. Benchmarking is the best way to figure out what resources you need to host your application.

+If you're looking to benchmark your disk, see the following articles:

+* For Windows: [Benchmark a disk](./disks-benchmarks.md)

+For SQL Server users, see the articles on performance best practices for SQL Server:

+* [Performance best practices for SQL Server in Azure VMs](/azure/azure-sql/virtual-machines/windows/performance-guidelines-best-practices-checklist)

+* [Azure premium storage provides highest performance for SQL Server in Azure VM](https://cloudblogs.microsoft.com/sqlserver/2015/04/23/azure-premium-storage-provides-highest-performance-for-sql-server-in-azure-vm/)

+ Title: Copy VM restore points to another region

+description: Learn how to copy virtual machine (VM) restore points to another region.

+# Cross-region copy of VM restore points

+In this article, you learn how to copy virtual machine (VM) restore points to another region.

+To copy a restore point across a region, you need to precreate a `restorePointCollection` resource in the target region.

+Learn more about [cross-region copy and its limitation](virtual-machines-restore-points-copy.md) before you copy restore points.

+### Create a restore point collection in a target region

+The first step in copying an existing VM restore point from one region to another is to create a `restorePointCollection` resource in the target region by referencing `restorePointCollection` from the source region.

+#### URI request

+#### Request body

+The request response includes a status code and a set of response headers.

+The operation returns 201 during creation and 200 during the update.

+### Create a VM restore point in a target region

+The next step is to trigger the copy of a restore point in the target `RestorePointCollection` resource by referencing the restore point in the source region that needs to be copied.

+> [!NOTE]

+> The location of `sourceRestorePoint` is inferred from that of the source `RestorePointCollection`.

+The request response includes a status code and a set of response headers.

+##### Status code

+This operation is long running, so the operation returns 201 during creation. The client is expected to poll for the status by using the operation. Both the `location` and `Azure-AsyncOperation` headers are provided for this purpose.

+During restore point creation, `ProvisioningState` appears as `Creating` in the GET restore point API response. If creation fails, `ProvisioningState` appears as `Failed`. `ProvisioningState` is set to `Succeeded` when the data copy across regions is initiated.

+> [!NOTE]

+> You can track the copy status by calling GET instance view (`?$expand=instanceView`) on the target VM restore point. For steps on how to do this, see the section "Get the VM restore points Copy/Replication status." The VM restore point is considered usable (can be used to restore a VM) only when a copy of all the disk restore points are successful.

+### Get the VM restore points Copy/Replication status

+After the copy of VM restore points is initiated, you can track the copy status by calling the GET instance view (`?$expand=instanceView`) on the target VM restore point.

+#### URI request

+- [Learn more](backup-recovery.md) about backup and restore options for VMs in Azure.

+ Title: Use virtual machine restore points

+description: Learn how to use virtual machine restore points.

+Business continuity and disaster recovery solutions are primarily designed to address sitewide data loss. Solutions that operate at this scale often manage and run across automated failovers and failbacks in multiple regions. You can use Azure virtual machine (VM) restore points to implement granular backup and retention policies.

+You can protect your data and guard against extended downtime by creating VM restore points at regular intervals. There are several backup options available for VMs, depending on your use case. For more information, see [Backup and restore options for VMs in Azure](backup-recovery.md).

+An individual VM restore point is a resource that stores VM configuration and point-in-time application-consistent snapshots of all the managed disks attached to the VM. You can use VM restore points to easily capture multidisk-consistent backups. VM restore points contain a disk restore point for each of the attached disks. A disk restore point consists of a snapshot of an individual managed disk.

+VM restore points support both application consistency and crash consistency (in preview). Fill out this [form](https://forms.office.com/r/LjLBt6tJRL) if you want to try crash-consistent restore points in preview.

+Application consistency is supported for VMs running Windows operating systems and support file system consistency for VMs running Linux operating systems. Application-consistent restore points use Volume Shadow Copy Service (VSS) writers (or pre- and postscripts for Linux) to ensure the consistency of the application data before a restore point is created. To get an application-consistent restore point, the application running in the VM needs to provide a VSS writer (for Windows) or pre- and postscripts (for Linux) to achieve application consistency.

+A multidisk crash-consistent VM restore point stores the VM configuration and point-in-time write-order-consistent snapshots for all managed disks attached to a VM. This information is the same as the status of data in the VM after a power outage or a crash. The `consistencyMode` optional parameter has to be set to `crashConsistent` in the creation request. This feature is currently in preview.

+> For disks configured with read/write host caching, multidisk crash consistency can't be guaranteed because writes that occur while the snapshot is taken might not be acknowledged by Azure Storage. If maintaining consistency is crucial, we recommend that you use the application-consistency mode.

+VM restore points are organized into restore point collections. A restore point collection is an Azure Resource Manager resource that contains the restore points for a specific VM. If you want to utilize Azure Resource Manager templates (ARM templates) for creating restore points and restore point collections, see the public [Virtual-Machine-Restore-Points](https://github.com/Azure/Virtual-Machine-Restore-Points) repository in GitHub.

+VM restore points are incremental. The first restore point stores a full copy of all disks attached to the VM. For each successive restore point for a VM, only the incremental changes to your disks are backed up. To reduce your costs, you can optionally exclude any disk when you create a restore point for your VM.

+## Restore points for VMs inside virtual machine scale set and availability set (AvSet)

+Currently, you can create restore points in only one VM at a time. You can't create a single restore point across multiple VMs. Because of this limitation, we currently support creating restore points for individual VMs with a virtual machine scale set in Flexible Orchestration mode or an availability set. If you want to back up instances within a virtual machine scale set instance or your availability set instance, you must individually create restore points for all the VMs that are part of the instance.

+> [!NOTE]

+> A virtual machine scale set with Uniform orchestration isn't supported by restore points. You can't create restore points of VMs inside a virtual machine scale set with Uniform orchestration.

+- Restore points are supported only for managed disks.

+- Ultra-disks, ephemeral OS disks, and shared disks aren't supported.

+- The API version for an application-consistent restore point is March 1, 2021, or later.

+- The API version for a crash-consistent restore point is July 1, 2021, or later (in preview).

+- A maximum of 500 VM restore points can be retained at any time for a VM, irrespective of the number of restore point collections.

+- Concurrent creation of restore points for a VM isn't supported.

+- Restore points for virtual machine scale sets in Uniform orchestration mode aren't supported.

+- Movement of VMs between resource groups or subscriptions isn't supported when the VM has restore points. Moving the VM between resource groups or subscriptions doesn't update the source VM reference in the restore point and causes a mismatch of Resource Manager IDs between the actual VM and the restore points.

+ > [!NOTE]

+ > Public preview of cross-region copying of VM restore points is available, with the following limitations:

+ >

+ > - Private links aren't supported when you copy restore points across regions or create restore points in a region other than the source VM.

+ > - Customer-managed key encrypted restore points, when copied to a target region, are created as platform-managed key encrypted restore points.

+Most common restore point failures are attributed to the communication with the VM agent and extension. To resolve failures, follow the steps in [Troubleshoot restore point failures](restore-point-troubleshooting.md).

+- [Learn more](backup-recovery.md) about backup and restore options for VMs in Azure.

+- [Learn more](virtual-machines-restore-points-copy.md) about how to copy VM restore points across regions.

+ Title: Use cross-region copy of virtual machine restore points

+description: Learn how to use cross-region copy of virtual machine (VM) restore points.

+# Overview of cross-region copy of VM restore points (in preview)

+Azure virtual machine (VM) restore point APIs are a lightweight option you can use to implement granular backup and retention policies. VM restore points support application consistency and crash consistency (in preview). You can copy a VM restore point from one region to another region. This capability can help you build business continuity and disaster recovery solutions for Azure VMs.

+* Extend multiple copies of restore points to different regions.

+* Extend local restore point solutions to support disaster recovery from region failures.

+> To copy a restore point across a region, you need to precreate a `RestorePointCollection` resource in the target region.

+* Private links aren't supported when you copy restore points across regions or create restore points in a region other than the source VM.

+* Azure confidential VMs aren't supported.

+* The API version for a cross-region copy of the VM restore point feature is 2021-03-01, or later.

+* Copy of a copy isn't supported. You can't copy a restore point that was already copied from another region. For example, if you copied RP1 from East US to West US as RRP1, you can't copy RRP1 from West US to another region (or back to East US).

+* Multiple copies of the same restore point in a single target region aren't supported. A single restore point in the source region can be copied only once to a target region.

+* Copying a restore point that's CMK encrypted in the source is encrypted by using CMK in the target region. This feature is currently in preview.

+* The target restore point only shows the creation time when the source restore point was created.

+* Currently, the replication progress is updated every 10 minutes. For disks that have low churn, there can be scenarios where only the initial (0) and the final replication progress (100) appear.

+* The maximum copy time supported is two weeks. For a huge amount of data to be copied to a target region, depending on the bandwidth available between the regions, the copy time could be a couple of days. If the copy time exceeds two weeks, the copy operation is terminated automatically.

+* No error details are provided when a disk restore point copy fails.

+* When a disk restore point copy fails, the intermediate completion percentage where the copy failed isn't shown.

+* Restoring a disk from the restore point doesn't automatically check if the disk restore point replication is completed. You need to manually check the percent completion if the replication status is 100% and then start restoring the disk.

+* Restore points that are copied to the target region don't have a reference to the source VM. They have a reference to the source restore points. So, if the source restore point is deleted, there's no way to identify the source VM by using the target restore points.

+* Copying restore points in a nonsequential order isn't supported. For example, you might have the three restore points RP1, RP2, and RP3. If you already successfully copied RP1 and RP3, you aren't allowed to copy RP2.

+* The full snapshot on the source side should always exist and can't be deleted to save cost. For example, if RP1 (full snapshot), RP2 (incremental), and RP3 (incremental) exist in the source and are successfully copied to the target, you can delete RP2 and RP3 on the source side to save cost. Deleting RP1 on the source side results in creating a full snapshot, say RRP1, the next time, and copying also results in a full snapshot. The storage layer maintains the relationship with each pair of source and target snapshot that needs to be preserved.

+Most common restore point failures are attributed to the communication with the VM agent and extension. To resolve failures, follow the steps in [Troubleshoot restore point failures](restore-point-troubleshooting.md).

+- [Learn more](backup-recovery.md) about backup and restore options for VMs in Azure.

+Application-consistent restore points use the Volume Shadow Copy Service (VSS) service (or pre-/postscripts for Linux) to verify application data consistency prior to creating a restore point. Achieving an application-consistent restore point involves the VM's running application providing a VSS service (for Windows) or pre- and postscripts (for Linux).

+For Windows images, the **VMSnapshot Windows** extension, and for Linux images, the **VMSnapshot Linux** extension, are used for taking application consistent restore points. When there's a create application consistent restore point request issued from a VM, Azure installs the VM snapshot extension if it's not already present. The extension is updated automatically.

+> Azure begins creating a restore point only after all extensions (including but not limited to VMSnapshot) provisioning state are complete.

+You can view logs for the VMSnapshot extension on the VM under

+```C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.RecoveryServices.VMSnapshot``` for Windows and under ```/var/log/azure/Microsoft.Azure.RecoveryServices.VMSnapshotLinux/extension.log``` for Linux.

+Most common restore point failures are attributed to the communication with the VM agent and extension. To resolve failures, follow the steps in [Troubleshoot restore point failures](restore-point-troubleshooting.md).

+During certain VSS writer failure, Azure takes file system-consistent restore points for the next three times (irrespective of the frequency at which the restore point creation is scheduled) upon failing the initial creation request. From the fourth time onward, an application-consistent restore point is attempted.

+> Avoid manually deleting the extension because it leads to failure of the subsequent creation of an application-consistent restore point.

+[Create a VM restore point](create-restore-points.md)