+This article assumes that you have an existing AKS 1.21 or above cluster. If you need an AKS cluster, see the AKS quickstart [using the Azure CLI][aks-quickstart-cli] or [using the Azure portal][aks-quickstart-portal].

+ csi:

+ driver: file.csi.azure.com

+ volumeAttributes:

+ secretName: azure-secret # required

+ shareName: aksshare # required

+ mountOptions: "dir_mode=0777,file_mode=0777,cache=strict,actimeo=30" # optional

+You now have a running pod with an Azure Files share mounted at */mnt/azure*. You can use `kubectl describe pod mypod` to verify the share is mounted successfully.

+> The default value for *fileMode* and *dirMode* is *0777* for Kubernetes version 1.15 and above.

+ persistentVolumeReclaimPolicy: Retain

+ csi:

+ driver: file.csi.azure.com

+ volumeHandle: unique-volumeid # make sure this volumeid is unique in the cluster

+ volumeAttributes:

+ resourceGroup: EXISTING_RESOURCE_GROUP_NAME # optional, only set this when storage account is not in the same resource group as agent node

+ shareName: aksshare

+ nodeStageSecretRef:

+ name: azure-secret

+ namespace: default

+ - dir_mode=0777

+ - file_mode=0777

+ - uid=0

+ - gid=0

+ - mfsymlinks

+ - cache=strict

+ - nosharesock

+For Azure File CSI driver parameters, see [CSI driver parameters][CSI driver parameters].

+For information about AKS 1.20 or below clusters interact with Azure Files, see the [Kubernetes plugin for Azure Files][kubernetes-files].

+For associated best practices, see [Best practices for storage and backups in AKS][operator-best-practices-storage].

+[CSI driver parameters]: https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/docs/driver-parameters.md#static-provisionbring-your-own-file-share

+[persistent-volume-example]: #mount-file-share-as-a-persistent-volume

+ Title: Import an SAP API using the Azure portal | Microsoft Docs

+description: Learn how to import OData metadata from SAP as an API to Azure API Management

+# Import SAP OData metadata as an API

+This article shows how to import an OData service using its metadata description. In this article, [SAP Gateway](https://help.sap.com/viewer/product/SAP_GATEWAY) serves as an example. However, you can apply the approach to any OData-compliant service.

+In this article, you'll:

+> [!div class="checklist"]

+> * Convert OData metadata to an OpenAPI specification

+> * Import the OpenAPI specification to API Management

+> * Complete API configuration

+> * Test the API in the Azure portal

+## Prerequisites

+- An existing API Management instance. [Create one if you haven't already](get-started-create-service-instance.md).

+- An SAP system and service exposed as OData v2 or v4.

+- If your SAP backend uses a self-signed certificate (for test purposes), you may need to disable the verification of the trust chain for SSL. To do so, configure a [backend](backends.md) in your API Management instance:

+ 1. In the Azure portal, under **APIs**, select **Backends** > **+ Add**.

+ 1. Add a **Custom URL** pointing to the SAP backend service.

+ 1. Uncheck **Validate certificate chain** and **Validate certificate name**.

+ > [!NOTE]

+ > For production scenarios, use proper certificates for end-to-end SSL verification.

+## Convert OData metadata to OpenAPI JSON

+1. Retrieve metadata XML from your SAP service. Use one of these methods:

+ * Use the SAP Gateway Client (transaction `/IWFND/GW_CLIENT`), or

+ * Make a direct HTTP call to retrieve the XML:

+ `http://<OData server URL>:<port>/<path>/$metadata`.

+1. Convert the OData XML to OpenAPI JSON format. Use an OASIS open-source tool for [OData v2](https://github.com/oasis-tcs/odata-openapi/tree/main/tools) or [OData v4](https://github.com/oasis-tcs/odata-openapi/tree/main/lib), depending on your metadata XML.

+ The following is an example command to convert OData v2 XML for the test service `epm_ref_apps_prod_man_srv`:

+ ```console

+ odata-openapi -p --basePath '/sap/opu/odata/sap/epm_ref_apps_prod_man_srv' \

+ --scheme https --host <your IP address>:<your SSL port> \

+ ./epm_ref_apps_prod_man_srv.xml

+ ```

+ > [!NOTE]

+ > * For test purposes with a single XML file, you can use a [web-based converter](https://aka.ms/ODataOpenAPI) based on the open-source tool.

+ > * With the tool or the web-based converter, specifying the \<IP address>:\<port> of your SAP OData server is optional. Alternatively, add this information later in your generated OpenAPI specification or after importing to API Management.

+1. Save the `openapi-spec.json` file locally for import to API Management.

+## Import and publish backend API

+1. From the side navigation menu, under the **APIs** section, select **APIs**.

+1. Under **Create a new definition**, select **OpenAPI specification**.

+ :::image type="content" source="./media/import-api-from-oas/oas-api.png" alt-text="OpenAPI specifiction":::

+1. Click **Select a file**, and select the `openapi-spec.json` file that you saved locally in a previous step.

+1. Enter API settings. You can set the values during creation or configure them later by going to the **Settings** tab.

+ * In **API URL suffix**, we recommend using the same URL path as in the original SAP service.

+ * For more information about API settings, see [Import and publish your first API](import-and-publish.md#import-and-publish-a-backend-api) tutorial.

+1. Select **Create**.

+> [!NOTE]

+> The API import limitations are documented in [another article](api-management-api-import-restrictions.md).

+## Complete API configuration

+[Add](add-api-manually.md#add-and-test-an-operation) the following three operations to the API that you imported.

+- `GET /$metadata`

+ |Operation |Description |Further configuration for operation |

+ ||||

+ |`GET /$metadata` | Enables API Management to reach the `$metadata` endpoint, which is required for client integration with the OData server.<br/><br/>This required operation isn't included in the OpenAPI specification that you generated and imported. | Add a `200 OK` response. |

+ :::image type="content" source="media/sap-api/get-metadata-operation.png" alt-text="Get metadata operation":::

+- `HEAD /`

+ |Operation |Description |Further configuration for operation |

+ ||||

+ |`HEAD /` | Enables the client to exchange Cross Site Request Forgery (CSRF) tokens with the SAP server, when required.<br/><br/>SAP also allows CSRF token exchange using the GET verb.<br/><br/> CSRF token exchange isnΓÇÖt covered in this article. See an example API Management [policy snippet](https://github.com/Azure/api-management-policy-snippets/blob/master/examples/Get%20X-CSRF%20token%20from%20SAP%20gateway%20using%20send%20request.policy.xml) to broker token exchange. | N/A |

+ :::image type="content" source="media/sap-api/head-root-operation.png" alt-text="Operation to fetch tokens":::

+- `GET /`

+ Operation |Description |Further configuration for operation |

+ ||||

+ |`GET /` | Enables policy configuration at service root. | Configure the following inbound [rewrite-uri](api-management-transformation-policies.md#RewriteURL) policy to append a trailing slash to requests that are forwarded to service root:<br/><br> `<rewrite-uri template="/" copy-unmatched-params="true" />` <br/><br/>This policy removes potential ambiguity of requests with or without trailing slashes, which are treated differently by some backends.|

+ :::image type="content" source="media/sap-api/get-root-operation.png" alt-text="Get operation for service root":::

+Also, configure authentication to your backend using an appropriate method for your environment. For examples, see [API Management authentication policies](api-management-authentication-policies.md).

+## Test your API

+1. Navigate to your API Management instance.

+1. From the side navigation menu, under the **APIs** section, select **APIs**.

+1. Under **All APIs**, select your imported API.

+1. Select the **Test** tab to access the test console.

+1. Select an operation, enter any required values, and select **Send**.

+ For example, test the `GET /$metadata` call to verify connectivity to the SAP backend

+1. View the response. To troubleshoot, [trace](api-management-howto-api-inspector.md) the call.

+1. When testing is complete, exit the test console.

+## Production considerations

+* See an [example end-to-end scenario](https://blogs.sap.com/2021/08/12/.net-speaks-odata-too-how-to-implement-azure-app-service-with-sap-odata-gateway/) to integrate API Management with an SAP gateway.

+* Control access to an SAP backend using API Management policies. See policy snippets for [SAP principal propagation](https://github.com/Azure/api-management-policy-snippets/blob/master/examples/Request%20OAuth2%20access%20token%20from%20SAP%20using%20AAD%20JWT%20token.xml) and [fetching an X-CSRF token](https://github.com/Azure/api-management-policy-snippets/blob/master/examples/Get%20X-CSRF%20token%20from%20SAP%20gateway%20using%20send%20request.policy.xml).

+* For guidance to deploy, manage, and migrate APIs at scale, see:

+ * [Automated API deployments with APIOps](/architecture/example-scenario/devops/automated-api-deployments-apiops)

+ * [CI/CD for API Management using Azure Resource Manager templates](devops-api-development-templates.md).

+## Next steps

+> [!div class="nextstepaction"]

+> [Transform and protect a published API](transform-api.md)

+- Must have CNAME mapped _directly_ to \<app-name\>.azurewebsites.net; using services that proxy the CNAME value will block certificate issuance and renewal

+ Title: 'Quickstart: Deploy a Python web app to Azure App Service'

+description: Get started with Azure App Service by deploying your first Python app to Azure App Service.

+# Quickstart: Deploy a Python (Django or Flask) web app to Azure App Service

+In this quickstart, you will deploy a Python web app (Django or Flask) to [Azure App Service](/azure/app-service/overview#app-service-on-linux). Azure App Service is a fully managed web hosting service that supports Python 3.7 and higher apps hosted in a Linux server environment.

+To complete this quickstart, you need:

+1. An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs&utm_campaign=visualstudio).

+1. <a href="https://www.python.org/downloads/" target="_blank">Python 3.9 or higher</a> installed locally.

+## 1 - Sample application

+This quickstart can be completed using either Flask or Django. A sample application in each framework is provided to help you follow along with this quickstart. Download or clone the sample application to your local workstation.

+### [Flask](#tab/flask)

+```Console

+git clone https://github.com/Azure-Samples/msdocs-python-flask-webapp-quickstart

+### [Django](#tab/django)

+```Console

+git clone https://github.com/Azure-Samples/msdocs-python-django-webapp-quickstart

+To run the application locally:

+### [Flask](#tab/flask)

+1. Navigate into in the application folder:

+ ```Console

+ cd msdocs-python-flask-webapp-quickstart

+ ```

+1. Create a virtual environment for the app:

+ [!INCLUDE [Virtual environment setup](<./includes/quickstart-python/virtual-environment-setup.md>)]

+1. Install the dependencies:

+ ```Console

+ pip install -r requirements.txt

+ ```

+1. Run the app:

+ ```Console

+ flask run

+ ```

+1. Browse to the sample application at `http://localhost:5000` in a web browser.

+ :::image type="content" source="./media/quickstart-python/run-flask-app-localhost.png" alt-text="Screenshot of the Flask app running locally in a browser":::

+### [Django](#tab/django)

+1. Navigate into in the application folder:

+ ```Console

+ cd msdocs-python-django-webapp-quickstart

+1. Create a virtual environment for the app:

+ [!INCLUDE [Virtual environment setup](<./includes/quickstart-python/virtual-environment-setup.md>)]

+1. Install the dependencies:

+ ```Console

+ pip install -r requirements.txt

+ ```

+1. Run the app:

+ ```Console

+ python manage.py runserver

+1. Browse to the sample application at `http://localhost:8000` in a web browser.

+ :::image type="content" source="./media/quickstart-python/run-django-app-localhost.png" alt-text="Screenshot of the Django app running locally in a browser":::

+Having issues? [Let us know](https://aka.ms/FlaskCLIQuickstartHelp).

+## 2 - Create a web app in Azure

+To host your application in Azure, you need to create Azure App Service web app in Azure. You can create a web app using the [Azure portal](https://portal.azure.com/), VS Code using the [Azure Tools extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack), or the Azure CLI.

+### [Azure portal](#tab/azure-portal)

+Sign in to the [Azure portal](https://portal.azure.com/) and follow these steps to create your Azure App Service resources.

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-azure-portal-1.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-azure-portal-1-240px.png" alt-text="A screenshot showing how to use the search box in the top tool bar to find App Services in Azure." lightbox="./media/quickstart-python/create-app-service-azure-portal-1.png"::: |

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-azure-portal-2.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-azure-portal-2-240px.png" alt-text="A screenshot showing the location of the Create button on the App Services page in the Azure portal." lightbox="./media/quickstart-python/create-app-service-azure-portal-2.png"::: |

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-azure-portal-3.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-azure-portal-3-240px.png" alt-text="A screenshot showing how to fill out the form to create a new App Service in the Azure portal." lightbox="./media/quickstart-python/create-app-service-azure-portal-3.png"::: |

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-azure-portal-4.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-azure-portal-4-240px.png" alt-text="A screenshot showing how to select the basic app service plan in the Azure portal." lightbox="./media/quickstart-python/create-app-service-azure-portal-4.png"::: |

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-azure-portal-5.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-azure-portal-5-240px.png" alt-text="A screenshot showing the location of the Review plus Create button in the Azure portal." lightbox="./media/quickstart-python/create-app-service-azure-portal-5.png"::: |

+### [VS Code](#tab/vscode-aztools)

+To create Azure resources in VS Code, you must have the [Azure Tools extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack) installed and be signed into Azure from VS Code.

+> [!div class="nextstepaction"]

+> [Download Azure Tools extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack)

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Create app service step 1](<./includes/quickstart-python/create-app-service-visual-studio-code-1.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-visual-studio-code-1-240px.png" alt-text="A screenshot showing the location of the Azure Tools icon in the left toolbar of VS Code." lightbox="./media/quickstart-python/create-app-service-visual-studio-code-1.png"::: |

+| [!INCLUDE [Create app service step 2](<./includes/quickstart-python/create-app-service-visual-studio-code-2.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-visual-studio-code-2-240px.png" alt-text="A screenshot showing the App Service section of Azure Tools extension and the context menu used to create a new web app." lightbox="./media/quickstart-python/create-app-service-visual-studio-code-2.png"::: |

+| [!INCLUDE [Create app service step 4](<./includes/quickstart-python/create-app-service-visual-studio-code-3.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-visual-studio-code-3-240px.png" alt-text="A screenshot of dialog box used to enter the name of the new web app in Visual Studio Code." lightbox="./media/quickstart-python/create-app-service-visual-studio-code-3.png"::: |

+| [!INCLUDE [Create app service step 5](<./includes/quickstart-python/create-app-service-visual-studio-code-4.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-visual-studio-code-4-240px.png" alt-text="A screenshot of the dialog box in VS Code used to select the runtime for the new web app." lightbox="./media/quickstart-python/create-app-service-visual-studio-code-4.png"::: |

+| [!INCLUDE [Create app service step 6](<./includes/quickstart-python/create-app-service-visual-studio-code-5.md>)] | :::image type="content" source="./media/quickstart-python/create-app-service-visual-studio-code-5-240px.png" alt-text="A screenshot of the dialog in VS Code used to select the App Service plan for the new web app." lightbox="./media/quickstart-python/create-app-service-visual-studio-code-5.png"::: |

+### [Azure CLI](#tab/azure-cli)

+## 3 - Deploy your application code to Azure

+Azure App service supports multiple methods to deploy your application code to Azure including support for GitHub Actions and all major CI/CD tools. This article focuses on how to deploy your code from your local workstation to Azure.

+### [Deploy using VS Code](#tab/vscode-deploy)

+To deploy a web app from VS Code, you must have the [Azure Tools extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack) installed and be signed into Azure from VS Code.

+> [!div class="nextstepaction"]

+> [Download Azure Tools extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack)

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [VS Code deploy step 1](<./includes/quickstart-python/deploy-visual-studio-code-1.md>)] | :::image type="content" source="./media/quickstart-python/deploy-visual-studio-code-1-240px.png" alt-text="A screenshot showing the location of the Azure Tools icon in the left toolbar of VS Code." lightbox="./media/quickstart-python/deploy-visual-studio-code-1.png"::: |

+| [!INCLUDE [VS Code deploy step 2](<./includes/quickstart-python/deploy-visual-studio-code-2.md>)] | :::image type="content" source="./media/quickstart-python/deploy-visual-studio-code-2-240px.png" alt-text="A screenshot showing the context menu of an App Service and the deploy to web app menu option." lightbox="./media/quickstart-python/deploy-visual-studio-code-2.png"::: |

+| [!INCLUDE [VS Code deploy step 3](<./includes/quickstart-python/deploy-visual-studio-code-3.md>)] | :::image type="content" source="./media/quickstart-python/deploy-visual-studio-code-3-240px.png" alt-text="A screenshot dialog in VS Code used to choose the app to deploy." lightbox="./media/quickstart-python/deploy-visual-studio-code-3.png"::: |

+| [!INCLUDE [VS Code deploy step 4](<./includes/quickstart-python/deploy-visual-studio-code-4.md>)] | :::image type="content" source="./media/quickstart-python/deploy-visual-studio-code-4-240px.png" alt-text="A screenshot of a dialog box in VS Code asking if you want to update your workspace to run build commands." lightbox="./media/quickstart-python/deploy-visual-studio-code-4.png"::: |

+| [!INCLUDE [VS Code deploy step 5](<./includes/quickstart-python/deploy-visual-studio-code-5.md>)] | :::image type="content" source="./media/quickstart-python/deploy-visual-studio-code-5-240px.png" alt-text="A screenshot showing the confirmation dialog when the app code has been deployed to Azure." lightbox="./media/quickstart-python/deploy-visual-studio-code-5.png"::: |

+### [Deploy using Local Git](#tab/local-git-deploy)

+### [Deploy using a ZIP file](#tab/zip-deploy)

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/FlaskCLIQuickstartHelp).

+## 4 - Browse to the app

+Browse to the deployed application in your web browser at the URL `http://<app-name>.azurewebsites.net`. It can take a minute or two for the app to start, so if you see a default app page, wait a minute and refresh the browser.

+The Python sample code is running a Linux container in App Service using a built-in image.

+**Congratulations!** You have deployed your Python app to App Service.

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/FlaskCLIQuickstartHelp).

+## 5 - Stream logs

+Azure App Service captures all messages output to the console to assist you in diagnosing issues with your application. The sample apps include `print()` statements to demonstrate this capability.

+### [Flask](#tab/flask)

+### [Django](#tab/django)

+The contents of the App Service diagnostic logs can be reviewed in the Azure portal, VS Code, or using the Azure CLI.

+### [Azure portal](#tab/azure-portal)

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Stream logs from Azure portal 1](<./includes/quickstart-python/stream-logs-azure-portal-1.md>)] | :::image type="content" source="./media/quickstart-python/stream-logs-azure-portal-1-240px.png" alt-text="A screenshot showing the location in the Azure portal where to enable streaming logs." lightbox="./media/quickstart-python/stream-logs-azure-portal-1.png"::: |

+| [!INCLUDE [Stream logs from Azure portal 2](<./includes/quickstart-python/stream-logs-azure-portal-2.md>)] | :::image type="content" source="./media/quickstart-python/stream-logs-azure-portal-2-240px.png" alt-text="A screenshot showing how to view logs in the Azure portal." lightbox="./media/quickstart-python/stream-logs-azure-portal-2.png"::: |

+### [VS Code](#tab/vscode-aztools)

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Stream logs from VS Code 1](<./includes/quickstart-python/stream-logs-visual-studio-code-1.md>)] | :::image type="content" source="./media/quickstart-python/stream-logs-vs-code-1-240px.png" alt-text="A screenshot showing how to start streaming logs with the VS Code extension." lightbox="./media/quickstart-python/stream-logs-vs-code-1.png"::: |

+| [!INCLUDE [Stream logs from VS Code 2](<./includes/quickstart-python/stream-logs-visual-studio-code-2.md>)] | :::image type="content" source="./media/quickstart-python/stream-logs-vs-code-2-240px.png" alt-text="A screenshot showing an example of streaming logs in the VS Code Output window." lightbox="./media/quickstart-python/stream-logs-vs-code-2.png"::: |

+### [Azure CLI](#tab/azure-cli)

+First, you need to configure Azure App Service to output logs to the App Service filesystem using the [az webapp log config](/cli/azure/webapp/log#az_webapp_log_config) command.

+To stream logs, use the [az webapp log tail](/cli/azure/webapp/log#az_webapp_log_tail) command.

+Refresh the home page in the app or attempt other requests to generate some log messages. The output should look similar to the following.

+```Output

+Starting Live Log Stream

+2021-12-23T02:15:52.740703322Z Request for index page received

+2021-12-23T02:15:52.740740222Z 169.254.130.1

+2021-12-23T02:15:52.841043070Z 169.254.130.1

+2021-12-23T02:15:52.884541951Z 169.254.130.1

+2021-12-23T02:15:53.043211176Z 169.254.130.1

+2021-12-23T02:16:01.304306845Z Request for hello page received with name=David

+2021-12-23T02:16:01.304335945Z 169.254.130.1

+2021-12-23T02:16:01.398399251Z 169.254.130.1

+2021-12-23T02:16:01.430740060Z 169.254.130.1

+```

+Having issues? Refer first to the [Troubleshooting guide](/azure/app-service/configure-language-python.md#troubleshooting), otherwise, [let us know](https://aka.ms/FlaskCLIQuickstartHelp).

+When you are finished with the sample app, you can remove all of the resources for the app from Azure to ensure you do not incur additional charges and keep your Azure subscription uncluttered. Removing the resource group also removes all resources in the resource group and is the fastest way to remove all Azure resources for your app.

+### [Azure portal](#tab/azure-portal)

+Follow these steps while signed-in to the Azure portal to delete a resource group.

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Remove resource group Azure portal 1](<./includes/quickstart-python/remove-resource-group-azure-portal-1.md>)] | :::image type="content" source="./media/quickstart-python/remove-resource-group-azure-portal-1-240px.png" alt-text="A screenshot showing how to search for and navigate to a resource group in the Azure portal." lightbox="./media/quickstart-python/remove-resource-group-azure-portal-1.png"::: |

+| [!INCLUDE [Remove resource group Azure portal 2](<./includes/quickstart-python/remove-resource-group-azure-portal-2.md>)] | :::image type="content" source="./media/quickstart-python/remove-resource-group-azure-portal-2-240px.png" alt-text="A screenshot showing the location of the Delete Resource Group button in the Azure portal." lightbox="./media/quickstart-python/remove-resource-group-azure-portal-2.png"::: |

+| [!INCLUDE [Remove resource group Azure portal 3](<./includes/quickstart-python/remove-resource-group-azure-portal-3.md>)] | :::image type="content" source="./media/quickstart-python/remove-resource-group-azure-portal-3-240px.png" alt-text="A screenshot of the confirmation dialog for deleting a resource group in the Azure portal." lightbox="./media/quickstart-python/remove-resource-group-azure-portal-3.png"::: |

+### [VS Code](#tab/vscode-aztools)

+| Instructions | Screenshot |

+|:-|--:|

+| [!INCLUDE [Remove resource group VS Code 1](<./includes/quickstart-python/remove-resource-group-visual-studio-code-1.md>)] | :::image type="content" source="./media/quickstart-python/remove-resource-group-visual-studio-code-1-240px.png" alt-text="A screenshot showing how to delete a resource group in VS Code using the Azure Tools extension." lightbox="./media/quickstart-python/remove-resource-group-visual-studio-code-1.png"::: |

+| [!INCLUDE [Remove resource group VS Code 2](<./includes/quickstart-python/remove-resource-group-visual-studio-code-2.md>)] | :::image type="content" source="./media/quickstart-python/remove-resource-group-visual-studio-code-2-240px.png" alt-text="A screenshot of the confirmation dialog for deleting a resource group from VS Code." lightbox="./media/quickstart-python/remove-resource-group-visual-studio-code-2.png"::: |

+### [Azure CLI](#tab/azure-cli)

+Delete the resource group by using the [az group delete](/cli/azure/group#az_group_delete) command.

+az group delete \

+ --name msdocs-python-webapp-quickstart \

+ --no-wait

+> [Tutorial: Python (Django) web app with PostgreSQL](/azure/app-service/tutorial-python-postgresql-app.md)

+> [Configure Python app](/azure/app-service/configure-language-python.md)

+> [Add user sign-in to a Python web app](/azure/active-directory/develop/quickstart-v2-python-webapp.md)

+> [Tutorial: Run Python app in custom container](/azure/app-service/tutorial-custom-container.md)

+1. On the Form Recognizer Studio home page, select **Identity documents**

+

+> [!NOTE]

+> To be able to publish your isolated function project to either a Windows or a Linux function app in Azure, you must set a value of `dotnet-isolated` in the remote [FUNCTIONS_WORKER_RUNTIME](functions-app-settings.md#functions_worker_runtime) application setting.

+- [Review ToDo API sample with Azure SQL bindings](/samples/azure-samples/azure-sql-binding-func-dotnet-todo/todo-backend-dotnet-azure-sql-bindings-azure-functions/)

+- [Learn how to connect Azure Function to Azure SQL with managed identity](./functions-identity-access-azure-sql-with-managed-identity.md)

+ Title: Connect a function app to Azure SQL with managed identity and SQL bindings

+description: Learn how to connect Azure SQL bindings through managed identity.

+#Customer intent: As a function developer, I want to learn how to use managed identities so that I can avoid having to handle connection strings in my application settings.

+# Tutorial: Connect a function app to Azure SQL with managed identity and SQL bindings

+Azure Functions provides a [managed identity](/azure/active-directory/managed-identities-azure-resources/overview.md), which is a turn-key solution for securing access to [Azure SQL Database](/azure/sql-database/) and other Azure services. Managed identities make your app more secure by eliminating secrets from your app, such as credentials in the connection strings. In this tutorial, you'll add managed identity to an Azure Function that utilizes [Azure SQL bindings](/azure/azure-functions/functions-bindings-azure-sql). A sample Azure Function project with SQL bindings is available in the [ToDo backend example](/samples/azure-samples/azure-sql-binding-func-dotnet-todo/todo-backend-dotnet-azure-sql-bindings-azure-functions/)).

+When you're finished with this tutorial, your Azure Function will connect to Azure SQL database without the need of username and password.

+An overview of the steps you'll take:

+> [!div class="checklist"]

+> * [Enable Azure AD authentication to the SQL database](#grant-database-access-to-azure-ad-user)

+> * [Enable Azure Function managed identity](#enable-system-assigned-managed-identity-on-azure-function)

+> * [Grant SQL Database access to the managed identity](#grant-sql-database-access-to-the-managed-identity)

+> * [Configure Azure Function SQL connection string](#configure-azure-function-sql-connection-string)

+## Grant database access to Azure AD user

+First enable Azure AD authentication to SQL database by assigning an Azure AD user as the Active Directory admin of the server. This user is different from the Microsoft account you used to sign up for your Azure subscription. It must be a user that you created, imported, synced, or invited into Azure AD. For more information on allowed Azure AD users, see [Azure AD features and limitations in SQL database](../azure-sql/database/authentication-aad-overview.md#azure-ad-features-and-limitations).

+Enabling Azure AD authentication can be completed via the Azure portal, PowerShell, or Azure CLI. Directions for Azure CLI are below and information completing this via Azure portal and PowerShell is available in the [Azure SQL documentation on Azure AD authentication](/azure/azure-sql/database/authentication-aad-configure).

+1. If your Azure AD tenant doesn't have a user yet, create one by following the steps at [Add or delete users using Azure Active Directory](../active-directory/fundamentals/add-users-azure-active-directory.md).

+1. Find the object ID of the Azure AD user using the [`az ad user list`](/cli/azure/ad/user#az_ad_user_list) and replace *\<user-principal-name>*. The result is saved to a variable.

+ ```azurecli-interactive

+ azureaduser=$(az ad user list --filter "userPrincipalName eq '<user-principal-name>'" --query [].objectId --output tsv)

+ ```

+ > [!TIP]

+ > To see the list of all user principal names in Azure AD, run `az ad user list --query [].userPrincipalName`.

+ >

+1. Add this Azure AD user as an Active Directory admin using [`az sql server ad-admin create`](/cli/azure/sql/server/ad-admin#az_sql_server_ad_admin_create) command in the Cloud Shell. In the following command, replace *\<server-name>* with the server name (without the `.database.windows.net` suffix).

+ ```azurecli-interactive

+ az sql server ad-admin create --resource-group myResourceGroup --server-name <server-name> --display-name ADMIN --object-id $azureaduser

+ ```

+For more information on adding an Active Directory admin, see [Provision an Azure Active Directory administrator for your server](../azure-sql/database/authentication-aad-configure.md#provision-azure-ad-admin-sql-database)

+## Enable system-assigned managed identity on Azure Function

+In this step we'll add a system-assigned identity to the Azure Function. In later steps, this identity will be given access to the SQL database.

+To enable system-assigned managed identity in the Azure portal:

+1. Create an Azure Function in the portal as you normally would. Navigate to it in the portal.

+1. Scroll down to the Settings group in the left navigation.

+1. Select Identity.

+1. Within the System assigned tab, switch Status to On. Click Save.

+

+For information on enabling system-assigned managed identity through Azure CLI or PowerShell, check out more information on [using managed identities with Azure Functions](/azure/app-service/overview-managed-identity?toc=%2Fazure%2Fazure-functions%2Ftoc.json&tabs=dotnet#add-a-system-assigned-identity).

+## Grant SQL database access to the managed identity

+In this step we'll connect to the SQL database with an Azure AD user account and grant the managed identity access to the database.

+1. Open your preferred SQL tool and login with an Azure AD user account (such as the Azure AD user we assigned as administrator). This can be accomplished in Cloud Shell with the SQLCMD command.

+ ```bash

+ sqlcmd -S <server-name>.database.windows.net -d <db-name> -U <aad-user-name> -P "<aad-password>" -G -l 30

+ ```

+1. In the SQL prompt for the database you want, run the following commands to grant permissions to your function. For example,

+ ```sql

+ CREATE USER [<identity-name>] FROM EXTERNAL PROVIDER;

+ ALTER ROLE db_datareader ADD MEMBER [<identity-name>];

+ ALTER ROLE db_datawriter ADD MEMBER [<identity-name>];

+ GO

+ ```

+ *\<identity-name>* is the name of the managed identity in Azure AD. If the identity is system-assigned, the name is always the same as the name of your Function app.

+## Configure Azure Function SQL connection string

+In the final step we'll configure the Azure Function SQL connection string to use Azure AD managed identity authentication.

+The connection string setting name is identified in our Functions code as the binding attribute "ConnectionStringSetting", as seen in the SQL input binding [attributes and annotations](/azure/azure-functions/functions-bindings-azure-sql-input?tabs=csharp#attributes-and-annotations).

+In the application settings of our Function App the SQL connection string setting should be updated to follow this format:

+`Server=demo.database.windows.net; Authentication=Active Directory Managed Identity; Database=testdb`

+*testdb* is the name of the database we're connecting to and *demo.database.windows.net* is the name of the server we're connecting to.

+## Next steps

+- [Read data from a database (Input binding)](./functions-bindings-azure-sql-input.md)

+- [Save data to a database (Output binding)](./functions-bindings-azure-sql-output.md)

+- [Review ToDo API sample with Azure SQL bindings](/samples/azure-samples/azure-sql-binding-func-dotnet-todo/todo-backend-dotnet-azure-sql-bindings-azure-functions/)

+1. Create data collection endpoint(s) using these [DCE REST APIs](/cli/azure/monitor/data-collection/endpoint).

+The four new preview features provided with AzAcSnap v5.1 are:

+- Backint coexistence

+This section explains how to enable communication with storage. Ensure the storage back-end you're using is correctly selected.

+ connection string and instead use an alias. If the connection information changes, it's a matter of changing the `tnsnames.ora` file instead

+ 1. Run the following commands on the Oracle Database Server.

+ 1. Create the Linux user to generate the Oracle Wallet and associated `*.ora` files using the output from the previous step.

+ > In these examples we are using the `bash` shell. If you're using a different shell (for example, csh), then ensure environment variables have been set correctly.

+## Backint coexistence

+> Support for coexistence with SAP HANA's Backint interface is a Preview feature.

+1. force a log backup flush to backint.

+1. wait for running backups to complete.

+1. disable the backint-based backup.

+1. put SAP HANA into a consistent state for backup.

+1. take a storage snapshot-based backup.

+1. release SAP HANA.

+JSON configuration file (for example, `azacsnap.json`). It's also possible to change this value by editing the configuration file directly.

+[Azure Storage types for SAP workload](/azure/virtual-machines/workloads/sap/planning-guide-storage) web page. Additionally there's a

+> [!CAUTION]

+> Take extra care to configure AzAcSnap with the correct mountpoints (filesystems) because `xfs_freeze` blocks I/O to the device specified by the Azure Managed Disk

+> mount-point. This could inadvertently block a running application until `azacsnap` finishes running.

+1. Azure Managed Disks attached to the VM using the Azure portal.

+1. Install and Configure AzAcSnap.

+ 1. Create an executable file called $HOME/bin/xfs_freeze with the following content.

+ > In this example we run each command twice to show it worked the first time as there's no command to confirm if `xfs_freeze` has frozen I/O.

+Here's an example config file, note the hierarchy for the dataVolume, mountpoint, azureManagedDisks:

+ 1. Using `dmesg`:

+ 1. Using `pvscan`:

+1. Mount the logical volume as the `root` user:

+1. Then access the data:

+- `$azSnapshotName` = the snapshot name generated by azacsnap.

+> There's only a value for `$azSnapshotName` in the `--runafter` option.

+The snapshots need to be mounted on the system doing the copy, with at a minimum read-only privilege. The base location of the mount point for the snapshots should

+5. From the Condition tab, select **Add condition**. From there, find a signal called ΓÇ£**is volume replication healthy**ΓÇ¥.

+6. There you'll see **Condition of the relationship, 1 or 0** and the **Configure Signal Logic** window is displayed.

+ * Set **Operator** to `Less than or equal to`.

+ * Set **Aggregation type** to `Average`.

+ * Set **Threshold** value to `0`.

+ * Set **Unit** to `Count`.

+8. To check if the replication is _healthy_:

+ * Set **Operator** to `Greater than or equal to`.

+ * Set **Aggregation** type to `Average`.

+ * Set **Threshold** value to `1`.

+ * Set **Unit** to `Count`.

+Cloud Shell entry points beside the Azure portal, such as Visual Studio Code & Windows Terminal, do not support various Cloud Shell functionalities:

+- Use of commands that modify UX components in Cloud Shell, such as `Code`

+- Fetching non-arm access tokens

+Individual container apps are deployed to a single Container Apps environment, which acts as a secure boundary around groups of container apps. Container Apps in the same environment are deployed in the same virtual network and write logs to the same Log Analytics workspace. You may provide an [existing virtual network](vnet-custom.md) when you create an environment.

+## Prerequisites

+- Azure account with an active subscription.

+ - If you don't have one, you [can create one for free](https://azure.microsoft.com/free/).

+- Install the [Azure CLI](/cli/azure/install-azure-cli).

+## Prerequisites

+- Azure account with an active subscription.

+ - If you don't have one, you [can create one for free](https://azure.microsoft.com/free/).

+- Install the [Azure CLI](/cli/azure/install-azure-cli).

+- [**Provide an existing virtual network**](vnet-custom.md) when creating an environment for your container apps.

+<!-- Create -->

+7. Select the **Create** button at the bottom of the *Create Container App Environment* page.

+<!-- Deploy the container app -->

+ Title: Provide a virtual network to an Azure Container Apps Preview environment

+description: Learn how to provide a VNET to an Azure Container Apps environment.

+zone_pivot_groups: azure-cli-or-portal

+# Provide a virtual network to an Azure Container Apps (Preview) environment

+As you create an Azure Container Apps [environment](environment.md), a virtual network (VNET) is created for you, or you can provide your own. Network addresses are assigned from a subnet range you define as the environment is created.

+- You control the subnet range used by the Container Apps environment.

+- Once the environment is created, the subnet range is immutable.

+- A single load balancer and single Kubernetes service are associated with each container apps environment.

+- Each [revision pod](revisions.md) is assigned an IP address in the subnet.

+- You can restrict inbound requests to the environment exclusively to the VNET by deploying the environment as internal.

+> [!IMPORTANT]

+> In order to ensure the environment deployment within your custom VNET is successful, configure your VNET with an "allow-all" configuration by default. The full list of traffic dependencies required to configure the VNET as "deny-all" is not yet available. Refer to the [custom VNET security sample](https://aka.ms/azurecontainerapps/customvnet) for additional details.

+## Subnet types

+As a Container Apps environment is created, you provide resource IDs for two different subnets. Both subnets must be defined in the same container apps.

+- **App subnet**: Subnet for user app containers. Subnet that contains IP ranges mapped to applications deployed as containers.

+- **Control plane subnet**: Subnet for [control plane infrastructure](/azure/azure-resource-manager/management/control-plane-and-data-plane) components and user app containers.

+If the [platformReservedCidr](#networking-parameters) range is defined, both subnets must not overlap with the IP range defined in `platformReservedCidr`.

+## Accessibility level

+You can deploy your Container Apps environment with an internet-accessible endpoint or with an IP address in your VNET. The accessibility level determines the type of load balancer used with your Container Apps instance.

+### External

+Container Apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address.

+### Internal

+When set to internal, the environment has no public endpoint. Internal environments are deployed with a virtual IP (VIP) mapped to an internal IP address. The internal endpoint is an Azure internal load balancer (ILB) and IP addresses are issued from the custom VNET's list of private IP addresses.

+To create an internal only environment, provide the `--internal-only` parameter to the `az containerapp env create` command.

+## Example

+The following example shows you how to create a Container Apps environment in an existing virtual network.

+<!-- Create -->

+7. Select the **Networking** tab to create a VNET.

+8. Select **Yes** next to *Use your own virtual network*.

+9. Next to the *Virtual network* box, select the **Create new** link.

+10. Enter **my-custom-vnet** in the name box.

+11. Select the **OK** button.

+12. Next to the *Control plane subnet* box, select the **Create new** link and enter the following values:

+ | Setting | Value |

+ |||

+ | Subnet name | Enter **my-control-plane-vnet**. |

+ | Virtual Network Address Block | Keep the default values. |

+ | Subnet Address Block | Keep the default values. |

+13. Select the **OK** button.

+14. Next to the *Control plane subnet* box, select the **Create new** link and enter the following values:

+ | Setting | Value |

+ |||

+ | Subnet name | Enter **my-apps-vnet**. |

+ | Virtual Network Address Block | Keep the default values. |

+ | Subnet Address Block | Keep the default values. |

+15. Under *Virtual IP*, select **External**.

+16. Select **Create**.

+<!-- Deploy -->

+## Prerequisites

+- Azure account with an active subscription.

+ - If you don't have one, you [can create one for free](https://azure.microsoft.com/free/).

+- Install the [Azure CLI](/cli/azure/install-azure-cli) version 2.28.0 or higher.

+Next, declare a variable to hold the VNET name.

+# [Bash](#tab/bash)

+```bash

+VNET_NAME="my-custom-vnet"

+```

+# [PowerShell](#tab/powershell)

+```powershell

+$VNET_NAME="my-custom-vnet"

+```

+Now create an instance of the virtual network to associate with the Container Apps environment. The virtual network must have two subnets available for the container apps instance.

+> [!NOTE]

+> You can use an existing virtual network, but two empty subnets are required to use with Container Apps.

+# [Bash](#tab/bash)

+```azurecli

+az network vnet create \

+ --resource-group $RESOURCE_GROUP \

+ --name $VNET_NAME \

+ --location $LOCATION \

+ --address-prefix 10.0.0.0/16

+```

+```azurecli

+az network vnet subnet create \

+ --resource-group $RESOURCE_GROUP \

+ --vnet-name $VNET_NAME \

+ --name control-plane \

+ --address-prefixes 10.0.0.0/21

+```

+```azurecli

+az network vnet subnet create \

+ --resource-group $RESOURCE_GROUP \

+ --vnet-name $VNET_NAME \

+ --name applications \

+ --address-prefixes 10.0.8.0/21

+```

+# [PowerShell](#tab/powershell)

+```powershell

+az network vnet create `

+ --resource-group $RESOURCE_GROUP `

+ --name $VNET_NAME `

+ --location $LOCATION `

+ --address-prefix 10.0.0.0/16

+```

+```powershell

+az network vnet subnet create `

+ --resource-group $RESOURCE_GROUP `

+ --vnet-name $VNET_NAME `

+ --name control-plane `

+ --address-prefixes 10.0.0.0/21

+```

+```powershell

+az network vnet subnet create `

+ --resource-group $RESOURCE_GROUP `

+ --vnet-name $VNET_NAME `

+ --name applications `

+ --address-prefixes 10.0.8.0/21

+```

+With the VNET established, you can now query for the VNET, control plane, and app subnet IDs.

+# [Bash](#tab/bash)

+```bash

+VNET_RESOURCE_ID=`az network vnet show --resource-group ${RESOURCE_GROUP} --name ${VNET_NAME} --query "id" -o tsv | tr -d '[:space:]'`

+```

+```bash

+CONTROL_PLANE_SUBNET=`az network vnet subnet show --resource-group ${RESOURCE_GROUP} --vnet-name $VNET_NAME --name control-plane --query "id" -o tsv | tr -d '[:space:]'`

+```

+```bash

+APP_SUBNET=`az network vnet subnet show --resource-group ${RESOURCE_GROUP} --vnet-name ${VNET_NAME} --name applications --query "id" -o tsv | tr -d '[:space:]'`

+```

+# [PowerShell](#tab/powershell)

+```powershell

+$VNET_RESOURCE_ID=(az network vnet show --resource-group $RESOURCE_GROUP --name $VNET_NAME --query "id" -o tsv)

+```

+```powershell

+$CONTROL_PLANE_SUBNET=(az network vnet subnet show --resource-group $RESOURCE_GROUP --vnet-name $VNET_NAME --name control-plane --query "id" -o tsv)

+```

+```powershell

+$APP_SUBNET=(az network vnet subnet show --resource-group $RESOURCE_GROUP --vnet-name $VNET_NAME --name applications --query "id" -o tsv)

+```

+Finally, create the Container Apps environment with the VNET and subnets.

+# [Bash](#tab/bash)

+```azurecli

+az containerapp env create \

+ --name $CONTAINERAPPS_ENVIRONMENT \

+ --resource-group $RESOURCE_GROUP \

+ --logs-workspace-id $LOG_ANALYTICS_WORKSPACE_CLIENT_ID \

+ --logs-workspace-key $LOG_ANALYTICS_WORKSPACE_CLIENT_SECRET \

+ --location "$LOCATION" \

+ --app-subnet-resource-id $APP_SUBNET \

+ --controlplane-subnet-resource-id $CONTROL_PLANE_SUBNET

+```

+# [PowerShell](#tab/powershell)

+```powershell

+az containerapp env create `

+ --name $CONTAINERAPPS_ENVIRONMENT `

+ --resource-group $RESOURCE_GROUP `

+ --logs-workspace-id $LOG_ANALYTICS_WORKSPACE_CLIENT_ID `

+ --logs-workspace-key $LOG_ANALYTICS_WORKSPACE_CLIENT_SECRET `

+ --location "$LOCATION" `

+ --app-subnet-resource-id $APP_SUBNET `

+ --controlplane-subnet-resource-id $CONTROL_PLANE_SUBNET

+```

+The following table describes the parameters used in for `containerapp env create`.

+| Parameter | Description |

+|||

+| `name` | Name of the container apps environment. |

+| `resource-group` | Name of the resource group. |

+| `logs-workspace-id` | The ID of the Log Analytics workspace. |

+| `logs-workspace-key` | The Log Analytics client secret. |

+| `location` | The Azure location where the environment is to deploy. |

+| `app-subnet-resource-id` | The resource ID of a subnet where containers are injected into the container app. This subnet must be in the same VNET as the subnet defined in `--control-plane-subnet-resource-id`. |

+| `controlplane-subnet-resource-id` | The resource ID of a subnet for control plane infrastructure components. This subnet must be in the same VNET as the subnet defined in `--app-subnet-resource-id`. |

+| `internal-only` | Optional parameter that scopes the environment to IP addresses only available the custom VNET. |

+With your environment created with your custom-virtual network, you can create container apps into the environment using the `az containerapp create` command.

+### Optional configuration

+You have the option of deploying a private DNS and defining custom networking IP ranges for your Container Apps environment.

+#### Deploy with a private DNS

+If you want to deploy your container app with a private DNS, run the following commands.

+First, extract identifiable information from the environment.

+# [Bash](#tab/bash)

+```bash

+ENVIRONMENT_DEFAULT_DOMAIN=`az containerapp env show --name ${CONTAINERAPPS_ENVIRONMENT} --resource-group ${RESOURCE_GROUP} --query defaultDomain --out json | tr -d '"'`

+```

+```bash

+ENVIRONMENT_STATIC_IP=`az containerapp env show --name ${CONTAINERAPPS_ENVIRONMENT} --resource-group ${RESOURCE_GROUP} --query staticIp --out json | tr -d '"'`

+```

+```bash

+VNET_ID=`az network vnet show --resource-group ${RESOURCE_GROUP} --name ${VNET_NAME} --query id --out json | tr -d '"'`

+```

+# [PowerShell](#tab/powershell)

+```powershell

+$ENVIRONMENT_DEFAULT_DOMAIN=(az containerapp env show --name $CONTAINERAPPS_ENVIRONMENT --resource-group $RESOURCE_GROUP --query defaultDomain -o tsv)

+```

+```powershell

+$ENVIRONMENT_STATIC_IP=(az containerapp env show --name $CONTAINERAPPS_ENVIRONMENT --resource-group $RESOURCE_GROUP --query staticIp -o tsv)

+```

+```powershell

+$VNET_ID=(az network vnet show --resource-group $RESOURCE_GROUP --name $VNET_NAME --query id -o tsv)

+```

+Next, set up the private DNS.

+# [Bash](#tab/bash)

+```azurecli

+az network private-dns zone create \

+ --resource-group $RESOURCE_GROUP \

+ --name $ENVIRONMENT_DEFAULT_DOMAIN

+```

+```azurecli

+az network private-dns link vnet create \

+ --resource-group $RESOURCE_GROUP \

+ --name $VNET_NAME \

+ --virtual-network $VNET_ID \

+ --zone-name $ENVIRONMENT_DEFAULT_DOMAIN -e true

+```

+```azurecli

+az network private-dns record-set a add-record \

+ --resource-group $RESOURCE_GROUP \

+ --record-set-name "*" \

+ --ipv4-address $ENVIRONMENT_STATIC_IP \

+ --zone-name $ENVIRONMENT_DEFAULT_DOMAIN

+```

+# [PowerShell](#tab/powershell)

+```powershell

+az network private-dns zone create `

+ --resource-group $RESOURCE_GROUP `

+ --name $ENVIRONMENT_DEFAULT_DOMAIN

+```

+```powershell

+az network private-dns link vnet create `

+ --resource-group $RESOURCE_GROUP `

+ --record-set-name $VNET_NAME `

+ --virtual-network $VNET_ID `

+ --zone-name $ENVIRONMENT_DEFAULT_DOMAIN -e true

+```

+```powershell

+az network private-dns record-set a add-record `

+ --resource-group $RESOURCE_GROUP `

+ --name "*" `

+ --ipv4-address $ENVIRONMENT_STATIC_IP `

+ --zone-name $ENVIRONMENT_DEFAULT_DOMAIN

+```

+#### Networking parameters

+There are three optional networking parameters you can choose to define when calling `containerapp env create`. You must either provide values for all three of these properties, or none of them. If they arenΓÇÖt provided, the CLI generates the values for you.

+| Parameter | Description |

+|||

+| `platform-reserved-cidr` | The address range used internally for environment infrastructure services. Must have a size between `/21` and `/12`. |

+| `platform-reserved-dns-ip` | An IP address from the `platform-reserved-cidr` range that is used for the internal DNS server. The address can't be the first address in the range, or the network address. For example, if `platform-reserved-cidr` is set to `10.2.0.0/16`, then `platform-reserved-dns-ip` can't be `10.2.0.0` (this is the network address), or `10.2.0.1` (infrastructure reserves use of this IP). In this case, the first usable IP for the DNS would be `10.2.0.2`. |

+| `docker-bridge-cidr` | The address range assigned to the Docker bridge network. This range must have a size between `/28` and `/12`. |

+- The `platform-reserved-cidr` and `docker-bridge-cidr` address ranges can't conflict with each other, or with the ranges of either provided subnet. Further, make sure these ranges don't conflict with any other address range in the VNET.

+- If these properties arenΓÇÖt provided, the CLI autogenerates the range values based on the address range of the VNET to avoid range conflicts.

+## Clean up resources

+If you're not going to continue to use this application, you can delete the Azure Container Apps instance and all the associated services by removing the **my-container-apps** resource group.

+# [Bash](#tab/bash)

+```azurecli

+az group delete \

+ --name $RESOURCE_GROUP

+```

+# [PowerShell](#tab/powershell)

+```azurecli

+az group delete `

+ --name $RESOURCE_GROUP

+```

+## Restrictions

+Subnet address ranges can't overlap with the following reserved ranges:

+- 169.254.0.0/16

+- 172.30.0.0/16

+- 172.31.0.0/16

+- 192.0.2.0/24

+Additionally, subnets must have a size between /21 and /12.

+## Additional resources

+- Refer to [What is Azure Private Endpoint](/azure/private-link/private-endpoint-overview) for more details on configuring your private endpoint.

+- To set up DNS name resolution for internal services, you must [set up your own DNS server](/azure/dns/).

+## Next steps

+> [!div class="nextstepaction"]

+> [Managing autoscaling behavior](scale-app.md)

+| Azure Container Instances | [Deploy to Azure Container Instances from Azure Container Registry using a managed identity](../container-instances/using-azure-container-registry-mi.md) | Yes, either system-assigned or user-assigned identity |

+Auto-Sync refers to the fully managed capability of Azure Cosmos DB where the inserts, updates, deletes to operational data are automatically synced from transactional store to analytical store in near real time. Auto-sync latency is usually within 2 minutes. In cases of shared throughput database with a large number of containers, auto-sync latency of individual containers could be higher and take up to 5 minutes. We would like to learn more how this latency fits your scenarios. For that, please reach out to the [Azure Cosmos DB Team](mailto:cosmosdbsynapselink@microsoft.com).

+ * If your documents have 10 levels with 400 properties in each one, only the two first levels will be fully represented in analytical store. Half of the third level will also be represented.

+ * Currently Azure Cosmos DB API for MongoDB isn't compatible with this possibility of changing the schema representation. All MongoDB accounts will always have full fidelity schema representation type.

+ * From `NULL` to any other data type. The first non-null occurrence defines the column data type. Any document not following the first non-null datatype won't be represented in analytical store.

+ > The condition above doesn't apply for `NULL` properties. For example, `{"a":123} and {"a":NULL}` is still well-defined.

+* Expect different behavior in regard to explicit `NULL` values:

+|NULL | ".NULL" | NULL|

+* Expect different behavior in regard to explicit `NULL` values:

+## <a id="analytical-ttl"></a> Analytical Time-to-Live (TTL)

+Analytical TTL (ATTL) indicates how long data should be retained in your analytical store, for a container.

+Analytical store is enabled when ATTL is set with a value other than `NULL` and `0`. When enabled, inserts, updates, deletes to operational data are automatically synced from transactional store to analytical store, irrespective of the transactional TTL (TTTL) configuration. The retention of this transactional data in analytical store can be controlled at container level by the `AnalyticalStoreTimeToLiveInSeconds` property.

+The possible ATTL configurations are:

+* If the value is set to `0` or set to `NULL`: the analytical store is disabled and no data is replicated from transactional store to analytical store

+* If the value is set to `-1`: the analytical store retains all historical data, irrespective of the retention of the data in the transactional store. This setting indicates that the analytical store has infinite retention of your operational data

+* If the value is set to any positive integer `n` number: items will expire from the analytical store `n` seconds after their last modified time in the transactional store. This setting can be leveraged if you want to retain your operational data for a limited period of time in the analytical store, irrespective of the retention of the data in the transactional store

+Some points to consider:

+* After the analytical store is enabled with an ATTL value, it can be updated to a different valid value later.

+* While TTTL can be set at the container or item level, ATTL can only be set at the container level currently.

+* You can achieve longer retention of your operational data in the analytical store by setting ATTL >= TTTL at the container level.

+* The analytical store can be made to mirror the transactional store by setting ATTL = TTTL.

+* If you have ATTL bigger than TTTL, at some point in time you'll have data that only exists in analytical store. This data is read only.

+How to enable analytical store on a container:

+* From the Azure portal, the ATTL option, when turned on, is set to the default value of -1. You can change this value to 'n' seconds, by navigating to container settings under Data Explorer.

+

+* From the Azure Management SDK, Azure Cosmos DB SDKs, PowerShell, or Azure CLI, the ATTL option can be enabled by setting it to either -1 or 'n' seconds.

+To learn more, see [how to configure analytical TTL on a container](configure-synapse-link.md#create-analytical-ttl).

+The data that was only in analytical store isn't restored, but it will be kept available for queries as long as you keep the original container. Analytical store is only deleted when you delete the container. Your analytical queries in Azure Synapse Analytics can read data from both original and restored container's analytical stores. Example:

+This article shows the Azure Cosmos DB Emulator released versions and it details the updates that were made. Only the latest version is made available to download and use and previous versions aren't actively supported by the Azure Cosmos DB Emulator developers.

+### 2.14.5 (January 18, 2022)

+ - This release updates the Azure Cosmos DB Emulator background services to match the latest online functionality of the Azure Cosmos DB. One other important update with this release is to reduce the number of services executed in the background and start them as needed.

+### 2.14.4 (October 25, 2021)

+ - This release updates the Azure Cosmos DB Emulator background services to match the latest online functionality of the Azure Cosmos DB.

+### 2.14.3 (September 8, 2021)

+ - This release updates the Azure Cosmos DB Emulator background services to match the latest online functionality of the Azure Cosmos DB. It also addresses couple issues with telemetry data that is collected and resets the base image for the Linux Cosmos emulator Docker image.

+### 2.14.2 (August 12, 2021)

+ - This release updates the local Data Explorer content to latest Microsoft Azure version and resets the base for the Linux Cosmos emulator Docker image.

+### 2.14.1 (June 18, 2021)

+### 2.14.0 (June 15, 2021)

+ - This release updates the local Data Explorer content to latest Microsoft Azure version. It also fixes an issue when importing multiple document items by using the JSON file upload feature.

+### 2.11.13 (April 21, 2021)

+ - This release updates the local Data Explorer content to latest Microsoft Azure version and adds a new MongoDB endpoint configuration, "4.0".

+### 2.11.11 (February 22, 2021)

+ - This release updates the local Data Explorer content to latest Microsoft Azure version.

+### 2.11.10 (January 5, 2021)

+ - This release updates the local Data Explorer content to latest Microsoft Azure version. It also adds a new public option, "/ExportPemCert", which allows the emulator user to directly export the public emulator's certificate as a .PEM file.

+### 2.11.9 (December 3, 2020)

+ - This release updates the Azure Cosmos DB Emulator background services to match the latest online functionality of the Azure Cosmos DB. It also addresses couple issues with the Azure Cosmos DB Emulator functionality:

+### 2.11.8 (November 6, 2020)

+ - This release includes an update for the Azure Cosmos DB Emulator Data Explorer and fixes an issue where "TLS 1.3" clients try to open the Data Explorer.

+### 2.11.6 (October 6, 2020)

+ - This release addresses a concurrency-related issue when multiple containers might be created at the same time. This can leave the emulator in a corrupted state and future API requests to the emulator's endpoint will fail with "service unavailable" errors. The work-around is to stop the emulator, reset of the emulator's local data and restart.

+### 2.11.5 (August 23, 2020)

+This release adds two new Azure Cosmos DB Emulator startup options:

+* "/EnablePreview" - it enables preview features for the Azure Cosmos DB Emulator. The preview features that are still under development and they can be accessed via CI and sample writing.

+### 2.11.2 (July 7, 2020)

+- This release changes how ETL traces required when troubleshooting the Azure Cosmos DB Emulator are collected. WPR (Windows Performance Runtime tools) is now the default tools for capturing ETL-based traces while old LOGMAN based capturing has been deprecated. With the latest Windows security update, LOGMAN stopped working as expected when executed through the Azure Cosmos DB Emulator.

+### 2.11.1 (June 10, 2020)

+This release fixes couple bugs related to Azure Cosmos DB Emulator Data Explorer:

+* Data Explorer fails to connect to the Azure Cosmos DB Emulator endpoint when hosted in some Web browser versions. Emulator users might not be able to create a database or a container through the Web page.

+* Address an issue that prevented emulator users from creating an item from a JSON file using Data Explorer upload action.

+- This release introduces support for autoscale provisioned throughput. The added features include the option to set a custom maximum provisioned throughput level in request units (RU/s), enable autoscale on existing databases and containers, and API support through Azure Cosmos DB SDK.

+- This release adds MongoDB version 3.6 server support to the Azure Cosmos DB Emulator. To start a MongoDB endpoint that target version 3.6 of the service, start the emulator from an Administrator command line with "/EnableMongoDBEndpoint=3.6" option.

+- This release fixes a regression in the Azure Cosmos DB Emulator that prevented users from executing SQL related queries. This issue impacts emulator users that configured SQL API endpoint and they're using .NET core or x86 .NET based client applications.

+When you copy data from/to Azure SQL Database with [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine), follow below steps:

+> Azure SQL Database [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) supports below scenarios:

+>[!NOTE]

+> Currently, Azure SQL Database [**Always Encrypted**](/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-ver15&preserve-view=true) is only supported for source transformation in mapping data flows.

+When you copy data from/to SQL Managed Instance with [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine), follow below steps:

+>SQL Managed Instance [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) supports below scenarios:

+>[!NOTE]

+>Currently, SQL Managed Instance [**Always Encrypted**](/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-ver15&preserve-view=true) is only supported for source transformation in mapping data flows.

+> Windows authentication is not supported in data flow.

+>[!NOTE]

+>Currently, SQL Server [**Always Encrypted**](/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-ver15&preserve-view=true) is only supported for source transformation in mapping data flows.

+> [!Note]

+> Currently, this connector doesn't support sandbox accounts.

+ Title: Configure export settings in Azure API for FHIR

+description: This article describes how to configure export settings in Azure API for FHIR

+# Configure export settings in Azure API for FHIR and set up a storage account

+Azure API for FHIR supports $export command that allows you to export the data out of Azure API for FHIR account to a storage account.

+There are three steps involved in configuring export in Azure API for FHIR:

+1. Enable Managed Identity on Azure API for FHIR.

+2. Create an Azure storage account (if not done before) and assign permissions to Azure API for FHIR to the storage account.

+3. Select the storage account in Azure API for FHIR as export storage account.

+## Enabling Managed Identity on Azure API for FHIR

+The first step in configuring Azure API for FHIR for export is to enable system wide managed identity on the service. For more information about managed identities in Azure, see [About managed identities for Azure resources](../../active-directory/managed-identities-azure-resources/overview.md).

+Browse to the Azure API for FHIR and select **Identity**. Changing the status to **On** will enable managed identity in Azure API for FHIR.

+[  ](media/export-data/fhir-mi-enabled.png#lightbox)

+In the next step, create a storage account and assign permission to our service.

+## Adding permission to storage account

+The next step in export data is to assign permission for Azure API for FHIR to write to the storage account.

+After you've created a storage account, browse to the **Access Control (IAM)** in the storage account, and then select **Add role assignment**.

+For more information about assigning roles in the Azure portal, see [Azure built-in roles](../../role-based-access-control/role-assignments-portal.md).

+It's here that you'll add the role [Storage Blob Data Contributor](../../role-based-access-control/built-in-roles.md#storage-blob-data-contributor) to our service name, and then select **Save**.

+[  ](../../../includes/role-based-access-control/media/add-role-assignment-page.png#lightbox)

+Now youΓÇÖre ready to select the storage account in Azure API for FHIR as a default storage account for $export.

+## Selecting the storage account for $export

+The final step is to assign the Azure storage account that Azure API for FHIR will use to export the data to. To do this, go to **Integration** in Azure API for FHIR and select the storage account.

+[  ](media/export-data/fhir-export-storage.png#lightbox)

+After you've completed this final step, youΓÇÖre now ready to export the data using $export command.

+> [!Note]

+> Only storage accounts in the same subscription as that for Azure API for FHIR are allowed to be registered as the destination for $export operations.

+## Next steps

+In this article, you learned the steps in configuring export settings that allow you to export data out the Azure API for FHIR to a storage account. For more information about configuring database settings, access control, enabling diagnostic logging, and using custom headers to add data to audit logs, see

+>[!div class="nextstepaction"]

+>[Additional Settings](azure-api-for-fhir-additional-settings.md)

+ Title: Copy data in Azure API for FHIR to Azure Synapse Analytics

+description: This article describes copying FHIR data into Synapse in Azure API for FHIR

+# Copy data from Azure API for FHIR to Azure Synapse Analytics

+In this article, you'll learn a couple of ways to copy data from Azure API for FHIR to [Azure Synapse Analytics](https://azure.microsoft.com/services/synapse-analytics/), which is a limitless analytics service that brings together data integration, enterprise data warehousing, and big data analytics.

+Copying data from the FHIR server to Synapse involves exporting the data using the FHIR `$export` operation followed by a series of steps to transform and load the data to Synapse. This article will walk you through two of the several approaches, both of which will show how to convert FHIR resources into tabular formats while copying them into Synapse.

+* **Load exported data to Synapse using T-SQL:** Use `$export` operation to copy FHIR resources into a **Azure Data Lake Gen 2 (ADL Gen 2) blob storage** in `NDJSON` format. Load the data from the storage into **serverless or dedicated SQL pools** in Synapse using T-SQL. Convert these steps into a robust data movement pipeline using [Synapse pipelines](../../synapse-analytics/get-started-pipelines.md).

+* **Use the tools from the FHIR Analytics Pipelines OSS repo:** The [FHIR Analytics Pipeline](https://github.com/microsoft/FHIR-Analytics-Pipelines) repo contains tools that can create an **Azure Data Factory (ADF) pipeline** to copy FHIR data into a **Common Data Model (CDM) folder**, and from the CDM folder to Synapse.

+## Load exported data to Synapse using T-SQL

+### `$export` for moving FHIR data into Azure Data Lake Gen 2 storage

+#### Configure your FHIR server to support `$export`

+Azure API for FHIR implements the `$export` operation defined by the FHIR specification to export all or a filtered subset of FHIR data in `NDJSON` format. In addition, it supports [de-identified export](./de-identified-export.md) to anonymize FHIR data during the export. If you use `$export`, you get de-identification feature by default its capability is already integrated in `$export`.

+To export FHIR data to Azure blob storage, you first need to configure your FHIR server to export data to the storage account. YouΓÇÖll need to (1) enable Managed Identity, (2) go to Access Control in the storage account and add role assignment, (3) select your storage account for `$export`. More step by step can be found [here](./configure-export-data.md).

+You can configure the server to export the data to any kind of Azure storage account, but we recommend exporting to ADL Gen 2 for best alignment with Synapse.

+#### Using `$export` command

+After configuring your FHIR server, you can follow the [documentation](./export-data.md#using-export-command) to export your FHIR resources at System, Patient, or Group level. For example, you can export all of your FHIR data related to the patients in a `Group` with the following `$export` command, in which you specify your ADL Gen 2 blob storage name in the field `{{BlobContainer}}`:

+```rest

+https://{{FHIR service base URL}}/Group/{{GroupId}}/$export?_container={{BlobContainer}}

+```

+You can also use `_type` parameter in the `$export` call above to restrict the resources we you want to export. For example, the following call will export only `Patient`, `MedicationRequest`, and `Observation` resources:

+```rest

+https://{{FHIR service base URL}}/Group/{{GroupId}}/$export?_container={{BlobContainer}}&

+_type=Patient,MedicationRequest,Condition

+```

+For more information on the different parameters supported, check out our `$export` page section on the [query parameters](./export-data.md#settings-and-parameters).

+### Create a Synapse workspace

+Before using Synapse, you'll need a Synapse workspace. YouΓÇÖll create an Azure Synapse Analytics service on Azure portal. More step-by-step guide can be found [here](../../synapse-analytics/get-started-create-workspace.md). You need an `ADLSGEN2` account to create a workspace. Your Azure Synapse workspace will use this storage account to store your Synapse workspace data.

+After creating a workspace, you can view your workspace on Synapse Studio by signing into your workspace on https://web.azuresynapse.net, or launching Synapse Studio in the Azure portal.

+#### Creating a linked service between Azure storage and Synapse

+To copy your data to Synapse, you need to create a linked service that connects your Azure Storage account with Synapse. More step-by-step instructions can be found [here](../../synapse-analytics/data-integration/data-integration-sql-pool.md#create-linked-services).

+1. In Synapse Studio, browse to the **Manage** tab and under **External connections**, select **Linked services**.

+2. Select **New** to add a new linked service.

+3. Select **Azure Data Lake Storage Gen2** from the list and select **Continue**.

+4. Enter your authentication credentials. Select **Create** when finished.

+Now that you have a linked service between your ADL Gen 2 storage and Synapse, youΓÇÖre ready to use Synapse SQL pools to load and analyze your FHIR data.

+### Decide between serverless and dedicated SQL pool

+Azure Synapse Analytics offers two different SQL pools, serverless SQL pool and dedicated SQL pool. Serverless SQL pool gives the flexibility of querying data directly in the blob storage using the serverless SQL endpoint without any resource provisioning. Dedicated SQL pool has the processing power for high performance and concurrency, and is recommended for enterprise-scale data warehousing capabilities. For more details on the two SQL pools, check out the [Synapse documentation page](../../synapse-analytics/sql/overview-architecture.md) on SQL architecture.

+#### Using serverless SQL pool

+Since itΓÇÖs serverless, there's no infrastructure to setup or clusters to maintain. You can start querying data from Synapse Studio as soon as the workspace is created.

+For example, the following query can be used to transform selected fields from `Patient.ndjson` into a tabular structure:

+```sql

+SELECT * FROM

+OPENROWSET(bulk 'https://{{youraccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson',

+FORMAT = 'csv',

+FIELDTERMINATOR ='0x0b',

+FIELDQUOTE = '0x0b')

+WITH (doc NVARCHAR(MAX)) AS rows

+CROSS APPLY OPENJSON(doc)

+WITH (

+ ResourceId VARCHAR(64) '$.id',

+ Active VARCHAR(10) '$.active',

+ FullName VARCHAR(100) '$.name[0].text',

+ Gender VARCHAR(20) '$.gender',

+ ...

+)

+```

+In the query above, the `OPENROWSET` function accesses files in Azure Storage, and `OPENJSON` parses JSON text and returns the JSON input properties as rows and columns. Every time this query is executed, the serverless SQL pool reads the file from the blob storage, parses the JSON, and extracts the fields.

+You can also materialize the results in Parquet format in an [External Table](../../synapse-analytics/sql/develop-tables-external-tables.md) to get better query performance, as shown below:

+```sql

+-- Create External data source where the parquet file will be written

+CREATE EXTERNAL DATA SOURCE [MyDataSource] WITH (

+ LOCATION = 'https://{{youraccount}}.blob.core.windows.net/{{exttblcontainer}}'

+);

+GO

+-- Create External File Format

+CREATE EXTERNAL FILE FORMAT [ParquetFF] WITH (

+ FORMAT_TYPE = PARQUET,

+ DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'

+);

+GO

+CREATE EXTERNAL TABLE [dbo].[Patient] WITH (

+ LOCATION = 'PatientParquet/',

+ DATA_SOURCE = [MyDataSource],

+ FILE_FORMAT = [ParquetFF]

+) AS

+SELECT * FROM

+OPENROWSET(bulk 'https://{{youraccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson'

+-- Use rest of the SQL statement from the previous example --

+```

+#### Using dedicated SQL pool

+Dedicated SQL pool supports managed tables and a hierarchical cache for in-memory performance. You can import big data with simple T-SQL queries, and then use the power of the distributed query engine to run high-performance analytics.

+The simplest and fastest way to load data from your storage to a dedicated SQL pool is to use the **`COPY`** command in T-SQL, which can read CSV, Parquet, and ORC files. As in the example query below, use the `COPY` command to load the `NDJSON` rows into a tabular structure.

+```sql

+-- Create table with HEAP, which is not indexed and does not have a column width limitation of NVARCHAR(4000)

+CREATE TABLE StagingPatient (

+Resource NVARCHAR(MAX)

+) WITH (HEAP)

+COPY INTO StagingPatient

+FROM 'https://{{yourblobaccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson'

+WITH (

+FILE_TYPE = 'CSV',

+ROWTERMINATOR='0x0a',

+FIELDQUOTE = '',

+FIELDTERMINATOR = '0x00'

+)

+GO

+```

+Once you have the JSON rows in the `StagingPatient` table above, you can create different tabular formats of the data using the `OPENJSON` function and storing the results into tables. HereΓÇÖs a sample SQL query to create a `Patient` table by extracting a few fields from the `Patient` resource:

+```sql

+SELECT RES.*

+INTO Patient

+FROM StagingPatient

+CROSS APPLY OPENJSON(Resource)

+WITH (

+ ResourceId VARCHAR(64) '$.id',

+ FullName VARCHAR(100) '$.name[0].text',

+ FamilyName VARCHAR(50) '$.name[0].family',

+ GivenName VARCHAR(50) '$.name[0].given[0]',

+ Gender VARCHAR(20) '$.gender',

+ DOB DATETIME2 '$.birthDate',

+ MaritalStatus VARCHAR(20) '$.maritalStatus.coding[0].display',

+ LanguageOfCommunication VARCHAR(20) '$.communication[0].language.text'

+) AS RES

+GO

+```

+## Use FHIR Analytics Pipelines OSS tools

+> [!Note]

+> [FHIR Analytics pipeline](https://github.com/microsoft/FHIR-Analytics-Pipelines) is an open source tool released under MIT license, and is not covered by the Microsoft SLA for Azure services.

+### ADF pipeline for moving FHIR data into CDM folder

+Common Data Model (CDM) folder is a folder in a data lake that conforms to well-defined and standardized metadata structures and self-describing data. These folders facilitate metadata interoperability between data producers and data consumers. Before you copy FHIR data into CDM folder, you can transform your data into a table configuration.

+### Generating table configuration

+Clone the repo get all the scripts and source code. Use `npm install` to install the dependencies. Run the following command from the `Configuration-Generator` folder to generate a table configuration folder using YAML format instructions:

+```bash

+Configuration-Generator> node .\generate_from_yaml.js -r {resource configuration file} -p {properties group file} -o {output folder}

+```

+You may use the sample `YAML` files, `resourcesConfig.yml` and `propertiesGroupConfig.yml` provided in the repo.

+### Generating ADF pipeline

+Now you can use the content of the generated table configuration and a few other configurations to generate an ADF pipeline. This ADF pipeline, when triggered, exports the data from the FHIR server using `$export` API and writes to a CDM folder along with associated CDM metadata.

+1. Create an Azure Active Directory (Azure AD) application and service principal. The ADF pipeline uses an Azure batch service to do the transformation, and needs an Azure AD application for the batch service. Follow [Azure AD documentation](../../active-directory/develop/howto-create-service-principal-portal.md).

+2. Grant access for export storage location to the service principal. In the `Access Control` of the export storage, grant `Storage Blob Data Contributor` role to the Azure AD application.

+3. Deploy the egress pipeline. Use the template `fhirServiceToCdm.json` for a custom deployment on Azure. This step will create the following Azure resources:

+ - An ADF pipeline with the name `{pipelinename}-df`.

+ - A key vault with the name `{pipelinename}-kv` to store the client secret.

+ - A batch account with the name `{pipelinename}batch` to run the transformation.

+ - A storage account with the name `{pipelinename}storage`.

+4. Grant access to the Azure Data Factory. In the access control panel of your FHIR service, grant `FHIR data exporter` and `FHIR data reader` roles to the data factory, `{pipelinename}-df`.

+5. Upload the content of the table configuration folder to the configuration container.

+6. Go to `{pipelinename}-df`, and trigger the pipeline. You should see the exported data in the CDM folder on the storage account `{pipelinename}storage`. You should see one folder for each table having a CSV file.

+### From CDM folder to Synapse

+Once you have the data exported in a CDM format and stored in your ADL Gen 2 storage, you can now copy your data in the CDM folder to Synapse.

+You can create CDM to Synapse pipeline using a configuration file, which would look like the following example:

+```json

+{

+ "ResourceGroup": "",

+ "TemplateFilePath": "../Templates/cdmToSynapse.json",

+ "TemplateParameters": {

+ "DataFactoryName": "",

+ "SynapseWorkspace": "",

+ "DedicatedSqlPool": "",

+ "AdlsAccountForCdm": "",

+ "CdmRootLocation": "cdm",

+ "StagingContainer": "adfstaging",

+ "Entities": ["LocalPatient", "LocalPatientAddress"]

+ }

+}

+```

+Run the following script with the configuration file above:

+```bash

+.\DeployCdmToSynapsePipeline.ps1 -Config: config.json

+```

+Add ADF Managed Identity as a SQL user into SQL database. Below is a sample SQL script to create a user and an assign role:

+```sql

+CREATE USER [datafactory-name] FROM EXTERNAL PROVIDER

+GO

+EXEC sp_addrolemember db_owner, [datafactory-name]

+GO

+```

+## Next steps

+In this article, you learned two different ways to copy your FHIR data into Synapse: (1) using `$export` to copy data into ADL Gen 2 blob storage then loading the data into Synapse SQL pools, and (2) using ADF pipeline for moving FHIR data into CDM folder then into Synapse.

+Next, you can learn about anonymization of your FHIR data while copying data to Synapse to ensure your healthcare information is protected:

+

+>[!div class="nextstepaction"]

+>[Exporting de-identified data](de-identified-export.md)

+ Title: Exporting de-identified data (preview) for Azure API for FHIR

+description: This article describes how to set up and use de-identified export for Azure API for FHIR

+# Exporting de-identified data (preview) for Azure API for FHIR

+> [!Note]

+> Results when using the de-identified export will vary based on factors such as data inputted, and functions selected by the customer. Microsoft is unable to evaluate the de-identified export outputs or determine the acceptability for customer's use cases and compliance needs. The de-identified export is not guaranteed to meet any specific legal, regulatory, or compliance requirements.

+The $export command can also be used to export de-identified data from the FHIR server. It uses the anonymization engine from [Tools for Health Data Anonymization](https://github.com/microsoft/Tools-for-Health-Data-Anonymization/blob/master/docs/FHIR-anonymization.md), and takes anonymization config details in query parameters. You can create your own anonymization config file or use the [sample config file](https://github.com/microsoft/FHIR-Tools-for-Anonymization#sample-configuration-file-for-hipaa-safe-harbor-method) for HIPAA Safe Harbor method as a starting point.

+ `https://<<FHIR service base URL>>/$export?_container=<<container_name>>&_anonymizationConfig=<<config file name>>&_anonymizationConfigEtag=<<ETag on storage>>`

+> [!Note]

+> Currently, Azure API for FHIR only supports de-identified export at the system level ($export).

+|Query parameter | Example |Optionality| Description|

+|||--||

+| _\_anonymizationConfig_ |DemoConfig.json|Required for de-identified export |Name of the configuration file. See the configuration file format [here](https://github.com/microsoft/Tools-for-Health-Data-Anonymization/blob/master/docs/FHIR-anonymization.md#configuration-file-format). This file should be kept inside a container named **anonymization** within the same Azure storage account that is configured as the export location. |

+| _\_anonymizationConfigEtag_|"0x8D8494A069489EC"|Optional for de-identified export|This is the Etag of the configuration file. You can get the Etag using Azure storage explorer from the blob property|

+> [!IMPORTANT]

+> Both raw export as well as de-identified export writes to the same Azure storage account specified as part of export configuration. It is recommended that you use different containers corresponding to different de-identified config and manage user access at the container level.

+## Next steps

+In this article, you've learned how to set up and use de-identified export. Next, to learn how to export FHIR data using $export for Azure API for FHIR, see

+

+>[!div class="nextstepaction"]

+>[Export data](export-data.md)

+ Title: Executing the export by invoking $export command on Azure API for FHIR

+description: This article describes how to export FHIR data using $export for Azure API for FHIR

+# How to export FHIR data in Azure API for FHIR

+The Bulk Export feature allows data to be exported from the FHIR Server per the [FHIR specification](https://hl7.org/fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html).

+Before using $export, you'll want to make sure that the Azure API for FHIR is configured to use it. For configuring export settings and creating Azure storage account, refer to [the configure export data page](configure-export-data.md).

+## Using $export command

+After configuring the Azure API for FHIR for export, you can use the $export command to export the data out of the service. The data will be stored into the storage account you specified while configuring export. To learn how to invoke $export command in FHIR server, read documentation on the [HL7 FHIR $export specification](https://hl7.org/Fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html).

+**Jobs stuck in a bad state**

+In some situations, thereΓÇÖs a potential for a job to be stuck in a bad state. This can occur especially if the storage account permissions havenΓÇÖt been set up properly. One way to validate if your export is successful is to check your storage account to see if the corresponding container (that is, `ndjson`) files are present. If they arenΓÇÖt present, and there are no other export jobs running, then thereΓÇÖs a possibility the current job is stuck in a bad state. You should cancel the export job by sending a cancellation request and try re-queuing the job again. Our default run time for an export in bad state is 10 minutes before it will stop and move to a new job or retry the export.

+The Azure API For FHIR supports $export at the following levels:

+* [System](https://hl7.org/Fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html#endpointsystem-level-export): `GET https://<<FHIR service base URL>>/$export>>`

+* [Patient](https://hl7.org/Fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html#endpointall-patients): `GET https://<<FHIR service base URL>>/Patient/$export>>`

+* [Group of patients*](https://hl7.org/Fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html#endpointgroup-of-patients) - Azure API for FHIR exports all related resources but doesn't export the characteristics of the group: `GET https://<<FHIR service base URL>>/Group/[ID]/$export>>`

+When data is exported, a separate file is created for each resource type. To ensure that the exported files don't become too large. We create a new file after the size of a single-exported file becomes larger than 64 MB. The result is that you may get multiple files for each resource type, which will be enumerated (that is, Patient-1.ndjson, Patient-2.ndjson).

+> [!Note]

+> `Patient/$export` and `Group/[ID]/$export` may export duplicate resources if the resource is in a compartment of more than one resource, or is in multiple groups.

+In addition, checking the export status through the URL returned by the location header during the queuing is supported along with canceling the actual export job.

+### Exporting FHIR data to ADLS Gen2

+Currently we support $export for ADLS Gen2 enabled storage accounts, with the following limitation:

+- User canΓÇÖt take advantage of [hierarchical namespaces](../../storage/blobs/data-lake-storage-namespace.md), yet there isn't a way to target export to a specific subdirectory within the container. We only provide the ability to target a specific container (where we create a new folder for each export).

+- Once an export is complete, we never export anything to that folder again, since subsequent exports to the same container will be inside a newly created folder.

+## Settings and parameters

+### Headers

+There are two required header parameters that must be set for $export jobs. The values are defined by the current [$export specification](https://hl7.org/Fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html#headers).

+* **Accept** - application/fhir+json

+* **Prefer** - respond-async

+### Query parameters

+The Azure API for FHIR supports the following query parameters. All of these parameters are optional:

+|Query parameter | Defined by the FHIR Spec? | Description|

+||||

+| \_outputFormat | Yes | Currently supports three values to align to the FHIR Spec: application/fhir+ndjson, application/ndjson, or just ndjson. All export jobs will return `ndjson` and the passed value has no effect on code behavior. |

+| \_since | Yes | Allows you to only export resources that have been modified since the time provided |

+| \_type | Yes | Allows you to specify which types of resources will be included. For example, \_type=Patient would return only patient resources|

+| \_typefilter | Yes | To request finer-grained filtering, you can use \_typefilter along with the \_type parameter. The value of the _typeFilter parameter is a comma-separated list of FHIR queries that further restrict the results |

+| \_container | No | Specifies the container within the configured storage account where the data should be exported. If a container is specified, the data will be exported into a folder into that container. If the container isnΓÇÖt specified, the data will be exported to a new container. |

+> [!Note]

+> Only storage accounts in the same subscription as that for Azure API for FHIR are allowed to be registered as the destination for $export operations.

+## Secure Export to Azure Storage

+Azure API for FHIR supports a secure export operation. Choose one of the two options below:

+* Allowing Azure API for FHIR as a Microsoft Trusted Service to access the Azure storage account.

+

+* Allowing specific IP addresses associated with Azure API for FHIR to access the Azure storage account.

+This option provides two different configurations depending on whether the storage account is in the same location as, or is in a different location from that of the Azure API for FHIR.

+### Allowing Azure API for FHIR as a Microsoft Trusted Service

+Select a storage account from the Azure portal, and then select the **Networking** blade. Select **Selected networks** under the **Firewalls and virtual networks** tab.

+> [!IMPORTANT]

+> Ensure that youΓÇÖve granted access permission to the storage account for Azure API for FHIR using its managed identity. For more information, see [Configure export setting and set up the storage account](../../healthcare-apis/fhir/configure-export-data.md).

+Under the **Exceptions** section, select the box **Allow trusted Microsoft services to access this storage account** and save the setting.

+You're now ready to export FHIR data to the storage account securely. Note that the storage account is on selected networks and isnΓÇÖt publicly accessible. To access the files, you can either enable and use private endpoints for the storage account, or enable all networks for the storage account for a short period of time.

+> [!IMPORTANT]

+> The user interface will be updated later to allow you to select the Resource type for Azure API for FHIR and a specific service instance.

+### Allowing specific IP addresses for the Azure storage account in a different region

+Select **Networking** of the Azure storage account from the

+portal.

+

+Select **Selected networks**. Under the Firewall section, specify the IP address in the **Address range** box. Add IP ranges to

+allow access from the internet or your on-premises networks. You can

+find the IP address in the table below for the Azure region where the

+Azure API for FHIR is provisioned.

+|**Azure Region** |**Public IP Address** |

+|:-|:-|

+| Australia East | 20.53.44.80 |

+| Canada Central | 20.48.192.84 |

+| Central US | 52.182.208.31 |

+| East US | 20.62.128.148 |

+| East US 2 | 20.49.102.228 |

+| East US 2 EUAP | 20.39.26.254 |

+| Germany North | 51.116.51.33 |

+| Germany West Central | 51.116.146.216 |

+| Japan East | 20.191.160.26 |

+| Korea Central | 20.41.69.51 |

+| North Central US | 20.49.114.188 |

+| North Europe | 52.146.131.52 |

+| South Africa North | 102.133.220.197 |

+| South Central US | 13.73.254.220 |

+| Southeast Asia | 23.98.108.42 |

+| Switzerland North | 51.107.60.95 |

+| UK South | 51.104.30.170 |

+| UK West | 51.137.164.94 |

+| West Central US | 52.150.156.44 |

+| West Europe | 20.61.98.66 |

+| West US 2 | 40.64.135.77 |

+> [!NOTE]

+> The above steps are similar to the configuration steps described in the document How to convert data to FHIR (Preview). For more information, see [Host and use templates](../../healthcare-apis/fhir/convert-data.md#host-and-use-templates)

+### Allowing specific IP addresses for the Azure storage account in the same region

+The configuration process is the same as above except a specific IP

+address range in CIDR format is used instead, 100.64.0.0/10. The reason why the IP address range, which includes 100.64.0.0 ΓÇô 100.127.255.255, must be specified is because the actual IP address used by the service varies, but will be within the range, for each $export request.

+> [!Note]

+> It is possible that a private IP address within the range of 10.0.2.0/24 may be used instead. In that case, the $export operation will not succeed. You can retry the $export request, but there is no guarantee that an IP address within the range of 100.64.0.0/10 will be used next time. That's the known networking behavior by design. The alternative is to configure the storage account in a different region.

+

+## Next steps

+In this article, you've learned how to export FHIR resources using $export command. Next, to learn how to export de-identified data, see

+

+>[!div class="nextstepaction"]

+>[Export de-identified data](de-identified-export.md)

+ Title: Move Azure API for FHIR instance to a different subscription or resource group

+description: This article describes how to move Azure an API for FHIR instance

+# Move Azure API for FHIR to a different subscription or resource group

+In this article, you'll learn how to move an Azure API for FHIR instance to a different subscription or another resource group.

+Moving to a different region isnΓÇÖt supported, though the option may be available from the list. For more information, see [Move operation support for resources](../../azure-resource-manager/management/move-support-resources.md).

+You can move an Azure API for FHIR instance to another subscription from the portal. However, the runtime and data for the service arenΓÇÖt moved. On average the **move** operation takes approximately 15 minutes or so, and the actual time may vary.

+In this article, you've learned how to move the Azure API for FHIR instance. For more information about the supported FHIR features in Azure API for FHIR, see

+>[Supported FHIR features](fhir-features-supported.md)

+>[Copy data from Azure API for FHIR to Azure Synapse Analytics](copy-to-synapse.md)

+ Title: Copy data from the FHIR service to Azure Synapse Analytics

+description: This article describes copying FHIR data into Synapse

+# Copy data from the FHIR service to Azure Synapse Analytics

+In this article, youΓÇÖll learn a couple of ways to copy data from the FHIR service to [Azure Synapse Analytics](https://azure.microsoft.com/services/synapse-analytics/), which is a limitless analytics service that brings together data integration, enterprise data warehousing, and big data analytics.

+Copying data from the FHIR server to Synapse involves exporting the data using the FHIR `$export` operation followed by a series of steps to transform and load the data to Synapse. This article will walk you through two of the several approaches, both of which will show how to convert FHIR resources into tabular formats while copying them into Synapse.

+* **Load exported data to Synapse using T-SQL:** Use `$export` operation to copy FHIR resources into a **Azure Data Lake Gen 2 (ADL Gen 2) blob storage** in `NDJSON` format. Load the data from the storage into **serverless or dedicated SQL pools** in Synapse using T-SQL. Convert these steps into a robust data movement pipeline using [Synapse pipelines](../../synapse-analytics/get-started-pipelines.md).

+* **Use the tools from the FHIR Analytics Pipelines OSS repo:** The [FHIR Analytics Pipeline](https://github.com/microsoft/FHIR-Analytics-Pipelines) repo contains tools that can create an **Azure Data Factory (ADF) pipeline** to copy FHIR data into a **Common Data Model (CDM) folder**, and from the CDM folder to Synapse.

+## Load exported data to Synapse using T-SQL

+### `$export` for moving FHIR data into Azure Data Lake Gen 2 storage

+#### Configure your FHIR server to support `$export`

+Azure API for FHIR implements the `$export` operation defined by the FHIR specification to export all or a filtered subset of FHIR data in `NDJSON` format. In addition, it supports [de-identified export](./de-identified-export.md) to anonymize FHIR data during the export. If you use `$export`, you get de-identification feature by default its capability is already integrated in `$export`.

+To export FHIR data to Azure blob storage, you first need to configure your FHIR server to export data to the storage account. YouΓÇÖll need to (1) enable Managed Identity, (2) go to Access Control in the storage account and add role assignment, (3) select your storage account for `$export`. More step-by-step instructions can be found [here](./configure-export-data.md).

+You can configure the server to export the data to any kind of Azure storage account, but we recommend exporting to ADL Gen 2 for best alignment with Synapse.

+#### Using `$export` command

+After configuring your FHIR server, you can follow the [documentation](./export-data.md#using-export-command) to export your FHIR resources at System, Patient, or Group level. For example, you can export all of your FHIR data related to the patients in a `Group` with the following `$export` command, in which you specify your ADL Gen 2 blob storage name in the field `{{BlobContainer}}`:

+```rest

+https://{{FHIR service base URL}}/Group/{{GroupId}}/$export?_container={{BlobContainer}}

+```

+You can also use `_type` parameter in the `$export` call above to restrict the resources you want to export. For example, the following call will export only `Patient`, `MedicationRequest`, and `Observation` resources:

+```rest

+https://{{FHIR service base URL}}/Group/{{GroupId}}/$export?_container={{BlobContainer}}&

+_type=Patient,MedicationRequest,Condition

+```

+For more information on the different parameters supported, check out our `$export` page section on the [query parameters](./export-data.md#settings-and-parameters).

+### Create a Synapse workspace

+Before using Synapse, youΓÇÖll need a Synapse workspace. YouΓÇÖll create an Azure Synapse Analytics service on Azure portal. More step-by-step guide can be found [here](../../synapse-analytics/get-started-create-workspace.md). You need an `ADLSGEN2` account to create a workspace. Your Azure Synapse workspace will use this storage account to store your Synapse workspace data.

+After creating a workspace, you can view your workspace on Synapse Studio by signing into your workspace on https://web.azuresynapse.net, or launching Synapse Studio in the Azure portal.

+#### Creating a linked service between Azure storage and Synapse

+To copy your data to Synapse, you need to create a linked service that connects your Azure Storage account with Synapse. More step-by-step instructions can be found [here](../../synapse-analytics/data-integration/data-integration-sql-pool.md#create-linked-services).

+1. On Synapse Studio, navigate to the **Manage** tab, and under **External connections**, select **Linked services**.

+2. Select **New** to add a new linked service.

+3. Select **Azure Data Lake Storage Gen2** from the list and select **Continue**.

+4. Enter your authentication credentials. Select **Create** when finished.

+Now that you have a linked service between your ADL Gen 2 storage and Synapse, youΓÇÖre ready to use Synapse SQL pools to load and analyze your FHIR data.

+### Decide between serverless and dedicated SQL pool

+Azure Synapse Analytics offers two different SQL pools, serverless SQL pool and dedicated SQL pool. Serverless SQL pool gives the flexibility of querying data directly in the blob storage using the serverless SQL endpoint without any resource provisioning. Dedicated SQL pool has the processing power for high performance and concurrency, and is recommended for enterprise-scale data warehousing capabilities. For more details on the two SQL pools, check out the [Synapse documentation page](../../synapse-analytics/sql/overview-architecture.md) on SQL architecture.

+#### Using serverless SQL pool

+Since itΓÇÖs serverless, there's no infrastructure to setup or clusters to maintain. You can start querying data from Synapse Studio as soon as the workspace is created.

+For example, the following query can be used to transform selected fields from `Patient.ndjson` into a tabular structure:

+```sql

+SELECT * FROM

+OPENROWSET(bulk 'https://{{youraccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson',

+FORMAT = 'csv',

+FIELDTERMINATOR ='0x0b',

+FIELDQUOTE = '0x0b')

+WITH (doc NVARCHAR(MAX)) AS rows

+CROSS APPLY OPENJSON(doc)

+WITH (

+ ResourceId VARCHAR(64) '$.id',

+ Active VARCHAR(10) '$.active',

+ FullName VARCHAR(100) '$.name[0].text',

+ Gender VARCHAR(20) '$.gender',

+ ...

+)

+```

+In the query above, the `OPENROWSET` function accesses files in Azure Storage, and `OPENJSON` parses JSON text and returns the JSON input properties as rows and columns. Every time this query is executed, the serverless SQL pool reads the file from the blob storage, parses the JSON, and extracts the fields.

+You can also materialize the results in Parquet format in an [External Table](../../synapse-analytics/sql/develop-tables-external-tables.md) to get better query performance, as shown below:

+```sql

+-- Create External data source where the parquet file will be written

+CREATE EXTERNAL DATA SOURCE [MyDataSource] WITH (

+ LOCATION = 'https://{{youraccount}}.blob.core.windows.net/{{exttblcontainer}}'

+);

+GO

+-- Create External File Format

+CREATE EXTERNAL FILE FORMAT [ParquetFF] WITH (

+ FORMAT_TYPE = PARQUET,

+ DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'

+);

+GO

+CREATE EXTERNAL TABLE [dbo].[Patient] WITH (

+ LOCATION = 'PatientParquet/',

+ DATA_SOURCE = [MyDataSource],

+ FILE_FORMAT = [ParquetFF]

+) AS

+SELECT * FROM

+OPENROWSET(bulk 'https://{{youraccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson'

+-- Use rest of the SQL statement from the previous example --

+```

+#### Using dedicated SQL pool

+Dedicated SQL pool supports managed tables and a hierarchical cache for in-memory performance. You can import big data with simple T-SQL queries, and then use the power of the distributed query engine to run high-performance analytics.

+The simplest and fastest way to load data from your storage to a dedicated SQL pool is to use the **`COPY`** command in T-SQL, which can read CSV, Parquet, and ORC files. As in the example query below, use the `COPY` command to load the `NDJSON` rows into a tabular structure.

+```sql

+-- Create table with HEAP, which is not indexed and does not have a column width limitation of NVARCHAR(4000)

+CREATE TABLE StagingPatient (

+Resource NVARCHAR(MAX)

+) WITH (HEAP)

+COPY INTO StagingPatient

+FROM 'https://{{yourblobaccount}}.blob.core.windows.net/{{yourcontainer}}/Patient.ndjson'

+WITH (

+FILE_TYPE = 'CSV',

+ROWTERMINATOR='0x0a',

+FIELDQUOTE = '',

+FIELDTERMINATOR = '0x00'

+)

+GO

+```

+Once you have the JSON rows in the `StagingPatient` table above, you can create different tabular formats of the data using the `OPENJSON` function and storing the results into tables. HereΓÇÖs a sample SQL query to create a `Patient` table by extracting a few fields from the `Patient` resource:

+```sql

+SELECT RES.*

+INTO Patient

+FROM StagingPatient

+CROSS APPLY OPENJSON(Resource)

+WITH (

+ ResourceId VARCHAR(64) '$.id',

+ FullName VARCHAR(100) '$.name[0].text',

+ FamilyName VARCHAR(50) '$.name[0].family',

+ GivenName VARCHAR(50) '$.name[0].given[0]',

+ Gender VARCHAR(20) '$.gender',

+ DOB DATETIME2 '$.birthDate',

+ MaritalStatus VARCHAR(20) '$.maritalStatus.coding[0].display',

+ LanguageOfCommunication VARCHAR(20) '$.communication[0].language.text'

+) AS RES

+GO

+```

+## Use FHIR Analytics Pipelines OSS tools

+> [!Note]

+> [FHIR Analytics pipeline](https://github.com/microsoft/FHIR-Analytics-Pipelines) is an open source tool released under MIT license, and is not covered by the Microsoft SLA for Azure services.

+### ADF pipeline for moving FHIR data into CDM folder

+Common Data Model (CDM) folder is a folder in a data lake that conforms to well-defined and standardized metadata structures and self-describing data. These folders facilitate metadata interoperability between data producers and data consumers. Before you copy FHIR data into CDM folder, you can transform your data into a table configuration.

+### Generating table configuration

+Clone the repo to get all the scripts and source code. Use `npm install` to install the dependencies. Run the following command from the `Configuration-Generator` folder to generate a table configuration folder using YAML format instructions:

+```bash

+Configuration-Generator> node .\generate_from_yaml.js -r {resource configuration file} -p {properties group file} -o {output folder}

+```

+You may use the sample `YAML` files, `resourcesConfig.yml` and `propertiesGroupConfig.yml` provided in the repo.

+### Generating ADF pipeline

+Now you can use the content of the generated table configuration and a few other configurations to generate an ADF pipeline. This ADF pipeline, when triggered, exports the data from the FHIR server using `$export` API and writes to a CDM folder along with associated CDM metadata.

+1. Create an Azure Active Directory (AD) application and service principal. The ADF pipeline uses an Azure batch service to do the transformation, and needs an Azure AD application for the batch service. Follow [Azure AD documentation](../../active-directory/develop/howto-create-service-principal-portal.md).

+2. Grant access for export storage location to the service principal. In the `Access Control` of the export storage, grant `Storage Blob Data Contributor` role to the Azure AD application.

+3. Deploy the egress pipeline. Use the template `fhirServiceToCdm.json` for a custom deployment on Azure. This step will create the following Azure resources:

+ - An ADF pipeline with the name `{pipelinename}-df`.

+ - A key vault with the name `{pipelinename}-kv` to store the client secret.

+ - A batch account with the name `{pipelinename}batch` to run the transformation.

+ - A storage account with the name `{pipelinename}storage`.

+4. Grant access to the Azure Data Factory. In the access control panel of your FHIR service, grant `FHIR data exporter` and `FHIR data reader` roles to the data factory, `{pipelinename}-df`.

+5. Upload the content of the table configuration folder to the configuration container.

+6. Go to `{pipelinename}-df`, and trigger the pipeline. You should see the exported data in the CDM folder on the storage account `{pipelinename}storage`. You should see one folder for each table having a CSV file.

+### From CDM folder to Synapse

+Once you have the data exported in a CDM format and stored in your ADL Gen 2 storage, you can now copy your data in the CDM folder to Synapse.

+You can create CDM to Synapse pipeline using a configuration file, which would look something like this:

+```json

+{

+ "ResourceGroup": "",

+ "TemplateFilePath": "../Templates/cdmToSynapse.json",

+ "TemplateParameters": {

+ "DataFactoryName": "",

+ "SynapseWorkspace": "",

+ "DedicatedSqlPool": "",

+ "AdlsAccountForCdm": "",

+ "CdmRootLocation": "cdm",

+ "StagingContainer": "adfstaging",

+ "Entities": ["LocalPatient", "LocalPatientAddress"]

+ }

+}

+```

+Run this script with the configuration file above:

+```bash

+.\DeployCdmToSynapsePipeline.ps1 -Config: config.json

+```

+Add ADF Managed Identity as a SQL user into SQL database. HereΓÇÖs a sample SQL script to create a user and an assign role:

+```sql

+CREATE USER [datafactory-name] FROM EXTERNAL PROVIDER

+GO

+EXEC sp_addrolemember db_owner, [datafactory-name]

+GO

+```

+## Next steps

+In this article, you learned two different ways to copy your FHIR data into Synapse: (1) using `$export` to copy data into ADL Gen 2 blob storage then loading the data into Synapse SQL pools, and (2) using ADF pipeline for moving FHIR data into CDM folder then into Synapse.

+Next, you can learn about anonymization of your FHIR data while copying data to Synapse to ensure your healthcare information is protected:

+

+>[!div class="nextstepaction"]

+>[Exporting de-identified data](./de-identified-export.md)

+>[Copy data from the FHIR service to Azure Synapse Analytics](copy-to-synapse.md)

+See alerts generated for devices across multiple IoT Hubs in **Alerts** tab of the [IoT Edge fleet view workbook](how-to-explore-curated-visualizations.md#fleet-view-workbook).

+You can visually explore metrics collected from IoT Edge devices using Azure Monitor workbooks. Curated monitoring workbooks for IoT Edge devices are provided in the form of public templates:

+* For devices connected to IoT Hub, from the **IoT Hub** page in the Azure portal, navigate to the **Workbooks** page in the **Monitoring** section.

+* For devices connected to IoT Central, from the **IoT Central** page in the Azure portal, navigate to the **Workbooks** page in the **Monitoring** section.

+Curated workbooks use [built-in metrics](how-to-access-built-in-metrics.md) from the IoT Edge runtime. They first need metrics to be [ingested](how-to-collect-and-transport-metrics.md) into a Log Analytics workspace. These views don't need any metrics instrumentation from the workload modules.

+Azure Monitor workbooks for IoT are a set of templates that you can use to visualize your device metrics. They can be customized to fit your solution.

+1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your IoT Hub or IoT Central application.

+ * **Fleet View**: Monitor your fleet of devices across multiple IoT Hubs or Central Apps, and drill into specific devices for a health snapshot.

+ * **Device Details**: Visualize device details around messaging, modules, and host components on an IoT Edge device.

+ * **Alerts**: View triggered [alerts](how-to-create-alerts.md) for devices across multiple IoT resources.

+Use the following sections to get a preview of the kind of data and visualizations that each workbook offers.

+>[!NOTE]

+> The screen captures that follow may not reflect the latest workbook design.

+## Fleet view workbook

+By default, this view shows the health of devices associated with the current IoT cloud resource. You can select multiple IoT resources using the dropdown control on the top left.

+Use the **Settings** tab to adjust the various thresholds to categorize the device as Healthy or Unhealthy.

+Click the **Details** button to see the device list with a snapshot of aggregated, primary metrics. Click the link in the **Status** column to view the trend of an individual device's health metrics or the device name to view its detailed metrics.

+## Device details workbook

+The device details workbook has three views:

+Switch between the views using the tabs at the top of the workbook.

+The device details workbook also integrates with the IoT Edge portal-based troubleshooting experience. You can pull **Live logs** from your device using this feature. Access this experience by selecting the **Troubleshoot \<device name> live** button above the workbook.

+The **Graph** section shows a visual representation of message flow between modules. Drag and zoom to adjust the graph.

+The **Health** section presents various metrics related to overall health of the messaging subsystem. Progressively drill-in to details if any errors are noted.

+* Modules restart count and restart timeline

+## Alerts workbook

+See the generated alerts from [pre-created alert rules](how-to-create-alerts.md) in the **Alerts** workbook. This view lets you see alerts from multiple IoT Hubs or IoT Central applications.

+Click on a severity row to see alerts details. The **Alert rule** link takes you to the alert context and the **Device** link opens the detailed metrics workbook. When opened from this view, the device details workbook is automatically adjusted to the time range around when the alert fired.

+[Azure Monitor workbooks](../azure-monitor/visualize/workbooks-overview.md) are very customizable. You can edit the public templates to suit your requirements. All the visualizations are driven by resource-centric [Kusto Query Language](/azure/data-explorer/kusto/query/) queries on the [InsightsMetrics](/azure/azure-monitor/reference/tables/insightsmetrics) table.

+To begin customizing a workbook, first enter editing mode. Select the **Edit** button in the menu bar of the workbook. Curated workbooks make extensive use of workbook groups. You may need to select **Edit** on several nested groups before being able to view a visualization query.

+These metrics are exposed automatically by both modules so that you can create your own solutions to access and report on these metrics. To make this process easier, Microsoft provides the [azureiotedge-metrics-collector module](https://azuremarketplace.microsoft.com/marketplace/apps/microsoft_iot_edge.metrics-collector?tab=overview) that handles this process for those who don't have or want a custom solution. The metrics collector module collects metrics from the two runtime modules and any other modules you may want to monitor, and transports them off-device.

+* The **Fleet View** workbook shows the health of devices across multiple IoT resources. The view allows configuring thresholds for determining device health and presents aggregations of primary metrics, per-device.

+* The **Device Details** workbook provides visualizations around three categories: messaging, modules, and host. The messaging view visualizes the message routes for a device and reports on the overall health of the messaging system. The modules view shows how the individual modules on a device are performing. The host view shows information about the host device including version information for host components and resource use.

+* The **Alerts** workbook View presents alerts for devices across multiple IoT resources.

+1. Select the **Fleet View** workbook.

+1. Select the device name to view detailed metrics from the device.

+```json

+* Access to an IoT Hub. It is recommended that you use a S1 (Standard) tier or higher.

+3. From the docs, follow the steps for how to prepare Azure Resources, Account, and register IoT devices to it.

+5. Add a new Device Update tag value to the root JSON object as shown below.

+2. Select the Updates option under "Device management" from the left-hand navigation bar.

+| Azure Service Bus namespace | - Service Bus queue to Service Bus queue <br>- Service Bus queue to Service Bus topic <br>- Service Bus topic to Service Bus topic <br>- Service Bus queue to Event Hubs instance <br>- Service Bus topic to Service Bus queue <br>- Service Bus topic to Event Hubs instance <p><p>**Important**: When a queue is the source, a replication task doesn't copy messages but *moves* them from the source to the target and deletes them from the source. <p><p>To mirror messages instead, use a topic as your source where the "main" subscription acts like a queue endpoint. That way, the target gets a copy of each message from the source. <p><p>To route messages across different regions, you can create a queue where messages are sent from an app. The replication task transfers messages from that queue to a target queue in a namespace that's in another region. You can also use a topic subscription as the entity that acts as the transfer queue. For more information, review [Replication topology for ServiceBusCopy](https://github.com/Azure-Samples/azure-messaging-replication-dotnet/tree/main/functions/config/ServiceBusCopy#replication-topology).|

+For Event Hubs, the following items obtained from the source Event Hubs namespace are replaced by new service-assigned values in the target Event Hubs namespace: service-assigned metadata of an event, original enqueue time, sequence number, and offset. However, for [helper functions](https://github.com/Azure-Samples/azure-messaging-replication-dotnet/tree/main/src/Azure.Messaging.Replication) and the replication tasks in the Azure-provided samples, the original values are preserved in the user properties: `repl-enqueue-time` (ISO8601 string), `repl-sequence`, and `repl-offset`. These properties have the `string` type and contain the stringified value of the respective original properties. If the event is forwarded multiple times, the service-assigned metadata of the immediate source is appended to any existing properties, with values separated by semicolons. For more information, review [Service-assigned metadata - Event replication task patterns](../event-hubs/event-hubs-federation-patterns.md#service-assigned-metadata).

+For Service Bus, the following items obtained from the source Service Bus queue or topic are replaced by new service-assigned values in the target Service Bus queue or topic: service-assigned metadata of a message, original enqueue time, and sequence number. However, for the default replication tasks in the Azure-provided samples, the original values are preserved in the user properties: `repl-enqueue-time` (ISO8601 string) and `repl-sequence`. These properties have the `string` type and contain the stringified value of the respective original properties. If the message is forwarded multiple times, the service-assigned metadata of the immediate source is appended to any existing properties, with values separated by semicolons. For more information, review [Service-assigned metadata - Message replication task patterns](../service-bus-messaging/service-bus-federation-patterns.md#service-assigned-metadata).

+ This example shows the prompt to create the connection to the target Service Bus namespace where the target queue exists. The connection exists for the source Service Bus namespace.

+ 1. Download, install, and open the latest [Azure Storage Explorer desktop client](https://azure.microsoft.com/features/storage-explorer/), if you don't have the most recent version.

+For information on deploying models with a custom DNS name or TLS security, see [Secure web services using TLS](how-to-secure-web-service.md).

+The integration runtime (IR) is the compute infrastructure that Azure Purview uses to power data scan across different network environments.

+A self-hosted integration runtime (SHIR) can be used to scan data source in an on-premises network or a virtual network. The installation of a self-hosted integration runtime needs an on-premises machine or a virtual machine inside a private network.

+This article describes how to create and manage a self-hosted integration runtime.

+ - Windows 11

+ - Windows Server 2022

+- Scan runs happen with a specific frequency per the schedule you've set up. Processor and RAM usage on the machine follows the same pattern with peak and idle times. Resource usage also depends heavily on the amount of data that is scanned. When multiple scan jobs are in progress, you see resource usage goes up during peak times.

+- Scanning some data sources requires additional setup on the self-hosted integration runtime machine. For example, JDK, Visual C++ Redistributable, or specific driver. Refer to [each source article](purview-connector-overview.md) for prerequisite details.

+> If you use the Self-Hosted Integration runtime to scan Parquet files, you need to install the **64-bit JRE 8 (Java Runtime Environment) or OpenJDK** on your IR machine. Check our [Java Runtime Environment section at the bottom of the page](#java-runtime-environment-installation) for an installation guide.

+### Considerations for using a self-hosted IR

+- You can use a single self-hosted integration runtime for scanning multiple data sources.

+- You can install only one instance of self-hosted integration runtime on any single machine. If you have two Azure Purview accounts that need to scan on-premises data sources, install the self-hosted IR on two machines, one for each Azure Purview account.

+- The self-hosted integration runtime doesn't need to be on the same machine as the data source, unless specially called out as a prerequisite in the respective source article. Having the self-hosted integration runtime close to the data source reduces the time for the self-hosted integration runtime to connect to the data source.

+### Create a self-hosted integration runtime

+3. On the **Integration runtime setup** page, select **Self-Hosted** to create a self-Hosted IR, and then select **Continue**.

+## Manage a self-hosted integration runtime

+You can edit a self-hosted integration runtime by navigating to **Integration runtimes** in the **Management center**, selecting the IR and then selecting edit. You can now update the description, copy the key, or regenerate new keys.

+You can delete a self-hosted integration runtime by navigating to **Integration runtimes** in the Management center, selecting the IR and then selecting **Delete**. Once an IR is deleted, any ongoing scans relying on it will fail.

+## Service account for Self-hosted integration runtime

+The default logon service account of self-hosted integration runtime is **NT SERVICE\DIAHostService**. You can see it in **Services -> Integration Runtime Service -> Properties -> Log on**.

+Make sure the account has the permission of Log on as a service. Otherwise self-hosted integration runtime can't start successfully. You can check the permission in **Local Security Policy -> Security Settings -> Local Policies -> User Rights Assignment -> Log on as a service**

+## Notification area icons and notifications

+If you move your cursor over the icon or message in the notification area, you can see details about the state of the self-hosted integration runtime.

+Your self-hosted integration runtime machine needs to connect to several resources to work correctly:

+* The Azure Purview services used to manage the self-hosted integration runtime.

+* The data sources you want to scan using the self-hosted integration runtime.

+* The managed Storage account and Event Hubs resource created by Azure Purview. Azure Purview uses these resources to ingest the results of the scan, among many other things, so the self-hosted integration runtime need to be able to connect with these resources.

+* The Azure Key Vault used to store credentials.

+There are two firewalls to consider:

+- The *corporate firewall* that runs on the central router of the organization

+- The *Windows firewall* that is configured as a daemon on the local machine where the self-hosted integration runtime is installed

+Here are the domains and outbound ports that you need to allow at both **corporate and Windows/machine firewalls**.

+> [!TIP]

+> For domains listed with '\<managed_storage_account>' and '\<managed_Event_Hub_resource>', add the name of the managed resources associated with your Azure Purview account. You can find them from Azure portal -> your Azure Purview account -> Managed resources tab.

+| `*.servicebus.windows.net` | 443 | Required for interactive authoring, for example, test connection on Azure Purview Studio. Currently wildcard is required as there is no dedicated resource. |

+| `*.frontend.clouddatahub.net` | 443 | Required to connect to the Azure Purview service. Currently wildcard is required as there is no dedicated resource. |

+| `<managed_storage_account>.blob.core.windows.net` | 443 | Required to connect to the Azure Purview managed Azure Blob storage account. |

+| `<managed_storage_account>.queue.core.windows.net` | 443 | Required to connect to the Azure Purview managed Azure Queue storage account. |

+| `<managed_Event_Hub_resource>.servicebus.windows.net` | 443 | Azure Purview uses this to connect with the associated service bus. It's covered by allowing the above domain. If you use private endpoint, you need to test access to this single domain.|

+| `download.microsoft.com` | 443 | Required to download the self-hosted integration runtime updates. If you have disabled auto-update, you can skip configuring this domain. |

+| `login.windows.net`<br>`login.microsoftonline.com` | 443 | Required to sign in to the Azure Active Directory. |

+Depending on the sources you want to scan, you also need to allow additional domains and outbound ports for other Azure or external sources. A few examples are provided here:

+| `<your_key_vault_name>.vault.azure.net` | 443 | Required if any credentials are stored in Azure Key Vault. |

+| `<your_storage_account>.dfs.core.windows.net` | 443 | When scan Azure Data Lake Store Gen 2. |

+| `<your_storage_account>.blob.core.windows.net` | 443 | When scan Azure Blob storage. |

+| `<your_sql_server>.database.windows.net` | 1433 | When scan Azure SQL Database. |

+| `<your_ADLS_account>.azuredatalakestore.net` | 443 | When scan Azure Data Lake Store Gen 1. |

+| Various domains | Dependent | Domains and ports for any other sources the SHIR will scan. |

+For some cloud data stores such as Azure SQL Database and Azure Storage, you need to allow IP address of self-hosted integration runtime machine on their firewall configuration.

+> [!IMPORTANT]

+> In most environments, you will also need to make sure that your DNS is correctly configured. To confirm, you can use **nslookup** from your SHIR machine to check connectivity to each of the domains. Each nslookup should return the IP of the resource. If you are using [Private Endpoints](catalog-private-link.md), the private IP should be returned and not the Public IP. If no IP is returned, or if when using Private Endpoints the public IP is returned, you need to address your DNS/VNet association, or your Private Endpoint/VNet peering.

+When configured, the self-hosted integration runtime uses the proxy server to connect to the services which use HTTP or HTTPS protocol. This is why you select **Change link** during initial setup.

+There are two supported configuration options by Azure Purview:

+- **Use system proxy**: The self-hosted integration runtime uses the proxy setting that is configured in the executable's configuration files. If no proxy is specified in these files, the self-hosted integration runtime connects to the services directly without going through a proxy.

+>

+> Currently, **custom proxy** is not supported in Azure Purview. In addition, system proxy is supported when scanning Azure data sources and SQL Server; scanning other sources doesn't support proxy.

+1. Select the **Settings** tab.

+If using system proxy, make sure your proxy server allow outbound traffic to the [network rules](#networking-requirements).

+### Configure proxy server settings

+If you select the **Use system proxy** option for the HTTP proxy, the self-hosted integration runtime uses the proxy settings in the following four files under the path C:\Program Files\Microsoft Integration Runtime\5.0\ to perform different operations:

+- .\Shared\diahost.exe.config

+- .\Shared\diawp.exe.config

+- .\Gateway\DataScan\Microsoft.DataMap.Agent.exe.config

+- .\Gateway\DataScan\DataTransfer\Microsoft.DataMap.Agent.Connectors.Azure.DataFactory.ServiceHost.exe.config

+When no proxy is specified in these files, the self-hosted integration runtime connects to the services directly without going through a proxy.

+The following procedure provides instructions for updating the **diahost.exe.config** file.

+1. In File Explorer, make a safe copy of C:\Program Files\Microsoft Integration Runtime\5.0\Shared\diahost.exe.config as a backup of the original file.

+1. Open Notepad running as administrator.

+1. In Notepad, open the text file C:\Program Files\Microsoft Integration Runtime\5.0\Shared\diahost.exe.config.

+1. Find the default **system.net** tag as shown in the following code:

+ ```xml

+ <system.net>

+ <defaultProxy useDefaultCredentials="true" />

+ </system.net>

+ ```

+ You can then add proxy server details as shown in the following example:

+ ```xml

+ <system.net>

+ <defaultProxy>

+ <proxy bypassonlocal="true" proxyaddress="<your proxy server e.g. http://proxy.domain.org:8888/>" />

+ </defaultProxy>

+ </system.net>

+ ```

+ The proxy tag allows additional properties to specify required settings like `scriptLocation`. See [\<proxy\> Element (Network Settings)](/dotnet/framework/configure-apps/file-schema/network/proxy-element-network-settings) for syntax.

+

+ ```xml

+ <proxy autoDetect="true|false|unspecified" bypassonlocal="true|false|unspecified" proxyaddress="uriString" scriptLocation="uriString" usesystemdefault="true|false|unspecified "/>

+ ```

+

+1. Save the configuration file in its original location.

+Repeat the same procedure to update **diawp.exe.config** and **Microsoft.DataMap.Agent.exe.config** files.

+Then go to path C:\Program Files\Microsoft Integration Runtime\5.0\Gateway\DataScan\DataTransfer, create a file named "**Microsoft.DataMap.Agent.Connectors.Azure.DataFactory.ServiceHost.exe.config**", and configure the proxy setting as follows. You can also extend the settings as described above.

+```xml

+<?xml version="1.0" encoding="utf-8"?>

+<configuration>

+ <system.net>

+ <defaultProxy>

+ <proxy bypassonlocal="true" proxyaddress="<your proxy server e.g. http://proxy.domain.org:8888/>" />

+ </defaultProxy>

+ </system.net>

+</configuration>

+```

+Restart the self-hosted integration runtime host service, which picks up the changes. To restart the service, use the services applet from Control Panel. Or from Integration Runtime Configuration Manager, select the **Stop Service** button, and then select **Start Service**. If the service doesn't start, you likely added incorrect XML tag syntax in the application configuration file that you edited.

+> [!IMPORTANT]

+> Don't forget to update all four files mentioned above.

+You also need to make sure that Microsoft Azure is in your company's allowlist. You can download the list of valid Azure IP addresses. IP ranges for each cloud, broken down by region and by the tagged services in that cloud are now available on MS Download:

+ - Public: https://www.microsoft.com/download/details.aspx?id=56519

+### Possible symptoms for issues related to the firewall and proxy server

+If you see error messages like the following ones, the likely reason is improper configuration of the firewall or proxy server. Such configuration prevents the self-hosted integration runtime from connecting to Azure Purview services. To ensure that your firewall and proxy server are properly configured, refer to the previous section.

+- When you try to register the self-hosted integration runtime, you receive the following error message: "Failed to register this Integration Runtime node! Confirm that the Authentication key is valid and the integration service host service is running on this machine."

+- When you open Integration Runtime Configuration Manager, you see a status of **Disconnected** or **Connecting**. When you view Windows event logs, under **Event Viewer** > **Application and Services Logs** > **Microsoft Integration Runtime**, you see error messages like this one:

+ ```output

+ Unable to connect to the remote server

+ A component of Integration Runtime has become unresponsive and restarts automatically. Component name: Integration Runtime (Self-hosted)

+ ```

+## Java Runtime Environment Installation

+If you scan Parquet files using the self-hosted integration runtime with Azure Purview, you will need to install either the Java Runtime Environment or OpenJDK on your self-hosted IR machine.

+When scanning Parquet files using the self-hosted IR, the service locates the Java runtime by firstly checking the registry *`(SOFTWARE\JavaSoft\Java Runtime Environment\{Current Version}\JavaHome)`* for JRE, if not found, secondly checking system variable *`JAVA_HOME`* for OpenJDK.

+- **To use JRE**: The 64-bit IR requires 64-bit JRE. You can find it from [here](https://go.microsoft.com/fwlink/?LinkId=808605).

+- **To use OpenJDK**: It's supported since IR version 3.13. Package the jvm.dll with all other required assemblies of OpenJDK into self-hosted IR machine, and set system environment variable JAVA_HOME accordingly.

+- [Azure Purview network architecture and best practices](concept-best-practices-network.md)

+When setting up scan, you can choose to scan one or more Snowflake database(s) entirely, or further scope the scan to a subset of schemas matching the given name(s) or name pattern(s).

+* You need to be a Data Source Administrator and Data Reader to register a source and manage it in the Azure Purview Studio. See our [Azure Purview Permissions page](catalog-permissions.md) for details.

+Here's a sample walkthrough to create a user specifically for Azure Purview scan and set up the permissions. If you choose to use an existing user, make sure it has adequate rights to the warehouse and database objects.

+1. Set up a `purview_reader` role. You need _ACCOUNTADMIN_ rights to do this.

+ 1. **Databases**: Specify one or more database instance names to import in capital case. Separate the names in the list with a semi-colon (;). The default role assigned to the user specified in the credential must have adequate rights on the database objects.

+- Check your account identifer in the source registration step. Don't include `https://` part at the front.

+>[!Important]

+> - Publish is a background operation. It can take up to **2 hours** for the changes to be reflected in Storage account(s).

+- Creating a policy at subscription or resource group level will enable the Subjects to access Azure Storage system containers e.g., *$logs*. If this is undesired, first scan the data source and then create finer-grained policies for each (i.e., at container or sub-container level).

+## Additional information

+- Creating a policy at Storage account level will enable the Subjects to access system containers e.g., *$logs*. If this is undesired, first scan the data source(s) and then create finer-grained policies for each (i.e., at container or sub-container level).

+### Limits

+## Regional availability

+The preview is only available on search services in the following regions:

+## Preview limitations

+There is a lot to be excited about with this preview, but there are a few limitations. This section describes the limitations that are specific to the current version of the preview.

+ Title: 'Tutorial: Index encrypted blobs'

+# Tutorial: Index and enrich encrypted blobs for full-text search in Azure Cognitive Search

+This tutorial shows you how to use [Azure Cognitive Search](search-what-is-azure-search.md) to index documents that have been previously encrypted with a customer-managed key in [Azure Blob Storage](../storage/blobs/storage-blobs-introduction.md).

+Normally, an indexer can't extract content from encrypted files because it doesn't have access to the customer-managed encryption key in [Azure Key Vault](../key-vault/general/overview.md). However, by leveraging the [DecryptBlobFile custom skill](https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Utils/DecryptBlobFile), followed by the [Document Extraction skill](cognitive-search-skill-document-extraction.md), you can provide controlled access to the key to decrypt the files and then extract content from them. This unlocks the ability to index and enrich these documents without compromising the encryption status of your stored documents.

+Starting with previously encrypted whole documents (unstructured text) such as PDF, HTML, DOCX, and PPTX in Azure Blob Storage, this tutorial uses Postman and the Search REST APIs to perform the following tasks:

+> + Define a pipeline that decrypts the documents and extracts text from them.

+> + Define an index to store the output.

+> + Execute the pipeline to create and load the index.

+> + Explore results using full text search and a rich query syntax.

+Custom skill deployment creates an Azure Function app and an Azure Storage account. Since these resources are created for you, they aren't listed as a prerequisite. When you're finished with this tutorial, remember to clean up the resources so that you aren't billed for services you're not using.

+> [!NOTE]

+> Skillsets often require [attaching a Cognitive Services resource](cognitive-search-attach-cognitive-services.md). As written, this skillset has no dependency on Cognitive Services and thus no key is required. If you later add enrichments that invoke built-in skills, remember to update your skillset accordingly.

+### Deploy the custom skill

+Operationally, the DecryptBlobFile skill takes the URL and SAS token for each blob as inputs, and it outputs the downloaded, decrypted file using the file reference contract that Azure Cognitive Search expects. Recall that DecryptBlobFile needs the encryption key to perform the decryption. As part of setup, you'll also create an access policy that grants DecryptBlobFile function access to the encryption key in Azure Key Vault.

+1. Choose the same subscription where your Azure Key Vault instance exists (this tutorial will not work if you select a different subscription).

+1. Select an existing resource group or create a new one. A dedicated resource group makes cleanup easier later.

+ :::image type="content" source="media/indexing-encrypted-blob-files/arm-template.png" alt-text="Screenshot of the arm template page in Azure portal." border="true":::

+You should have an Azure Function app that contains the decryption logic and an Azure Storage resource that will store application data. In the next several steps, you'll give the app permissions to access the key vault and collect information that you'll need for the REST calls.

+### Grant permissions in Azure Key Vault

+1. Navigate to your Azure Key Vault service in the portal. [Create an access policy](../key-vault/general/assign-access-policy-portal.md) in the Azure Key Vault that grants key access to the custom skill.

+1. On the left navigation pane, select **Access policies**, and then select **+ Create** to start the **Create an access policy** wizard.

+ :::image type="content" source="media/indexing-encrypted-blob-files/keyvault-access-policies.png" alt-text="Screenshot of the Access Policy command in the left navigation pane." border="true":::

+1. On the **Permissions** page under **Configure from template**, select **Azure Data Lake Storage or Azure Storage**.

+1. Select **Next**.

+1. On the **Principal** page, select the Azure Function instance that you deployed. You can search for it using the resource prefix that was used to create it in step 2, which has a default prefix value of **psdbf-function-app**.

+1. Select **Next**.

+1. On **Review + create**, select **Create**.

+### Collect app information

+1. Navigate to the **psdbf-function-app** function in the portal, and make a note of the following properties you'll need for the REST calls:

+1. Get the function URL, which can be found under **Essentials** on the main page for the function.

+ :::image type="content" source="media/indexing-encrypted-blob-files/function-uri.png" alt-text="Screenshot of the overview page and Essentials section of the Azure Function app." border="true":::

+1. Get the host key code, which can be found by navigating to **App keys**, clicking to show the **default** key, and copying the value.

+ :::image type="content" source="media/indexing-encrypted-blob-files/function-host-key.png" alt-text="Screenshot of the App Keys page of the Azure Function app." border="true":::

+ 

+1. On the **Variables** tab, provide the values that you've collected in the previous steps. Postman swaps in a value every time it encounters a specific variable inside double braces. For example, Postman replaces the symbol `{{admin-key}}` with the current value that you set for the search service admin API key.

+ | Variable | Where to get it |

+ |-|--|

+ | `admin-key` | On the **Keys** page of the Azure Cognitive Search service. |

+ | `search-service-name` | The name of the Azure Cognitive Search service. The URL is `https://{{search-service-name}}.search.windows.net`. |

+ | `storage-connection-string` | In the storage account, on the **Access Keys** tab, select **key1** > **Connection string**. |

+ | `storage-container-name` | The name of the blob container that has the encrypted files to be indexed. |

+ | `function-uri` | In the Azure Function under **Essentials** on the main page. |

+ | `function-code` | In the Azure Function, by navigating to **App keys**, clicking to show the **default** key, and copying the value. |

+ | `api-version` | Leave as **2020-06-30**. |

+ | `datasource-name` | Leave as **encrypted-blobs-ds**. |

+ | `index-name` | Leave as **encrypted-blobs-idx**. |

+ | `skillset-name` | Leave as **encrypted-blobs-ss**. |

+ | `indexer-name` | Leave as **encrypted-blobs-ixr**. |

+### Review and run each request

+In this section, you'll issue four HTTP requests:

+To issue the requests, in Postman, select the tab for the requests and select **Send** for each of them.

+## Clean up resources

+When you're working in your own subscription, at the end of a project, it's a good idea to remove the resources that you no longer need. Resources left running can cost you money. You can delete resources individually or delete the resource group to delete the entire set of resources.

+You can find and manage resources in the portal, using the All resources or Resource groups link in the left-navigation pane.

+If you are working with doubly encrypted data, you might want to investigate the index encryption features available in Azure Cognitive Search. Although the indexer needs decrypted data for indexing purposes, once the index exists, it can be encrypted in a search index using a customer-managed key. This will ensure that your data is always encrypted when at rest. For more information, see [Configure customer-managed keys for data encryption in Azure Cognitive Search](search-security-manage-encryption-keys.md).

+1. Select the **Access policies** on the left, and select **+ Create** to start the **Create an access policy** wizard.

+|**Cybersecurity Maturity Model Certification (CMMC)** | Analytics rules, workbook, playbook | Compliance | Microsoft|

+| **IoT/OT Threat Monitoring with Defender for IoT** | [Analytics rules, playbooks, workbook](iot-solution.md) | Internet of Things (IoT), Security - Threat Protection | Microsoft |

+|**Maturity Model for Event Log Management M2131** | [Analytics rules, hunting queries, playbooks, workbook](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/modernize-log-management-with-the-maturity-model-for-event-log/ba-p/3072842) | Compliance | Microsoft|

+|**Microsoft Insider Risk Management** (IRM) |[Data connector](data-connectors-reference.md#microsoft-365-insider-risk-management-irm-preview), workbook, analytics rules, hunting queries, playbook |Security - Insider threat | Microsoft|

+|**Zero Trust** (TIC3.0) |[Analytics rules, playbook, workbooks](https://techcommunity.microsoft.com/t5/public-sector-blog/announcing-the-azure-sentinel-zero-trust-tic3-0-workbook/ba-p/2313761) |Identity, Security - Others |Microsoft |

+## July 2021

+- [Microsoft Threat Intelligence Matching Analytics (Public preview)](#microsoft-threat-intelligence-matching-analytics-public-preview)

+- [Use Azure AD data with Azure Sentinel's IdentityInfo table (Public preview)](#use-azure-ad-data-with-azure-sentinels-identityinfo-table-public-preview)

+- [Enrich Entities with geolocation data via API (Public preview)](#enrich-entities-with-geolocation-data-via-api-public-preview)

+- [Support for ADX cross-resource queries (Public preview)](#support-for-adx-cross-resource-queries-public-preview)

+- [Watchlists are in general availability](#watchlists-are-in-general-availability)

+- [Support for data residency in more geos](#support-for-data-residency-in-more-geos)

+- [Bidirectional sync in Azure Defender connector (Public preview)](#bidirectional-sync-in-azure-defender-connector-public-preview)

+### Microsoft Threat Intelligence Matching Analytics (Public preview)

+Azure Sentinel now provides the built-in **Microsoft Threat Intelligence Matching Analytics** rule, which matches Microsoft-generated threat intelligence data with your logs. This rule generates high-fidelity alerts and incidents, with appropriate severities based on the context of the logs detected. After a match is detected, the indicator is also published to your Azure Sentinel threat intelligence repository.

+The **Microsoft Threat Intelligence Matching Analytics** rule currently matches domain indicators against the following log sources:

+- [CEF](connect-common-event-format.md)

+- [DNS](./data-connectors-reference.md#windows-dns-server-preview)

+- [Syslog](connect-syslog.md)

+For more information, see [Detect threats using matching analytics (Public preview)](work-with-threat-indicators.md#detect-threats-using-matching-analytics-public-preview).

+### Use Azure AD data with Azure Sentinel's IdentityInfo table (Public preview)

+As attackers often use the organization's own user and service accounts, data about those user accounts, including the user identification and privileges, are crucial for the analysts in the process of an investigation.

+Now, having [UEBA enabled](enable-entity-behavior-analytics.md) in your Azure Sentinel workspace also synchronizes Azure AD data into the new **IdentityInfo** table in Log Analytics. Synchronizations between your Azure AD and the **IdentifyInfo** table create a snapshot of your user profile data that includes user metadata, group information, and the Azure AD roles assigned to each user.

+Use the **IdentityInfo** table during investigations and when fine-tuning analytics rules for your organization to reduce false positives.

+For more information, see [IdentityInfo table](ueba-enrichments.md#identityinfo-table-public-preview) in the UEBA enrichments reference and [Use UEBA data to analyze false positives](investigate-with-ueba.md#use-ueba-data-to-analyze-false-positives).

+### Enrich entities with geolocation data via API (Public preview)

+Azure Sentinel now offers an API to enrich your data with geolocation information. Geolocation data can then be used to analyze and investigate security incidents.

+For more information, see [Enrich entities in Azure Sentinel with geolocation data via REST API (Public preview)](geolocation-data-api.md) and [Classify and analyze data using entities in Azure Sentinel](entities.md).

+### Support for ADX cross-resource queries (Public preview)

+The hunting experience in Azure Sentinel now supports [ADX cross-resource queries](../azure-monitor/logs/azure-monitor-data-explorer-proxy.md#cross-query-your-log-analytics-or-application-insights-resources-and-azure-data-explorer).

+

+Although Log Analytics remains the primary data storage location for performing analysis with Azure Sentinel, there are cases where ADX is required to store data due to cost, retention periods, or other factors. This capability enables customers to hunt over a wider set of data and view the results in the [Azure Sentinel hunting experiences](hunting.md), including hunting queries, [livestream](livestream.md), and the Log Analytics search page.

+To query data stored in ADX clusters, use the adx() function to specify the ADX cluster, database name, and desired table. You can then query the output as you would any other table. See more information in the pages linked above.

+### Watchlists are in general availability

+The [watchlists](watchlists.md) feature is now generally available. Use watchlists to enrich alerts with business data, to create allowlists or blocklists against which to check access events, and to help investigate threats and reduce alert fatigue.

+### Support for data residency in more geos

+Azure Sentinel now supports full data residency in the following additional geos:

+Brazil, Norway, South Africa, Korea, Germany, United Arab Emirates (UAE), and Switzerland.

+See the [complete list of supported geos](quickstart-onboard.md#geographical-availability-and-data-residency) for data residency.

+### Bidirectional sync in Azure Defender connector (Public preview)

+The Azure Defender connector now supports bi-directional syncing of alerts' status between Defender and Azure Sentinel. When you close a Sentinel incident containing a Defender alert, the alert will automatically be closed in the Defender portal as well.

+See this [complete description of the updated Azure Defender connector](connect-defender-for-cloud.md).

+- [Maturity Model for Event Log Management (M-21-31) Solution (Public preview)](#maturity-model-for-event-log-management-m-21-31-solution-public-preview)

+### Maturity Model for Event Log Management (M-21-31) Solution (Public preview)

+The Microsoft Sentinel content hub now includes the **Maturity Model for Event Log Management (M-21-31)** solution, which integrates Microsoft Sentinel and Microsoft Defender for Cloud to provide an industry differentiator for meeting challenging requirements in regulated industries.

+The Maturity Model for Event Log Management (M-21-31) solution provides a quantifiable framework to measure maturity. Use the analytics rules, hunting queries, playbooks, and workbook provided with the solution to do any of the following:

+- Design and build log management architectures

+- Monitor and alert on log health issues, coverage, and blind spots

+- Respond to notifications with Security Orchestration Automation & Response (SOAR) activities

+- Remediate with Cloud Security Posture Management (CSPM)

+For more information, see:

+- [Modernize Log Management with the Maturity Model for Event Log Management (M-21-31) Solution](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/modernize-log-management-with-the-maturity-model-for-event-log/ba-p/3072842) (blog)

+- [Centrally discover and deploy Microsoft Sentinel out-of-the-box content and solutions](sentinel-solutions-deploy.md)

+- [The Microsoft Sentinel content hub catalog](sentinel-solutions-catalog.md#domain-solutions)

+**Recovery point retention** | Specifies how long Site Recovery keeps recovery points | 1 day

+**App-consistent snapshot frequency** | How often Site Recovery takes an app-consistent snapshot. | 0 hours (Disabled)

+>[!NOTE]

+>High recovery point retention period may have an implication on the storage cost since more recovery points may need to be saved.

+- Retain recovery points for 1 day.

+- App-consistent snapshots are disabled and are not created by default.

+- A replication policy retains recovery points for one day, and takes an app-consistent snapshot every hour.

+- Site Recovery prunes recovery points after two hours, saving one point per hour.

+So, for the recent two hours, you can choose from 24 crash-consistent points, and two app-consistent points, as shown in the graphic.

+The oldest recovery point that you can use is 15 days with Managed disk and 3 days with Unmanaged disk.

+### How does the pruning of recovery points happen?

+Crash-consistent recovery points are generated in every five minutes. App-consistent snapshots are generated based on the input frequency entered by you. Beyond two hours, pruning of recovery points may happen based on the retention period that you input. Following are the scenarios:

+|**Retention Period input** | **Pruning mechanism** |

+|-|--|

+|0 day|No recovery point saved. You can failover only to the latest point|

+|1 day|One recovery point saved per hour beyond the last two hours|

+|2 - 7 days|One recovery point saved per two hours beyond the last two hours|

+|8 - 15 days|One recovery point saved per two hours beyond the last two hours for 7 days. Post that, one recovery point saved per four hours.<p>App-consistent snapshots will also be pruned based on the duration mentioned above in the table even if you had input lesser app-consistent snapshot frequency.|

+### What happens if Site Recovery can't generate recovery points for more than one day?

+If you have a replication policy of one day, and Site Recovery can't generate recovery points for more than one day, your old recovery points remain. Site Recovery only replaces the oldest point if it generates new points. Until there are new recovery points, all the old points remain after you reach the retention window.

+Yes. For example, if you increase retention from 1 day to 3 days, Site Recovery saves recovery points for an additional two days.The added time incurs storage changes. Earlier, it was saving recovery points per hour for 1 day. Now, it is saving recovery points per two hours for 3 days. Refer [pruning of recovery points](#how-does-the-pruning-of-recovery-points-happen). So additional 12 recovery points are saved. As an example only, if a single recovery point had delta changes of 10 GB, with a per-GB cost of $0.16 per month, then additional charges would be $1.60 × 12 per month.

+ - If the resource group created by Site Recovery already exists, it's reused.

+ - **Target storage accounts (source VM doesn't use managed disks)**: By default, Site Recovery creates a new target storage account mimicking your source VM storage configuration. In case storage account already exists, it's reused.

+ - **Target availability sets**: By default, Site Recovery creates a new availability set in the target region with the "asr" suffix in the name, for VMs that are part of an availability set in the source region. If the availability set created by Site Recovery already exists, it's reused.

+ If the target region does not support availability zones, the target VMs are configured as single instances by default. If necessary, you can configure such VMs to be part of availability sets in target region by clicking 'Customize'.

+ - **Replication Policy**: It defines the settings for retention period of recovery points and app-consistent snapshot frequency. By default, Azure Site Recovery creates a default replication policy with the following settings:

+ - One day of retention for recovery points.

+ - No app-consistent snapshots.

+4. In **Recovery point retention**, specify how long (in days) the retention window is for each recovery point. Replicated VMs can be recovered to any point in a window. Up to 15 days retention is supported.

+5. In **App-consistent snapshot frequency**, specify how often (in hours) recovery points containing application-consistent snapshots will be created. Click **OK** to create the policy.

+By default, a matching policy is automatically created for failback. For example, if the replication policy is **rep-policy** then a failback policy **rep-policy-failback** is created. This policy isn't used until you initiate a failback from Azure.

+ - **Recovery point retention**. This setting specifies how far back in time you want to go when a disruption occurs. Maximum retention is 15 days.

+ >[!NOTE]

+ >High recovery point retention period may have an implication on the storage cost since more recovery points may need to be saved.

+

+**Recovery point retention** | Specifies how long Site Recovery keeps recovery points | 3 days

+ - **Recovery point retention**. This setting specifies how far back in time you want to go when a disruption occurs. Maximum retention is 15 days on Managed disk.

+ >[!NOTE]

+ >High recovery point retention period may have an implication on the storage cost since more recovery points may need to be saved.

+

+- You can customize the settings of replication policies as you enable replication.

+For VMware to Azure, the oldest recovery point you can use is 15 days.

+### How does the pruning of recovery points happen?

+Crash-consistent recovery points are generated in every five minutes. App-consistent snapshots are generated based on the input frequency entered by you. Beyond two hours, pruning of recovery points may happen based on the retention period that you input. Following are the scenarios:

+|**Retention Period input** | **Pruning mechanism** |

+|-||

+|0 day|No recovery point saved. You can failover only to the latest point|

+|1 day|One recovery point saved per hour beyond the last two hours|

+|2 - 7 days|One recovery point saved per two hours beyond the last two hours|

+|8 - 15 days|One recovery point saved per two hours beyond last two hours for 7 days. Post that, one recovery point saved per four hours.<p>App-consistent snapshots will also be pruned based on duration mentioned above even if you input lesser app-consistent snapshot frequency.|

+### Do increases in recovery point retention increase storage costs?

+Yes. For example, if you increase retention from 1 day to 3 days, Site Recovery saves recovery points for an additional 2 days.The added time incurs storage changes. Earlier, it was saving recovery points per hour for 1 day. Now, it is saving recovery points per two hours for 3 days. Refer [pruning of recovery points](#how-does-the-pruning-of-recovery-points-happen). So additional 12 recovery points are saved. As an example only, if a single recovery point had delta changes of 10 GB, with a per-GB cost of $0.16 per month, then additional charges would be $1.60 × 12 per month.

+ A default replication policy gets created under the vault with 3 days recovery point retention and 4 hours app consistency frequency. You can create a new replication policy as per your RPO requirements.

+ - Enter **Recovery point retention** in days.

+5. In **Recovery point retention**, specify (in days) the duration of the retention window for each recovery point. Protected machines can be recovered to any point within a retention window. Up to 15 days of retention is supported.

+6. In **App-consistent snapshot frequency**, you can choose to enable app-consistent snapshot frequency and input the frequency from 0 - 12 (in hours) that will determine how frequently application-consistent snapshots should be created.

+>[!NOTE]

+>High recovery point retention period in a policy may have an implication on storage cost since more recovery points may need to be saved.

+1. Select the replication policy.

+

+ 

+2. Click **Associate**.

+

+ 

+3. Select the configuration server.

+ 

+3. Click **OK**. The configuration server should be associated in one to two minutes.

+ 

+If version-level immutability support is enabled for the storage account and the account contains one or more containers, then you must delete all containers before you delete the storage account, even if there are no immutability policies in effect for the account or containers.

+If version-level immutability support is enabled for a container and the container contains one or more blobs, then you must delete all blobs in the container before you can delete the container, even if there are no immutability policies in effect for the container or its blobs.

+## Check for SAS expiration policy violations

+You can monitor your storage accounts with Azure Policy to ensure that storage accounts in your subscription have configured SAS expiration policies. Azure Storage provides a built-in policy for ensuring that accounts have this setting configured. For more information about the built-in policy, see **Storage accounts should have shared access signature (SAS) policies configured** in [List of built-in policy definitions](../../governance/policy/samples/built-in-policies.md#storage).

+### Assign the built-in policy for a resource scope

+Follow these steps to assign the built-in policy to the appropriate scope in the Azure portal:

+1. In the Azure portal, search for *Policy* to display the Azure Policy dashboard.

+1. In the **Authoring** section, select **Assignments**.

+1. Choose **Assign policy**.

+1. On the **Basics** tab of the **Assign policy** page, in the **Scope** section, specify the scope for the policy assignment. Select the **More** button to choose the subscription and optional resource group.

+1. For the **Policy definition** field, select the **More** button, and enter *storage account keys* in the **Search** field. Select the policy definition named **Storage account keys should not be expired**.

+ :::image type="content" source="media/sas-expiration-policy/policy-definition-select-portal.png" alt-text="Screenshot showing how to select the built-in policy to monitor validity intervals for shared access signatures for your storage accounts":::

+1. Select **Review + create** to assign the policy definition to the specified scope.

+ :::image type="content" source="media/sas-expiration-policy/policy-assignment-create.png" alt-text="Screenshot showing how to create the policy assignment":::

+### Monitor compliance with the key expiration policy

+To monitor your storage accounts for compliance with the key expiration policy, follow these steps:

+1. On the Azure Policy dashboard, locate the built-in policy definition for the scope that you specified in the policy assignment. You can search for *Storage accounts should have shared access signature (SAS) policies configured* in the **Search** box to filter for the built-in policy.

+1. Select the policy name with the desired scope.

+1. On the **Policy assignment** page for the built-in policy, select **View compliance**. Any storage accounts in the specified subscription and resource group that do not meet the policy requirements appear in the compliance report.

+ :::image type="content" source="media/sas-expiration-policy/policy-compliance-report-portal-inline.png" alt-text="Screenshot showing how to view the compliance report for the SAS expiration built-in policy" lightbox="media/sas-expiration-policy/policy-compliance-report-portal-expanded.png":::

+To bring a storage account into compliance, configure a SAS expiration policy for that account, as described in [Create a SAS expiration policy](#create-a-sas-expiration-policy).

+ :::image type="content" source="media/storage-account-keys-manage/policy-definition-select-portal.png" alt-text="Screenshot showing how to select the built-in policy to monitor key rotation intervals for your storage accounts":::

+**A:** Please follow this page [Increase limits by VM series](../azure-portal/supportability/per-vm-quota-requests.md). NP VMs are available in East US, West US2, West Europe, SouthEast Asia, and SouthCentral US.

+**A:** Xilinx recommends [Vitis 2021.1](https://www.xilinx.com/products/design-tools/vitis/vitis-platform.html), you can also use the Development VM marketplace options (Vitis 2021.1 Development VM for Ubuntu 18.04, Ubuntu 20.04, and CentOS 7.8)

+**A:** No, you can develop on-premise and deploy to the cloud. Please make sure to follow the [attestation documentation](./field-programmable-gate-arrays-attestation.md) to deploy on NP VMs.

+**Q:** Where should I get all the XRT / Platform files?

+**A:** xrt_202110.2.11.680

+**Q:** What are the supported Operating Systems?

+**A:** Xilinx and Microsoft have validated Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and CentOS 7.8.

+[Xilinx Alveo U250 2021.1 Deployment VM ΓÇô Ubuntu18.04](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/xilinx.xilinx_xrt2021_1_ubuntu1804_deployment_image)

+[Xilinx Alveo U250 2021.1 Deployment VM ΓÇô Ubuntu20.04](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/xilinx.xilinx_xrt2021_1_ubuntu2004_deployment_image)

+[Xilinx Alveo U250 2021.1 Deployment VM ΓÇô CentOS7.8](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/xilinx.xilinx_xrt2021_1_centos78_deployment_image)

+**Q:** Can I deploy my Own Ubuntu / CentOS VMs and install XRT / Deployment Target Platform?

+**A:** Use Kernel 4.15 per [Xilinx XRT documentation](https://www.xilinx.com/support/documentation/sw_manuals/xilinx2021_1/ug1451-xrt-release-notes.pdf)

+- xrt_202110.2.11.680_18.04-amd64-xrt.deb

+- xrt_202110.2.11.680_18.04-amd64-azure.deb

+**Q:** On Ubuntu, after rebooting my VM I can't find my FPGA(s):

+**A:** Please verify that your kernel hasn't been upgraded (uname -a). If so, please downgrade to kernel 4.1X.

+**Q:** If I deploy my own Ubuntu20.04 VM then what are the required packages and steps?

+**A:** Use Kernel 5.4 per [Xilinx XRT documentation](https://www.xilinx.com/support/documentation/sw_manuals/xilinx2021_1/ug1451-xrt-release-notes.pdf)

+

+Install the following packages.

+- xrt_202110.2.11.680_20.04-amd64-xrt.deb

+

+- xrt_202110.2.11.680_20.04-amd64-azure.deb

+

+- xilinx-u250-gen3x16-xdma-platform-2.1-3_all_18.04.deb.tar.gz

+

+- xilinx-u250-gen3x16-xdma-validate_2.1-3005608.1_all.deb

+ - xrt_202110.2.11.680_7.8.2003-x86_64-xrt.rpm

+ - xrt_202110.2.11.680_7.8.2003-x86_64-azure.rpm

+To enable Host_Mem(SB) (up to 1 Gb RAM): sudo xbutil host_mem --enable --size 1g

+**A:** No, on Azure VMs there's no management support directly from the Azure VM.

+**A:** No, the PLP is loaded automatically for you, so there's no need to load via xbmgmt commands.

+Pricing Calculator: [Pricing Calculator](https://azure.microsoft.com/pricing/calculator/)