+#Customer intent: As a developer, I want to enable my users to reset their passwords without the need for admin intervention, so that they can recover their accounts if they forget their passwords.

+> [!TIP]

+> A user can change their password by using the self-service password reset flow if they forget their password and want to reset it. You can also choose one of the following user flow options to change a user's password:

+> - If a user knows their password and wants to change it, use a [password change flow](add-password-change-policy.md).

+> - If you want to force a user to reset their password (for example, when they sign in for the first time, when their passwords have been reset by an admin, or after they've been migrated to Azure AD B2C with random passwords), use a [force password reset](force-password-reset.md) flow.

+1. The user can then enter a new password. (After the email is verified, the user can still select the **Change e-mail** button; see [Hide the change email button](#hide-the-change-email-button-optional) if you wish to remove it.)

+### Hide the change email button (Optional)

+After the email is verified, the user can still select **Change email**, enter another email address, and then repeat email verification. If you'd prefer to hide the **Change email** button, you can modify the CSS to hide the associated HTML elements in the dialog. For example, you can add the following CSS entry to selfAsserted.html and [customize the user interface by using HTML templates](customize-ui-with-html.md):

+```html

+<style type="text/css">

+ .changeClaims

+ {

+ visibility: hidden;

+ }

+</style>

+```

+You can see a basic demonstration of how user flows link in our [ASP.NET sample](https://github.com/AzureADQuickStarts/B2C-WebApp-OpenIDConnect-DotNet-SUSI).

+[**Package (MVN)**](https://mvnrepository.com/artifact/com.azure/azure-ai-documentintelligence/1.0.0-beta.1)

+**This content applies to:**  **v3.1 (GA)** **Earlier versions:**  [v3.0](?view=doc-intel-3.0.0&preserve-view=true)  [v2.1](?view=doc-intel-2.1.0&preserve-view=true)

+ Title: Document Intelligence (formerly Form Recognizer) SDK target REST API v2.1 (GA)

+description: Document Intelligence v2.1 (GA) software development kits (SDKs) expose Document Intelligence models, features and capabilities, using C#, Java, JavaScript, and Python programming language.

+ - devx-track-python

+ - ignite-2023

+monikerRange: 'doc-intel-2.1.0'

+<!-- markdownlint-disable MD024 -->

+<!-- markdownlint-disable MD036 -->

+<!-- markdownlint-disable MD001 -->

+<!-- markdownlint-disable MD051 -->

+# SDK target: REST API v2.1 (GA)

+ **REST API version v2.1 (GA) 21-06-08**

+Azure AI Document Intelligence is a cloud service that uses machine learning to analyze text and structured data from documents. The Document Intelligence software development kit (SDK) is a set of libraries and tools that enable you to easily integrate Document Intelligence models and capabilities into your applications. Document Intelligence SDK is available across platforms in C#/.NET, Java, JavaScript, and Python programming languages.

+## Supported programming languages

+Document Intelligence SDK supports the following languages and platforms:

+| Language → Document Intelligence SDK version | Package| Supported API version| Platform support |

+|:-:|:-|:-| :-|

+| [.NET/C# → 3.1.x (GA)](/dotnet/api/azure.ai.formrecognizer?view=azure-dotnet&preserve-view=true)|[NuGet](https://www.nuget.org/packages/Azure.AI.FormRecognizer)|[v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux, Docker](https://dotnet.microsoft.com/download)|

+|[Java → 3.1.x (GA)](https://azuresdkdocs.blob.core.windows.net/$web/java/azure-ai-formrecognizer/3.1.1/https://docsupdatetracker.net/index.html) |[MVN repository](https://mvnrepository.com/artifact/com.azure/azure-ai-formrecognizer/3.1.1) |[v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux](/java/openjdk/install)|

+|[JavaScript → 3.1.0 (GA)](https://azuresdkdocs.blob.core.windows.net/$web/javascript/azure-ai-form-recognizer/3.1.0/https://docsupdatetracker.net/index.html)| [npm](https://www.npmjs.com/package/@azure/ai-form-recognizer/v/3.1.0)|[v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) | [Browser, Windows, macOS, Linux](https://nodejs.org/en/download/) |

+|[Python → 3.1.0 (GA)](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-ai-formrecognizer/3.1.0/https://docsupdatetracker.net/index.html) | [PyPI](https://pypi.org/project/azure-ai-formrecognizer/3.1.0/)|[v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux](/azure/developer/python/configure-local-development-environment?tabs=windows%2Capt%2Ccmd#use-the-azure-cli)|

+## Supported Clients

+| Language| SDK version | API version | Supported clients|

+| : | :--|:- | :--|

+|.NET/C#</br> Java</br> JavaScript</br>| 3.1.x | v2.1 (default)</br>v2.0 | **FormRecognizerClient**</br>**FormTrainingClient** |

+|.NET/C#</br> Java</br> JavaScript</br>| 3.0.x| v2.0 | **FormRecognizerClient**</br>**FormTrainingClient** |

+| Python | 3.1.x | v2.1 (default)</br>v2.0 |**FormRecognizerClient**</br>**FormTrainingClient** |

+| Python | 3.0.0 | v2.0 |**FormRecognizerClient**</br>**FormTrainingClient** |

+## Use Document Intelligence SDK in your applications

+The Document Intelligence SDK enables the use and management of the Document Intelligence service in your application. The SDK builds on the underlying Document Intelligence REST API allowing you to easily use those APIs within your programming language paradigm. Here's how you use the Document Intelligence SDK for your preferred language:

+### 1. Install the SDK client library

+### [C#/.NET](#tab/csharp)

+```dotnetcli

+dotnet add package Azure.AI.FormRecognizer --version 3.1.0

+```

+```powershell

+Install-Package Azure.AI.FormRecognizer -Version 3.1.0

+```

+### [Java](#tab/java)

+```xml

+<dependency>

+<groupId>com.azure</groupId>

+<artifactId>azure-ai-formrecognizer</artifactId>

+<version>3.1.0</version>

+</dependency>

+```

+```kotlin

+implementation("com.azure:azure-ai-formrecognizer:3.1.0")

+```

+### [JavaScript](#tab/javascript)

+```javascript

+npm i @azure/ai-form-recognizer@3.1.0

+```

+### [Python](#tab/python)

+```python

+pip install azure-ai-formrecognizer==3.1.0

+```

+### 2. Import the SDK client library into your application

+### [C#/.NET](#tab/csharp)

+```csharp

+using Azure;

+using Azure.AI.FormRecognizer.Models;

+```

+### [Java](#tab/java)

+```java

+import com.azure.ai.formrecognizer.*;

+import com.azure.ai.formrecognizer.models.*;

+import com.azure.core.credential.AzureKeyCredential;

+```

+### [JavaScript](#tab/javascript)

+```javascript

+const { FormRecognizerClient, AzureKeyCredential } = require("@azure/ai-form-recognizer");

+```

+### [Python](#tab/python)

+```python

+ from azure.ai.formrecognizer import FormRecognizerClient

+ from azure.core.credentials import AzureKeyCredential

+```

+### 3. Set up authentication

+There are two supported methods for authentication

+* Use a [Document Intelligence API key](#use-your-api-key) with AzureKeyCredential from azure.core.credentials.

+* Use a [token credential from azure-identity](#use-an-azure-active-directory-azure-ad-token-credential) to authenticate with [Microsoft Entra ID](../../active-directory/fundamentals/active-directory-whatis.md).

+#### Use your API key

+Here's where to find your Document Intelligence API key in the Azure portal:

+### [C#/.NET](#tab/csharp)

+```csharp

+//set `<your-endpoint>` and `<your-key>` variables with the values from the Azure portal to create your `AzureKeyCredential` and `FormRecognizerClient` instance

+ string key = "<your-key>";

+ string endpoint = "<your-endpoint>";

+ FormRecognizerClient client = new FormRecognizerClient(new Uri(endpoint), new AzureKeyCredential(key));

+```

+### [Java](#tab/java)

+```java

+// create your `FormRecognizerClient` instance and `AzureKeyCredential` variable

+FormRecognizerClient formRecognizerClient = new FormRecognizerClientBuilder()

+ .credential(new AzureKeyCredential("<your-key>"))

+ .endpoint("<your-endpoint>")

+ .buildClient();

+```

+### [JavaScript](#tab/javascript)

+```javascript

+// create your `FormRecognizerClient` instance and `AzureKeyCredential` variable

+async function main() {

+ const client = new FormRecognizerClient("<your-endpoint>", new AzureKeyCredential("<your-key>"));

+```

+### [Python](#tab/python)

+```python

+# create your `FormRecognizerClient` instance and `AzureKeyCredential` variable

+ form_recognizer_client = FormRecognizerClient(endpoint="<your-endpoint>", credential=AzureKeyCredential("<your-key>"))

+```

+<a name='use-an-azure-active-directory-azure-ad-token-credential'></a>

+#### Use a Microsoft Entra token credential

+> [!NOTE]

+> Regional endpoints do not support Microsoft Entra authentication. Create a [custom subdomain](../../ai-services/authentication.md?tabs=powershell#create-a-resource-with-a-custom-subdomain) for your resource in order to use this type of authentication.

+Authorization is easiest using the `DefaultAzureCredential`. It provides a default token credential, based upon the running environment, capable of handling most Azure authentication scenarios.

+### [C#/.NET](#tab/csharp)

+Here's how to acquire and use the [DefaultAzureCredential](/dotnet/api/azure.identity.defaultazurecredential?view=azure-dotnet&preserve-view=true) for .NET applications:

+1. Install the [Azure Identity library for .NET](/dotnet/api/overview/azure/identity-readme):

+ ```console

+ dotnet add package Azure.Identity

+ ```

+ ```powershell

+ Install-Package Azure.Identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret in the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`FormRecognizerClient`** instance including the **`DefaultAzureCredential`**:

+ ```csharp

+ string endpoint = "<your-endpoint>";

+ var client = new FormRecognizerClient(new Uri(endpoint), new DefaultAzureCredential());

+ ```

+For more information, *see* [Authenticate the client](https://github.com/Azure/azure-sdk-for-net/tree/Azure.AI.FormRecognizer_4.0.0-beta.4/sdk/formrecognizer/Azure.AI.FormRecognizer#authenticate-the-client)

+### [Java](#tab/java)

+Here's how to acquire and use the [DefaultAzureCredential](/java/api/com.azure.identity.defaultazurecredential?view=azure-java-stable&preserve-view=true) for Java applications:

+1. Install the [Azure Identity library for Java](/java/api/overview/azure/identity-readme?view=azure-java-stable&preserve-view=true):

+ ```xml

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-identity</artifactId>

+ <version>1.5.3</version>

+ </dependency>

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`FormRecognizerClient`** instance and **`TokenCredential`** variable:

+ ```java

+ TokenCredential credential = new DefaultAzureCredentialBuilder().build();

+ FormRecognizerClient formRecognizerClient = new FormRecognizerClientBuilder()

+ .endpoint("{your-endpoint}")

+ .credential(credential)

+ .buildClient();

+ ```

+For more information, *see* [Authenticate the client](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/formrecognizer/azure-ai-formrecognizer#authenticate-the-client)

+### [JavaScript](#tab/javascript)

+Here's how to acquire and use the [DefaultAzureCredential](/javascript/api/@azure/identity/defaultazurecredential?view=azure-node-latest&preserve-view=true) for JavaScript applications:

+1. Install the [Azure Identity library for JavaScript](/javascript/api/overview/azure/identity-readme?view=azure-node-latest&preserve-view=true):

+ ```javascript

+ npm install @azure/identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`FormRecognizerClient`** instance including the **`DefaultAzureCredential`**:

+ ```javascript

+ const { FormRecognizerClient } = require("@azure/ai-form-recognizer");

+ const { DefaultAzureCredential } = require("@azure/identity");

+ const client = new FormRecognizerClient("<your-endpoint>", new DefaultAzureCredential());

+ ```

+For more information, *see* [Create and authenticate a client](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/formrecognizer/ai-form-recognizer#create-and-authenticate-a-client).

+### [Python](#tab/python)

+Here's how to acquire and use the [DefaultAzureCredential](/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python&preserve-view=true) for Python applications.

+1. Install the [Azure Identity library for Python](/python/api/overview/azure/identity-readme?view=azure-python&preserve-view=true):

+ ```python

+ pip install azure-identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`FormRecognizerClient`** instance including the **`DefaultAzureCredential`**:

+ ```python

+ from azure.identity import DefaultAzureCredential

+ from azure.ai.formrecognizer import FormRecognizerClient

+ credential = DefaultAzureCredential()

+ form_recognizer_client = FormRecognizerClient(

+ endpoint="https://<my-custom-subdomain>.cognitiveservices.azure.com/",

+ credential=credential

+ )

+ ```

+For more information, *see* [Authenticate the client](https://github.com/Azure/azure-sdk-for-python/tree/azure-ai-formrecognizer_3.2.0b5/sdk/formrecognizer/azure-ai-formrecognizer#authenticate-the-client)

+### 4. Build your application

+Create a client object to interact with the Document Intelligence SDK, and then call methods on that client object to interact with the service. The SDKs provide both synchronous and asynchronous methods. For more insight, try a [quickstart](quickstarts/get-started-sdks-rest-api.md?view=doc-intel-3.0.0&preserve-view=true) in a language of your choice.

+## Help options

+The [Microsoft Q&A](/answers/topics/azure-form-recognizer.html) and [Stack Overflow](https://stackoverflow.com/questions/tagged/azure-form-recognizer) forums are available for the developer community to ask and answer questions about Azure AI Document Intelligence and other services. Microsoft monitors the forums and replies to questions that the community has yet to answer. To make sure that we see your question, tag it with **`azure-form-recognizer`**.

+## Next steps

+>[!div class="nextstepaction"]

+> [**Explore Document Intelligence REST API v2.1**](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)

+> [!div class="nextstepaction"]

+> [**Try a Document Intelligence quickstart**](quickstarts/get-started-sdks-rest-api.md?view=doc-intel-2.1.0&preserve-view=true)

+ Title: Document Intelligence (formerly Form Recognizer) SDK target REST API 2022ΓÇô08ΓÇô31 (GA)

+description: Document Intelligence 2022ΓÇô08ΓÇô31 (GA) software development kits (SDKs) expose Document Intelligence models, features and capabilities, using C#, Java, JavaScript, and Python programming language.

+# SDK target: REST API 2022ΓÇô08ΓÇô31 (GA)

+ **REST API version 2022ΓÇô08ΓÇô31 (GA)**

+|[Java → 4.0.6 (GA)](https://azuresdkdocs.blob.core.windows.net/$web/java/azure-ai-formrecognizer/4.0.0/https://docsupdatetracker.net/index.html) |[MVN repository](https://mvnrepository.com/artifact/com.azure/azure-ai-formrecognizer/4.0.6) |[v3.0](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux](/java/openjdk/install)|

+ Title: Document Intelligence (formerly Form Recognizer) SDK target REST API 2023-07-31 (GA) latest.

+description: The Document Intelligence 2023-07-31 (GA) software development kits (SDKs) expose Document Intelligence models, features and capabilities that are in active development for C#, Java, JavaScript, or Python programming language.

+

+# SDK target: REST API 2023-07-31 (GA) latest

+ **REST API version 2023-07-31 (GA)**

+ Title: Document Intelligence (formerly Form Recognizer) SDK target REST API 2023-10-31-preview

+description: The Document Intelligence 2023-10-31-preview software development kits (SDKs) expose Document Intelligence models, features and capabilities that are in active development for C#, Java, JavaScript, or Python programming language.

+ - devx-track-python

+ - ignite-2023

+monikerRange: 'doc-intel-4.0.0'

+

+<!-- markdownlint-disable MD024 -->

+<!-- markdownlint-disable MD036 -->

+<!-- markdownlint-disable MD001 -->

+<!-- markdownlint-disable MD051 -->

+# SDK target: REST API 2023-10-31-preview

+ **REST API version 2023-10-31-preview**

+Azure AI Document Intelligence is a cloud service that uses machine learning to analyze text and structured data from documents. The Document Intelligence software development kit (SDK) is a set of libraries and tools that enable you to easily integrate Document Intelligence models and capabilities into your applications. Document Intelligence SDK is available across platforms in C#/.NET, Java, JavaScript, and Python programming languages.

+## Supported programming languages

+Document Intelligence SDK supports the following languages and platforms:

+| Language → Document Intelligence SDK version &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;| Package| Supported API version &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;| Platform support |

+|:-:|:-|:-| :-:|

+| [**.NET/C# → 1.0.0-beta.1 (preview)**](/dotnet/api/azure.ai.documentintelligence.documentintelligenceadministrationclient?view=azure-dotnet-preview&preserve-view=true)|[NuGet](https://www.nuget.org/packages/Azure.AI.DocumentIntelligence/1.0.0-beta.1)|[&bullet; 2023-10-31 (preview)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-10-31-preview&preserve-view=true&tabs=HTTP)</br>[&bullet; 2023-07-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; 2022-08-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[&bullet; v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux, Docker](https://dotnet.microsoft.com/download)|

+ |[**Java → 1.0.0-beta.1 (preview)**](/java/api/overview/azure/ai-documentintelligence-readme?view=azure-java-preview&preserve-view=true) |[MVN repository](https://mvnrepository.com/artifact/com.azure/azure-ai-documentintelligence/1.0.0-beta.1) |[&bullet; 2023-10-31 (preview)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-10-31-preview&preserve-view=true&tabs=HTTP)</br>[&bullet; 2023-07-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; 2022-08-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[&bullet; v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux](/java/openjdk/install)|

+|[**JavaScript → 1.0.0-beta.1 (preview)**](/javascript/api/overview/azure/ai-document-intelligence-rest-readme?view=azure-node-preview&preserve-view=true)| [npm](https://www.npmjs.com/package/@azure-rest/ai-document-intelligence)|[&bullet; 2023-10-31 (preview)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-10-31-preview&preserve-view=true&tabs=HTTP)</br>[&bullet; 2023-07-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> &bullet; [2022-08-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[&bullet; v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) | [Browser, Windows, macOS, Linux](https://nodejs.org/en/download/) |

+|[**Python → 1.0.0b1 (preview)**](/python/api/overview/azure/ai-documentintelligence-readme?view=azure-python-preview&preserve-view=true) | [PyPI](https://pypi.org/project/azure-ai-documentintelligence/)|[&bullet; 2023-10-31 (preview)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-10-31-preview&preserve-view=true&tabs=HTTP)</br>[&bullet; 2023-07-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> &bullet; [2022-08-31 (GA)](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP)</br> [&bullet; v2.1](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1/operations/AnalyzeBusinessCardAsync)</br>[&bullet; v2.0](https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync) |[Windows, macOS, Linux](/azure/developer/python/configure-local-development-environment?tabs=windows%2Capt%2Ccmd#use-the-azure-cli)

+## Supported Clients

+The following tables present the correlation between each SDK version the supported API versions of the Document Intelligence service.

+### [C#/.NET](#tab/csharp)

+| Language| SDK alias | API version (default) &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; | Supported clients|

+| : | :--|:- | :--|

+ |**.NET/C# 1.0.0-beta.1 (preview)**| v4.0 (preview)| 2023-10-31-preview|**DocumentIntelligenceClient**</br>**DocumentIntelligenceAdministrationClient**|

+|**.NET/C# 4.1.0**| v3.1 latest (GA)| 2023-07-31|**DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**.NET/C# 4.0.0**| v3.0 (GA)| 2022-08-31| **DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**.NET/C# 3.1.x**| v2.1 | v2.1 | **FormRecognizerClient**</br>**FormTrainingClient** |

+|**.NET/C# 3.0.x**| v2.0 | v2.0 | **FormRecognizerClient**</br>**FormTrainingClient** |

+### [Java](#tab/java)

+| Language| SDK alias | API version &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; | Supported clients|

+| : | :--|:- | :--|

+|**Java 1.0.0-beta.1 (preview)**| v4.0 preview| 2023-10-31 (default)|**DocumentIntelligenceClient**</br>**DocumentIntelligenceAdministrationClient**|

+|**Java 4.1.0**| v3.1 latest (GA)| 2023-07-31 (default)|**DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**Java 4.0.0**</br>| v3.0 (GA)| 2022-08-31| **DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**Java 3.1.x**| v2.1 | v2.1 | **FormRecognizerClient**</br>**FormTrainingClient** |

+|**Java 3.0.x**| v2.0| v2.0 | **FormRecognizerClient**</br>**FormTrainingClient** |

+### [JavaScript](#tab/javascript)

+| Language| SDK alias | API version (default) &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; | Supported clients|

+| : | :--|:- | :--|

+|**JavaScript 1.0.0-beta.1**| v4.0 (preview)| 2023-10-31 (default)|**DocumentIntelligenceClient**</br>**DocumentIntelligenceAdministrationClient**|

+|**JavaScript 5.0.0**| v3.1 latest (GA)| 2023-07-31 (default)|**DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**JavaScript 4.0.0**</br>| v3.0 (GA)| 2022-08-31| **DocumentAnalysisClient**</br>**DocumentModelAdministrationClient** |

+|**JavaScript 3.1.x**</br>| v2.1 | v2.1 | **FormRecognizerClient**</br>**FormTrainingClient** |

+|**JavaScript 3.0.x**</br>| v2.0| v2.0 | **FormRecognizerClient**</br>**FormTrainingClient** |

+### [Python](#tab/python)

+| Language| SDK alias | API version (default) &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; | Supported clients|

+| : | :--|:- | :--|

+| **Python 1.0.0b1**| v4.0 (preview)| 2023-10-31 (default) |**DocumentIntelligenceClient**</br>**DocumentIntelligenceAdministrationClient**|

+| **Python 3.3.0**| v3.1 latest (GA)| 2023-07-31 (default) | **DocumentAnalysisClient**</br>**DocumentModelAdministrationClient**|

+| **Python 3.2.x**| v3.0 (GA)| 2022-08-31| **DocumentAnalysisClient**</br>**DocumentModelAdministrationClient**|

+| **Python 3.1.x**| v2.1 | v2.1 | **FormRecognizerClient**</br>**FormTrainingClient** |

+| **Python 3.0.0** | v2.0 | v2.0 |**FormRecognizerClient**</br>**FormTrainingClient** |

+## Use Document Intelligence SDK in your applications

+The Document Intelligence SDK enables the use and management of the Document Intelligence service in your application. The SDK builds on the underlying Document Intelligence REST API allowing you to easily use those APIs within your programming language paradigm. Here's how you use the Document Intelligence SDK for your preferred language:

+### 1. Install the SDK client library

+### [C#/.NET](#tab/csharp)

+```dotnetcli

+dotnet add package Azure.AI.DocumentIntelligence --version 1.0.0-beta.1

+```

+```powershell

+Install-Package Azure.AI.FormRecognizer -Version 1.0.0-beta.1

+```

+### [Java](#tab/java)

+```xml

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-ai-documentintelligence</artifactId>

+ <version>1.0.0-beta.1</version>

+ </dependency>

+```

+```kotlin

+implementation("com.azure:azure-ai-documentintelligence:1.0.0-beta.1")

+```

+### [JavaScript](#tab/javascript)

+```console

+npm i @azure-rest/ai-document-intelligence@1.0.0-beta.1

+```

+### [Python](#tab/python)

+```python

+pip install azure-ai-documentintelligence==1.0.0b1

+```

+### 2. Import the SDK client library into your application

+### [C#/.NET](#tab/csharp)

+```csharp

+using Azure;

+using Azure.AI.DocumentIntelligence;

+```

+### [Java](#tab/java)

+```java

+import com.azure.ai.documentintelligence.*;

+import com.azure.ai.documentintelligence.models.*;

+import com.azure.core.credential.AzureKeyCredential;

+```

+### [JavaScript](#tab/javascript)

+```javascript

+const { AzureKeyCredential, DocumentIntelligence } = require("@azure-rest/ai-document-intelligence@1.0.0-beta.1");

+```

+### [Python](#tab/python)

+```python

+from azure.ai.documentintelligence import DocumentIntelligenceClient

+from azure.core.credentials import AzureKeyCredential

+```

+### 3. Set up authentication

+There are two supported methods for authentication

+* Use a [Document Intelligence API key](#use-your-api-key) with AzureKeyCredential from azure.core.credentials.

+* Use a [token credential from azure-identity](#use-an-azure-active-directory-azure-ad-token-credential) to authenticate with [Microsoft Entra ID](../../active-directory/fundamentals/active-directory-whatis.md).

+#### Use your API key

+Here's where to find your Document Intelligence API key in the Azure portal:

+### [C#/.NET](#tab/csharp)

+```csharp

+//set `<your-endpoint>` and `<your-key>` variables with the values from the Azure portal to create your `AzureKeyCredential` and `DocumentIntelligenceClient` instance

+string key = "<your-key>";

+string endpoint = "<your-endpoint>";

+AzureKeyCredential credential = new AzureKeyCredential(key);

+DocumentIntelligenceClient client = new DocumentIntelligenceClient(new Uri(endpoint), new AzureKeyCredential(key));

+```

+### [Java](#tab/java)

+```java

+// create your `DocumentIntelligenceClient` instance and `AzureKeyCredential` variable

+DocumentIntelligenceClient documentIntelligenceClient = new DocumentIntelligenceClientBuilder()

+ .credential(new AzureKeyCredential("<your-key>"))

+ .endpoint("<your-endpoint>")

+ .buildClient();

+```

+### [JavaScript](#tab/javascript)

+```javascript

+// create your `DocumentIntelligenceClient` instance and `AzureKeyCredential` variable

+async function main() {

+ const client = DocumentIntelligence(process.env["your-endpoint>"], {

+ key: process.env["<your-key>"],

+});

+```

+### [Python](#tab/python)

+```python

+# create your `DocumentIntelligenceClient` instance and `AzureKeyCredential` variable

+ endpoint = "<your-endpoint>"

+ credential = AzureKeyCredential("<your-key>")

+ document_analysis_client = DocumentIntelligenceClient(endpoint, credential)

+```

+<a name='use-an-azure-active-directory-azure-ad-token-credential'></a>

+#### Use a Microsoft Entra token credential

+> [!NOTE]

+> Regional endpoints do not support Microsoft Entra authentication. Create a [custom subdomain](../../ai-services/authentication.md?tabs=powershell#create-a-resource-with-a-custom-subdomain) for your resource in order to use this type of authentication.

+Authorization is easiest using the `DefaultAzureCredential`. It provides a default token credential, based upon the running environment, capable of handling most Azure authentication scenarios.

+### [C#/.NET](#tab/csharp)

+Here's how to acquire and use the [DefaultAzureCredential](/dotnet/api/azure.identity.defaultazurecredential?view=azure-dotnet&preserve-view=true) for .NET applications:

+1. Install the [Azure Identity library for .NET](/dotnet/api/overview/azure/identity-readme):

+ ```console

+ dotnet add package Azure.Identity

+ ```

+ ```powershell

+ Install-Package Azure.Identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret in the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`DocumentIntelligenceClient`** instance including the **`DefaultAzureCredential`**:

+ ```csharp

+ string endpoint = "<your-endpoint>";

+ var client = new DocumentIntelligenceClient(new Uri(endpoint), new DefaultAzureCredential());

+ ```

+For more information, *see* [Authenticate the client](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/documentintelligence/Azure.AI.DocumentIntelligence/README.md#authenticate-the-client)

+### [Java](#tab/java)

+Here's how to acquire and use the [DefaultAzureCredential](/java/api/com.azure.identity.defaultazurecredential?view=azure-java-stable&preserve-view=true) for Java applications:

+1. Install the [Azure Identity library for Java](/java/api/overview/azure/identity-readme?view=azure-java-stable&preserve-view=true):

+ ```xml

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-identity</artifactId>

+ <version>1.5.3</version>

+ </dependency>

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`DocumentIntelligenceClient`** instance and **`TokenCredential`** variable:

+ ```java

+ TokenCredential credential = new DefaultAzureCredentialBuilder().build();

+ DocumentIntelligenceClient documentIntelligenceClient = new DocumentIntelligenceClientBuilder()

+ .endpoint("{your-endpoint}")

+ .credential(credential)

+ .buildClient();

+ ```

+For more information, *see* [Authentication](https://github.com/Azure/azure-sdk-for-jav#authentication)

+### [JavaScript](#tab/javascript)

+Here's how to acquire and use the [DefaultAzureCredential](/javascript/api/@azure/identity/defaultazurecredential?view=azure-node-latest&preserve-view=true) for JavaScript applications:

+1. Install the [Azure Identity library for JavaScript](/javascript/api/overview/azure/identity-readme?view=azure-node-latest&preserve-view=true):

+ ```javascript

+ npm install @azure/identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`DocumentIntelligenceClient`** instance including the **`DefaultAzureCredential`**:

+ ```javascript

+ const { DocumentIntelligenceClient } = require("@azure-rest/ai-document-intelligence@1.0.0-beta.1");

+ const { DefaultAzureCredential } = require("@azure/identity");

+ const client = new DocumentIntelligenceClient("<your-endpoint>", new DefaultAzureCredential());

+ ```

+For more information, *see* [Create and authenticate a client](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/documentintelligence/ai-document-intelligence-rest#create-and-authenticate-a-documentintelligenceclient).

+### [Python](#tab/python)

+Here's how to acquire and use the [DefaultAzureCredential](/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python&preserve-view=true) for Python applications.

+1. Install the [Azure Identity library for Python](/python/api/overview/azure/identity-readme?view=azure-python&preserve-view=true):

+ ```python

+ pip install azure-identity

+ ```

+1. [Register a Microsoft Entra application and create a new service principal](../../ai-services/authentication.md?tabs=powershell#assign-a-role-to-a-service-principal).

+1. Grant access to Document Intelligence by assigning the **`Cognitive Services User`** role to your service principal.

+1. Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables: **`AZURE_CLIENT_ID`**, **`AZURE_TENANT_ID`**, and **`AZURE_CLIENT_SECRET`**, respectively.

+1. Create your **`DocumentIntelligenceClient`** instance including the **`DefaultAzureCredential`**:

+ ```python

+ from azure.identity import DefaultAzureCredential

+ from azure.ai.documentintelligence import DocumentIntelligenceClient

+ credential = DefaultAzureCredential()

+ client = DocumentIntelligenceClient(

+ endpoint="<your-endpoint>",

+ credential=credential

+ )

+ ```

+For more information, *see* [Authenticate the client](https://github.com/Azure/azure-sdk-for-python/blob/7c42462ac662522a6fd21b17d2a20f4cd40d0356/sdk/documentintelligence/azure-ai-documentintelligence/README.md#authenticate-the-client)

+### 4. Build your application

+Create a client object to interact with the Document Intelligence SDK, and then call methods on that client object to interact with the service. The SDKs provide both synchronous and asynchronous methods. For more insight, try a [quickstart](quickstarts/get-started-sdks-rest-api.md?view=doc-intel-3.0.0&preserve-view=true) in a language of your choice.

+## Help options

+The [Microsoft Q&A](/answers/tags/440/document-intelligence) and [Stack Overflow](https://stackoverflow.com/questions/tagged/azure-ai-document-intelligence) forums are available for the developer community to ask and answer questions about Azure AI Document Intelligence and other services. Microsoft monitors the forums and replies to questions that the community has yet to answer. To make sure, use the following tags so that we see your question

+* Microsoft Q&A: **`Azure AI Document Intelligence`**.

+* Stack Overflow: **`azure-ai-document-intelligence`**

+## Next steps

+> [!div class="nextstepaction"]

+>Explore [**Document Intelligence REST API 2023-10-31-rest**](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-2023-07-31&preserve-view=true&tabs=HTTP) operations.

+- **Cost savings:** High throughput workloads may provide cost savings vs token-based consumption.

+While we make every attempt to ensure that quota is always deployable, quota does not represent a guarantee that the underlying capacity is available for the customer to use. The service assigns capacity to the customer at deployment time and if capacity is unavailable the deployment will fail with an out of capacity error.

+## Azure Role-based access controls (Azure RBAC) for adding data sources

+To add a new data source to Azure OpenAI on your data, you need the following Azure RBAC roles.

+|Azure RBAC role | Which resource needs this role? | Needed when |

+||||

+| [Cognitive Services OpenAI Contributor](../how-to/role-based-access-control.md#cognitive-services-openai-contributor) | The Azure AI Search resource, to access Azure OpenAI resource. | You want to use Azure OpenAI on your data. |

+|[Search Index Data Reader](/azure/role-based-access-control/built-in-roles#search-index-data-reader) | The Azure OpenAI resource, to access the Azure AI Search resource. | You want to use Azure OpenAI on your data. |

+|[Search Service Contributor](/azure/role-based-access-control/built-in-roles#search-service-contributor) | The Azure OpenAI resource, to access the Azure AI Search resource. | You plan to create a new Azure AI Search index. |

+|[Storage Blob Data Contributor](/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) | You have an existing Blob storage container that you want to use, instead of creating a new one. | The Azure AI Search and Azure OpenAI resources, to access the storage account. |

+| [Cognitive Services OpenAI User](../how-to/role-based-access-control.md#cognitive-services-openai-user) | The web app, to access the Azure OpenAI resource. | You want to deploy a web app. |

+| [Contributor](/azure/role-based-access-control/built-in-roles#contributor) | Your subscription, to access Azure Resource Manager. | You want to deploy a web app. |

+| [Cognitive Services Contributor Role](/azure/role-based-access-control/built-in-roles#cognitive-services-contributor) | The Azure AI Search resource, to access Azure OpenAI resource. | You want to deploy a [web app](#using-the-web-app). |

+## Virtual network support & private endpoint support (Azure AI Search only)

+> [!TIP]

+> For instructions on setting up your resources to work on a virtual private network or private endpoint, see [Use Azure OpenAI on your data securely](../how-to/use-your-data-securely.md)

+ Title: 'Using your data with Azure OpenAI securely'

+description: Use this article to learn about securely using your data for text generation in Azure OpenAI.

+#

+recommendations: false

+# Securely use Azure OpenAI on your data

+Use this article to learn how to use Azure OpenAI on Your Data securely by protecting data with virtual networks and private endpoints.

+## Data ingestion architecture

+When you ingest data into Azure OpenAI on your data, the following process is used to process the data and store it in blob storage. This applies to the following data sources:

+* local files

+* Azure blob storage

+* URLs

+1. The ingestion process is started when a client sends data to be processed.

+1. Ingestion assets (indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) and container in the search resource) are created in the Azure AI Search resource and Azure storage account.

+1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), the ingestion process starts at `[3]`.

+1. Azure OpenAI's `preprocessing-jobs` API implements the [Azure AI Search customer skill web API protocol](/azure/search/cognitive-search-custom-skill-web-api), and processes the documents in a queue.

+1. Azure OpenAI:

+ 1. Internally uses the indexer created earlier to crack the documents.

+ 1. Uses a heuristic-based algorithm to perform chunking, honoring table layouts and other formatting elements in the chunk boundary to ensure the best chunking quality.

+ 1. If you choose to enable vector search, uses the current embedding model to vectorize the chunks, if `embeddingDeploymentName` is specified in the request header.

+1. When all the data that the service is monitoring are processed, Azure OpenAI triggers another indexer.

+1. The indexer stores the processed data into an Azure AI Search service.

+For the managed identities used in service calls, only system assigned managed identities are supported. User assigned managed identities aren't supported.

+## Inference architecture

+When you send API calls to chat with an Azure OpenAI model on your data, the service needs to retrieve the index fields during inference to perform fields mapping automatically if the fields mapping isn't explicitly set in the request. Therefore the service requires the Azure OpenAI identity to have the `Search Service Contributor` role for the search service even during inference.

+## Resources setup

+Use the following sections to set your resources for secure usage. If you plan to set up security for your resources, you should complete all of the following sections. For more information on inbound and outbound data flow, see the [Azure AI search documentation](/azure/search/search-security-overview).

+## Security support for Azure OpenAI

+### Inbound security: networking

+

+You can set the Azure OpenAI service networking by allowing access from the **Selected Networks and Private Endpoints** section in the Azure portal.

+If you use [Azure Management REST API](/rest/api/cognitiveservices/accountmanagement/accounts/update), you can set `networkAcls.defaultAction` as `Deny`

+```json

+...

+"networkAcls": {

+ "defaultAction": "Deny",

+ "ipRules": [

+ {

+ "value": "4.155.49.0/24"

+ }

+ ]

+},

+"privateEndpointConnections": [],

+"publicNetworkAccess": "Enabled"

+...

+```

+> [!NOTE]

+> To use Azure OpenAI Studio, you cannot set `publicNetworkAccess` as `Disabled`, because you need to add your local IP to the IP rules, so Azure OpenAI Studio can call the Azure OpenAI API for both ingestion and inference from your browser.

+### Inbound security: trusted service

+To allow Azure AI Search to call Azure OpenAI `preprocessing-jobs` as custom skill web API, while Azure OpenAI is network restricted, you'll need to set up Azure OpenAI to bypass Azure AI Search as a trusted service. Azure OpenAI will identify the traffic from Azure AI Search by verifying the claims in the JSON Web Token (JWT). Azure AI Search must use the system assigned managed identity authentication to call the custom skill web API. Set `networkAcls.bypass` as `AzureServices` from the management API. See [Virtual networks article](/azure/ai-services/cognitive-services-virtual-networks?tabs=portal#grant-access-to-trusted-azure-services-for-azure-openai) for more information.

+### Outbound security: managed identity

+To allow other services to recognize Azure OpenAI via Azure Active Directory (Azure AD) authentication, you need to assign a managed identity for your Azure OpenAI service. The easiest way is to toggle on system assigned managed identity on Azure portal.

+You can also add a user assigned managed identity, but using user assigned managed identities is only supported by the inference API, not in the ingestion API.

+> [!TIP]

+> Unless you are in an advanced stage of development and ready for production, we recommend using the system assigned managed identity.

+To set the managed identities via the management API, see [the management API reference documentation](/rest/api/cognitiveservices/accountmanagement/accounts/update#identity).

+```json

+"identity": {

+ "principalId": "12345678-abcd-1234-5678-abc123def", "tenantId": "1234567-abcd-1234-1234-abcd1234", "type": "SystemAssigned, UserAssigned",

+ "userAssignedIdentities": {

+ "/subscriptions/1234-5678-abcd-1234-1234abcd/resourceGroups/my-resource-group",

+ "principalId": "12345678-abcd-1234-5678-abcdefg1234",

+ "clientId": "12345678-abcd-efgh-1234-12345678"

+ }

+ }

+```

+## Security support for Azure AI Search

+### Inbound security: authentication

+As Azure OpenAI will use managed identity to access Azure AI Search, you need to enable Azure AD based authentication in your Azure AI Search. To do it on Azure portal, select **Both** in the **Keys** tab in the Azure portal.

+To enable Azure AD via the REST API, set `authOptions` as `aadOrApiKey`. See the [Azure AI Search RBAC article](/azure/search/search-security-rbac?tabs=config-svc-rest%2Croles-portal%2Ctest-portal%2Ccustom-role-portal%2Cdisable-keys-portal#configure-role-based-access-for-data-plane) for more information.

+```json

+"disableLocalAuth": false,

+"authOptions": {

+ "aadOrApiKey": {

+ "aadAuthFailureMode": "http401WithBearerChallenge"

+ }

+}

+```

+To use Azure OpenAI Studio, you can't disable the API key based authentication for Azure AI Search, because Azure OpenAI Studio uses the API key to call the Azure AI Search API from your browser.

+> [!TIP]

+> For the best security, when you are ready for production and no longer need to use Azure OpenAI Studio for testing, we recommend that you disable the API key. See the [Azure AI Search RBAC article](/azure/search/search-security-rbac?tabs=config-svc-portal%2Croles-portal%2Ctest-portal%2Ccustom-role-portal%2Cdisable-keys-portal#disable-api-key-authentication) for details.

+### Inbound security: networking

+Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for theAzure OpenAI on your data (as a multitenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](https://aka.ms/applyacsvpnaoaioyd).

+> [!NOTE]

+> To use Azure OpenAI Studio, you cannot disable public network access, and you need to add your local IP to the IP rules, because Azure AI Studio calls the search API from your browser to list available indexes.

+### Outbound security: managed identity

+To allow other services to recognize the Azure AI Search using Azure AD authentication, you need to assign a managed identity for your Azure AI Search service. The easiest way is to toggle the system assigned managed identity in the Azure portal to **on**.

+User assigned managed identities aren't supported.

+## Security support for Azure blob storage

+### Inbound security: networking

+In the Azure portal, navigate to your storage account networking tab and select **Enabled from selected virtual networks and IP addresses**.

+Make sure **Allow Azure services on the trusted services list to access this storage account** is selected, so Azure OpenAI and Azure AI Search can bypass the network restriction of your storage account when using a managed identity for authentication.

+To use Azure OpenAI Studio, make sure to add your local IP to the IP rules, so the Azure OpenAI Studio can upload files to the storage account from your browser.

+## Role assignments

+So far you have already setup each resource work independently. Next you will need to allow the services to authorize each other. The minimum requirements are listed below.

+|Role| Assignee | Resource | Description |

+|--|--|--|--|

+| `Search Index Data Reader` | Azure OpenAI | Azure AI Search | Inference service queries the data from the index. |

+| `Search Service Contributor` | Azure OpenAI | Azure AI Search | Inference service queries the index schema for auto fields mapping. Data ingestion service creates index, data sources, skill set, indexer, and queries the indexer status. |

+| `Storage Blob Data Contributor` | Azure OpenAI | Storage Account | Reads from the input container, and writes the pre-process result to the output container. |

+| `Cognitive Services OpenAI Contributor` | Azure AI Search | Azure OpenAI | Custom skill |

+| `Storage Blob Data Contributor` | Azure AI Search | Storage Account | Reads blob and writes knowledge store |

+| `Cognitive Services OpenAI Contributor` | Signed-in User | Azure OpenAI | Calls public ingestion or inference API from Azure OpenAI Studio.|

+See the [Azure RBAC documentation](/azure/role-based-access-control/role-assignments-portal) for instructions on setting these roles in the Azure portal. You can use the [available script on GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/scripts/role_assignment.sh) to add the role assignments programmatically. You need to have the `Owner` role on these resources to do role assignments.

+## Using the API

+### Local test setup

+Make sure your sign-in credential has `Cognitive Services OpenAI Contributor` role on your Azure OpenAI resource.

+Also, make sure that the IP your development machine is allowlisted in the IP rules, so you can call the Azure OpenAI API.

+### Ingestion API

+See the [ingestion API reference article](/azure/ai-services/openai/reference#start-an-ingestion-job) for details on the request and response objects used by the ingestion API.

+Additional notes:

+* `JOB_NAME` in the API path will be used as the index name in Azure AI Search.

+* Use the `Authorization` header rather than api-key.

+* Explicitly set `storageEndpoint` header, this is required if the `storageConnectionString` is in keyless format. It starts with `ResourceId=`.

+* Use `ResourceId=` format for `storageConnectionString`. This indicates that Azure OpenAI and Azure AI Search will use managed identity to authenticate the storage account, which is required to bypass network restrictions.

+* **Do not** set the `searchServiceAdminKey` header. The system-assigned identity of the Azure OpenAI resource will be used to authenticate Azure AI Search.

+* **Do not** set `embeddingEndpoint` or `embeddingKey`. Instead, use the `embeddingDeploymentName` header to enable text vectorization.

+**Submit job example**

+```bash

+accessToken=$(az account get-access-token --resource https://cognitiveservices.azure.com/ --query "accessToken" --output tsv)

+curl -i -X PUT https://my-resource.openai.azure.com/openai/extensions/on-your-data/ingestion-jobs/vpn1025a?api-version=2023-10-01-preview \

+-H "Content-Type: application/json" \

+-H "Authorization: Bearer $accessToken" \

+-H "storageEndpoint: https://mystorage.blob.core.windows.net/" \

+-H "storageConnectionString: ResourceId=/subscriptions/1234567-abcd-1234-5678-1234abcd/resourceGroups/my-resource/providers/Microsoft.Storage/storageAccounts/mystorage" \

+-H "storageContainer: my-container" \

+-H "searchServiceEndpoint: https://mysearch.search.windows.net" \

+-H "embeddingDeploymentName: ada" \

+-d \

+'

+{

+}

+'

+```

+**Get job status example**

+```bash

+accessToken=$(az account get-access-token --resource https://cognitiveservices.azure.com/ --query "accessToken" --output tsv)

+curl -i -X GET https://my-resource.openai.azure.com/openai/extensions/on-your-data/ingestion-jobs/abc1234?api-version=2023-10-01-preview \

+-H "Content-Type: application/json" \

+-H "Authorization: Bearer $accessToken"

+```

+### Inference API

+See the [inference API reference article](/azure/ai-services/openai/reference#completions-extensions) for details on the request and response objects used by the inference API.

+Additional notes:

+* **Do not** set `dataSources[0].parameters.key`. The service will use system assigned managed identity to authenticate the Azure AI Search.

+* **Do not** set `embeddingEndpoint` or `embeddingKey`. Instead, to enable vector search (with `queryType` set properly), use `embeddingDeploymentName`.

+Example:

+```bash

+accessToken=$(az account get-access-token --resource https://cognitiveservices.azure.com/ --query "accessToken" --output tsv)

+curl -i -X POST https://my-resource.openai.azure.com/openai/deployments/turbo/extensions/chat/completions?api-version=2023-10-01-preview \

+-H "Content-Type: application/json" \

+-H "Authorization: Bearer $accessToken" \

+-d \

+'

+{

+ "dataSources": [

+ {

+ "type": "AzureCognitiveSearch",

+ "parameters": {

+ "endpoint": "https://my-search-service.search.windows.net",

+ "indexName": "my-index",

+ "queryType": "vector",

+ "embeddingDeploymentName": "ada"

+ }

+ }

+ ],

+ "messages": [

+ {

+ "role": "user",

+ "content": "Who is the primary DRI for QnA v2 Authoring service?"

+ }

+ ]

+}

+'

+```

+## Azure OpenAI Studio

+You should be able to use all Azure OpenAI Studio features, including both ingestion and inference.

+## Web app

+The web app published from the Studio will communicate with Azure OpenAI. If Azure OpenAI is network restricted, the web app need to be set up correctly for outbound networking.

+1. Set Azure OpenAI allow inbound traffic from your virtual network.

+ :::image type="content" source="../media/use-your-data/web-app-configure-inbound-traffic.png" alt-text="A screenshot showing inbound traffic configuration for the web app." lightbox="../media/use-your-data/web-app-configure-inbound-traffic.png":::

+1. Configure the web app for outbound virtual network integration

+ :::image type="content" source="../media/use-your-data/web-app-configure-outbound-traffic.png" alt-text="A screenshot showing outbound traffic configuration for the web app." lightbox="../media/use-your-data/web-app-configure-outbound-traffic.png":::

+## December 2023

+### Azure OpenAI on your data

+- Full VPN and private endpoint support for Azure OpenAI on your data, including security support for: storage accounts, Azure OpenAI resources, and Azure AI Search service resources.

+- New article for using [Azure OpenAI on your data securely](./how-to/use-your-data-securely.md) by protecting data with virtual networks and private endpoints.

+### New data source support in Azure OpenAI on your data

+- You can now use [Azure Cosmos DB for MongoDB vCore](./concepts/use-your-data.md?tabs=mongo-db.md#ingesting-your-data) as well as URLs/web addresses as data sources to ingest your data and chat with a supported Azure OpenAI model.

+### Azure OpenAI on your data

+- New [custom parameters](./concepts/use-your-data.md#custom-parameters) for determining the number of retrieved documents and strictness.

+ - The strictness setting sets the threshold to categorize documents as relevant to your queries.

+ - The retrieved documents setting specifies the number of top-scoring documents from your data index used to generate responses.

+- You can see data ingestion/upload status in the Azure OpenAI Studio.

+- Support for [private endpoints & VPNs for blob containers](./how-to/use-your-data-securely.md#security-support-for-azure-blob-storage)

+If the language of the content in the source document is known, we recommend that you specify the source language in the request to get a better translation. If the document has content in multiple languages or the language is unknown, then don't specify the source language in the request. Document Translation automatically identifies language for each text segment and translates.

+When text is translated from the source to target language, the overall length of translated text can differ from source. The result could be reflow of text across pages. The same fonts aren't always available in both source and target language. In general, the same font style is applied in target language to retain formatting closer to source.

+No. The text in an image within a document isn't translated.

+#### Can encrypted or password-protected documents be translated?

+No. The service can't translate encrypted or password-protected documents. If your scanned or text-embedded PDFs are password-locked, you must remove the lock before submission.

+No. Don't include SAS token-appended URLS. Managed identities eliminate the need for you to include shared access signature tokens (SAS) with your HTTP requests.

+#### Which PDF format renders the best results?

+PDF documents generated from digital file formats (also known as "native" PDFs) provide optimal output. Scanned PDFs are images of printed documents scanned into an electronic format. Translating scanned PDF files can result in loss of the original formatting, layout, and style, and affect the quality of the translation.

+## Limitations

+Azure CNI Overlay networking in AKS currently has the following limitations:

+* In case you are using your own subnet to deploy the cluster, the names of the subnet, VNET and resource group which contains the VNET, must be 63 characters or less. This comes from the fact that these names will be used as labels in AKS worker nodes, and are therefore subjected to [Kubernetes label syntax rules](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set).

+Create a file named `nfs-sc.yaml` and copy the manifest below. For a list of supported `mountOptions`, see [NFS mount options][nfs-file-share-mount-options]

+[nfs-file-share-mount-options]: ../storage/files/storage-files-how-to-mount-nfs-shares.md#mount-options

+ - protocol: TCP

+ - protocol: TCP

+ hardware: highmem

+[use-gpus-aks]: gpu-cluster.md

+AKS allows you to create a cluster without specifying the exact patch version. When you create a cluster without designating a patch, the cluster runs the minor version's latest GA patch. For example, if you create a cluster with **`1.21`**, your cluster runs **`1.21.7`**, which is the latest GA patch version of *1.21*. If you want to upgrade your patch version in the same minor version, please use [auto-upgrade](./auto-upgrade-cluster.md#use-cluster-auto-upgrade).

+To see what patch you're on, run the `az aks show --resource-group myResourceGroup --name myAKSCluster` command. The `currentKubernetesVersion` property shows the whole Kubernetes version.

+When a new minor version is introduced, the oldest minor version and patch releases supported are deprecated and removed. For example, let's say the current supported version list is:

+### What happens when you scale a Kubernetes cluster with a minor version that isn't supported?

+### Can you stay on a Kubernetes version forever?

+If a cluster has been out of support for more than three (3) minor versions and has been found to carry security risks, Azure proactively contacts you to upgrade your cluster. If you don't take further action, Azure reserves the right to automatically upgrade your cluster on your behalf.

+### What happens if you scale a Kubernetes cluster with a minor version that isn't supported?

+For minor versions not supported by AKS, scaling in or out should continue to work. Since there are no guarantees with quality of service, we recommend upgrading to bring your cluster back into support.

+ var args = context.GraphQL.Arguments;

+| * / 1886, 443 | Outbound | TCP | VirtualNetwork / AzureMonitor | **Publish [Diagnostics Logs and Metrics](api-management-howto-use-azure-monitor.md), [Resource Health](../service-health/resource-health-overview.md), and [Application Insights](api-management-howto-app-insights.md)** | External & Internal |

+| * / 1886, 443 | Outbound | TCP | VirtualNetwork / AzureMonitor | **Publish [Diagnostics Logs and Metrics](api-management-howto-use-azure-monitor.md), [Resource Health](../service-health/resource-health-overview.md), and [Application Insights](api-management-howto-app-insights.md)** | External & Internal |

+### To configure continuous deployment from Azure DevOps

+Create the following variables containing your Azure DevOps information.

+## 1. Install the Service Connector passwordless extension

+## 2. Create a passwordless connection

+Next, create a passwordless connection with Service Connector.

+> [!TIP]

+> The Azure portal can help you compose the commands below. In the Azure portal, go to your [Azure App Service](../service-connector/quickstart-portal-app-service-connection.md) resource, select **Service Connector** from the left menu and select **Create**. Fill out the form with all required parameters. Azure automaticaly generates the connection creation command, which you can copy to use in the CLI or execute in Azure Cloud Shell.

+# [Azure SQL Database](#tab/sqldatabase-sc)

+The following Azure CLI command uses a `--client-type` parameter.

+1. Optionally run the `az webapp connection create sql -h` to get the supported client types.

+1. Choose a client type and run the corresponding command. Replace the placeholders below with your own information.

+ # [User-assigned managed identity](#tab/userassigned-sc)

+ az webapp connection create sql \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <sql-group-name> \

+ --server <sql-name> \

+ --database <database-name> \

+ --user-identity client-id=<client-id> subs-id=<subscription-id> \

+ --client-type <client-type>

+ # [System-assigned managed identity](#tab/systemassigned-sc)

+ az webapp connection create sql \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <group-name> \

+ --server <sql-name> \

+ --database <database-name> \

+ --system-identity \

+ --client-type <client-type>

+ --

+# [Azure Database for MySQL](#tab/mysql-sc)

+> [!NOTE]

+> For Azure Database for MySQL - Flexible Server, you must first [manually set up Microsoft Entra authentication](../mysql/flexible-server/how-to-azure-ad.md), which requires a separate user-assigned managed identity and specific Microsoft Graph permissions. This step can't be automated.

+1. Manually [set up Microsoft Entra authentication for Azure Database for MySQL - Flexible Server](../mysql/flexible-server/how-to-azure-ad.md).

+1. Optionally run the command `az webapp connection create mysql-flexible -h` to get the supported client types.

+1. Choose a client type and run the corresponding command. The following Azure CLI command uses a `--client-type` parameter.

+ # [User-assigned managed identity](#tab/userassigned-sc)

+ az webapp connection create mysql-flexible \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <group-name> \

+ --server <mysql-name> \

+ --database <database-name> \

+ --user-identity client-id=XX subs-id=XX mysql-identity-id=$IDENTITY_RESOURCE_ID \

+ --client-type <client-type>

+ # [System-assigned managed identity](#tab/systemassigned-sc)

+ az webapp connection create mysql-flexible \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <group-name> \

+ --server <mysql-name> \

+ --database <database-name> \

+ --system-identity mysql-identity-id=$IDENTITY_RESOURCE_ID \

+ --client-type <client-type>

+ --

+# [Azure Database for PostgreSQL](#tab/postgresql-sc)

+The following Azure CLI command uses a `--client-type` parameter.

+1. Optionally run the command `az webapp connection create postgres-flexible -h` to get a list of all supported client types.

+1. Choose a client type and run the corresponding command.

+ # [User-assigned managed identity](#tab/userassigned-sc)

+ az webapp connection create postgres-flexible \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <group-name> \

+ --server <postgresql-name> \

+ --database <database-name> \

+ --user-identity client-id=XX subs-id=XX \

+ --client-type java

+ # [System-assigned managed identity](#tab/systemassigned-sc)

+ az webapp connection create postgres-flexible \

+ --resource-group <group-name> \

+ --name <server-name> \

+ --target-resource-group <group-name> \

+ --server <postgresql-name> \

+ --database <database-name> \

+ --system-identity \

+ --client-type <client-type>

+1. Grant permission to pre-created tables

+--

+This Service Connector command completes the following tasks in the background:

+* Enable system-assigned managed identity, or assign a user identity for the app `<server-name>` hosted by Azure App Service.

+* Set the Microsoft Entra admin to the current signed-in user.

+* Add a database user for the system-assigned managed identity or user-assigned managed identity. Grant all privileges of the database `<database-name>` to this user. The username can be found in the connection string in preceding command output.

+* Set configurations named `AZURE_MYSQL_CONNECTIONSTRING`, `AZURE_POSTGRESQL_CONNECTIONSTRING`, or `AZURE_SQL_CONNECTIONSTRING` to the Azure resource based on the database type.

+* For App Service, the configurations are set in the **App Settings** blade.

+If you encounter any problem when creating a connection, refer to [Troubleshooting](../service-connector/tutorial-passwordless.md#troubleshooting) for help.

+1. Instantiate a `DefaultAzureCredential` from the Azure Identity client library. If you're using a user-assigned identity, specify the client ID of the identity.

+2. Get an access token for the resource URI respective to the database type.

+ * For Azure SQL Database: `https://database.windows.net/.default`

+ * For Azure Database for MySQL: `https://ossrdbms-aad.database.windows.net/.default`

+ * For Azure Database for PostgreSQL: `https://ossrdbms-aad.database.windows.net/.default`

+3. Add the token to your connection string.

+4. Open the connection.

+# [Azure SQL Database](#tab/sqldatabase-sc)

+# [Azure Database for MySQL](#tab/mysql-sc)

+# [Azure Database for PostgreSQL](#tab/postgresql-sc)

+Without any further changes, your code is ready to be run in Azure. To debug your code locally, however, your develop environment needs a signed-in Microsoft Entra user. In this step, you configure your environment of choice by signing in with your Microsoft Entra user.

+1. Run your code in your dev environment. Your code uses the signed-in Microsoft Entra user in your environment to connect to the back-end database. The user can access the database because it's configured as a Microsoft Entra administrator for the database.

+## 1. Run the sample

+For your convenience, the sample repository, [Hibernate ORM with Panache and RESTEasy](https://github.com/Azure-Samples/msdocs-quarkus-postgresql-sample-app), includes a [dev container](https://docs.github.com/codespaces/setting-up-your-project-for-codespaces/adding-a-dev-container-configuration/introduction-to-dev-containers) configuration. The dev container has everything you need to develop an application, including the database, cache, and all environment variables needed by the sample application. The dev container can run in a [GitHub codespace](https://docs.github.com/en/codespaces/overview), which means you can run the sample on any computer with a web browser.

+| [PowerShell](#powershell-runbooks) |Textual runbook based on Windows PowerShell scripting. The currently supported versions are: PowerShell 7.2 (GA), PowerShell 5.1 (GA), and PowerShell 7.1 (preview). |

+* Learn how [Azure Arc-enabled SCVMM extends Azure's governance and management capabilities to System Center managed infrastructure](../system-center-virtual-machine-manager/overview.md).

+As servers no longer require ESUs because they've been migrated to Azure, Azure VMware Solution (AVS), or Azure Stack HCI **where theyΓÇÖre eligible for free ESUs**, or updated to Windows Server 2016 or higher, you can modify the number of cores associated with a license or delete/deactivate licenses. You can also link the license to a new scope of additional servers. See [Programmatically deploy and manage Azure Arc Extended Security Updates licenses](api-extended-security-updates.md) to learn more. For information about no-cost ESUs through Azure Stack HCI, see [Free Extended Security Updates through Azure Stack HCI](/azure-stack/hci/manage/azure-benefits-esu?tabs=windows-server-2012).

+In this scenario, you could either license the entire cluster with 1024 Windows Server 2012 Datacenter ESU physical cores or license each VM individually with a total of 506 standard edition virtual cores. In this case, it's cheaper to purchase an Arc ESU Windows Server 2012 Standard edition license associated with 506 virtual cores. You'll need to onboard each of the 44 VMs to Azure Arc and then link the license to the Arc machines.

+> [!IMPORTANT]

+> If you migrate the VMs to Azure VMware Solution (AVS), these servers become eligible for free WS2012 ESUs and should not enroll in ESUs enabled through Azure Arc.

+>

+> [!NOTE]

+> Azure VMware Solutions (AVS) machines are eligible for free ESUs and should not enroll in ESUs enabled through Azure Arc.

+>

+- **Environment:** The connected machine should not be running on Azure Stack HCI, Azure VMware solution (AVS), or as an Azure virtual machine. **In these scenarios, WS2012 ESUs are available for free**. For information about no-cost ESUs through Azure Stack HCI, see [Free Extended Security Updates through Azure Stack HCI](/azure-stack/hci/manage/azure-benefits-esu?tabs=windows-server-2012).

+The following example shows a [JavaScript function](functions-reference-node.md) with a warmup trigger that runs on each new instance when added to your app:

+The following example shows a [TypeScript function](functions-reference-node.md) with a warmup trigger that runs on each new instance when added to your app:

+- An [Azure supported Java Development Kit (JDK)](/azure/developer/java/fundamentals/java-support-on-azure) for Java, version 8, 11, or 17

+1. Click on line *20* of the file *src/main/java/org/example/functions/HttpTriggerFunction.java* to add a breakpoint. Access the endpoint `http://localhost:7071/api/HttpTrigger-Java?name=Azure` again and you'll find the breakpoint is hit. You can then try more debug features like **Step**, **Watch**, and **Evaluation**. Stop the debug session by clicking the **Stop** button.

+1. Click and expand the Azure icon in IntelliJ Project explorer, then select **Deploy to Azure -> Deploy to Azure Functions**.

+## [1.0.3]

+### Other changes (1.0.3)

+- Updated CDN links in the readme.

+[1.0.3]: https://www.npmjs.com/package/azure-maps-drawing-tools/v/1.0.3

+### [3.0.3] (November 29, 2023)

+#### New features (3.0.3)

+- Included ESM support.

+#### Other changes (3.0.3)

+- The accessibility feature for screen reader has been upgraded to utilize the Search V2 API (reverse geocoding).

+- Enhanced accessibility in the Compass and Pitch controls.

+### [2.3.5] (November 29, 2023)

+#### Other changes (2.3.5)

+- The accessibility feature for screen reader has been upgraded to utilize the Search V2 API (reverse geocoding).

+[3.0.3]: https://www.npmjs.com/package/azure-maps-control/v/3.0.3

+[2.3.5]: https://www.npmjs.com/package/azure-maps-control/v/2.3.5

+> [!NOTE]

+> With _Spring Boot native image applications_, you can use [this project](https://aka.ms/AzMonSpringNative).

+ "^User account with userId (?<redactedUserId>[\\da-zA-Z]+)[\\w\\s]+"

+1. Add `opentelemetry-instrumentation-annotations-1.32.0.jar` (or later) to your application:

+ <version>1.32.0</version>

+ <version>3.4.18</version>

+ Title: Azure Monitor REST API index

+description: Lists the operation groups for the Azure Monitor REST API, which includes Application Insights, Log Analytics, and Monitor.

+# Azure Monitor REST API index

+Organized by subject area.

+| Operation groups | Description |

+|-|-|

+| [Operations](/rest/api/monitor/alertsmanagement/operations) | Lists the available REST API operations for Azure Monitor. |

+| ***Activity Log*** | |

+| [Activity log(s)](/rest/api/monitor/activity-logs) | Get a list of event entries in the [activity log](./essentials/platform-logs-overview.md). |

+| [(Activity log) event categories](/rest/api/monitor/event-categories) | Lists the types of Activity Log Entries. |

+| [Activity log profiles](/rest/api/monitor/log-profiles) | Operations to manage [activity log profiles](./essentials/platform-logs-overview.md) so you can route activity log events to other locations. |

+| [Activity log tenant events](/rest/api/monitor/tenant-activity-logs) | Gets the [Activity Log](./essentials/platform-logs-overview.md) event entries for a specific tenant. |

+| ***Alerts Management and Action Groups*** | |

+| [Action groups](/rest/api/monitor/action-groups) | Manages and lists [action groups](./alerts/action-groups.md). |

+| [Activity log alerts](/rest/api/monitor/activity-log-alerts) | Manages and lists [activity log alert rules](./alerts/alerts-types.md#activity-log-alerts). |

+| [Alert management](/rest/api/monitor/alertsmanagement/alerts) | Lists and updates [fired alerts](./alerts/alerts-overview.md). |

+| [Alert processing rules](/rest/api/monitor/alertsmanagement/alert-processing-rules) | Manages and lists [alert processing rules](./alerts/alerts-processing-rules.md). |

+| [Metric alert baseline](/rest/api/monitor/baselines) | List the metric baselines used in alert rules with [dynamic thresholds](./alerts/alerts-dynamic-thresholds.md). |

+| [Metric alerts](/rest/api/monitor/metric-alerts) | Manages and lists [metric alert rules](./alerts/alerts-overview.md). |

+| [Metric alerts status](/rest/api/monitor/metric-alerts-status) | Lists the status of [metric alert rules](./alerts/alerts-overview.md). |

+| [Prometheus rule groups](/rest/api/monitor/prometheus-rule-groups) | Manages and lists [Prometheus rule groups](./essentials/prometheus-rule-groups.md) (alert rules and recording rules). |

+| [Scheduled query rules - 2023-03-15 (preview)](/rest/api/monitor/scheduled-query-rules?view=rest-monitor-2023-03-15-preview&preserve-view=true) | Manages and lists [log alert rules](./alerts/alerts-types.md#log-alerts). |

+| [Scheduled query rules - 2018-04-16](/rest/api/monitor/scheduled-query-rules?view=rest-monitor-2018-04-16&preserve-view=true) | Manages and lists [log alert rules](./alerts/alerts-types.md#log-alerts). |

+| [Scheduled query rules - 2021-08-01](/rest/api/monitor/scheduled-query-rules?view=rest-monitor-2021-08-01&preserve-view=true) | Manages and lists [log alert rules](./alerts/alerts-types.md#log-alerts). |

+| [Smart Detector alert rules](/rest/api/monitor/smart-detector-alert-rules) | Manages and lists [smart detection alert rules](./alerts/alerts-types.md#smart-detection-alerts). |

+| ***Application Insights*** | |

+| [Components](/rest/api/application-insights/components) | Enables you to manage components that contain Application Insights data. |

+| [Data Access](./logs/api/overview.md) | Query Application Insights data. |

+| [Events](/rest/api/application-insights/events) | Retrieve the data for a single event or multiple events by event type and retrieve the Odata EDMX metadata for an application. |

+| [Metadata](/rest/api/application-insights/metadata) | Retrieve and export the metadata information for an application. |

+| [Metrics](/rest/api/application-insights/metrics) | Retrieve or export the metric data for an application and retrieve the metadata describing the available metrics for an application. |

+| [Query](/rest/api/application-insights/query) | The Query operation group, which includes Execute and Get operations, enables running analytics queries on resources and retrieving the results, even for large data sets that require extended processing time. |

+| [Web Tests](/rest/api/application-insights/web-tests) | Set up web tests to monitor a web endpointΓÇÖs availability and responsiveness. |

+| [Workbooks](/rest/api/application-insights/workbooks) | Manage Azure workbooks for an Application Insights component resource and retrieve workbooks within resource group or subscription by category. |

+| ***Autoscale Settings*** | |

+| [Autoscale settings](/rest/api/monitor/autoscale-settings) | Operations to manage autoscale settings. |

+| [Predictive metric](/rest/api/monitor/predictive-metric) | Retrieves predicted autoscale metric data. |

+| ***Data Collection Endpoints*** | |

+| [Data collection endpoints](/rest/api/monitor/data-collection-endpoints) | Create and manage a data collection endpoint and retrieve the data collection endpoints within a resource group or subscription. |

+| ***Data Collection Rules*** | Create and manage a data collection rule and retrieve the data collection rules within a resource group or subscription. |

+| [Data collection rule associations](/rest/api/monitor/data-collection-rule-associations) | Create and manage a data collection rule association and retrieve the data collection rule associations for a data collection endpoint, resource, or data collection rule. |

+| [Data collection rules](/rest/api/monitor/data-collection-rules) | Create and manage a data collection rule and retrieve the data collection rules within a resource group or subscription. |

+| ***Diagnostic Settings*** | |

+| [Diagnostic settings](/rest/api/monitor/diagnostic-settings) | Operations to create, update, and retrieve the [diagnostic settings](./essentials/platform-logs-overview.md) for a resource. Controls the routing of metric data and diagnostic logs. |

+| [Diagnostic settings category](/rest/api/monitor/diagnostic-settings-category) | Relates to the [possible categories](./essentials/resource-logs-schema.md) for a given resource. |

+| [Management group diagnostic settings](/rest/api/monitor/management-group-diagnostic-settings) | Manage the management group diagnostic settings for a resource and retrieve the management group diagnostic settings list for a management group. |

+| [Subscription diagnostic settings](/rest/api/monitor/subscription-diagnostic-settings) | Manage the subscription diagnostic settings for a resource and retrieve the subscription diagnostic settings list for a subscriptionId. |

+| ***Manage Log Analytics workspaces and related resources*** | |

+| [Available service tiers](/rest/api/loganalytics/available-service-tiers) | Retrieve the available service tiers for a Log Analytics workspace. |

+| [Clusters](/rest/api/loganalytics/clusters) | Manage Log Analytics clusters. |

+| [Data Collector Logs (Preview)](/rest/api/loganalytics/data%20collector%20logs%20%28preview%29) | Delete or retrieve a data collector log tables for a Log Analytics workspace and retrieve all data collector log tables for a Log Analytics workspace. |

+| [Data exports](/rest/api/loganalytics/data-exports) | Manage a data export for a Log Analytics workspace or retrieve the data export instances within a Log Analytics workspace. |

+| [Data Sources](/rest/api/loganalytics/data-sources) | Create or update data sources. |

+| [Deleted workspaces](/rest/api/loganalytics/deleted-workspaces) | Retrieve the recently deleted workspaces within a subscription or resource group. |

+| [Gateways](/rest/api/loganalytics/gateways) | Delete a Log Analytics gateway. |

+| [Intelligence Packs](/rest/api/loganalytics/intelligence-packs) | Enable or disable an intelligence pack for a Log Analytics workspace or retrieve all intelligence packs for a Log Analytics workspace. |

+| [Linked Services](/rest/api/loganalytics/linked-services) | Create or update linked services. |

+| [Linked Storage Accounts](/rest/api/loganalytics/linked-storage-accounts) | Manage a link relation between a workspace and storage accounts and retrieve all linked storage accounts associated with a workspace. |

+| [Management Groups](/rest/api/loganalytics/management-groups) | Retrieve all management groups connected to a Log Analytics workspace. |

+| [Metadata](/rest/api/loganalytics/metadata) | Retrieve the metadata information for a Log Analytics workspace. |

+| [Operation Statuses](/rest/api/loganalytics/operation-statuses) | Retrieve the status of a long running azure asynchronous operation. |

+| [Operations](/rest/api/loganalytics/operations) | Retrieve all of the available OperationalInsights Rest API operations. |

+| [Query](/rest/api/loganalytics/query) | Execute a batch of Analytics queries for data. |

+| [Query pack queries](/rest/api/monitor/query-pack-queries) | Manage a query defined within a Log Analytics QueryPack and retrieve or search the list of queries defined within a Log Analytics QueryPack. |

+| [Query packs](/rest/api/monitor/query-packs) | Manage a Log Analytics QueryPack including updating its tags and retrieve a list of all Log Analytics QueryPacks within a subscription or resource group. |

+| [Saved Searches](/rest/api/loganalytics/saved-searches) | Create or update saved searches. |

+| [Storage Insights](/rest/api/loganalytics/storage-insights) | Create or update storage insights. |

+| [Tables](/rest/api/loganalytics/tables) | Manage Log Analytics workspace tables. |

+| [Workspace purge](/rest/api/loganalytics/workspace-purge) | Retrieve the status of an ongoing purge operation or purge the data in a Log Analytics workspace. |

+| [Workspace schema](/rest/api/loganalytics/workspace-schema) | Retrieves the schema for a Log Analytics workspace. |

+| [Workspace shared keys](/rest/api/loganalytics/workspace-shared-keys) | Retrieve or regenerate the shared keys for a Log Analytics workspace. |

+| [Workspace usages](/rest/api/loganalytics/workspace-usages) | Retrieve the usage metrics for a Log Analytics workspace. |

+| [Workspaces](/rest/api/loganalytics/workspaces) | Manage Log Analytics workspaces. |

+| ***Metrics*** | |

+| [Azure Monitor Workspaces](/rest/api/monitor/azure-monitor-workspaces) | Manage an Azure Monitor workspace and retrieve the Azure Monitor workspaces within a resource group or subscription. |

+| [Metric definitions](/rest/api/monitor/metric-definitions) | Lists the metric definitions available for the resource. That is, what [specific metrics](/azure/azure-monitor/reference/supported-metrics/metrics-index) can you collect. |

+| [Metric namespaces](/rest/api/monitor/metric-namespaces) | Lists the metric namespaces. Most relevant when using [custom metrics](./essentials/metrics-custom-overview.md). |

+| [Metrics Batch](/rest/api/monitor/metrics-batch) | List the metric values for multiple resources. |

+| [Metrics](/rest/api/monitor/metrics) | Lists the metric values for a resource you identify. |

+| [Metrics ΓÇô Custom](/rest/api/monitor/metrics-custom) | Post the metric values for a resource. |

+| ***Private Link Networking*** | |

+| [Private endpoint connections (preview)](/rest/api/monitor/private-endpoint-connections) | Approve, reject, delete, and retrieve a private endpoint connection and retrieve all project endpoint connections on a private link scope. |

+| [Private link resources (preview)](/rest/api/monitor/private-link-resources) | Retrieve a single private link resource that needs to be created for an Azure Monitor PrivateLinkScope or all private link resources within an Azure Monitor PrivateLinkScope resource that need to be created for an Azure Monitor PrivateLinkScope. |

+| [Private link scope operation status (preview)](/rest/api/monitor/private-link-scope-operation-status) | Retrieves the status of an Azure asynchronous operation associated with a private link scope operation. |

+| [Private link scoped resources (preview)](/rest/api/monitor/private-link-scoped-resources) | Approve, reject, delete, and retrieve a scoped resource object and retrieve all scoped resource objects within an Azure Monitor PrivateLinkScope resource. |

+| [Private link scopes (preview)](/rest/api/monitor/private-link-scopes) | Manage an Azure Monitor PrivateLinkScope including its tags and retrieve a list of all Azure Monitor PrivateLinkScopes within a subscription or resource group. |

+| ***Query log data*** | |

+| [Data Access](./logs/api/overview.md) | Query Log Analytics data. |

+| ***Send Custom Log Data to Log Analytics*** | |

+| [Logs Ingestion](./logs/logs-ingestion-api-overview.md) | Lets you send data to a Log Analytics workspace using either a [REST API call](./logs/logs-ingestion-api-overview.md#rest-api-call) or [client libraries](./logs/logs-ingestion-api-overview.md#client-libraries). |

+| ***Retired or being retired*** | |

+| [Alerts (classic) rule incidents](/rest/api/monitor/alert-rule-incidents) | [Being retired in 2019](/previous-versions/azure/azure-monitor/alerts/monitoring-classic-retirement) in the public cloud. Older classic alerts functions. Gets an incident associated to a [classic metric alert rule](./alerts/alerts-classic.overview.md). When an alert rule fires because the threshold is crossed in the up or down direction, an incident is created and an entry added to the [Activity Log](./essentials/platform-logs-overview.md). |

+| [Alert (classic) rules](/rest/api/monitor/alert-rules) | [Being retired in 2019](/previous-versions/azure/azure-monitor/alerts/monitoring-classic-retirement) in the public cloud. Provides operations for managing [classic alert](./alerts/alerts-classic.overview.md) rules. |

+| [Data Collector](/rest/api/loganalytics/create-request) | Data Collector API Reference. |

+# Customer intent: As a Log Analytics workspace administrator, I want to manage table schemas and be able create a table with a custom schema to store logs from an Azure or non-Azure data source.

+> [!IMPORTANT]

+> Whenever you update a table schema, be sure to [update any data collection rules](../essentials/data-collection-rule-overview.md) that send data to the table. The table schema you define in your data collection rule determines how Azure Monitor streams data to the destination table. Azure Monitor does not update data collection rules automatically when you make table schema changes.

+> [!IMPORTANT]

+> Custom tables have a suffix of **_CL**; for example, *tablename_CL*. The Azure portal adds the **_CL** suffix to the table name automatically. When you create a custom table using a different method, you need to add the **_CL** suffix yourself. The *tablename_CL* in the [DataFlows Streams](../essentials/data-collection-rule-structure.md#dataflows) properties in your data collection rules must match the *tablename_CL* name in the Log Analytics workspace.

+The following image identifies four Log Analytics components. These four Log Analytics components are:

+1. [Top action bar](#top-action-bar)

+1. [Left sidebar](#left-sidebar)

+1. [Query window](#query-window)

+1. [Results window](#results-window)

+To view and manage your settings, select the **Settings** menu icon in the top right section of the global page header to open the **Portal settings** page.

+Within **Portal settings**, you'll see different sections. This article describes the available options for each section.

+In the **Directories** section, you'll see your **Current directory** (the directory that you're currently signed in to).

+The **Startup directory** shows the default directory when you sign in to the Azure portal (or **Last visited** if you've chosen that option). To choose a different startup directory, select **change** to open the [Appearance + startup views](#appearance--startup-views) page, where you can change your selection.

+### Subscription filters

+After you continue, the **Advanced filters** page appears in the left navigation menu of **Portal settings**. You can create and manage multiple subscription filters on this page. Your currently selected subscriptions are saved as an imported filter that you can use again. You'll see this filter selected on the **Directories + subscriptions** page.

+If you want to stop using advanced filters, select the toggle again to restore the default subscription view. Any custom filters you've created are saved and will be available to use if you enable **Advanced filters** in the future.

+## Advanced filters

+To change the filter that is currently in use, select **Activate** next to that filter.

+After you've named your filter, enter at least one condition. In the **Filter type** field, select **Management group**, **Subscription ID**, **Subscription name**, or **Subscription state**. Then select an operator and the value to filter on.

+To delete a filter, select the trash can icon in that filter's row. You can't delete the **Default** filter or a filter that is currently active.

+The **Appearance + startup views** pane has two sections. The **Appearance** section lets you choose menu behavior, your color theme, and whether to use a high-contrast theme.

+- **Flyout**: The menu is hidden until you need it. You can select the menu icon in the upper left hand corner to open or close the menu.

+- **Docked**: The menu is always visible. You can collapse the menu to provide more working space.

+The **Startup views** section lets you set options for what you see when you first sign in to the Azure portal.

+Choose one of the following options for **Startup page**. This setting determines which page you see when you first sign in to the Azure portal.

+- **Dashboard**: Displays your most recently used dashboard. Dashboards can be customized to create a workspace designed just for you. For more information, see [Create and share dashboards in the Azure portal](azure-portal-dashboards.md).

+- **Last visited**: When you sign in to the Azure portal, you'll start in the same directory from your previous visit.

+- **Select a directory**: Choose this option to select a specific directory. You'll start in that directory every time you sign in to the Azure portal, even if you had been working in a different directory last time.

+Here, you can choose the language used in the Azure portal. You can also select a regional format to determine the format for dates, time, and currency.

+The options shown in the **Regional format** drop-down list correspond to the **Language** options. For example, if you select **English** as your language, and then select **English (United States)** as the regional format, currency is shown in U.S. dollars. If you select **English** as your language and then select **English (Europe)** as the regional format, currency is shown in euros. You can also select a regional format that is different from your language selection.

+Once you have made the desired changes to your language and regional format settings, select **Apply**.

+To export your portal settings, select **Export settings** from the top of the **My information** pane. This creates a JSON file that contains your user settings data.

+Due to the dynamic nature of user settings and risk of data corruption, you can't import settings from the JSON file. However, you can use this file to review the settings you selected. It can be useful to have a backup of your selections if you choose to delete your settings and private dashboards.

+- User settings, such as favorite subscriptions or directories

+It's a good idea to export and review your settings before you delete them, as described in the previous section. Rebuilding [dashboards](azure-portal-dashboards.md) or redoing custom settings can be time-consuming.

+To read all notifications received during your current session, select the **Notifications** icon from the global header.

+To deploy Zerto on Azure VMware Solution, follow these [instructions](https://help.zerto.com/bundle/Install.AVS.HTML/page/Prerequisites_Zerto_AVS.htm).

+ [WebPubSubTrigger("<hub>", WebPubSubEventType.User, "message")] UserEventRequest request, ILogger log)

+ log.LogInformation($"Request from: {request.ConnectionContext.UserId}");

+ log.LogInformation($"Request message data: {request.Data}");

+ log.LogInformation($"Request message dataType: {request.DataType}");

+ context.log('Request from: ', context.bindingData.request.connectionContext.userId);

+ context.log('Request message data: ', data);

+ context.log('Request message dataType: ', context.bindingData.request.dataType);

+ if (wpsContext.hasError || wpsContext.isPreflight)

+ # BackupHook is a list of hooks to execute before and after backing up a resource.

+ backupHook:

+ # BackupHook Name. This is the name of the hook that will be executed during backup.

+ # compulsory

+ - name: hook1

+ # Namespaces where this hook will be executed.

+ includedNamespaces:

+ - hrweb

+ excludedNamespaces:

+ labelSelector:

+ # PreHooks is a list of BackupResourceHooks to execute prior to backing up an item.

+ preHooks:

+ - exec:

+ # Container is the container in the pod where the command should be executed.

+ container: webcontainer

+ # Command is the command and arguments to execute.

+ command:

+ - /bin/uname

+ - -a

+ # OnError specifies how Velero should behave if it encounters an error executing this hook

+ onError: Continue

+ # Timeout is the amount of time to wait for the hook to complete before considering it failed.

+ timeout: 10s

+ - exec:

+ command:

+ - /bin/bash

+ - -c

+ - echo hello > hello.txt && echo goodbye > goodbye.txt

+ container: webcontainer

+ onError: Continue

+ # PostHooks is a list of BackupResourceHooks to execute after backing up an item.

+ postHooks:

+ - exec:

+ container: webcontainer

+ command:

+ - /bin/uname

+ - -a

+ onError: Continue

+ timeout: 10s

+ # RestoreHook is a list of hooks to execute after restoring a resource.

+ restoreHook:

+ # Name is the name of this hook.

+ - name: myhook-1

+ # Restored Namespaces where this hook will be executed.

+ includedNamespaces:

+ excludedNamespaces:

+ labelSelector:

+ # PostHooks is a list of RestoreResourceHooks to execute during and after restoring a resource.

+ postHooks:

+ - exec:

+ # Container is the container in the pod where the command should be executed.

+ container: webcontainer

+ # Command is the command and arguments to execute from within a container after a pod has been restored.

+ command:

+ - /bin/bash

+ - -c

+ - echo hello > hello.txt && echo goodbye > goodbye.txt

+ # OnError specifies how Velero should behave if it encounters an error executing this hook

+ # default value is Continue

+ onError: Continue

+ # Timeout is the amount of time to wait for the hook to complete before considering it failed.

+ execTimeout: 30s

+ # WaitTimeout defines the maximum amount of time Velero should wait for the container to be ready before attempting to run the command.

+ waitTimeout: 5m

+ -g <aksclusterrg> \

+ --cluster-name <aksclustername> \

+ -n <randomRoleBindingName> \

+ --source-resource-id $(az dataprotection backup-vault show -g <vaultrg> -v <VaultName> --query id -o tsv) \

+## Configure multistreaming data backups for higher throughput using Backint

+To configure multistreaming data backups, see the [SAP documentation](https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/18db704959a24809be8d01cc0a409681.html).

+### Support matrix

+- **Supported HANA versions**: SAP HANA 2.0 SP05 and prior.

+- **Parameters to enable SAP HANA settings for multistreaming**:

+ - *parallel_data_backup_backint_channels*

+ - *data_backup_buffer_size (optional)*

+ >[!Note]

+ >By setting the above HANA parameters will lead to increased memory and CPU utilization. We recommend that you monitor the memory consumption and CPU utilization as overutilization might negatively impact the backup and other HANA operations.

+- **Backup performance for databases**: The performance gain will be more prominent for larger databases.

+- **Database size applicable for multistreaming**: The number of multistreaming channels applies to all data backups *larger than 128 GB*. Data backups smaller than 128 GB always use only one channel.

+- **Supported backup throughput**: Multistreaming currently supports the data backup throughput of up to *1.5 GBps*. Recovery throughput is slower than the backup throughput.

+- **VM configuration applicable for multistreaming**: To utilize the benefits of multistreaming, the VM needs to have a minimum configuration of *16 vCPUs* and *128 GB* of RAM.

+- **Limiting factors**: Throughput of *total disk LVM striping* and *VM network*, whichever hits first.

+Learn more about [SAP HANA Azure Virtual Machine storage](/azure/sap/workloads/hana-vm-operations-storage) and [SAP HANA Azure virtual machine Premium SSD storage configurations](/azure/sap/workloads/hana-vm-premium-ssd-v1) configurations.

+<a name="tvm-backup">Back up trusted launch VMs</a> | Backup is supported. <br><br> Backup of trusted launch VMs is supported through [Enhanced policy](backup-azure-vms-enhanced-policy.md). You can enable backup through a [Recovery Services vault](./backup-azure-arm-vms-prepare.md), the [pane for managing a VM](./backup-during-vm-creation.md#start-a-backup-after-creating-the-vm), and the [pane for creating a VM](backup-during-vm-creation.md#create-a-vm-with-backup-configured). <br><br> **Feature details** <br><br> - Backup is supported in all regions where trusted launch VMs are available. <br><br> - Configuration of backups, alerts, and monitoring for trusted launch VMs is currently not supported through the backup center. <br><br> - Migration of an existing [Gen2 VM](../virtual-machines/generation-2.md) (protected with Azure Backup) to a trusted launch VM is currently not supported. [Learn how to create a trusted launch VM](../virtual-machines/trusted-launch-portal.md?tabs=portal#deploy-a-trusted-launch-vm). <br><br> - Item-level restore is supported for the scenarios mentioned [here](backup-support-matrix-iaas.md#support-for-file-level-restore). <br><br> Note that if the trusted launch VM was created by converting a Standard VM, ensure that you remove all the recovery points created using Standard policy before enabling the backup operation for the VM.

+> The **UninstallConfiguration** parameter uninstalls any extension configuration that is applied to the service. Every extension configuration is associated with the service configuration. Calling the *remove* cmdlet without **UninstallConfiguration** disassociates the **deployment** from the extension configuration, thus effectively removing the extension. However, the extension configuration remains associated with the service.

+ * Review the Databricks runtime version, the Spark version. Then find the [maven coordinates](https://mvnrepository.com/artifact/com.datastax.spark/spark-cassandra-connector-assembly) that are compatible with the Cassandra Spark connector, and attach it to the cluster. See ["Upload a Maven package or Spark package"](https://docs.databricks.com/libraries) article to attach the connector library to the cluster. We recommend selecting Databricks runtime version 10.4 LTS, which supports Spark 3.2.1. To add the Apache Spark Cassandra Connector, your cluster, select **Libraries** > **Install New** > **Maven**, and then add `com.datastax.spark:spark-cassandra-connector-assembly_2.12:3.2.0` in Maven coordinates. If using Spark 2.x, we recommend an environment with Spark version 2.4.5, using spark connector at maven coordinates `com.datastax.spark:spark-cassandra-connector_2.11:2.4.3`.

+for all of your learning & evaluation needs. Users can provision a single free DB server per supported Azure region for a given subscription. This feature is currently available for our users in the East US, West Europe and Southeast Asia regions.

+* Free tier is currently available in East US, West Europe and Southeast Asia regions only.

+First, we create and attach the required [SQL connector](/connectors/sql/) and [Azure Cosmos DB connector](/azure/databricks/external-data/cosmosdb-connector) libraries to our Azure Databricks cluster. Restart the cluster to make sure libraries are loaded.

+| | Schema-based sharding | Row-based sharding |

+| | | |

+| **Multi-tenancy model** | Separate schema per tenant | Shared tables with tenant ID columns |

+| **Citus version** | 12.0+ | All versions |

+| **Extra steps compared to vanilla PostgreSQL** | None, only a config change | Use create_distributed_table on each table to distribute & colocate tables by tenant ID |

+| **Number of tenants** | 1-10k | 1-1 M+ |

+| **Data modeling requirement** | No foreign keys across distributed schemas | Need to include a tenant ID column (a distribution column, also known as a sharding key) in each table, and in primary keys, foreign keys |

+| **SQL requirement for single node queries** | Use a single distributed schema per query | Joins and WHERE clauses should include tenant_id column |

+| **Parallel cross-tenant queries** | No | Yes |

+| **Custom table definitions per tenant** | Yes | No |

+| **Access control** | Schema permissions | Schema permissions |

+| **Data sharing across tenants** | Yes, using reference tables (in a separate schema) | Yes, using reference tables |

+| **Tenant to shard isolation** | Every tenant has its own shard group by definition | Can give specific tenant IDs their own shard group via isolate_tenant_to_new_shard |

+This error means that Cost Management is unable to list the object in the S3 bucket where the CUR is located. AWS IAM policy requires a permission on the bucket and on the objects in the bucket. See [Create a role and policy in AWS](aws-integration-set-up-configure.md#create-a-policy-and-role-in-aws).

+This error means that Cost Management is unable to access and download the CUR files stored in the Amazon S3 bucket. Make sure that the AWS JSON policy attached to the role resembles the example shown at the bottom of the [Create a role and policy in AWS](aws-integration-set-up-configure.md#create-a-policy-and-role-in-aws) section.

+This error means that Cost Management can't find the Cost and Usage report that was defined in the connector. Make sure it isn't deleted and that the AWS JSON policy attached to the role resembles the example shown at the bottom of the [Create a role and policy in AWS](aws-integration-set-up-configure.md#create-a-policy-and-role-in-aws) section.

+11. (Optional) For Report path prefix, enter the report path prefix that you want prepended to the name of your report.

+ If skipped, the default prefix is the name that you specified for the report. The date range has the `/report-name/date-range/` format.

+17. After you review the settings for your report, select **Review and Complete**.

+ Note the report name. You use it in later steps.

+## Create a policy and role in AWS

+### Use the Create Policy wizard

+1. Sign in into your AWS console and select **Services**.

+2. In the list of services, select **IAM**.

+3. Select **Policies**.

+4. Select **Create policy**.

+5. Select **Choose a service**.

+### Configure permission for the Cost and Usage report

+3. Select **Add more permissions**.

+### Configure permission for your S3 bucket and objects

+5. Select **Resources** > **Specific**.

+6. In **bucket**, select the **Add ARNs** link to open another window.

+7. In **Resource Bucket name**, enter the bucket used to store the CUR files.

+8. Select **Add ARNs**.

+9. In **object**, select **Any**.

+10. Select **Add more permissions**.

+### Configure permission for Cost Explorer

+4. Select **Add more permissions**.

+### Add permission for AWS Organizations

+3. Select **Add more permissions**.

+### Configure permissions for Policies

+1. Select **Next**.

+### Review and create

+1. In Review Policy, enter a name for the new policy. Verify that you entered the correct information.

+1. Add tags. You can enter tags you wish to use or skip this step. This step isn't required to create a connector in Cost Management.

+1. Select **Create policy** to complete this procedure.

+### Use the Create a New Role wizard

+1. Sign in to your AWS console and select **Services**.

+2. In the list of services, select **IAM**.

+3. Select **Roles** and then select **Create Role**.

+4. On the **Select trusted entity** page, select **AWS account** and then under **An AWS account**, select **Another AWS account**.

+5. Under **Account ID**, enter **432263259397**.

+6. Under **Options**, select **Require external ID (Best practice when a third party will assume this role)**.

+7. Under **External ID**, enter the external ID, which is a shared passcode between the AWS role and Cost Management. Note the external ID, because you use it on the **New Connector** page in Cost Management. Microsoft recommends that you use a strong passcode policy when entering the external ID. The external ID should comply with AWS restrictions:

+ - Type: String

+ - Length constraints: Minimum length of 2. Maximum length of 1224.

+ - Must satisfy regular expression pattern: `[\w+=,.@: /-]*`

+ > [!NOTE]

+ > Don't change the selection for **Require MFA**. It should remain cleared.

+8. Select **Next**.

+9. On the search bar, search for your new policy and select it.

+10. Select **Next**.

+11. In **Role details**, enter a role name. Verify that you entered the correct information. Note the name entered because you use it later when you set up the Cost Management connector.

+12. Optionally, add tags. You can enter any tags like or skip this step. This step isn't required to create a connector in Cost Management.

+13. Select **Create role**.

+1. Optionally, select the default management group. It stores all discovered linked accounts. You can set it up later.

+Place your Azure subscriptions and AWS linked accounts in the same management group to create a single location where you can see cross-cloud provider information. If you want to configure your Azure environment with management groups, see [Initial setup of management groups](../../governance/management-groups/overview.md#initial-setup-of-management-groups).

+- Now that you set up and configured AWS Cost and Usage report integration, continue to [Manage AWS costs and usage](aws-integration-manage.md).

+Often, partners don't have their own Azure subscriptions in the tenant associated with their own Microsoft Partner Agreement. Partners with a Microsoft Partner Agreement plan who are global admins of their billing account can export and copy cost data into a storage account in a different tenant using a shared access service (SAS) key. In other words, a storage account with a SAS key allows the partner to use a storage account that's outside of their partner agreement to receive exported information. This article helps partners create a SAS key and configure Cost Management exports.

+- You need must be a partner with a Microsoft Partner Agreement. Your customers on the Azure plan must have Microsoft Customer Agreement that is signed.

+ - SAS key-based export isn't supported for indirect enterprise agreements.

+1. Select **key1** for _Signing key_. If you rotate or update the key used to sign the SAS token, you must regenerate a new SAS token.

+1. Configure the Export details as you would for a normal export. You can configure the export to use an existing directory or container or you can specify a new directory or container. The export process creates them for you.

+The SAS token-based export only works while the token remains valid. Reset the token before the current one expires, or your export stops working. Because the token provides access to your storage account, protect the token as carefully as you would any other sensitive information. You're responsible to maintain permissions and data access when your export data to your storage account.

+ - The behavior is expected. If the storage account is in another tenant, you need to navigate to that tenant first in the Azure portal to find the storage account.

+- Don't configure exports to a storage container when configured as a destination in an [object replication rule](../../storage/blobs/object-replication-overview.md#object-replication-policies-and-rules).

+ :::image type="content" source="./media/tutorial-export-acm-data/permitted-scope-copy-operations.png" alt-text="Screenshot showing From any storage account option set." lightbox="./media/tutorial-export-acm-data/permitted-scope-copy-operations.png" :::

+Enable **Allow trusted Azure services access** on the storage account. You can turn that on while configuring the firewall of the storage account, from the Networking page. Here's a screenshot showing the page.

+If you missed enabling that setting, you can easily do so from the **Exports** page when creating a new export.

+A system-assigned managed identity is created for a new job export when created or modified. You must have permissions because Cost Management uses the privilege to assign the *StorageBlobDataContributor* role to the managed identity. The permission is restricted to the storage account container scope. After the export job is created or updated, the user doesn't require Owner permissions for regular runtime operations.

+Scheduled exports get affected by the time and day of week of when you initially create the export. When you create a scheduled export, the export runs at the same frequency for each export that runs later. For example, for a daily export of month-to-date costs export set at a daily frequency, the export runs during once each UTC day. Similarly for a weekly export, the export runs every week on the same UTC day as it is scheduled. Individual export runs can occur at different times throughout the day. So, avoid taking a firm dependency on the exact time of the export runs. Run timing depends on the active load present in Azure during a given UTC day. When an export run begins, your data should be available within 4 hours.

+You can use a management group to aggregate subscription cost information in a single container. Exports support management group scope for Enterprise Agreement but not for Microsoft Customer Agreement or other subscription types. Multiple currencies are also not supported in management group exports.

+Exports at the management group scope support only usage charges, purchases (including reservations and savings plans). Amortized cost reports aren't supported. When you create an export from the Azure portal for a management group scope, the metric field isn't shown because it defaults to the usage type. When you create a management group scope export using the REST API, choose [ExportType](/rest/api/cost-management/exports/create-or-update#exporttype) as `Usage`.

+1. Create one management group and assign subscriptions to it, if you haven't already.

+If you have existing exports and you want to set up file partitioning, create a new export. File partitioning is only available with the latest Exports version. There might be minor changes to some of the fields in the usage files that get created.

+If you enable file partitioning on an existing export, you might see minor changes to the fields in file output. Any changes are due to updates that were made to Exports after you initially set yours up.

+The file opens with the program or application set to open CSV file extensions. Here's an example in Excel.

+There are two runs per day for the first five days of each month after you create a daily export. One run executes and creates a file with the current monthΓÇÖs cost data. It's the run that's available for you to see in the run history. A second run also executes to create a file with all the costs from the prior month. The second run isn't currently visible in the run history. Azure executes the second run to ensure that your latest file for the past month contains all charges exactly as seen on your invoice. It runs because there are cases where latent usage and charges are included in the invoice up to 72 hours after the calendar month is closed. To learn more about Cost Management usage data updates, see [Cost and usage data updates and retention](understand-cost-mgt-data.md#cost-and-usage-data-updates-and-retention).

+## Exports FAQ

+Here are some frequently asked questions and answers about exports.

+### Why do I see garbled characters when I open exported cost files with Microsoft Excel?

+If you see garbled characters in Excel and you use an Asian-based language, such as Japanese or Chinese, you can resolve this issue with the following steps:

+For new versions of Excel:

+1. Open Excel.

+1. Select the **Data** tab at the top.

+1. Select the **From Text/CSV** option.

+ :::image type="content" source="./media/tutorial-export-acm-data/new-excel-from-text.png" alt-text="Screenshot showing the Excel From Text/CSV option." lightbox="./media/tutorial-export-acm-data/new-excel-from-text.png" :::

+1. Select the CSV file that you want to import.

+1. In the next box, set **File origin** to **65001: Unicode (UTF-8)**.

+ :::image type="content" source="./media/tutorial-export-acm-data/new-excel-file-origin.png" alt-text="Screenshot showing the Excel File origin option." lightbox="./media/tutorial-export-acm-data/new-excel-file-origin.png" :::

+1. Select **Load**.

+For older versions of MS Excel:

+1. Open Excel.

+1. Select the **Data** tab at the top.

+1. Select the **From Text** option and then select the CSV file that you want to import.

+1. Excel shows the Text Import Wizard.

+1. In the wizard, select the **Delimited** option.

+1. In the **File origin** field, select **65001 : Unicode (UTF-8)**.

+1. Select **Next**.

+1. Next, select the **Comma** option and then select **Finish**.

+1. In the dialog window that appears, select **OK**.

+### Why does the aggregated cost from the exported file differ from the cost displayed in Cost Analysis?

+You might have discrepancies between the aggregated cost from the exported file and the cost displayed in Cost Analysis. Determine if the tool you use to read and aggregate the total cost is truncating decimal values. This issue can happen in tools like Power BI and Microsoft Excel. Determine if decimal places are getting dropped when cost values are converted into integers. Losing decimal values can result in a loss of precision and misrepresentation of the aggregated cost.

+To manually transform a column to a decimal number in Power BI, follow these steps:

+1. Go to the Table view.

+1. Select **Transform data**.

+1. Right-click the required column.

+1. Change the type to a decimal number.

+- The new reservation's lifetime commitment should equal or be greater than the returned reservation's remaining commitment. Example: for a three-year reservation that's 100 USD per month and exchanged after the 18th payment, the new reservation's lifetime commitment should be 1,800 USD or more (paid monthly or upfront).

+Let's look at an example with the previous points in mind. If you bought a 300,000 USD reservation, you can exchange it at any time for another reservation that equals or costs more (of the remaining reservation balance, not the original purchase price). For this example:

+|[Copy activity](copy-activity-overview.md) (source/sink)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Mapping data flow](concepts-data-flow-overview.md) (source/sink)|&#9312; |Γ£ô Exclude storage account V1|

+|[Lookup activity](control-flow-lookup-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[GetMetadata activity](control-flow-get-metadata-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Delete activity](delete-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Copy activity](copy-activity-overview.md) (source/sink)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Lookup activity](control-flow-lookup-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[GetMetadata activity](control-flow-get-metadata-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Delete activity](delete-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Copy activity](copy-activity-overview.md) (source/sink)|&#9312; &#9313;|Γ£ô Public preview |

+|[Mapping data flow](concepts-data-flow-overview.md) (source/sink)|&#9312; |Γ£ô Public preview |

+|[Lookup activity](control-flow-lookup-activity.md)|&#9312; &#9313;|Γ£ô Public preview |

+|[GetMetadata activity](control-flow-get-metadata-activity.md)|&#9312; &#9313;|Γ£ô Public preview |

+|[Script activity](transform-data-using-script.md)|&#9312; &#9313;|Γ£ô Public preview |

+|[Stored procedure activity](transform-data-using-stored-procedure.md)|&#9312; &#9313;|Γ£ô Public preview |

+| useTempDB | Specify whether to use a global temporary table or physical table as the interim table for upsert. <br>By default, the service uses global temporary table as the interim table. value is `true`. | No |

+|[Copy activity](copy-activity-overview.md) (source/sink)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+|[Lookup activity](control-flow-lookup-activity.md)|&#9312; &#9313;|Γ£ô Exclude storage account V1|

+| Source\Sink | Boolean | Byte array | Decimal | Date/Time (1) | Float-point (2) | GUID | Integer (3) | String | TimeSpan |

+In this sample, the output dataset has a structure and it points to a table in Salesforce.

+* If you select the **SentryOne's Task Factory** component, you can install the [Task Factory](https://www.solarwinds.com/resources/it-glossary/ssis-components) suite of components from SentryOne on your Azure-SSIS IR by entering the product license key that you purchased from them in the **License key** box. The current integrated version is **2020.21.2**.

+- [SAP CDC advanced topics](sap-change-data-capture-advanced-topics.md)

+1. Enable the compute on a port that has internet access. For example, in this case, port 2 that was connected to the internet is enabled for compute. Internet access allows you to retrieve container images from AKS.

+ The Azure consistent services virtual IP must be able to reach this compute virtual switch network either via external routing or by creating an Azure consistent services virtual IP on the same network.

+

+For Azure consistent services and NFS, you'll also need to define a virtual IP that allows you to connect to a clustered device as opposed to a specific node. A virtual IP is an available IP in the cluster network and any client connecting to the cluster network on the two-node device should be able to access this IP. Azure consistent services and NFS must be on the same network.

+For Azure consistent services and NFS, you'll also need to define a virtual IP that allows you to connect to a clustered device as opposed to a specific node. A virtual IP is an available IP in the cluster network and any client connecting to the cluster network on the two-node device should be able to access this IP. Azure consistent services and NFS must be on the same network.

+> [!NOTE]

+> Access to index tags requires permissions. For more information see [Get, set, and update blob index tags](/azure/storage/blobs/storage-blob-index-how-to#get-set-and-update-blob-index-tags).

+With agentless scanning for VMs, you get wide visibility on installed software and software CVEs. You get the visibility without the challenges of agent installation and maintenance, network connectivity requirements, and performance affect on your workloads. The analysis is powered by Microsoft Defender Vulnerability Management.

+- Microsoft Defender Vulnerability Management

+description: Learn how to authenticate to Azure Deployment Environments REST APIs by using Microsoft Entra ID.

+# Authenticate to Azure Deployment Environments REST APIs

+> [!TIP]

+> Before authenticating, ensure that the user or identity has the appropriate permissions to perform the desired action. For more information, see [Provide access for dev team leads](./how-to-configure-project-admin.md) and [Provide access for developers](./how-to-configure-deployment-environments-user.md).

+## Use Microsoft Entra ID authentication for REST APIs

+Use the following procedures to access Azure Deployment Environments REST APIs by using Microsoft Entra ID. You can follow along in [Azure Cloud Shell](../../articles/cloud-shell/quickstart.md), on an Azure virtual machine, or on your local machine.

+### Sign in to your Azure subscription

+The command opens a browser window to the Microsoft Azure authentication page, where you can choose an account. The page requires you to give your Microsoft Entra ID username and password.

+Next, set the correct subscription context. If you authenticate from an incorrect subscription or tenant, you might receive unexpected **403 Forbidden** errors.

+### Retrieve the Microsoft Entra ID access token

+Use the Azure CLI to acquire an access token for the Microsoft Entra ID authenticated user. The resource ID is different depending on if you access administrator (control plane) APIs or developer (data plane) APIs.

+After authentication is successful, Microsoft Entra ID returns an access token for the current Azure subscription:

+The token is a Base64 string. The token is valid for at least five minutes. The maximum duration is 90 minutes. The `expiresOn` defines the actual token expiration time.

+> Developer API tokens for the service are encrypted and can't be decoded using JWT decoding tools. They can only be processed by the service.

+### Use a bearer token to access REST APIs

+To access REST APIs, you must set the authorization header on your request. The header value should be the string `Bearer` followed by a space and the token you received in the previous step.

+- [Review Microsoft Entra ID fundamentals](../../articles/active-directory/fundamentals/whatis.md)

+The following steps show you how to create a dev box pool associated with a project. You use an existing dev box definition and network connection in the dev center to configure the pool.

+## Manage dev boxes in a pool

+You can manage existing dev boxes in a dev box pool through the Azure portal. You can start, stop, or delete dev boxes. You must be a member of the Project Admin role at the project level to manage dev boxes in pools.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. In the search box, enter **projects**, in the list of results, select **Projects**.

+

+1. Select the project that contains the dev box pool that you want to manage.

+1. Select **Dev box pools**.

+

+1. Select the pool that contains the dev box that you want to manage.

+ :::image type="content" source="media/how-to-manage-dev-box-pools/manage-dev-box-pool.png" alt-text="Screenshot showing a list of dev box pools in Azure portal." lightbox="media/how-to-manage-dev-box-pools/manage-dev-box-pool.png":::

+

+1. Scroll to the far right, and select the Dev box operations menu (**...**) for the dev box that you want to manage.

+

+ :::image type="content" source="media/how-to-manage-dev-box-pools/manage-dev-box-in-azure-portal.png" alt-text="Screenshot of the Azure portal, showing dev boxes in a dev box pool." lightbox="media/how-to-manage-dev-box-pools/manage-dev-box-in-azure-portal.png":::

+1. Depending on the current state of the dev box, you can select **Start**, **Stop**, or **Delete**.

+ :::image type="content" source="media/how-to-manage-dev-box-pools/dev-box-operations-menu.png" alt-text="Screenshot of the Azure portal, showing the menu for managing a dev box." lightbox="media/how-to-manage-dev-box-pools/dev-box-operations-menu.png":::

+This [Dev Box with customized image](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.devcenter/devbox-with-customized-image) template deploys a simple Dev Box environment that you can use for testing and exploring the service.

+It creates the following Dev Box resources: dev center, project, network connection, dev box definition, and dev box pool. Once the template is deployed, you can go to the [developer portal](https://aka.ms/devbox-portal) to [create your dev box](quickstart-create-dev-box.md).

+- Microsoft Intune subscription. Your organization must use Microsoft Intune for device management.

+The template used in this QuickStart is fromΓÇ»[Azure Quickstart Templates](/samples/azure/azure-quickstart-templates/devbox-with-customized-image/).

+The template for this article is too long to show here. To view the template, see [azuredeploy.json](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.devcenter/devbox-with-customized-image/azuredeploy.json)

+- [Microsoft.DevCenter/devcenters/galleries](/azure/templates/microsoft.devcenter/devcenters/galleries): create an Azure Compute Gallery.

+ $userPrincipalName = Read-Host "Please enter user principal name e.g. alias@xxx.com"

+ $resourceGroupName = Read-Host "Please enter resource group name e.g. rg-devbox-dev"

+ $location = Read-Host "Please enter region name e.g. eastus"

+ $templateUri = "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/quickstarts/microsoft.devcenter/devbox-with-customized-image/azuredeploy.json"

+ $userPrincipalId=(Get-AzADUser -UserPrincipalName $userPrincipalName).Id

+ if($userPrincipalId){

+ Write-Host "Start provisioning..."

+ az group create -l $location -n $resourceGroupName

+ az group deployment create -g $resourceGroupName --template-uri $templateUri --parameters userPrincipalId=$userPrincipalId

+ }else {

+ Write-Host "User Principal Name cannot be found."

+ }

+ Write-Host "Provisioning Completed."

+It takes about 30 minutes to deploy the template.

+### Required parameters:

+- *User Principal ID*: The user principal ID of the user or group that is granted the *Devcenter Dev Box User* role.

+- *User Principal Type*: The type of user principal. Valid values are *User* or *Group*.

+- *Location*: The location where the resources are deployed. Choose a location close to the dev boxes users to reduce latency.

+Alternatively, you can provide access to a dev box project in the Azure portal, see [Provide user-level access to projects for developers](how-to-dev-box-user.md).

+### Virtual network security considerations

+Planning for a Microsoft Dev Box deployment covers many areas, including securing the virtual network (VNet). For more information, see [Azure network security overview](../security/fundamentals/network-overview.md).

+ :::image type="content" source="media/quickstart-configure-dev-box-arm-template/dev-box-template-resources.png" alt-text="Screenshot showing the newly created dev box resource group and the resources it contains in the Azure portal." lightbox="media/quickstart-configure-dev-box-arm-template/dev-box-template-resources.png":::

+## Find more templates

+To find more templates that are related to Microsoft Dev Box, see [Azure Quickstart Templates](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.devcenter).

+For example, you can use a template to [add other customized images for Base, Java, .NET and Data](https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.devcenter/devbox-with-customized-image#add-other-customized-image-for-base-java-net-and-data). These images have the following software and tools installed:

+|Image type |Software and tools |

+|||

+|Base |Git, Azure CLI, VS Code, VS Code Extension for GitHub Copilot |

+|Java |Git, Azure CLI, VS Code, Maven, OpenJdk11, VS Code Extension for Java Pack |

+|.NET |Git, Azure CLI, VS Code，.NET SDK, Visual Studio |

+|Data |Git, Azure CLI, VS Code，Python 3, VS Code Extension for Python and Jupyter |

+ The template deployment creates a zone with one `A` record pointing to two IP addresses. The resource group name is the project name with `rg` appended.

+1. Select the resource group that you created in the previous section. The default resource group name is the project name with `rg` appended.

+description: Learn how to create a DNS zone and record in Azure DNS. This article is a step-by-step quickstart to create and manage your first DNS zone and record using Azure PowerShell.

+A DNS zone is used to host the DNS records for a particular domain. To start hosting your domain in Azure DNS, you need to create a DNS zone for that domain name. Each DNS record for your domain is then created inside this DNS zone. Finally, to publish your DNS zone to the Internet, you need to configure the name servers for the domain. Each of these steps is described in this article.

+Create record sets by using the `New-AzDnsRecordSet` cmdlet. The following example creates a record with the relative name `www` in the DNS Zone `contoso.xyz`, in resource group `MyResourceGroup`. The fully qualified name of the record set is `www.contoso.xyz`. The record type is `A`, with IP address `10.10.10.10`, and the TTL is 3600 seconds.

+Now that your first DNS zone and record is created using Azure PowerShell, you can create records for a web app in a custom domain.

+For example, suppose the Public IP address resource has the DNS name `contosoapp1.northus.cloudapp.azure.com` and IP address `23.96.52.53`. The reverse FQDN for the Public IP address can be specified as:

+1. Confirm that the DNS records are configured correctly in Azure DNS. Review the DNS records in the Azure portal, checking that the zone name, record name, and record type are correct.

+3. Confirm that the DNS domain name is correctly [delegated to the Azure DNS name servers](dns-domain-delegation.md). There are a [many 3rd-party web sites that offer DNS delegation validation](https://www.bing.com/search?q=dns+check+tool). This test is a *zone* delegation test, so you should only enter the DNS zone name and not the fully qualified record name.

+A primary zone contains NS delegation records, which help delegate traffic from the primary to the child zones. If any NS delegation record is present in the parent zone, the DNS server is supposed to mask all other records below the NS delegation record (except glue records) and direct traffic to the respective child zone based on the user query. If a parent zone contains other records meant for the child zones (delegated zones) below the NS delegation record, the zone will be marked unhealthy and its status is **Degraded**.

+**How to locate unhealthy delegation records?** - A script is provided to find the unhealthy delegation records in your zone. The script will report records, which are unhealthy.

+6. The access token of the app id and client secret has the Infra Admin access to the instance.

+ 1. Add the users to the `users@<data-partition-id>.<domain>` OSDU group.

+ 1. Get the OSDU group such as `service.legal.editor@<data-partition-id>.<domain>` you want to add the user to.

+ 1. Add the users to that group.

+By default, Event Grid namespaces and entities in them such as Message Queuing Telemetry Transport (MQTT) topic spaces are accessible from internet as long as the request comes with valid authentication (access key) and authorization. With IP firewall, you can restrict it further to only a set of IPv4 addresses or IPv4 address ranges in [CIDR (Classless Inter-Domain Routing)](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) notation. Only the MQTT clients that fall into the allowed IP range can connect to publish and subscribe. Clients originating from any other IP address are rejected and receive a 403 (Forbidden) response. For more information about network security features supported by Event Grid, see [Network security for Event Grid](network-security.md).

+ Title: Configure IP firewall for Azure Event Grid namespaces

+description: This article describes how to configure firewall settings for Azure Event Grid namespaces.

+# Configure IP firewall for Azure Event Grid namespaces

+By default, Event Grid namespaces and entities are accessible from internet as long as the request comes with valid authentication (access key) and authorization. With IP firewall, you can restrict it further to only a set of IPv4 addresses or IPv4 address ranges in [CIDR (Classless Inter-Domain Routing)](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) notation. Only the clients that fall into the allowed IP range can connect to Azure Event Grid to publish events or pull events. Clients originating from any other IP address are rejected and receive a 403 (Forbidden) response. For more information about network security features supported by Event Grid, see [Network security for Event Grid](network-security.md).

+This article describes how to configure IP firewall settings for an Event Grid namespace. For complete steps for creating a namespace, see [Create and manage namespaces](create-view-manage-namespaces.md).

+## Create a namespace with IP firewall settings

+1. On the **Networking** page, if you want to allow clients to connect to the namespace endpoint via a public IP address, select **Public access** for **Connectivity method** if it's not already selected.

+2. You can restrict access to the topic from specific IP addresses by specifying values for the **Address range** field. Specify a single IPv4 address or a range of IP addresses in Classless inter-domain routing (CIDR) notation.

+ :::image type="content" source="./media/configure-firewall-namespaces/ip-firewall-settings.png" alt-text="Screenshot that shows IP firewall settings on the Networking page of the Create namespace wizard.":::

+## Update a namespace with IP firewall settings

+1. Sign-in to the [Azure portal](https://portal.azure.com).

+1. In the **search box**, enter **Event Grid Namespaces** and select **Event Grid Namespaces** from the results.

+ :::image type="content" source="./media/create-view-manage-namespaces/portal-search-box-namespaces.png" alt-text="Screenshot showing Event Grid Namespaces in the search results.":::

+1. Select your Event Grid namespace in the list to open the **Event Grid Namespace** page for your namespace.

+1. On the **Event Grid Namespace** page, select **Networking** on the left menu.

+1. Specify values for the **Address range** field. Specify a single IPv4 address or a range of IP addresses in Classless inter-domain routing (CIDR) notation.

+ :::image type="content" source="./media/configure-firewall-namespaces/namespace-ip-firewall-settings.png" alt-text="Screenshot that shows IP firewall settings on the Networking page of an existing namespace.":::

+## Next steps

+See [Allow access via private endpoints](configure-private-endpoints-pull.md).

+You can use [private endpoints](../private-link/private-endpoint-overview.md) to allow ingress of events directly from your virtual network to entities in your Event Grid namespaces securely over a [private link](../private-link/private-link-overview.md) without going through the public internet. The private endpoint uses an IP address from the virtual network address space for your namespace. When an MQTT client on a private network connects to the MQTT broker on a private link, the client can publish and subscribe to MQTT messages. For more conceptual information, see [Network security](network-security.md).

+1. If there are any connections that are pending, you see a connection listed with **Pending** in the provisioning state.

+You can use [private endpoints](../private-link/private-endpoint-overview.md) to allow clients from only your virtual network to connect to your Event Grid namespace securely over a [private link](../private-link/private-link-overview.md) without going through the public internet. The private endpoint uses an IP address from the virtual network address space for your namespace. A client in a private network can connect to the Event Grid namespace and publish events or pull events. For more conceptual information, see [Network security](network-security-namespaces.md).

+1. If there are any connections that are pending, you see a connection listed with **Pending** in the provisioning state.

+To learn about how to configure IP firewall settings, see [Configure IP firewall for Azure Event Grid namespaces](configure-firewall-namespaces.md).

+ 1. Sign in to [Azure portal](https://portal.azure.com).

+ 2. In the search box at the top, type **Event Grid System Topics**, and then press **ENTER**.

+ :::image type="content" source="./media/create-view-manage-system-topics/search-system-topics.png" alt-text="Screenshot that shows the Azure portal with Event Grid System Topics in the search box.":::

+ 3. On the **Event Grid System Topics** page, select **+ Create** on the toolbar.

+

+ :::image type="content" source="./media/create-view-manage-system-topics/add-system-topic-menu.png" alt-text="Screenshot that shows in the Event Grid System Topics page with the Create button selected." lightbox="./media/create-view-manage-system-topics/add-system-topic-menu.png":::

+ 4. On the **Create Event Grid System Topic** page, do the following steps:

+ 1. Select the **topic type**. In the following example, **Storage Accounts** option is selected.

+ 2. Select the **Azure subscription** that has your storage account resource.

+ 3. Select the **resource group** that has the storage account.

+ 4. Select the **storage account**.

+ 5. Enter a **name** for the system topic to be created.

+ > [!NOTE]

+ > You can use this system topic name to search metrics and diagnostic logs.

+ 6. Select **Review + create**.

+

+ :::image type="content" source="./media/create-view-manage-system-topics/create-system-topic-page.png" alt-text="Screenshot that shows the Create System Topic page.":::

+ 5. Review settings and select **Create**.

+

+ :::image type="content" source="./media/create-view-manage-system-topics/system-topic-review-create.png" alt-text="Screenshot that shows the Review & Create page.":::

+ 6. After the deployment succeeds, select **Go to resource** to see the **Event Grid System Topic** page for the system topic you created.

+ :::image type="content" source="./media/create-view-manage-system-topics/system-topic-page.png" alt-text="Screenshot that shows the System Topic home page." lightbox="./media/create-view-manage-system-topics/system-topic-page.png":::

+ :::image type="content" source="./media/create-view-manage-system-topics/system-topic-delete-button.png" alt-text="Screenshot that shows the System Topic page with the Delete button selected.":::

+ :::image type="content" source="./media/create-view-manage-system-topics/add-event-subscription-button.png" alt-text="Screenshot that shows the System Topic page with Add Event Subscription button selected.":::

+ Title: Azure Maintenance Configuration as an Event Grid source

+description: The article provides details on Azure Maintenance Configuration as an Event Grid source.

+# Azure Maintenance Configuration as an Event Grid source

+This article provides the properties and schema for Azure Maintenance Configurations events. For an introduction to event schemas, see [Azure Event Grid event schema](./event-schema.md). It also gives you links to articles to use Maintenance Configuration as an event source.

+## Available event types

+Maintenance Configuration emits the following event types:

+**Event type** | **Description**

+| |

+Microsoft.Maintenance.PreMaintenanceEvent | Raised before maintenance job start and gives user an opportunity to perform pre maintenance operations. |

+Microsoft.Maintenance.PostMaintenanceEvent | Raised after maintenance job completes and gives an opportunity to perform post maintenance operations.

+## Example event

+# [Event Grid event schema](#tab/event-grid-event-schema)

+Following is an example of a schema for the Pre-Maintenance event:

+```json

+[{

+ "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "topic": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "subject": "contosomaintenanceconfiguration",

+"data":

+{

+ "correlationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "maintenanceConfigurationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "startDateTime": "2023-05-09T15:00:00Z",

+ "endDateTime": "2023-05-09T18:55:00Z",

+ "cancellationCutOffDateTime": "2023-05-09T14:59:00Z",

+ "resourceSubscriptionIds": ["subscription guid 1", "subscription guid 2"]

+}

+"eventType": "Microsoft.Maintenance.PreMaintenanceEvent",

+"eventTime": "2023-05-09T14:25:00.3717473Z",

+ "dataVersion": "1.0",

+ "metadataVersion": "1"

+}]

+```

+# [Cloud event schema](#tab/cloud-event-schema)

+Following is an example for a schema of a pre-maintenance event:

+```json

+[{

+ "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "source": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "subject": "contosomaintenanceconfiguration",

+"data":

+{

+ "correlationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "maintenanceConfigurationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "startDateTime": "2023-05-09T15:00:00Z",

+ "endDateTime": "2023-05-09T18:55:00Z",

+ "cancellationCutOffDateTime": "2023-05-09T14:59:00Z",

+ "resourceSubscriptionIds": ["subscription guid 1", "subscription guid 2"]

+}

+"type": "Microsoft.Maintenance.PreMaintenanceEvent",

+"time": "2023-05-09T14:25:00.3717473Z",

+ "specversion": "1.0"

+}]

+```

+# [Event Grid event schema](#tab/event-grid-event-schema)

+Following is an example of a schema for a post-maintenance event:

+```json

+[{

+ "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "topic": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "subject": "contosomaintenanceconfiguration",

+"data":

+{

+ "correlationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "maintenanceConfigurationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "status": "Succeeded",

+ "startDateTime": "2023-05-09T15:00:00Z",

+ "endDateTime": "2023-05-09T18:55:00Z",

+ "resourceSubscriptionIds": ["subscription guid 1", "subscription guid 2"]

+}

+"eventType": "Microsoft.Maintenance.PostMaintenanceEvent",

+"eventTime": "2023-05-09T15:55:00.3717473Z",

+ "dataVersion": "1.0",

+ "metadataVersion": "1"

+}]

+```

+# [Cloud event schema](#tab/cloud-event-schema)

+Following is an example for a post maintenance event:

+```json

+[{

+ "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "source": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "subject": "contosomaintenanceconfiguration",

+"data":

+{

+ "correlationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration/providers/microsoft.maintenance/applyupdates/20230509150000",

+ "maintenanceConfigurationId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testrg/providers/Microsoft.Maintenance/maintenanceConfigurations/contosomaintenanceconfiguration",

+ "status": "Succeeded",

+ "startDateTime": "2023-05-09T15:00:00Z",

+ "endDateTime": "2023-05-09T18:55:00Z",

+ "resourceSubscriptionIds": ["subscription guid 1", "subscription guid 2"]

+}

+"type": "Microsoft.Maintenance.PostMaintenanceEvent",

+"time": "2023-05-09T15:55:00.3717473Z",

+ "specversion": "1.0"

+}]

+```

+## Event properties

+# [Event Grid event schema](#tab/event-grid-event-schema)

+An event has the following top-level data:

+**Property** | **Type** | **Description** |

+ | | |

+topic | string | Full resource path to the event source. This field isn't writeable. Event Grid provides this value. |

+subject | string | Publisher-defined path to the event subject. |

+eventType | string | One of the registered event types for this event source. |

+eventTime | string | The time the event is generated based on the provider's UTC time. |

+ID | string | Unique identifier for the event |

+data | object | App Configuration event data. |

+dataVersion | string | The schema version of the data object. The publisher defines the schema version. |

+metadataVersion | string | The schema version of the event metadata. Event Grid defines the schema of the top-level properties. Event Grid provides this value. |

+# [Cloud event schema](#tab/cloud-event-schema)

+An event has the following top-level data:

+**Property** | **Type** | **Description** |

+ | | |

+source | string | Full resource path to the event source. This field isn't writeable. Event Grid provides this value. |

+subject | string | Publisher-defined path to the event subject. |

+type | string | One of the registered event types for this event source. |

+time | string | The time the event is generated based on the provider's UTC time. |

+ID | string | Unique identifier for the event. |

+data | object | App Configuration event data. |

+specversion | string | CloudEvents schema specification version.

+The data object has the following properties:

+**Property** | **Type** | **Description** |

+ | | |

+correlationId | string | The resource ID of specific maintenance schedule instance. |

+maintenanceConfigurationId | string | The resource ID of maintenance configuration. |

+startDateTime | string | The maintenance schedule start time. |

+endDateTime | string | The maintenance schedule end time. |

+cancellationCutOffDateTime | string | The maintenance schedule instance cancellation cut-off time. |

+resourceSubscriptionIds | string | The subscription IDs from which VMs are included in this schedule instance. |

+status | string | The completion status of maintenance schedule instance.

+## Next steps

+- For an introduction to Azure Event Grid, see [What is Event Grid?](./overview.md)

+- For more information about creating an Azure Event Grid subscription, see [Event Grid subscription schema](./subscription-creation-schema.md).

+ Title: Network security for Azure Event Grid namespaces

+description: This article describes how to use service tags for egress, IP firewall rules for ingress, and private endpoints for ingress with Azure Event Grid namespaces.

+ - ignite-2023

+# Network security for Azure Event Grid namespaces

+This article describes how to use the following security features with Azure Event Grid:

+- Service tags for egress

+- IP Firewall rules

+- Private endpoints

+## Service tags

+A service tag represents a group of IP address prefixes from a given Azure service. Microsoft manages the address prefixes encompassed by the service tag and automatically updates the service tag as addresses change, minimizing the complexity of frequent updates to network security rules. For more information about service tags, see [Service tags overview](../virtual-network/service-tags-overview.md).

+You can use service tags to define network access controls on network security groups or Azure firewall. Use service tags in place of specific IP addresses when you create security rules. By specifying the service tag name (for example, Azure Event Grid) in the appropriate source or destination fields of a rule, you can allow or deny the traffic for the corresponding service.

+| Service tag | Purpose | Can use inbound or outbound? | Can be regional? | Can use with Azure Firewall? |

+| | -- |::|::|::|

+| AzureEventGrid | Azure Event Grid. | Both | No | No |

+## IP firewall

+By default, Event Grid namespaces and entities are accessible from internet as long as the request comes with valid authentication (access key) and authorization. With IP firewall, you can restrict it further to only a set of IPv4 addresses or IPv4 address ranges in [CIDR (Classless Inter-Domain Routing)](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) notation. Clients originating from any other IP address are rejected and receive a 403 (Forbidden) response.

+For Message Queueing Telemetry Transport (MQTT) scenarios, only the clients that fall into the allowed IP range can connect to publish and subscribe for events. For more information, see [Configure IP firewall for Event Grid namespaces in MQTT scenarios](configure-firewall-mqtt.md).

+For non-MQTT scenarios, only the clients that fall into the allowed IP range can connect to Azure Event Grid to publish events or pull events. For more information, see [Configure IP firewall for Event Grid namespaces in non-MQTT scenarios](configure-firewall-namespaces.md).

+The steps are same for both scenarios. These articles provide some additional information specific to the MQTT or non-MQTT scenarios.

+## Private endpoints

+A private endpoint is a special network interface for an Azure service in your VNet. When you create a private endpoint for your namespace, it provides secure connectivity between clients on your VNet and your Event Grid namespace. The private endpoint is assigned an IP address from the IP address range of your VNet. The connection between the private endpoint and the Event Grid service uses a secure private link. Using private endpoints for your Event Grid namespace enables you to:

+- Secure access to your namespace from a virtual network over the Microsoft backbone network as opposed to the public internet.

+- Securely connect from on-premises networks that connect to the virtual network using VPN or Express Routes with private-peering.

+When you create a private endpoint for a namespace in your virtual network, a consent request is sent for approval to the resource owner. If the user requesting the creation of the private endpoint is also an owner of the resource, this consent request is automatically approved. Otherwise, the connection is in **pending** state until approved. Applications in the virtual network can connect to the Event Grid service over the private endpoint seamlessly, using the same connection strings and authorization mechanisms that they would use otherwise. Resource owners can manage consent requests and the private endpoints, through the **Private endpoints** tab for the resource in the Azure portal.

+When an MQTT client in a private network connects to the MQTT broker on a private link, the client can publish and subscribe to MQTT messages. For more information, see [Configure private endpoints for namespaces in MQTT scenarios](configure-private-endpoints-mqtt.md).

+In non-MQTT scenarios, a client in a private network can connect to the Event Grid namespace and publish events or pull events. For more information, see [Configure private endpoints for namespaces in non-MQTT scenarios](configure-private-endpoints-pull.md).

+### Connect to private endpoints

+You can use [private endpoints](../private-link/private-endpoint-overview.md) to allow ingress of events directly from your virtual network to entities in your Event Grid namespaces securely over a [private link](../private-link/private-link-overview.md) without going through the public internet. The private endpoint uses an IP address from the virtual network address space for your namespace.

+Clients on a virtual network using the private endpoint should use the same connection string for the namespace as clients connecting to the public endpoint. Domain Name System (DNS) resolution automatically routes connections from the virtual network to the namespace over a private link. Event Grid creates a [private DNS zone](../dns/private-dns-overview.md) attached to the virtual network with the necessary update for the private endpoints, by default. However, if you're using your own DNS server, you might need to make additional changes to your DNS configuration.

+When an MQTT client on a private network connects to the MQTT broker on a private link, the client can publish and subscribe to MQTT messages.

+### DNS changes for private endpoints

+When you create a private endpoint, the DNS CNAME record for the resource is updated to an alias in a subdomain with the prefix `privatelink`. By default, a private DNS zone is created that corresponds to the private link's subdomain.

+When you resolve the namespace endpoint URL from outside the virtual network with the private endpoint, it resolves to the public endpoint of the service. The DNS resource records for 'namespaceA', when resolved from **outside the VNet** hosting the private endpoint, are:

+| Name | Type | Value |

+| | -| |

+| `namespaceA.westus.eventgrid.azure.net` | CNAME | `namespaceA.westus.privatelink.eventgrid.azure.net` |

+| `namespaceA.westus.privatelink.eventgrid.azure.net` | CNAME | \<Azure traffic manager profile\>

+You can deny or control access for a client outside the virtual network through the public endpoint using the [IP firewall](#ip-firewall).

+When resolved from the virtual network hosting the private endpoint, the namespace endpoint URL resolves to the private endpoint's IP address. The DNS resource records for the namespace 'namespaceA', when resolved from **inside the VNet** hosting the private endpoint, are:

+| Name | Type | Value |

+| | -| |

+| `namespaceA.westus.eventgrid.azure.net` | CNAME | `namespaceA.westus.privatelink.eventgrid.azure.net` |

+| `namespaceA.westus.privatelink.eventgrid.azure.net` | A | 10.0.0.5

+This approach enables access to the namespace using the same connection string for clients on the virtual network hosting the private endpoints, and clients outside the virtual network.

+If you're using a custom DNS server on your network, clients can resolve the Fully Qualified Domain Name (FQDN) for the namespace endpoint to the private endpoint IP address. Configure your DNS server to delegate your private link subdomain to the private DNS zone for the virtual network, or configure the A records for `namespaceName.regionName.privatelink.eventgrid.azure.net` with the private endpoint IP address.

+The recommended DNS zone name is `privatelink.eventgrid.azure.net`.

+### Private endpoints and publishing

+The following table describes the various states of the private endpoint connection and the effects on publishing:

+| Connection State | Successfully publish (Yes/No) |

+| | -|

+| Approved | Yes |

+| Rejected | No |

+| Pending | No |

+| Disconnected | No |

+For publishing to be successful, the private endpoint connection state should be **approved**. If a connection is rejected, it can't be approved using the Azure portal. The only possibility is to delete the connection and create a new one instead.

+## Quotas and limits

+There's a limit on the number of IP firewall rules and private endpoint connections per namespace. See [Event Grid quotas and limits](quotas-limits.md).

+## Related articles

+If you're using MQTT, see the following articles:

+- [Configure IP firewall for Event Grid namespaces in MQTT scenarios](configure-firewall-mqtt.md)

+- [Configure private endpoints for Event Grid namespaces in MQTT scenarios](configure-private-endpoints-mqtt.md)

+For pull-based event delivery, see the following articles:

+- [Configure IP firewall for Event Grid namespaces in non-MQTT scenarios](configure-firewall-namespaces.md)

+- [Configure private endpoints for Event Grid namespaces in non-MQTT scenarios](configure-private-endpoints-pull.md)

+ :::image type="content" source="./media/subscribe-through-portal/select-all-services.png" alt-text="Screenshot that shows the Azure portal with All Services selected on the left menu.":::

+ :::image type="content" source="./media/subscribe-through-portal/search.png" alt-text="Screenshot that shows Event Grid Subscription in the search box in the Azure portal.":::

+ :::image type="content" source="./media/subscribe-through-portal/add-subscription.png" alt-text="Screenshot that shows the select of Add Event Subscription menu on the Event Grid Subscriptions page.":::

+1. On the **Create Event Subscription** page, follow these steps:

+ 1. Enter a name for the event subscription.

+ 1. Select the type of event source (**topic type**) on which you want to create a subscription. For example, to subscribe to events for your Azure storage account, select **Storage Accounts**.

+

+ :::image type="content" source="./media/subscribe-through-portal/azure-subscription.png" alt-text="Screenshot that shows the Create Event Subscription page.":::

+ 1. Select the Azure subscription that contains the storage account.

+ 1. Select the resource group that has the storage account.

+ 1. Then, select the storage account.

+ :::image type="content" source="./media/subscribe-through-portal/create-event-subscription.png" alt-text="Screenshot that shows the Create Event Subscription page with the storage account selected.":::

+1. Select the event types that you want to receive on the event subscription.

+ :::image type="content" source="./media/subscribe-through-portal/select-event-types.png" alt-text="Screenshot that shows the selection of event types.":::

+ :::image type="content" source="./media/subscribe-through-portal/select-end-point.png" alt-text="Screenshot that shows the selection of an endpoint.":::

+ :::image type="content" source="./media/subscribe-through-portal/select-additional-features.png" alt-text="Screenshot that shows the Additional features tab of the Create Event Subscription page.":::

+> [!TIP]

+> Activate Top flows logs only when troubleshooting a specific issue to avoid excessive CPU usage of Azure Firewall.

+>

+> [!TIP]

+> To avoid excessive disk usage caused by Flow trace logs in Azure Firewall with many short-lived connections, activate the logs only when troubleshooting a specific issue for diagnostic purposes.

+For assignments with a Resource Provider mode, select the _Non-compliant_ resource to view its component compliance records. The **Component Compliance** tab shows more information specific to the [Resource Provider mode](../concepts/definition-structure.md#resource-provider-modes) like **Component Name**, **Component ID**, and **Type**.

+ The _visual diff_ aides in identifying changes to a resource. The changes detected might not be

+ Title: Use Apache Flink on HDInsight on AKS with Azure Service Bus

+description: Use Apache Flink DataStream API on HDInsight on AKS with Azure Service Bus

+# Use Apache Flink on HDInsight on AKS with Azure Service Bus

+This article provides an overview and demonstration of Apache Flink DataStream API on HDInsight on AKS for Azure Service Bus. A Flink job demonstration is designed to read messages from an [Azure Service Bus](/azure/service-bus-messaging/service-bus-messaging-overview) and writes them to [Azure Data Lake Storage Gen2](./assign-kafka-topic-event-message-to-azure-data-lake-storage-gen2.md) (ADLS Gen2).

+## Prerequisites

+- [Flink Cluster 1.16.0 on HDInsight on AKS](./flink-create-cluster-portal.md)

+- For this demonstration, we use a Window VM as maven project develop env in the same VNET as HDInsight on AKS.

+- During the [creation](./flink-create-cluster-portal.md) of the Flink cluster, you are required to ensure that SSH access is selected. This enables you to access the cluster using Secure Shell (SSH).

+- Set up an [Azure Service Bus](/azure/service-bus-messaging/service-bus-messaging-overview) instance.

+- To proceed with the integration, obtain the necessary connection string, topic name, and subscription name for your [Azure Service Bus](/azure/service-bus-messaging/service-bus-messaging-overview).

+## Develop Apache Flink job

+This job is designed to read messages from an [Azure Service Bus](/azure/service-bus-messaging/service-bus-messaging-overview) and writes them to [Data Lake Storage Gen2](./assign-kafka-topic-event-message-to-azure-data-lake-storage-gen2.md) (ADLS Gen2).

+### Submit the JAR into Azure HDInsight on AKS with Flink

+To initiate a job, transfer the JAR file into the webssh pod and submit the job using the following command:

+```

+bin/flink run -c contoso.example.ServiceBusToAdlsGen2 -j AzureServiceBusDemo-1.0-SNAPSHOT.jar

+Job has been submitted with JobID fc5793361a914821c968b5746a804570

+```

+### Confirm job submission on Flink UI

+After submitting the job, access the Flink Dashboard UI and click on the running job for further details.

+### Sending message from Azure Service Bus Explorer

+Navigate to the Service Bus Explorer on the Azure portal and send messages to the corresponding Service Bus.

+### Check job run details on Apache Flink UI

+Review the running details of the job on the Flink UI for insights.

+### Confirm output file in ADLS gen2 on Portal

+After successfully publishing messages, verify the output file generated by the Flink job in ADLS Gen2 storage on the Azure portal.

+## Source Code

+In the POM.xml file, we define the project's dependencies using Maven, ensuring a smooth and organized management of libraries and plugins. Here's a snippet illustrating how to include dependencies for a project, such as one involving Apache Flink:

+```Maven Pom.xml

+<?xml version="1.0" encoding="UTF-8"?>

+<project xmlns="http://maven.apache.org/POM/4.0.0"

+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

+ <modelVersion>4.0.0</modelVersion>

+ <groupId>contoso.example</groupId>

+ <artifactId>AzureServiceBusDemo</artifactId>

+ <version>1.0-SNAPSHOT</version>

+ <properties>

+ <maven.compiler.source>1.8</maven.compiler.source>

+ <maven.compiler.target>1.8</maven.compiler.target>

+ <flink.version>1.16.0</flink.version>

+ <java.version>1.8</java.version>

+ </properties>

+ <dependencies>

+ <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java -->

+ <dependency>

+ <groupId>org.apache.flink</groupId>

+ <artifactId>flink-streaming-java</artifactId>

+ <version>${flink.version}</version>

+ </dependency>

+ <dependency>

+ <groupId>org.apache.flink</groupId>

+ <artifactId>flink-java</artifactId>

+ <version>${flink.version}</version>

+ </dependency>

+ <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients -->

+ <dependency>

+ <groupId>org.apache.flink</groupId>

+ <artifactId>flink-clients</artifactId>

+ <version>${flink.version}</version>

+ </dependency>

+ <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-files -->

+ <dependency>

+ <groupId>org.apache.flink</groupId>

+ <artifactId>flink-connector-files</artifactId>

+ <version>${flink.version}</version>

+ </dependency>

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-core</artifactId>

+ <version>1.26.0</version>

+ </dependency>

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-identity</artifactId>

+ <version>1.4.1</version>

+ </dependency>

+<!-- <dependency>-->

+<!-- <groupId>com.microsoft.azure</groupId>-->

+<!-- <artifactId>azure-servicebus</artifactId>-->

+<!-- <version>3.6.0</version>-->

+<!-- </dependency>-->

+ <dependency>

+ <groupId>com.azure</groupId>

+ <artifactId>azure-messaging-servicebus</artifactId>

+ <version>7.5.0</version>

+ </dependency>

+ </dependencies>

+ <build>

+ <plugins>

+ <plugin>

+ <groupId>org.apache.maven.plugins</groupId>

+ <artifactId>maven-assembly-plugin</artifactId>

+ <version>3.0.0</version>

+ <configuration>

+ <appendAssemblyId>false</appendAssemblyId>

+ <descriptorRefs>

+ <descriptorRef>jar-with-dependencies</descriptorRef>

+ </descriptorRefs>

+ </configuration>

+ <executions>

+ <execution>

+ <id>make-assembly</id>

+ <phase>package</phase>

+ <goals>

+ <goal>single</goal>

+ </goals>

+ </execution>

+ </executions>

+ </plugin>

+ </plugins>

+ </build>

+</project>

+```

+**main function: ServiceBusToAdlsGen2.java**

+```

+package contoso.example;

+import org.apache.flink.api.common.serialization.SimpleStringEncoder;

+import org.apache.flink.configuration.MemorySize;

+import org.apache.flink.connector.file.sink.FileSink;

+import org.apache.flink.core.fs.Path;

+import org.apache.flink.streaming.api.datastream.DataStream;

+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;

+import org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy;

+import java.time.Duration;

+public class ServiceBusToAdlsGen2 {

+ public static void main(String[] args) throws Exception {

+ // Set up the execution environment

+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

+ final String connectionString = "Endpoint=sb://contososervicebus.servicebus.windows.net/;SharedAccessKeyName=policy1;SharedAccessKey=<key>";

+ final String topicName = "topic1";

+ final String subName = "subscription1";

+ // Create a source function for Azure Service Bus

+ SessionBasedServiceBusSource sourceFunction = new SessionBasedServiceBusSource(connectionString, topicName, subName);

+ // Create a data stream using the source function

+ DataStream<String> stream = env.addSource(sourceFunction);

+ // Process the data (this is where you'd put your processing logic)

+ DataStream<String> processedStream = stream.map(value -> processValue(value));

+ processedStream.print();

+ // 3. sink to gen2

+ String outputPath = "abfs://<container>@<account>.dfs.core.windows.net/data/ServiceBus/Topic1";

+// String outputPath = "src/ServiceBugOutput/";

+ final FileSink<String> sink = FileSink

+ .forRowFormat(new Path(outputPath), new SimpleStringEncoder<String>("UTF-8"))

+ .withRollingPolicy(

+ DefaultRollingPolicy.builder()

+ .withRolloverInterval(Duration.ofMinutes(2))

+ .withInactivityInterval(Duration.ofMinutes(3))

+ .withMaxPartSize(MemorySize.

+ ofMebiBytes(5))

+ .build())

+ .build();

+ // Add the sink function to the processed stream

+ processedStream.sinkTo(sink);

+ // Execute the job

+ env.execute("ServiceBusToDataLakeJob");

+ }

+ private static String processValue(String value) {

+ // Implement your processing logic here

+ return value;

+ }

+}

+```

+**input source: SessionBasedServiceBusSource.java**

+```

+package contoso.example;

+import com.azure.messaging.servicebus.ServiceBusClientBuilder;

+import com.azure.messaging.servicebus.ServiceBusSessionReceiverAsyncClient;

+import org.apache.flink.streaming.api.functions.source.RichParallelSourceFunction;

+import org.apache.flink.streaming.api.functions.source.SourceFunction;

+public class SessionBasedServiceBusSource extends RichParallelSourceFunction<String> {

+ private final String connectionString;

+ private final String topicName;

+ private final String subscriptionName;

+ private volatile boolean isRunning = true;

+ private ServiceBusSessionReceiverAsyncClient sessionReceiver;

+ public SessionBasedServiceBusSource(String connectionString, String topicName, String subscriptionName) {

+ this.connectionString = connectionString;

+ this.topicName = topicName;

+ this.subscriptionName = subscriptionName;

+ }

+ @Override

+ public void run(SourceFunction.SourceContext<String> ctx) throws Exception {

+ ServiceBusSessionReceiverAsyncClient sessionReceiver = new ServiceBusClientBuilder()

+ .connectionString(connectionString)

+ .sessionReceiver()

+ .topicName(topicName)

+ .subscriptionName(subscriptionName)

+ .buildAsyncClient();

+ sessionReceiver.acceptNextSession()

+ .flatMapMany(session -> session.receiveMessages())

+ .doOnNext(message -> {

+ try {

+ ctx.collect(message.getBody().toString());

+ } catch (Exception e) {

+ System.out.printf("An error occurred: %s.", e.getMessage());

+ }

+ })

+ .doOnError(error -> System.out.printf("An error occurred: %s.", error.getMessage()))

+ .blockLast();

+ }

+ @Override

+ public void cancel() {

+ isRunning = false;

+ if (sessionReceiver != null) {

+ sessionReceiver.close();

+ }

+ }

+}

+```

+### Key components and functionality of the provided code

+#### Main code: ServiceBusToAdlsgen2.java

+This Java class, `ServiceBusToAdlsgen2`, orchestrates the entire Flink job for the `DStreamAPI AzureServiceBusDemo`, shows a robust integration between Apache Flink and Azure Service Bus.

+1. **Setting up the execution environment**

+ The `StreamExecutionEnvironment.getExecutionEnvironment()` method is used to set up the execution environment for the Flink job.

+1. **Creating a source function for Azure Service Bus**

+ A `SessionBasedServiceBusSource` object is created with the connection string, topic name, and subscription name for your Azure Service Bus. This object is a source function that can be used to create a data stream.

+1. **Creating a data stream**

+ The `env.addSource(sourceFunction)` method is used to create a data stream from the source function. Each message from the Azure Service Bus topic becomes an element in this stream.

+1. **Processing the data**

+ The `stream.map(value -> processValue(value))` method is used to process each element in the stream. In this case, the `processValue` method is applied to each element. This is where youΓÇÖd put your processing logic.

+1. **Creating a sink for Azure Data Lake Storage Gen2**

+ A `FileSink object` is created with the output path and a `SimpleStringEncoder`. The `withRollingPolicy` method is used to set a rolling policy for the sink.

+1. **Adding the sink function to the processed stream**

+

+ The `processedStream.sinkTo(sink)` method is used to add the sink function to the processed stream. Each processed element is written to a file in Azure Data Lake Storage Gen2.

+1. **Executing the job**

+ Finally, the `env.execute("ServiceBusToDataLakeJob")` method is used to execute the Flink job. This starts reading messages from the Azure Service Bus topic, process them, and write them to Azure Data Lake Storage Gen2.

+#### Flink source function: SessionBasedServiceBusSource.java

+This Flink source function, encapsulated within the `SessionBasedServiceBusSource.java` class, establishing a connection with Azure Service Bus, retrieving messages, and integrating with Apache Flink for parallel data processing. The following is key aspects of this source function:

+1. **Class Definition**

+ The `SessionBasedServiceBusSource` class extends `RichParallelSourceFunction<String>`, which is a base class for implementing a parallel data source in Flink.

+1. **Instance Variables**

+ The `connectionString`, `topicName`, and `subscriptionName` variables hold the connection string, topic name, and subscription name for your Azure Service Bus. The isRunning flag is used to control the execution of the source function. The `sessionReceiver` is an instance of `erviceBusSessionReceiverAsyncClient`, which is used to receive messages from the Service Bus.

+1. **Constructor**

+ The constructor initializes the instance variables with the provided values.

+1. **run() Method**

+ This method is where the source function starts to emit data to Flink. It creates a `ServiceBusSessionReceiverAsyncClient`, accepts the next available session, and starts receiving messages from that session. Each messageΓÇÖs body is then collected into the Flink source context.

+1. **cancel() Method**

+ This method is called when the source function needs to be stopped. It sets the isRunning flag to false and closes the `sessionReceiver`.

+

+## Reference

+- To learn more about Azure Service Bus, refer to the [What is Azure Service Bus?](/azure/service-bus-messaging/service-bus-messaging-overview).

+- For guidance on creating topics, consult the [Service Bus Explorer](/azure/service-bus-messaging/explorer).

+- Apache, Apache Flink, Flink, and associated open source project names are [trademarks](../trademarks.md) of the [Apache Software Foundation](https://www.apache.org/) (ASF).

+ Title: Monitor performance metrics for the MedTech service in Azure Health Data Services

+description: Learn how to monitor the performance metrics of the MedTech service in Azure Health Data Services. Find out how to configure, display, and save the metrics in an Azure portal dashboard.

+# Monitor performance metrics for the MedTech service

+Gain insights into the health, availability, latency, traffic, and errors of your organization's MedTech services by monitoring MedTech service metrics in the Azure portal. To help you identify patterns or trends, pin tiles for the metrics to an Azure portal dashboard for easy access and visualization.

+## Configure service metrics

+1. In the Azure portal, go to the Azure Health Data Services workspace. Go to **Services** > **MedTech service**.

+ :::image type="content" source="media\configure-metrics\select-medtech-service.png" alt-text="Screenshot showing how to open the MedTech service in a workspace." lightbox="media\configure-metrics\select-medtech-service.png":::

+2. Select the MedTech service that you want to monitor metrics for. In this example, the MedTech service is named **mt-azuredocsdemo**.

+ :::image type="content" source="media\configure-metrics\select-medtech-service2.png" alt-text="Screenshot showing the MedTech service to display metrics for." lightbox="media\configure-metrics\select-medtech-service2.png":::

+3. In the left pane, select **Monitoring** > **Metrics**.

+ :::image type="content" source="media\configure-metrics\monitor-metrics.png" alt-text="Screenshot showing the selection of the Metrics menu item in the MedTech service." lightbox="media\configure-metrics\monitor-metrics.png":::

+4. Choose **Add metric**.

+5. Select a metric from the drop-down list.

+The service performance metrics you can monitor are:

+Metric category|Metric name|Metric description|

+|--|--|--|

+|Availability|IotConnector Health Status|The overall health of the MedTech service.|

+|Errors|Total Error Count|The total number of errors.|

+|Latency|Average Group Stage Latency|The average latency of the group stage. The [group stage](overview-of-device-data-processing-stages.md#groupoptional) performs buffering, aggregating, and grouping on normalized messages.|

+|Latency|Average Normalize Stage Latency|The average latency of the normalized stage. The [normalized stage](overview-of-device-data-processing-stages.md#normalize) performs normalization on raw incoming messages.|

+|Traffic|Number of FHIR resources saved|The total number of FHIR&reg; resources [updated or persisted](overview-of-device-data-processing-stages.md#persist) by the MedTech service.|

+|Traffic|Number of Incoming Messages|The number of received raw [incoming messages](overview-of-device-data-processing-stages.md#ingest) (for example, the device events) from the configured source event hub.|

+|Traffic|Number of Measurements|The number of normalized value readings received by the FHIR [transformation stage](overview-of-device-data-processing-stages.md#transform) of the MedTech service.|

+|Traffic|Number of Message Groups|The number of groups that have messages aggregated in the designated time window.|

+|Traffic|Number of Normalized Messages|The number of normalized messages.|

+The screenshot shows an example of a line chart that monitors the **Number of Incoming Messages**.

+## Save metrics as a tile on an Azure dashboard

+To keep your MedTech service metrics settings and view the metrics again later, pin them as a tile on an Azure dashboard. For steps, see [Create a dashboard in the Azure portal](../../azure-portal/azure-portal-dashboards.md).

+To learn more about advanced metrics display and sharing options, see [Analyze metrics with Azure Monitor metrics explorer](../../azure-monitor/essentials/analyze-metrics.md).

+## Next steps

+[Enable diagnostic settings for the MedTech service](how-to-enable-diagnostic-settings.md)

+ > To learn how to customize and save metrics settings to an Azure portal dashboard and tile, see [How to configure the MedTech service metrics](configure-metrics.md).

+[How to configure the MedTech service metrics](configure-metrics.md)

+> [How to configure the MedTech service metrics](configure-metrics.md)

+>For load tests with more than 45 engine instances or a greater than 3-hour test run duration, the results file is not available for download. You can [configure a JMeter Backend Listener to export the results](#export-test-results-using-jmeter-backend-listeners) to a data store of your choice or [copy the results from a storage account container](#copy-test-artifacts-from-a-storage-account-container).

+## Copy test artifacts from a storage account container

+>[!IMPORTANT]

+>Copying test artifacts from a storage account container is only enabled for load tests with more than 45 engine instances or with a test run duration greater than three hours.

+To copy the test results and log files for a test run from a storage account, in the Azure portal:

+1. In the [Azure portal](https://portal.azure.com), go to your Azure Load Testing resource.

+1. On the left pane, select **Tests** to view a list of tests, and then select your test.

+ :::image type="content" source="media/how-to-export-test-results/test-list.png" alt-text="Screenshot that shows the list of tests for an Azure Load Testing resource.":::

+1. From the list of test runs, select your test run.

+ :::image type="content" source="media/how-to-export-test-results/test-runs-list.png" alt-text="Screenshot that shows the list of test runs for a test in an Azure Load Testing resource.":::

+ >[!TIP]

+ > To limit the number of tests to display in the list, you can use the search box and the **Time range** filter.

+1. On the **Test run details** pane, select **Copy artifacts**.

+ :::image type="content" source="media/how-to-export-test-results/test-run-page-copy-artifacts.png" alt-text="Screenshot that shows how to copy the test artifacts from the 'Test run details' pane.":::

+ > [!NOTE]

+ > A load test run needs to be in the *Done*, *Stopped*, or *Failed* status for the results file to be available for download.

+1. Copy the SAS URL of the storage account container.

+ You can use the SAS URL in the [Azure Storage Explorer](/azure/vs-azure-tools-storage-manage-with-storage-explorer?tabs=windows#shared-access-signature-sas-url) or [AzCopy](/azure/storage/common/storage-use-azcopy-blobs-copy#copy-a-container) to copy the results CSV files and the log files for the test run to your storage account.

+

+ The SAS URL is valid for 60 minutes from the time it gets generated. If the URL expires, select **Copy artifacts** to generate a new SAS URL.

+This article shows how to create a managed identity for Azure Load Testing. You can use a managed identity to securely read secrets or certificates from Azure Key Vault in your load test.

+A managed identity from Microsoft Entra ID allows your load testing resource to easily access Microsoft Entra protected Azure Key Vault. The identity is managed by the Azure platform and doesn't require you to manage or rotate any secrets. For more information about managed identities in Microsoft Entra ID, see [Managed identities for Azure resources](/azure/active-directory/managed-identities-azure-resources/overview).

+| Plugins | Azure Load Testing lets you use plugins from https://jmeter-plugins.org, or upload a Java archive (JAR) file with your own plugin code.| [Customize a load test with plugins](./how-to-use-jmeter-plugins.md) |

+| Web Driver sampler | Due to the resource intensive nature of WebDriver tests, you can run tests with a load of up to four virtual users associated with the [Web Driver sampler](https://jmeter-plugins.org/wiki/WebDriverSampler/). Tests with higher load associated with the Web Driver sampler can result in errors. In such a case, reduce the load and try again.<br/>You can have a higher load associated with other samplers, like HTTP sampler, in the same test. | |

+- If the connection ID is incorrectly formatted, an error is thrown. Before you export your workflow, make sure that the connection IDs for your connectors match the following format:

+ `subscriptionId/{subscription-ID}/resourceGroups/{resource-group-name}/providers/Microsoft.Logic/integrationServiceEnvironments/{integration-service-environment-name}/managedApis/{managed-api-name}`

+In this article, you learn how to troubleshoot common workload (including training jobs and endpoints) errors on the [Kubernetes compute](./how-to-attach-kubernetes-to-workspace.md).

+ The common error types in **compute scope** that you might encounter when using Kubernetes compute to create online endpoints and online deployments for real-time model inference, which you can trouble shoot by following the guidelines:

+The error message is as:

+* Check if the instance types are information is correct. You can check the supported instance types in the [Kubernetes compute](./how-to-attach-kubernetes-to-workspace.md) documentation.

+There is a compute target validation process when deploying models to your Kubernetes cluster. This error should occur when the compute information is invalid. For example, the compute target is not found, or the configuration of Azure Machine Learning extension has been updated in your Kubernetes cluster.

+ * If the AKS cluster has enabled authorized IP ranges, make sure all the **Azure Machine Learning control plane IP ranges** have been enabled for the AKS cluster. More information you can see this [document](how-to-deploy-kubernetes-extension.md#limitations).

+This error occurs when the extension is installed but the extension identity is not correctly assigned. You can try to reinstall the extension to fix it.

+In order to allow for any known errors to be surfaced, you can use the commands to run a baseline check for your cert and key. Expect the second command to return "RSA key ok" without prompting you for password.

+Run the commands to verify whether sslCertPemFile and sslKeyPemFile are matched:

+For sslCertPemFile, it is the public certificate. It should include the certificate chain which includes the following certificates and should be in the sequence of the server certificate, the intermediate CA certificate and the root CA certificate:

+* The server certificate: the server presents to the client during the TLS handshake. It contains the serverΓÇÖs public key, domain name, and other information. The server certificate is signed by an intermediate certificate authority (CA) that vouches for the serverΓÇÖs identity.

+* The intermediate CA certificate: the intermediate CA presents to the client to prove its authority to sign the server certificate. It contains the intermediate CAΓÇÖs public key, name, and other information. The intermediate CA certificate is signed by a root CA that vouches for the intermediate CAΓÇÖs identity.

+* The root CA certificate: the root CA presents to the client to prove its authority to sign the intermediate CA certificate. It contains the root CAΓÇÖs public key, name, and other information. The root CA certificate is self-signed and trusted by the client.

+When the training job is running, you can check the job status in the workspace portal. When you encounter some abnormal job status, such as the job retried multiple times, or the job has been stuck in initializing state, or even the job has eventually failed, you can follow the guide to troubleshoot the issue.

+If the training job pod running in the cluster was terminated due to the node running to node OOM (out of memory), the job is **automatically retried** to another available node.

+* Each retry log is recorded in a new log folder with the format of "retry-<retry number\>"(such as: retry-001).

+Then you can get the retry job-node mapping information, to figure out which node the retry-job has been running on.

+The host name of the node, which the job pod is running on is indicated in this log, for example:

+Check whether you have `enableTraining=True` set when doing the Azure Machine Learning extension installation. More details could be found at [Deploy Azure Machine Learning extension on AKS or Arc Kubernetes cluster](how-to-deploy-kubernetes-extension.md)

+In the above training scenario, this **computing identity** is necessary for Kubernetes compute to be used as a credential to communicate between the ARM resource bound to the workspace and the Kubernetes computing cluster. So without this identity, the training job fails and reports missing account key or sas token. Take accessing storage account, for example, if you don't specify a managed identity to your Kubernetes compute, the job fails with the following error message:

+The cause is machine learning workspace default storage account without any credentials is not accessible for training jobs in Kubernetes compute.

+The cause is the authorization failed when the job tries to upload the project files to the AzureBlob. You can check the following items to troubleshoot the issue:

+We could use the method to check private link setup by logging into one pod in the Kubernetes cluster and then check related network settings.

+When the proxy and workspace are correctly set up with a private link, you should observe an attempt to connect to an internal IP. A response with an HTTP 401 status code is expected in this scenario if a token is not provided.

+At this time, the CLI v2 and SDK v2 do not allow updating any configuration of an existing Kubernetes compute. For example, changing the namespace does not take effect.

+|Nov 21, 2023 | 1.1.39| Fixed vulnerabilities. Refined error message. Increased stability for relayserver API. |

+|Nov 1, 2023 | 1.1.37| Update data plane envoy version. |

+Learn how to create a [time-series forecasting model](concept-automated-ml.md#time-series-forecasting) without writing a single line of code using automated machine learning in the Azure Machine Learning studio. This model predicts rental demand for a bike sharing service.

+You don't write any code in this tutorial, you use the studio interface to perform training. You learn how to do the following tasks:

+For this tutorial, you create your automated ML experiment run in Azure Machine Learning studio, a consolidated web interface that includes machine learning tools to perform data science scenarios for data science practitioners of all skill levels. The studio isn't supported on Internet Explorer browsers.

+ 1. On the **Datastore and file selection** form, select the default datastore that was automatically set up during your workspace creation, **workspaceblobstore (Azure Blob Storage)**. This is the storage location where you upload your data file.

+ 1. Select **Upload files** from the **Upload** drop-down.

+ Min / Max nodes| To profile data, you must specify one or more nodes.|Min nodes: 1<br>Max nodes: 6

+ Additional forecasting settings| These settings help improve the accuracy of your model. <br><br> _**Forecast target lags:**_ how far back you want to construct the lags of the target variable <br> _**Target rolling window**_: specifies the size of the rolling window over which features, such as the *max, min* and *sum*, is generated. | <br><br>Forecast&nbsp;target&nbsp;lags: None <br> Target&nbsp;rolling&nbsp;window&nbsp;size: None

+In this article, learn how to use a custom Docker image when you're training models with Azure Machine Learning. You use the example scripts in this article to classify pet images by creating a convolutional neural network.

+- **Enhanced replica provisioning experience**

+ Replica provisioning experience will now provide extra flexibility to modify the replica compute and storage settings during the provisioning workflow. You can choose to modify the compute settings of the replica server at the time of provisioning instead of having to make the changes post provisioning of replica server. The feature will also enable modifying the backup retention days of the replica server and configure it to have a different value than that of the source server

+- **Modify multiple server parameters using Azure CLI**

+

+ You can now conveniently update multiple server parameters for your Azure Database for MySQL - Flexible Server using Azure CLI. [Learn more](./how-to-configure-server-parameters-cli.md#modify-a-server-parameter-value)

+#CustomerIntent: As an Azure administrator, I want learn how to create a connection monitor in Network Watcher so I can monitor the communication between one VM and another.

+# Create a connection monitor using the Azure portal

+ * In the previously mentioned scenario, you can consent to an auto upgrade of a virtual machine scale set with auto enabling of the Network Watcher extension during the creation of the connection monitor for Virtual Machine Scale Sets with manual upgrading. This would eliminate having to manually upgrade the virtual machine scale set after you install the Network Watcher extension.

+ :::image type="content" source="./media/connection-monitor-2-preview/consent-vmss-auto-upgrade.png" alt-text="Screenshot that shows where to set up test groups and consent for autoupgrading of a virtual machine scale set in the connection monitor.":::

+- **Create alert**: You can select this checkbox to create a metric alert in Azure Monitor. When you select this checkbox, the other fields are enabled for editing. Additional charges for the alert will be applicable, based on the [pricing for alerts](https://azure.microsoft.com/pricing/details/monitor/).

+* [Learn how to analyze monitoring data and set alerts](./connection-monitor-overview.md#analyze-monitoring-data-and-set-alerts).

+* [Learn how to diagnose problems in your network](./connection-monitor-overview.md#diagnose-issues-in-your-network).

+This tutorial shows you how to use Azure Network Watcher [VPN troubleshoot](vpn-troubleshoot-overview.md) capability to diagnose and troubleshoot a connectivity issue between two virtual networks. The virtual networks are connected via VPN gateways using VNet-to-VNet connections.

+Diagnostic Toolkit provides access to all the diagnostic features available for troubleshooting the network. You can use this drop-down list to access features like [packet capture](../network-watcher/network-watcher-packet-capture-overview.md), [VPN troubleshooting](../network-watcher/vpn-troubleshoot-overview.md), [connection troubleshooting](../network-watcher/network-watcher-connectivity-overview.md), [next hop](../network-watcher/network-watcher-next-hop-overview.md), and [IP flow verify](../network-watcher/network-watcher-ip-flow-verify-overview.md):

+To diagnose your connection, connect to Azure PowerShell and initiate the `Start-AzNetworkWatcherResourceTroubleshooting` cmdlet. You can find the details on using this cmdlet at [Troubleshoot Virtual Network Gateway and connections - PowerShell](vpn-troubleshoot-powershell.md). This cmdlet may take up to few minutes to complete.

+ Title: NSG flow logs overview

+#CustomerIntent: As an Azure administrator, I want to learn about NSG flow logs so that I can log my network traffic to analyze and optimize the network performance.

+- **Subscription**: The storage account must be in the same subscription of the network security group or in a subscription associated with the same Microsoft Entra tenant of the network security group's subscription.

+**VPN troubleshoot** enables you to troubleshoot virtual network gateways and their connections. For more information, see [VPN troubleshoot overview](vpn-troubleshoot-overview.md) and [Diagnose a communication problem between networks](diagnose-communication-problem-between-networks.md).

+description: Learn about Azure Network Watcher VNet flow logs feature and how to use them to record your virtual networks traffic.

+- **Subscription**: The storage account must be in the same subscription of the virtual network or in a subscription associated with the same Microsoft Entra tenant of the virtual network's subscription.

+ Title: Troubleshoot VPN gateways and connections - Azure CLI

+description: Learn how to use Azure Network Watcher VPN troubleshoot capability to troubleshoot VPN virtual network gateways and their connections using the Azure CLI.

+#CustomerIntent: As a network administrator, I want to determine why resources in a virtual network can't communicate with resources in a different virtual network over a VPN connection.

+# Troubleshoot VPN virtual network gateways and connections using the Azure CLI

+> [!div class="op_single_selector"]

+> - [Portal](diagnose-communication-problem-between-networks.md)

+> - [PowerShell](vpn-troubleshoot-powershell.md)

+> - [Azure CLI](vpn-troubleshoot-cli.md)

+In this article, you learn how to use Network Watcher VPN troubleshoot capability to diagnose and troubleshoot VPN virtual network gateways and their connections to solve connectivity issues between your virtual network and on-premises network. VPN troubleshoot requests are long running requests, which could take several minutes to return a result. The logs from troubleshooting are stored in a container on a storage account that is specified.

+## Prerequisites

+- An Azure account with an active subscription. [create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- A Network Watcher enabled in the region of the virtual network gateway. For more information, see [Enable or disable Azure Network Watcher](network-watcher-create.md?tabs=cli).

+- A virtual network gateway. For more information, see [Supported gateway types](vpn-troubleshoot-overview.md#supported-gateway-types).

+- Azure Cloud Shell or Azure CLI.

+

+ The steps in this article run the Azure CLI commands interactively in [Azure Cloud Shell](/azure/cloud-shell/overview). To run the commands in the Cloud Shell, select **Open Cloudshell** at the upper-right corner of a code block. Select **Copy** to copy the code, and paste it into Cloud Shell to run it. You can also run the Cloud Shell from within the Azure portal.

+

+ You can also [install Azure CLI locally](/cli/azure/install-azure-cli) to run the commands. If you run Azure CLI locally, sign in to Azure using the [az login](/cli/azure/reference-index#az-login) command.

+## Troubleshoot using an existing storage account

+In this section, you learn how to troubleshoot a VPN virtual network gateway or a VPN connection using an existing storage account.

+# [**Gateway**](#tab/gateway)

+Use [az storage account show](/cli/azure/storage/account#az-storage-account-show) to retrieve the resource ID of the storage account. Then use [az network watcher troubleshooting start](/cli/azure/network/watcher/troubleshooting#az-network-watcher-troubleshooting-start) to start troubleshooting the VPN gateway.

+```azurecli-interactive

+# Place the storage account ID into a variable.

+storageId=$(az storage account show --name 'mystorageaccount' --resource-group 'myResourceGroup' --query 'id' --output tsv)

+# Start VPN troubleshoot session.

+az network watcher troubleshooting start --resource-group 'myResourceGroup' --resource 'myGateway' --resource-type 'vnetGateway' --storage-account $storageId --storage-path 'https://mystorageaccount.blob.core.windows.net/{containerName}'

+```

+# [**Connection**](#tab/connection)

+Use [az storage account show](/cli/azure/storage/account#az-storage-account-show) to retrieve the resource ID of the storage account. Then use [az network watcher troubleshooting start](/cli/azure/network/watcher/troubleshooting#az-network-watcher-troubleshooting-start) to start troubleshooting the VPN connection.

+```azurecli-interactive

+# Place the storage account ID into a variable.

+storageId=$(az storage account show --name 'mystorageaccount' --resource-group 'myResourceGroup' --query 'id' --output tsv)

+# Start VPN troubleshoot session.

+az network watcher troubleshooting start --resource-group 'myResourceGroup' --resource 'myConnection' --resource-type 'vpnConnection' --storage-account $storageId --storage-path 'https://mystorageaccount.blob.core.windows.net/{containerName}'

+```

+After the troubleshooting request is completed, ***Healthy*** or ***UnHealthy*** is returned with action text that provides general guidance on how to resolve the issue. If an action can be taken for the issue, a link is provided with more guidance.

+Additionally, detailed logs are stored in the storage account container you specified in the previous command. For more information, see [Log files](vpn-troubleshoot-overview.md#log-files). You can use Storage explorer or any other way you prefer to access and download the logs. For more information, see [Get started with Storage Explorer](../vs-azure-tools-storage-manage-with-storage-explorer.md).

+## Troubleshoot using a new storage account

+In this section, you learn how to troubleshoot a VPN virtual network gateway or a VPN connection using a new storage account.

+# [**Gateway**](#tab/gateway)

+Use [az storage account create](/cli/azure/storage/account#az-storage-account-create) and [az storage container create](/cli/azure/storage/container#az-storage-container-create) to create a new storage account and a container respectively. Then, use [az network watcher troubleshooting start](/cli/azure/network/watcher/troubleshooting#az-network-watcher-troubleshooting-start) to start troubleshooting the VPN gateway.

+```azurecli-interactive

+# Create a new storage account.

+az storage account create --name 'mystorageaccount' --resource-group 'myResourceGroup' --location 'eastus' --sku 'Standard_LRS'

+# Get the storage account keys.

+az storage account keys list --resource-group 'myResourceGroup' --account-name 'mystorageaccount'

+# Create a container.

+az storage container create --account-name 'mystorageaccount' --account-key {storageAccountKey} --name 'vpn'

+# Start VPN troubleshoot session.

+az network watcher troubleshooting start --resource-group 'myResourceGroup' --resource 'myGateway' --resource-type 'vnetGateway' --storage-account 'mystorageaccount' --storage-path 'https://mystorageaccount.blob.core.windows.net/vpn'

+```

+# [**Connection**](#tab/connection)

+Use [az storage account create](/cli/azure/storage/account#az-storage-account-create) and [az storage container create](/cli/azure/storage/container#az-storage-container-create) to create a new storage account and a container respectively. Then, use [az network watcher troubleshooting start](/cli/azure/network/watcher/troubleshooting#az-network-watcher-troubleshooting-start) to start troubleshooting the VPN connection.

+```azurecli-interactive

+# Create a new storage account.

+az storage account create --name 'mystorageaccount' --resource-group 'myResourceGroup' --location 'eastus' --sku 'Standard_LRS'

+# Get the storage account keys.

+az storage account keys list --resource-group 'myResourceGroup' --account-name 'mystorageaccount'

+# Create a container.

+az storage container create --account-name 'mystorageaccount' --account-key {storageAccountKey} --name 'vpn'

+# Start VPN troubleshoot session.

+az network watcher troubleshooting start --resource-group 'myResourceGroup' --resource 'myConnection' --resource-type 'vpnConnection' --storage-account 'mystorageaccount' --storage-path 'https://mystorageaccount.blob.core.windows.net/vpn'

+```

+After the troubleshooting request is completed, ***Healthy*** or ***UnHealthy*** is returned with action text that provides general guidance on how to resolve the issue. If an action can be taken for the issue, a link is provided with more guidance.

+Additionally, detailed logs are stored in the storage account container you specified in the previous command. For more information, see [Log files](vpn-troubleshoot-overview.md#log-files). You can use Storage explorer or any other way you prefer to access and download the logs. For more information, see [Get started with Storage Explorer](../vs-azure-tools-storage-manage-with-storage-explorer.md).

+## Related content

+- [Tutorial: Diagnose a communication problem between virtual networks using the Azure portal](diagnose-communication-problem-between-networks.md).

+- [VPN troubleshoot overview](vpn-troubleshoot-overview.md).

+ Title: VPN troubleshoot overview

+description: Learn about Azure Network Watcher VPN troubleshoot capability and how to use it to troubleshoot VPN virtual network gateways and their connections.

+#CustomerIntent: As an Azure administrator, I want to learn about VPN troubleshoot so I can use it to troubleshoot my VPN virtual network gateways and their connections whenever resources in a virtual network can't communicate with on-premises machines over a VPN connection.

+# VPN troubleshoot overview

+Virtual network gateways provide connectivity between on-premises resources and Azure Virtual Networks. Monitoring virtual network gateways and their connections are critical to ensure communication isn't broken. Azure Network Watcher provides the capability to troubleshoot virtual network gateways and their connections. The capability can be called through the Azure portal, Azure PowerShell, Azure CLI, or REST API. When called, Network Watcher diagnoses the health of the gateway, or connection, and returns the appropriate results. The request is a long running transaction. The results are returned once the diagnosis is complete.

+## Supported Gateway types

+The following table lists which gateways and connections are supported with Network Watcher troubleshooting:

+| Gateway or connection | Supported |

+|||

+|**Gateway types** | |

+|VPN | Supported |

+|ExpressRoute | Not Supported |

+|**VPN types** | |

+|Route Based | Supported|

+|Policy Based | Not Supported|

+|**Connection types**||

+|IPSec| Supported|

+|VNet2VNet| Supported|

+|ExpressRoute| Not Supported|

+|VPNClient| Not Supported|

+## Results

+The preliminary results returned give an overall picture of the health of the resource. Deeper information can be provided for resources as shown in the following section:

+The following list is the values returned with the troubleshoot API:

+* **startTime** - This value is the time the troubleshoot API call started.

+* **endTime** - This value is the time when the troubleshooting ended.

+* **code** - This value is UnHealthy, if there's a single diagnosis failure.

+* **results** - Results is a collection of results returned on the Connection or the virtual network gateway.

+ * **id** - This value is the fault type.

+ * **summary** - This value is a summary of the fault.

+ * **detailed** - This value provides a detailed description of the fault.

+ * **recommendedActions** - This property is a collection of recommended actions to take.

+ * **actionText** - This value contains the text describing what action to take.

+ * **actionUri** - This value provides the URI to documentation on how to act.

+ * **actionUriText** - This value is a short description of the action text.

+The following tables show the different fault types (**id** under results from the preceding list) that are available and if the fault creates logs.

+### Gateway

+| Fault Type | Reason | Log|

+||||

+| NoFault | When no error is detected |Yes|

+| GatewayNotFound | Can't find gateway or gateway isn't provisioned |No|

+| PlannedMaintenance | Gateway instance is under maintenance |No|

+| UserDrivenUpdate | This fault occurs when a user update is in progress. The update could be a resize operation. | No |

+| VipUnResponsive | This fault occurs when the primary instance of the gateway can't be reached due to a health probe failure. | No |

+| PlatformInActive | There's an issue with the platform. | No|

+| ServiceNotRunning | The underlying service isn't running. | No|

+| NoConnectionsFoundForGateway | No connections exist on the gateway. This fault is only a warning.| No|

+| ConnectionsNotConnected | Connections aren't connected. This fault is only a warning.| Yes|

+| GatewayCPUUsageExceeded | The current gateway CPU usage is > 95%. | Yes |

+### Connection

+| Fault Type | Reason | Log|

+||||

+| NoFault | When no error is detected |Yes|

+| GatewayNotFound | Can't find gateway or gateway isn't provisioned |No|

+| PlannedMaintenance | Gateway instance is under maintenance |No|

+| UserDrivenUpdate | This fault occurs when a user update is in progress. The update could be a resize operation. | No |

+| VipUnResponsive | This fault occurs when the primary instance of the gateway can't be reached due to a health probe failure. | No |

+| ConnectionEntityNotFound | Connection configuration is missing | No |

+| ConnectionIsMarkedDisconnected | The connection is marked "disconnected" |No|

+| ConnectionNotConfiguredOnGateway | The underlying service doesn't have the connection configured. | Yes |

+| ConnectionMarkedStandby | The underlying service is marked as standby.| Yes|

+| Authentication | Preshared key mismatch | Yes|

+| PeerReachability | The peer gateway isn't reachable. | Yes|

+| IkePolicyMismatch | The peer gateway has IKE policies that aren't supported by Azure. | Yes|

+| WfpParse Error | An error occurred parsing the WFP log. |Yes|

+## Log files

+The resource troubleshooting log files are stored in a storage account after resource troubleshooting is finished. The following image shows the example contents of a call that resulted in an error.

+> [!NOTE]

+> 1. In some cases, only a subset of the logs files is written to storage.

+> 2. For newer gateway versions, the IkeErrors.txt, Scrubbed-wfpdiag.txt and wfpdiag.txt.sum have been replaced by an IkeLogs.txt file that contains the whole IKE activity (not just errors).

+For instructions on downloading files from Azure storage accounts, see [Download a block blob](../storage/blobs/storage-quickstart-blobs-portal.md#download-a-block-blob). Another tool that can be used is Storage Explorer. For information about Azure Storage Explorer, see [Use Azure Storage Explorer to download blobs](../storage/blobs/quickstart-storage-explorer.md#download-blobs)

+### ConnectionStats.txt

+The **ConnectionStats.txt** file contains overall stats of the Connection, including ingress and egress bytes, Connection status, and the time the Connection was established.

+> [!NOTE]

+> If the call to the troubleshooting API returns healthy, the only thing returned in the zip file is a **ConnectionStats.txt** file.

+The contents of this file are similar to the following example:

+```

+Connectivity State : Connected

+Remote Tunnel Endpoint :

+Ingress Bytes (since last connected) : 288 B

+Egress Bytes (Since last connected) : 288 B

+Connected Since : 2/1/2017 8:22:06 PM

+```

+### CPUStats.txt

+The **CPUStats.txt** file contains CPU usage and memory available at the time of testing. The contents of this file is similar to the following example:

+```

+Current CPU Usage : 0 % Current Memory Available : 641 MBs

+```

+### IKElogs.txt

+The **IKElogs.txt** file contains any IKE activity that was found during monitoring.

+The following example shows the contents of an IKElogs.txt file.

+```

+Remote <IPaddress>:500: Local <IPaddress>:500: [RECEIVED][SA_AUTH] Received IKE AUTH message

+Remote <IPaddress>:500: Local <IPaddress>:500: Received Traffic Selector payload request- [Tsid 0x729 ]Number of TSIs 2: StartAddress 10.20.0.0 EndAddress 10.20.255.255 PortStart 0 PortEnd 65535 Protocol 0, StartAddress 192.168.100.0 EndAddress 192.168.100.255 PortStart 0 PortEnd 65535 Protocol 0 Number of TSRs 1:StartAddress 0.0.0.0 EndAddress 255.255.255.255 PortStart 0 PortEnd 65535 Protocol 0

+Remote <IPaddress>:500: Local <IPaddress>:500: [SEND] Proposed Traffic Selector payload will be (Final Negotiated) - [Tsid 0x729 ]Number of TSIs 2: StartAddress 10.20.0.0 EndAddress 10.20.255.255 PortStart 0 PortEnd 65535 Protocol 0, StartAddress 192.168.100.0 EndAddress 192.168.100.255 PortStart 0 PortEnd 65535 Protocol 0 Number of TSRs 1:StartAddress 0.0.0.0 EndAddress 255.255.255.255 PortStart 0 PortEnd 65535 Protocol 0

+Remote <IPaddress>:500: Local <IPaddress>:500: [RECEIVED]Received IPSec payload: Policy1:Cipher=DESIntegrity=Md5

+IkeCleanupQMNegotiation called with error 13868 and flags a

+Remote <IPaddress>:500: Local <IPaddress>:500: [SEND][NOTIFY] Sending Notify Message - Policy Mismatch

+```

+### IKEErrors.txt

+The **IKEErrors.txt** file contains any IKE errors that were found during monitoring.

+The following example shows the contents of an IKEErrors.txt file. Your errors might be different depending on the issue.

+```

+Error: Authentication failed. Check shared key. Check crypto. Check lifetimes.

+ based on log : Peer failed with Windows error 13801(ERROR_IPSEC_IKE_AUTH_FAIL)

+Error: On-prem device sent invalid payload.

+ based on log : IkeFindPayloadInPacket failed with Windows error 13843(ERROR_IPSEC_IKE_INVALID_PAYLOAD)

+```

+### Scrubbed-wfpdiag.txt

+The **Scrubbed-wfpdiag.txt** log file contains the wfp log. This log contains logging of packet drop and IKE/AuthIP failures.

+The following example shows the contents of the Scrubbed-wfpdiag.txt file. In this example, the pre-shared key of a Connection wasn't correct as can be seen from the third line from the bottom. The following example is just a snippet of the entire log, as the log can be lengthy depending on the issue.

+```

+...

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|Deleted ICookie from the high priority thread pool list

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|IKE diagnostic event:

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|Event Header:

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Timestamp: 1601-01-01T00:00:00.000Z

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Flags: 0x00000106

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Local address field set

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Remote address field set

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| IP version field set

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| IP version: IPv4

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| IP protocol: 0

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Local address: 13.78.238.92

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Remote address: 52.161.24.36

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Local Port: 0

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Remote Port: 0

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Application ID:

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| User SID: <invalid>

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|Failure type: IKE/Authip Main Mode Failure

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|Type specific info:

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Failure error code:0x000035e9

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| IKE authentication credentials are unacceptable

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36|

+[0]0368.03A4::02/02/2017-17:36:01.496 [ikeext] 3038|52.161.24.36| Failure point: Remote

+...

+```

+### wfpdiag.txt.sum

+The **wfpdiag.txt.sum** file is a log showing the buffers and events processed.

+The following example is the contents of a wfpdiag.txt.sum file.

+```

+Files Processed:

+ C:\Resources\directory\924336c47dd045d5a246c349b8ae57f2.GatewayTenantWorker.DiagnosticsStorage\2017-02-02T17-34-23\wfpdiag.etl

+Total Buffers Processed 8

+Total Events Processed 2169

+Total Events Lost 0

+Total Format Errors 0

+Total Formats Unknown 486

+Elapsed Time 330 sec

+|EventCount EventName EventType TMF |

+| 36 ikeext ike_addr_utils_c844 a0c064ca-d954-350a-8b2f-1a7464eef8b6|

+| 12 ikeext ike_addr_utils_c857 a0c064ca-d954-350a-8b2f-1a7464eef8b6|

+| 96 ikeext ike_addr_utils_c832 a0c064ca-d954-350a-8b2f-1a7464eef8b6|

+| 6 ikeext ike_bfe_callbacks_c133 1dc2d67f-8381-6303-e314-6c1452eeb529|

+| 6 ikeext ike_bfe_callbacks_c61 1dc2d67f-8381-6303-e314-6c1452eeb529|

+| 12 ikeext ike_sa_management_c5698 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+| 6 ikeext ike_sa_management_c8447 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+| 12 ikeext ike_sa_management_c494 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+| 12 ikeext ike_sa_management_c642 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+| 6 ikeext ike_sa_management_c3162 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+| 12 ikeext ike_sa_management_c3307 7857a320-42ee-6e90-d5d9-3f414e3ea2d3|

+```

+## Considerations

+- Only one VPN troubleshoot operation can be run at a time per subscription. To run another VPN troubleshoot operation, wait for the previous one to complete. Triggering a new operation while a previous one hasn't completed causes the subsequent operations to fail.

+- CLI Bug: If you're using Azure CLI to run the command, the VPN Gateway and the Storage account need to be in same resource group. Customers with the resources in different resource groups can use PowerShell or the Azure portal instead.

+## Next step

+To learn how to diagnose a problem with a virtual network gateway or gateway connection, see [Diagnose communication problems between virtual networks](diagnose-communication-problem-between-networks.md).

+ Title: Troubleshoot VPN gateways and connections - PowerShell

+description: Learn how to use Azure Network Watcher VPN troubleshoot capability to troubleshoot VPN virtual network gateways and their connections using PowerShell.

+#CustomerIntent: As a network administrator, I want to determine why resources in a virtual network can't communicate with resources in a different virtual network over a VPN connection.

+# Troubleshoot VPN virtual network gateways and connections using PowerShell

+> [!div class="op_single_selector"]

+> - [Portal](diagnose-communication-problem-between-networks.md)

+> - [PowerShell](vpn-troubleshoot-powershell.md)

+> - [Azure CLI](vpn-troubleshoot-cli.md)

+In this article, you learn how to use Network Watcher VPN troubleshoot capability to diagnose and troubleshoot VPN virtual network gateways and their connections to solve connectivity issues between your virtual network and on-premises network. VPN troubleshoot requests are long running requests, which could take several minutes to return a result. The logs from troubleshooting are stored in a container on a storage account that is specified.

+## Prerequisites

+- An Azure account with an active subscription. [create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- A Network Watcher enabled in the region of the virtual network gateway. For more information, see [Enable or disable Azure Network Watcher](network-watcher-create.md?tabs=powershell).

+- A virtual network gateway. For more information about supported gateway types, see [Supported gateway types](vpn-troubleshoot-overview.md#supported-gateway-types).

+- Azure Cloud Shell or Azure PowerShell.

+ The steps in this article run the Azure PowerShell cmdlets interactively in [Azure Cloud Shell](/azure/cloud-shell/overview). To run the commands in the Cloud Shell, select **Open Cloudshell** at the upper-right corner of a code block. Select **Copy** to copy the code and then paste it into Cloud Shell to run it. You can also run the Cloud Shell from within the Azure portal.

+ You can also [install Azure PowerShell locally](/powershell/azure/install-azure-powershell) to run the cmdlets. If you run PowerShell locally, sign in to Azure using the [Connect-AzAccount](/powershell/module/az.accounts/connect-azaccount) cmdlet.

+## Troubleshoot using an existing storage account

+In this section, you learn how to troubleshoot a VPN virtual network gateway or a VPN connection using an existing storage account.

+# [**Gateway**](#tab/gateway)

+Use [Start-AzNetworkWatcherResourceTroubleshooting](/powershell/module/az.network/start-aznetworkwatcherresourcetroubleshooting) to start troubleshooting the VPN gateway.

+```azurepowershell-interactive

+# Place the virtual network gateway configuration into a variable.

+$vng = Get-AzVirtualNetworkGateway -Name 'myGateway' -ResourceGroupName 'myResourceGroup'

+# Place the storage account configuration into a variable.

+$sa = Get-AzStorageAccount -ResourceGroupName 'myResourceGroup' -Name 'mystorageaccount'

+# Start VPN troubleshoot session.

+Start-AzNetworkWatcherResourceTroubleshooting -Location 'eastus' -TargetResourceId $vng.Id -StorageId $sa.Id -StoragePath 'https://mystorageaccount.blob.core.windows.net/{containerName}'

+```

+# [**Connection**](#tab/connection)

+Use [Start-AzNetworkWatcherResourceTroubleshooting](/powershell/module/az.network/start-aznetworkwatcherresourcetroubleshooting) to start troubleshooting the VPN connection.

+```azurepowershell-interactive

+# Place the virtual network gateway configuration into a variable.

+$connection = Get-AzVirtualNetworkGatewayConnection -Name 'myConnection' -ResourceGroupName 'myResourceGroup'

+# Place the storage account configuration into a variable.

+$sa = Get-AzStorageAccount -ResourceGroupName 'myResourceGroup' -Name 'mystorageaccount'

+# Start VPN troubleshoot session.

+Start-AzNetworkWatcherResourceTroubleshooting -Location 'eastus' -TargetResourceId $connection.Id -StorageId $sa.Id -StoragePath 'https://mystorageaccount.blob.core.windows.net/{containerName}'

+```

+After the troubleshooting request is completed, ***healthy*** or ***unhealthy*** is returned. Detailed logs are stored in the storage account container you specified in the previous command. For more information, see [Log files](vpn-troubleshoot-overview.md#log-files). You can use Storage explorer or any other way you prefer to access and download the logs. For more information, see [Get started with Storage Explorer](../vs-azure-tools-storage-manage-with-storage-explorer.md).

+## Troubleshoot using a new storage account

+In this section, you learn how to troubleshoot a VPN virtual network gateway or a VPN connection using a new storage account.

+# [**Gateway**](#tab/gateway)

+Use [New-AzStorageAccount](/powershell/module/az.storage/new-azstorageaccount) and [New-AzStorageContainer](/powershell/module/az.storage/new-azstoragecontainer) to create a new storage account and a container. Then, use [Start-AzNetworkWatcherResourceTroubleshooting](/powershell/module/az.network/start-aznetworkwatcherresourcetroubleshooting) to start troubleshooting the VPN gateway.

+```azurepowershell-interactive

+# Place the virtual network gateway configuration into a variable.

+$vng = Get-AzVirtualNetworkGateway -Name 'myGateway' -ResourceGroupName 'myResourceGroup'

+# Create a new storage account.

+$sa = New-AzStorageAccount -Name 'mystorageaccount' -SKU 'Standard_LRS' -ResourceGroupName 'myResourceGroup' -Location 'eastus'

+# Create a container.

+Set-AzCurrentStorageAccount -ResourceGroupName $sa.ResourceGroupName -Name $sa.StorageAccountName

+$sc = New-AzStorageContainer -Name 'vpn'

+# Start VPN troubleshoot session.

+Start-AzNetworkWatcherResourceTroubleshooting -Location 'eastus' -TargetResourceId $vng.Id -StorageId $sa.Id -StoragePath 'https://mystorageaccount.blob.core.windows.net/vpn'

+```

+# [**Connection**](#tab/connection)

+Use [New-AzStorageAccount](/powershell/module/az.storage/new-azstorageaccount) and [New-AzStorageContainer](/powershell/module/az.storage/new-azstoragecontainer) to create a new storage account and a container. Then, use [Start-AzNetworkWatcherResourceTroubleshooting](/powershell/module/az.network/start-aznetworkwatcherresourcetroubleshooting) to start troubleshooting the VPN gateway.

+```azurepowershell-interactive

+# Place the virtual network gateway configuration into a variable.

+$connection = Get-AzVirtualNetworkGatewayConnection -Name 'myConnection' -ResourceGroupName 'myResourceGroup'

+# Create a new storage account.

+$sa = New-AzStorageAccount -Name 'mystorageaccount' -SKU 'Standard_LRS' -ResourceGroupName 'myResourceGroup' -Location 'eastus'

+# Create a container.

+Set-AzCurrentStorageAccount -ResourceGroupName $sa.ResourceGroupName -Name $sa.StorageAccountName

+$sc = New-AzStorageContainer -Name 'vpn'

+# Start VPN troubleshoot session.

+Start-AzNetworkWatcherResourceTroubleshooting -Location 'eastus' -TargetResourceId $connection.Id -StorageId $sa.Id -StoragePath 'https://mystorageaccount.blob.core.windows.net/vpn'

+```

+After the troubleshooting request is completed, ***healthy*** or ***unhealthy*** is returned. Detailed logs are stored in the storage account container you specified in the previous command. For more information, see [Log files](vpn-troubleshoot-overview.md#log-files). You can use Storage explorer or any other way you prefer to access and download the logs. For more information, see [Get started with Storage Explorer](../vs-azure-tools-storage-manage-with-storage-explorer.md).

+## Related content

+- [Tutorial: Diagnose a communication problem between virtual networks using the Azure portal](diagnose-communication-problem-between-networks.md).

+- [VPN troubleshoot overview](vpn-troubleshoot-overview.md).

+When you're creating a Nexus Kubernetes cluster, you can schedule the cluster onto specific racks or distribute it across multiple racks. This technique can improve resource utilization and fault tolerance.

+### Unable to install Astro using a personal email

+Installing Apache Airflow on Astro from the Azure Marketplace using a personal email from a generic domain isn't supported. To install this service, use an email address with a unique domain, such as an email address associated with work or school, or start by creating a new user in Azure and make this user a subscription owner. For more information, see [Install Astro from the Azure Marketplace using a personal email](https://docs.astronomer.io/astro/install-azure#install-astro-from-the-azure-marketplace-using-a-personal-email).

+## Diagnostic settings are active even after disabling the Datadog resource or applying necessary tag rules

+If logs are being emitted and diagnostic settings remain active on monitored resources even after the Datadog resource is disabled or tag rules have been modified to exclude certain resources, it's likely that there's a delete lock applied to the resource(s) or the resource group containing the resource. This lock prevents the cleanup of the diagnostic settings, and hence, logs continue to be forwarded for those resources. To resolve this, remove the delete lock from the resource or the resource group. If the lock is removed after the Datadog resource is deleted, the diagnostic settings have to be cleaned up manually to stop log forwarding.

+### Diagnostic settings are active even after disabling the Dynatrace resource or applying necessary tag rules

+If logs are being emitted and diagnostic settings remain active on monitored resources even after the Dynatrace resource is disabled or tag rules have been modified to exclude certain resources, it's likely that there's a delete lock applied to the resource(s) or the resource group containing the resource. This lock prevents the cleanup of the diagnostic settings, and hence, logs continue to be forwarded for those resources. To resolve this, remove the delete lock from the resource or the resource group. If the lock is removed after the Dynatrace resource is deleted, the diagnostic settings have to be cleaned up manually to stop log forwarding.

+## Diagnostic settings are active even after disabling the Elastic resource or applying necessary tag rules

+If logs are being emitted and diagnostic settings remain active on monitored resources even after the Elastic resource is disabled or tag rules have been modified to exclude certain resources, it's likely that there's a delete lock applied to the resource(s) or the resource group containing the resource. This lock prevents the cleanup of the diagnostic settings, and hence, logs continue to be forwarded for those resources. To resolve this, remove the delete lock from the resource or the resource group. If the lock is removed after the Elastic resource is deleted, the diagnostic settings have to be cleaned up manually to stop log forwarding.

+### Diagnostic settings are active even after disabling the New Relic resource or applying necessary tag rules

+If logs are being emitted and diagnostic settings remain active on monitored resources even after the New Relic resource is disabled or tag rules have been modified to exclude certain resources, it's likely that there's a delete lock applied to the resource(s) or the resource group containing the resource. This lock prevents the cleanup of the diagnostic settings, and hence, logs continue to be forwarded for those resources. To resolve this, remove the delete lock from the resource or the resource group. If the lock is removed after the New Relic resource is deleted, the diagnostic settings have to be cleaned up manually to stop log forwarding.

+## Release: December 2023

+* Public preview of [Server logs](./how-to-server-logs-portal.md).

+* General availability of [TLS Version 1.3 support](./concepts-networking-ssl-tls.md#tls-versions).

+This article applies to the AP5GC 2310 release (2310.0-8). This release is compatible with the Azure Stack Edge Pro 1 GPU and Azure Stack Edge Pro 2 running the ASE 2309 release and supports the 2023-09-01, 2023-06-01 and 2022-11-01 [Microsoft.MobileNetwork](/rest/api/mobilenetwork) API versions.

+ |No. |Feature | Issue | SKU Fixed In |

+ |--|--|--|--|-|

+ | 1 | Packet Forwarding | In scenarios of sustained high load (for example, continuous setup of 100s of TCP flows per second) in 4G setups, AP5GC might encounter an internal error, leading to a short period of service disruption resulting in some call failures. | 2310.0-4 |

+ | 2 | Packet Forwarding | An intermittent fault at the network layer causes an outage of packet forwarding | 2310.0-8 |

+ | 3 | Diagnosability | During packet capture, uplink userplane packets can be omitted from packet captures | 2310.0-8 |

+ | 4 | Packet Forwarding | Errors in userplane packet detection rules can cause incorrect packet handling | 2310.0-8 |

+ | 5 | Diagnosability | Procedures from different subscribers appear in the same trace | 2310.0-8 |

+|Americas | Brazil South* | |

+(*) These regions are restricted in supporting customer scenarios for disaster recovery. For more information please contact your Microsoft sales or customer representatives.

+- If you are assigning a role with permission to create role assignments, consider adding a condition to constrain the role assignment. For more information, see [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md).

+> Delegating Azure role assignment management with conditions is currently in PREVIEW.

+- [Examples to delegate Azure role assignment management with conditions (preview)](delegate-role-assignments-examples.md)

+- [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md)

+ Title: Examples to delegate Azure role assignment management with conditions (preview) - Azure ABAC

+description: Examples to delegate Azure role assignment management to other users by using Azure attribute-based access control (Azure ABAC).

+# Examples to delegate Azure role assignment management with conditions (preview)

+> Delegating Azure role assignment management with conditions is currently in PREVIEW.

+This article lists examples of how to delegate Azure role assignment management to other users with conditions.

+description: Overview of how to delegate Azure role assignment management to other users by using Azure attribute-based access control (Azure ABAC).

+#Customer intent: As a dev, devops, or it admin, I want to delegate Azure role assignment management to other users who are closer to the decision, but want to limit the scope of the role assignments.

+Assigning Azure roles to grant access to Azure resources is a common task. As an administrator, you might get several requests to grant access that you want to delegate to someone else. However, you want to make sure the delegate has just the permissions they need to do their job. This article describes a more secure way to delegate role assignment management to other users in your organization.

+## Why delegate role assignment management?

+Here are some reasons why you might want to delegate role assignment management to others:

+## How you currently can delegate role assignment management

+The [Owner](built-in-roles.md#owner) and [User Access Administrator](built-in-roles.md#user-access-administrator) roles are built-in roles that allow users to create role assignments. Members of these roles can decide who can have write, read, and delete permissions for any resource in a subscription. To delegate role assignment management to another user, you can assign the Owner or User Access Administrator role to a user.

+Here are the primary issues with the current method of delegating role assignment management to others in your organization.

+## A more secure method: Delegate role assignment management with conditions (preview)

+> Delegating Azure role assignment management with conditions is currently in PREVIEW.

+Delegating role assignment management with conditions is a way to restrict the role assignments a user can create. In the preceding example, Alice can allow Dara to create some role assignments on her behalf, but not all role assignments. For example, Alice can constrain the roles that Dara can assign and constrain the principals that Dara can assign roles to. This delegation with conditions is sometimes referred to as *constrained delegation* and is implemented using [Azure attribute-based access control (Azure ABAC) conditions](conditions-overview.md).

+This video provides an overview of delegating role assignment management with conditions.

+## Why delegate role assignment management with conditions?

+Here are some reasons why delegating role assignment management to others with conditions is more secure:

+The [Role Based Access Control Administrator (Preview)](built-in-roles.md#role-based-access-control-administrator-preview) role is a built-in role that has been designed for delegating role assignment management to others. It has fewer permissions than [User Access Administrator](built-in-roles.md#user-access-administrator), which follows least privilege best practices. The Role Based Access Control Administrator role has following permissions:

+## How to delegate role assignment management with conditions

+To delegate role assignment management with conditions, you assign roles as you currently do, but you also add a [condition to the role assignment](delegate-role-assignments-portal.md).

+ Select the user that you want to delegate role assignment management to.

+ For more information, see [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md).

+ For examples, see [Examples to delegate Azure role assignment management with conditions (preview)](delegate-role-assignments-examples.md).

+ :::image type="content" source="./media/shared/delegate-role-assignments-expression.png" alt-text="Screenshot of condition editor in Azure portal showing a role assignment condition to delegate role assignment management." lightbox="./media/shared/delegate-role-assignments-expression.png":::

+Here are the known issues related to delegating role assignment management with conditions (preview):

+- You can't delegate role assignment management with conditions using [Privileged Identity Management](../active-directory/privileged-identity-management/pim-resource-roles-assign-roles.md).

+- [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md)

+- [Examples to delegate Azure role assignment management with conditions (preview)](delegate-role-assignments-examples.md)

+ Title: Delegate Azure role assignment management to others with conditions (preview) - Azure ABAC

+description: How to delegate Azure role assignment management to other users by using Azure attribute-based access control (Azure ABAC).

+#Customer intent: As a dev, devops, or it admin, I want to delegate Azure role assignment management to other users who are closer to the decision, but want to limit the scope of the role assignments.

+# Delegate Azure role assignment management to others with conditions (preview)

+> Delegating Azure role assignment management with conditions is currently in PREVIEW.

+As an administrator, you might get several requests to grant access to Azure resources that you want to delegate to someone else. You could assign a user the [Owner](built-in-roles.md#owner) or [User Access Administrator](built-in-roles.md#user-access-administrator) roles, but these are highly privileged roles. This article describes a more secure way to [delegate role assignment management](delegate-role-assignments-overview.md) to other users in your organization, but add restrictions for those role assignments. For example, you can constrain the roles that can be assigned or constrain the principals the roles can be assigned to.

+Once you know the permissions that delegate needs, you use the following steps to add a condition to the delegate's role assignment. For example conditions, see [Examples to delegate Azure role assignment management with conditions (preview)](delegate-role-assignments-examples.md).

+ :::image type="content" source="./media/delegate-role-assignments-portal/delegate-role-assignments-actions-select.png" alt-text="Screenshot of Select an action pane to delegate role assignment management with conditions." lightbox="./media/delegate-role-assignments-portal/delegate-role-assignments-actions-select.png":::

+ :::image type="content" source="./media/shared/delegate-role-assignments-expression.png" alt-text="Screenshot of Build expression section to delegate role assignment management with conditions." lightbox="./media/shared/delegate-role-assignments-expression.png":::

+ > When you add multiple expressions to delegate role assignment management with conditions, you typically use the **And** operator between expressions instead of the default **Or** operator.

+ On the **Manage privileged role assignments** page, you can add a condition to constrain the privileged role assignment or remove the role assignment. For more information, see [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md).

+> Delegating Azure role assignment management with conditions is currently in PREVIEW.

+1. Follow the steps in [Delegate Azure role assignment management to others with conditions (preview)](delegate-role-assignments-portal.md#step-3-add-a-condition).

+In this quickstart, you learn how a skillset in Azure AI Search adds Optical Character Recognition (OCR), image analysis, language detection, text translation, and entity recognition to generate text-searchable content in a search index.

+You can run the **Import data** wizard in the Azure portal to apply skills that create and transform textual content during indexing. Input is your raw data, usually blobs in Azure Storage. Output is a searchable index containing AI-generated image text, captions, and entities. Generated content is queryable in the portal using [**Search explorer**](search-explorer.md).

+1. In Azure portal, open your Azure Storage page and create a container. You can use the default access level.

+1. In Container, select **Upload** to upload the sample files. Notice that you have a wide range of content types, including images and application files that aren't full text searchable in their native formats.

+1. [Find your search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Storage%2storageAccounts/) and on the Overview page, select **Import data** on the command bar to create searchable content in four steps.

+An index contains your searchable content and the **Import data** wizard can usually create the schema by sampling the data source. In this step, review the generated schema and potentially revise any settings.

+Marking a field as **Retrievable** doesn't mean that the field *must* be present in the search results. You can control search results composition by using the **select** query parameter to specify which fields to include.

+1. In the **Indexer** page, accept the default name and select **Once**.

+Select **Indexers** from the left navigation pane to monitor status, and then select the indexer. Skills-based indexing takes longer than text-based indexing, especially OCR and image analysis.

+To view details about execution status, select **Success** (or **Failed**) to view execution details.

+In this demo, there are a few warnings: `"Could not execute skill because one or more skill input was invalid."` It tells you that a PNG file in the data source doesn't provide a text input to Entity Recognition. This warning occurs because the upstream OCR skill didn't recognize any text in the image, and thus couldn't provide a text input to the downstream Entity Recognition skill.

+After an index is created, use **Search explorer** to return results.

+1. On the left, select **Indexes** and then select the index. **Search explorer** is on the first tab.

+1. Enter a search string to query the index, such as `satya nadella`. The search bar accepts keywords, quote-enclosed phrases, and operators (`"Satya Nadella" +"Bill Gates" +"Steve Ballmer"`).

+Here's some JSON you can paste into the view:

+ ```json

+ {

+ "search": "\"Satya Nadella\" +\"Bill Gates\" +\"Steve Ballmer\"",

+ "count": true,

+ "select": "content, people"

+ }

+ ```

+> [!TIP]

+> Query strings are case-sensitive so if you get an "unknown field" message, check **Fields** or **Index Definition (JSON)** to verify name and case.

+You've now created your first skillset and learned the basic steps of skills-based indexing.

+Some key concepts that we hope you picked up include the dependencies. A skillset is bound to an indexer, and indexers are Azure and source-specific. Although this quickstart uses Azure Blob Storage, other Azure data sources are possible. For more information, see [Indexers in Azure AI Search](search-indexer-overview.md).

+Output is routed to a search index, and there's a mapping between name-value pairs created during indexing and individual fields in your index. Internally, the wizard sets up [an enrichment tree](cognitive-search-concept-annotations-syntax.md) and defines a [skillset](cognitive-search-defining-skillset.md), establishing the order of operations and general flow. These steps are hidden in the wizard, but when you start writing code, these concepts become important.

+If you use a free service, remember that you're limited to three indexes, indexers, and data sources. You can delete individual items in the portal to stay under the limit.

+1. Select **JSON view** so that you can enter text for your vector query in the **text** vector query parameter.

+- [Cisco Firepower eStreamer](data-connectors/cisco-firepower-estreamer.md)

+- [Citrix WAF (Web App Firewall)](data-connectors/deprecated-citrix-waf-web-app-firewall-via-legacy-agent.md)

+- [Contrast Protect](data-connectors/contrast-protect.md)

+- [CyberArk Enterprise Password Vault (EPV) Events](data-connectors/cyberark-enterprise-password-vault-epv-events.md)

+- [AI Analyst Darktrace](data-connectors/ai-analyst-darktrace.md)

+- [Delinea Secret Server](data-connectors/delinea-secret-server.md)

+- [ExtraHop Reveal(x)](data-connectors/extrahop-reveal-x.md)

+- [F5 Networks](data-connectors/f5-networks.md)

+- [iboss](data-connectors/iboss.md)

+- [Illusive Platform](data-connectors/illusive-platform.md)

+- [Morphisec UTPP](data-connectors/morphisec-utpp.md)

+- [SonicWall Firewall](data-connectors/sonicwall-firewall.md)

+- [vArmour Application Controller](data-connectors/varmour-application-controller.md)

+- [WireX Network Forensics Platform](data-connectors/wirex-network-forensics-platform.md)

+ Title: "AI Analyst Darktrace connector for Microsoft Sentinel"

+description: "Learn how to install the connector AI Analyst Darktrace to connect your data source to Microsoft Sentinel."

+# AI Analyst Darktrace connector for Microsoft Sentinel

+The Darktrace connector lets users connect Darktrace Model Breaches in real-time with Microsoft Sentinel, allowing creation of custom Dashboards, Workbooks, Notebooks and Custom Alerts to improve investigation. Microsoft Sentinel's enhanced visibility into Darktrace logs enables monitoring and mitigation of security threats.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (Darktrace)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Darktrace](https://www.darktrace.com/en/contact/) |

+## Query samples

+**first 10 most recent data breaches**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Darktrace"

+ | order by TimeGenerated desc

+ | limit 10

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-premises environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Configure Darktrace to forward Syslog messages in CEF format to your Azure workspace via the Syslog agent.

+ 1) Within the Darktrace Threat Visualizer, navigate to the System Config page in the main menu under Admin.

+ 2) From the left-hand menu, select Modules and choose Microsoft Sentinel from the available Workflow Integrations.\n 3) A configuration window will open. Locate Microsoft Sentinel Syslog CEF and click New to reveal the configuration settings, unless already exposed.

+ 4) In the Server configuration field, enter the location of the log forwarder and optionally modify the communication port. Ensure that the port selected is set to 514 and is allowed by any intermediary firewalls.

+ 5) Configure any alert thresholds, time offsets or additional settings as required.

+ 6) Review any additional configuration options you may wish to enable that alter the Syslog syntax.

+ 7) Enable Send Alerts and save your changes.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+Connect to Azure DDoS Protection Standard logs via Public IP Address Diagnostic Logs. In addition to the core DDoS protection in the platform, Azure DDoS Protection Standard provides advanced DDoS mitigation capabilities against network attacks. It's automatically tuned to protect your specific Azure resources. Protection is simple to enable during the creation of new virtual networks. It can also be done after creation and requires no application or resource changes. For more information, see the [Microsoft Sentinel documentation](https://go.microsoft.com/fwlink/p/?linkid=2219760&wt.mc_id=sentinel_dataconnectordocs_content_cnl_csasci).

+ Title: "Cisco Firepower eStreamer connector for Microsoft Sentinel"

+description: "Learn how to install the connector Cisco Firepower eStreamer to connect your data source to Microsoft Sentinel."

+# Cisco Firepower eStreamer connector for Microsoft Sentinel

+eStreamer is a Client Server API designed for the Cisco Firepower NGFW Solution. The eStreamer client requests detailed event data on behalf of the SIEM or logging solution in the Common Event Format (CEF).

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (CiscoFirepowerEstreamerCEF)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Cisco](https://www.cisco.com/c/en_in/support/https://docsupdatetracker.net/index.html) |

+## Query samples

+**Firewall Blocked Events**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Cisco"

+ | where DeviceProduct == "Firepower"

+ | where DeviceAction != "Allow"

+ ```

+**File Malware Events**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Cisco"

+ | where DeviceProduct == "Firepower"

+ | where Activity == "File Malware Event"

+ ```

+**Outbound Web Traffic Port 80**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Cisco"

+ | where DeviceProduct == "Firepower"

+ | where DestinationPort == "80"

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 25226 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Install the Firepower eNcore client

+Install and configure the Firepower eNcore eStreamer client, for more details see full install [guide](https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/eStreamer_enCore/eStreamereNcoreSentinelOperationsGuide_409.html)

+2.1 Download the Firepower Connector from github

+Download the latest version of the Firepower eNcore connector for Microsoft Sentinel [here](https://github.com/CiscoSecurity/fp-05-microsoft-sentinel-connector). If you plan on using python3 use the [python3 eStreamer connector](https://github.com/CiscoSecurity/fp-05-microsoft-sentinel-connector/tree/python3)

+2.2 Create a pkcs12 file using the Azure/VM Ip Address

+Create a pkcs12 certificate using the public IP of the VM instance in Firepower under System->Integration->eStreamer, for more information please see install [guide](https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/eStreamer_enCore/eStreamereNcoreSentinelOperationsGuide_409.html#_Toc527049443)

+2.3 Test Connectivity between the Azure/VM Client and the FMC

+Copy the pkcs12 file from the FMC to the Azure/VM instance and run the test utility (./encore.sh test) to ensure a connection can be established, for more details please see the setup [guide](https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/eStreamer_enCore/eStreamereNcoreSentinelOperationsGuide_409.html#_Toc527049430)

+2.4 Configure encore to stream data to the agent

+Configure encore to stream data via TCP to the Microsoft Agent, this should be enabled by default, however, additional ports and streaming protocols can configured depending on your network security posture, it is also possible to save the data to the file system, for more information please see [Configure Encore](https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/eStreamer_enCore/eStreamereNcoreSentinelOperationsGuide_409.html#_Toc527049433)

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/cisco.cisco-firepower-estreamer?tab=Overview) in the Azure Marketplace.

+ Title: "Citrix WAF (Web App Firewall) connector for Microsoft Sentinel"

+description: "Learn how to install the connector Citrix WAF (Web App Firewall) to connect your data source to Microsoft Sentinel."

+# Citrix WAF (Web App Firewall) connector for Microsoft Sentinel

+ Citrix WAF (Web App Firewall) is an industry leading enterprise-grade WAF solution. Citrix WAF mitigates threats against your public-facing assets, including websites, apps, and APIs. From layer 3 to layer 7, Citrix WAF includes protections such as IP reputation, bot mitigation, defense against the OWASP Top 10 application threats, built-in signatures to protect against application stack vulnerabilities, and more.

+Citrix WAF supports Common Event Format (CEF) which is an industry standard format on top of Syslog messages . By connecting Citrix WAF CEF logs to Microsoft Sentinel, you can take advantage of search & correlation, alerting, and threat intelligence enrichment for each log.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (CitrixWAFLogs)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Citrix Systems](https://www.citrix.com/support/) |

+## Query samples

+**Citrix WAF Logs**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Citrix"

+ | where DeviceProduct == "NetScaler"

+ ```

+**Citrix Waf logs for cross site scripting**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Citrix"

+ | where DeviceProduct == "NetScaler"

+ | where Activity == "APPFW_XSS"

+ ```

+**Citrix Waf logs for SQL Injection**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Citrix"

+ | where DeviceProduct == "NetScaler"

+ | where Activity == "APPFW_SQL"

+ ```

+**Citrix Waf logs for Bufferoverflow**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Citrix"

+ | where DeviceProduct == "NetScaler"

+ | where Activity == "APPFW_STARTURL"

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Configure Citrix WAF to send Syslog messages in CEF format to the proxy machine using the steps below.

+1. Follow [this guide](https://support.citrix.com/article/CTX234174) to configure WAF.

+2. Follow [this guide](https://support.citrix.com/article/CTX136146) to configure CEF logs.

+3. Follow [this guide](https://docs.citrix.com/en-us/citrix-adc/13/system/audit-logging/configuring-audit-logging.html) to forward the logs to proxy . Make sure you to send the logs to port 514 TCP on the Linux machine's IP address.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/citrix.citrix_waf_mss?tab=Overview) in the Azure Marketplace.

+ Title: "Contrast Protect connector for Microsoft Sentinel"

+description: "Learn how to install the connector Contrast Protect to connect your data source to Microsoft Sentinel."

+# Contrast Protect connector for Microsoft Sentinel

+Contrast Protect mitigates security threats in production applications with runtime protection and observability. Attack event results (blocked, probed, suspicious...) and other information can be sent to Microsoft Sentinel to blend with security information from other systems.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (ContrastProtect)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Contrast Protect](https://docs.contrastsecurity.com/) |

+## Query samples

+**All attacks**

+ ```kusto

+let extract_data=(a:string, k:string) { parse_urlquery(replace(@';', @'&', a))["Query Parameters"][k] }; CommonSecurityLog

+ | where DeviceVendor == 'Contrast Security'

+ | extend Outcome = replace(@'INEFFECTIVE', @'PROBED', tostring(coalesce(column_ifexists("EventOutcome", ""), extract_data(AdditionalExtensions, 'outcome'), "")))

+ | where Outcome != 'success'

+ | extend Rule = extract_data(AdditionalExtensions, 'pri')

+ | project TimeGenerated, ApplicationProtocol, Rule, Activity, Outcome, RequestURL, SourceIP

+ | order by TimeGenerated desc

+ ```

+**Effective attacks**

+ ```kusto

+let extract_data=(a:string, k:string) {

+ parse_urlquery(replace(@';', @'&', a))["Query Parameters"][k]

+};

+CommonSecurityLog

+ | where DeviceVendor == 'Contrast Security'

+ | extend Outcome = tostring(coalesce(column_ifexists("EventOutcome", ""), extract_data(AdditionalExtensions, 'outcome'), ""))

+ | where Outcome in ('EXPLOITED','BLOCKED','SUSPICIOUS')

+ | extend Rule = extract_data(AdditionalExtensions, 'pri')

+ | project TimeGenerated, ApplicationProtocol, Rule, Activity, Outcome, RequestURL, SourceIP

+ | order by TimeGenerated desc

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Configure the Contrast Protect agent to forward events to syslog as described here: https://docs.contrastsecurity.com/en/output-to-syslog.html. Generate some attack events for your application.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/contrast_security.contrast_protect_azure_sentinel_solution?tab=Overview) in the Azure Marketplace.

+ Title: "CyberArk Enterprise Password Vault (EPV) Events connector for Microsoft Sentinel"

+description: "Learn how to install the connector CyberArk Enterprise Password Vault (EPV) Events to connect your data source to Microsoft Sentinel."

+# CyberArk Enterprise Password Vault (EPV) Events connector for Microsoft Sentinel

+CyberArk Enterprise Password Vault generates an xml Syslog message for every action taken against the Vault. The EPV will send the xml messages through the Microsoft Sentinel.xsl translator to be converted into CEF standard format and sent to a syslog staging server of your choice (syslog-ng, rsyslog). The Log Analytics agent installed on your syslog staging server will import the messages into Microsoft Log Analytics. Refer to the [CyberArk documentation](https://docs.cyberark.com/Product-Doc/OnlineHelp/PAS/Latest/en/Content/PASIMP/DV-Integrating-with-SIEM-Applications.htm) for more guidance on SIEM integrations.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (CyberArk)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Cyberark](https://www.cyberark.com/services-support/technical-support/) |

+## Query samples

+**CyberArk Alerts**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Cyber-Ark"

+ | where DeviceProduct == "Vault"

+ | where LogSeverity == "7" or LogSeverity == "10"

+ | sort by TimeGenerated desc

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python installed on your machine.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+On the EPV configure the dbparm.ini to send Syslog messages in CEF format to the proxy machine. Make sure you to send the logs to port 514 TCP on the machines IP address.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python installed on your machine using the following command: python -version

+>

+> 2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machines security according to your organizations security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/cyberark.cyberark_epv_events_mss?tab=Overview) in the Azure Marketplace.

+ Title: "Delinea Secret Server connector for Microsoft Sentinel"

+description: "Learn how to install the connector Delinea Secret Server to connect your data source to Microsoft Sentinel."

+# Delinea Secret Server connector for Microsoft Sentinel

+Common Event Format (CEF) from Delinea Secret Server

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog(DelineaSecretServer)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Delinea](https://delinea.com/support/) |

+## Query samples

+**Get records create new secret**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Delinea Software" or DeviceVendor == "Thycotic Software"

+ | where DeviceProduct == "Secret Server"

+ | where Activity has "SECRET - CREATE"

+ ```

+**Get records where view secret**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "Delinea Software" or DeviceVendor == "Thycotic Software"

+ | where DeviceProduct == "Secret Server"

+ | where Activity has "SECRET - VIEW"

+ ```

+## Prerequisites

+To integrate with Delinea Secret Server make sure you have:

+- **Delinea Secret Server**: must be configured to export logs via Syslog

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-premises environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Set your security solution to send Syslog messages in CEF format to the proxy machine. Make sure you to send the logs to port 514 TCP on the machine's IP address.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/delineainc1653506022260.delinea_secret_server_mss?tab=Overview) in the Azure Marketplace.

+2.1 Docker Implementation

+Leverages docker images where the integration component is already installed with all necessary dependencies.

+[Integration Guide >](https://frcpnt.com/csg-sentinel)

+2.2 Traditional Implementation

+[Integration Guide >](https://frcpnt.com/csg-sentinel)

+The [Trend Micro Apex One](https://www.trendmicro.com/en_us/business/products/user-protection/sps/endpoint.html) data connector provides the capability to ingest [Trend Micro Apex One events](https://aka.ms/sentinel-TrendMicroApex-OneEvents) into Microsoft Sentinel. Refer to [Trend Micro Apex Central](https://aka.ms/sentinel-TrendMicroApex-OneCentral) for more information.

+- **Dynatrace tenant (ex. xyz.dynatrace.com)**: You need a valid Dynatrace tenant with [Application Security](https://www.dynatrace.com/platform/application-security/) enabled, learn more about the [Dynatrace platform](https://www.dynatrace.com/).

+Configure and Enable Dynatrace [Application Security](https://www.dynatrace.com/platform/application-security/).

+ Follow [these instructions](https://docs.dynatrace.com/docs/shortlink/token#create-api-token) to generate an access token.

+This connector uses the [Dynatrace Audit Logs REST API](https://docs.dynatrace.com/docs/dynatrace-api/environment-api/audit-logs) to ingest tenant audit logs into Microsoft Sentinel Log Analytics

+Enable Dynatrace Audit [Logging](https://docs.dynatrace.com/docs/shortlink/audit-logs#enable-audit-logging).

+ Follow [these instructions](https://docs.dynatrace.com/docs/shortlink/token#create-api-token) to generate an access token.

+This connector uses the [Dynatrace Problem REST API](https://docs.dynatrace.com/docs/dynatrace-api/environment-api/problems-v2) to ingest problem events into Microsoft Sentinel Log Analytics

+Follow [these instructions](https://docs.dynatrace.com/docs/shortlink/token#create-api-token) to generate an access token.

+This connector uses the [Dynatrace Security Problem REST API](https://docs.dynatrace.com/docs/dynatrace-api/environment-api/application-security/vulnerabilities/get-vulnerabilities) to ingest detected runtime vulnerabilities into Microsoft Sentinel Log Analytics.

+- **Dynatrace tenant (ex. xyz.dynatrace.com)**: You need a valid Dynatrace tenant with [Application Security](https://www.dynatrace.com/platform/application-security/) enabled, learn more about the [Dynatrace platform](https://www.dynatrace.com/).

+Configure and Enable Dynatrace [Application Security](https://www.dynatrace.com/platform/application-security/).

+ Follow [these instructions](https://docs.dynatrace.com/docs/shortlink/token#create-api-token) to generate an access token.

+ Title: "ExtraHop Reveal(x) connector for Microsoft Sentinel"

+description: "Learn how to install the connector ExtraHop Reveal(x) to connect your data source to Microsoft Sentinel."

+# ExtraHop Reveal(x) connector for Microsoft Sentinel

+The ExtraHop Reveal(x) data connector enables you to easily connect your Reveal(x) system with Microsoft Sentinel to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (ΓÇÿExtraHopΓÇÖ)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [ExtraHop](https://www.extrahop.com/support/) |

+## Query samples

+**All logs**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "ExtraHop"

+

+ | sort by TimeGenerated

+ ```

+**All detections, de-duplicated**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "ExtraHop"

+

+ | extend categories = iif(DeviceCustomString2 != "", split(DeviceCustomString2, ","),dynamic(null))

+    

+ | extend StartTime = extract("start=([0-9-]+T[0-9:.]+Z)", 1, AdditionalExtensions,typeof(datetime))

+    

+ | extend EndTime = extract("end=([0-9-]+T[0-9:.]+Z)", 1, AdditionalExtensions,typeof(datetime))

+    

+ | project      

+     DeviceEventClassID="ExtraHop Detection",

+     Title=Activity,

+     Description=Message,

+     riskScore=DeviceCustomNumber2,     

+     SourceIP,

+     DestinationIP,

+     detectionID=tostring(DeviceCustomNumber1),

+     updateTime=todatetime(ReceiptTime),

+     StartTime,

+     EndTime,

+     detectionURI=DeviceCustomString1,

+     categories,

+     Computer

+    

+ | summarize arg_max(updateTime, *) by detectionID

+    

+ | sort by detectionID desc

+ ```

+## Prerequisites

+To integrate with ExtraHop Reveal(x) make sure you have:

+- **ExtraHop**: ExtraHop Discover or Command appliance with firmware version 7.8 or later with a user account that has Unlimited (administrator) privileges.

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-premises environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python --version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward ExtraHop Networks logs to Syslog agent

+1. Set your security solution to send Syslog messages in CEF format to the proxy machine. Make sure to send the logs to port 514 TCP on the machine IP address.

+2. Follow the directions to install the [ExtraHop Detection SIEM Connector bundle](https://learn.extrahop.com/extrahop-detection-siem-connector-bundle) on your Reveal(x) system. The SIEM Connector is required for this integration.

+3. Enable the trigger for **ExtraHop Detection SIEM Connector - CEF**

+4. Update the trigger with the ODS syslog targets you created 

+5. The Reveal(x) system formats syslog messages in Common Event Format (CEF) and then sends data to Microsoft Sentinel.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python --version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/extrahop.extrahop_revealx_mss?tab=Overview) in the Azure Marketplace.

+ Title: "F5 Networks connector for Microsoft Sentinel"

+description: "Learn how to install the connector F5 Networks to connect your data source to Microsoft Sentinel."

+# F5 Networks connector for Microsoft Sentinel

+The F5 firewall connector allows you to easily connect your F5 logs with Microsoft Sentinel, to view dashboards, create custom alerts, and improve investigation. This gives you more insight into your organization's network and improves your security operation capabilities.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (F5)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [F5](https://www.f5.com/services/support) |

+## Query samples

+**All logs**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "F5"

+

+ | sort by TimeGenerated

+ ```

+**Summarize by time**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "F5"

+

+ | summarize count() by TimeGenerated

+

+ | sort by TimeGenerated

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python --version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Configure F5 to forward Syslog messages in CEF format to your Microsoft Sentinel workspace via the Syslog agent.

+Go to [F5 Configuring Application Security Event Logging](https://aka.ms/asi-syslog-f5-forwarding), follow the instructions to set up remote logging, using the following guidelines:

+1. Set the **Remote storage type** to CEF.

+2. Set the **Protocol setting** to UDP.

+3. Set the **IP address** to the Syslog server IP address.

+4. Set the **port number** to 514, or the port your agent uses.

+5. Set the **facility** to the one that you configured in the Syslog agent (by default, the agent sets this to local4).

+6. You can set the **Maximum Query String Size** to be the same as you configured.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python --version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5_networks_data_mss?tab=Overview) in the Azure Marketplace.

+**NOTE:** This data connector depends on a parser based on a Kusto Function to work as expected which is deployed as part of the solution. To view the function code in Log Analytics, open Log Analytics/Microsoft Sentinel Logs blade, click Functions and search for the alias GWorkspaceReports and load the function code or click [here](https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/GoogleWorkspaceReports/Parsers/GWorkspaceActivityReports), on the second line of the query, enter the hostname(s) of your GWorkspaceReports device(s) and any other unique identifiers for the logstream. The function usually takes 10-15 minutes to activate after solution installation/update.

+ Title: "iboss connector for Microsoft Sentinel"

+description: "Learn how to install the connector iboss to connect your data source to Microsoft Sentinel."

+# iboss connector for Microsoft Sentinel

+The [iboss](https://www.iboss.com) data connector enables you to seamlessly connect your Threat Console to Microsoft Sentinel and enrich your instance with iboss URL event logs. Our logs are forwarded in Common Event Format (CEF) over Syslog and the configuration required can be completed on the iboss platform without the use of a proxy. Take advantage of our connector to garner critical data points and gain insight into security threats.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | ibossUrlEvent<br/> |

+| **Data collection rules support** | Not currently supported |

+| **Supported by** | [iboss](https://www.iboss.com/contact-us/) |

+## Query samples

+**Logs Received from the past week**

+ ```kusto

+ibossUrlEvent

+ | where TimeGenerated > ago(7d)

+ ```

+## Vendor installation instructions

+1. Configure a dedicated proxy Linux machine

+If using the iboss gov environment or there is a preference to forward the logs to a dedicated proxy Linux machine, proceed with this step. In all other cases, please advance to step two.

+1.1 Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.2 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the dedicated proxy Linux machine between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.3 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version

+> 2. You must have elevated permissions (sudo) on your machine

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs

+Set your Threat Console to send Syslog messages in CEF format to your Azure workspace. Make note of your Workspace ID and Primary Key within your Log Analytics Workspace (Select the workspace from the Log Analytics workspaces menu in the Azure portal. Then select Agents management in the Settings section).

+>1. Navigate to Reporting & Analytics inside your iboss Console

+>2. Select Log Forwarding -> Forward From Reporter

+>3. Select Actions -> Add Service

+>4. Toggle to Microsoft Sentinel as a Service Type and input your Workspace ID/Primary Key along with other criteria. If a dedicated proxy Linux machine has been configured, toggle to Syslog as a Service Type and configure the settings to point to your dedicated proxy Linux machine

+>5. Wait one to two minutes for the setup to complete

+>6. Select your Microsoft Sentinel Service and verify the Microsoft Sentinel Setup Status is Successful. If a dedicated proxy Linux machine has been configured, you may proceed with validating your connection

+3. Validate connection

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy (Only applicable if a dedicated proxy Linux machine has been configured).

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/iboss.iboss-sentinel-connector?tab=Overview) in the Azure Marketplace.

+ Title: "Illusive Platform connector for Microsoft Sentinel"

+description: "Learn how to install the connector Illusive Platform to connect your data source to Microsoft Sentinel."

+# Illusive Platform connector for Microsoft Sentinel

+The Illusive Platform Connector allows you to share Illusive's attack surface analysis data and incident logs with Microsoft Sentinel and view this information in dedicated dashboards that offer insight into your organization's attack surface risk (ASM Dashboard) and track unauthorized lateral movement in your organization's network (ADS Dashboard).

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (illusive)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Illusive Networks](https://illusive.com/support) |

+## Query samples

+**Number of Incidents in the last 30 days in which Trigger Type is found**

+ ```kusto

+union CommonSecurityLog

+ | where (DeviceEventClassID == "illusive:login" or DeviceEventClassID == "illusive:access" or DeviceEventClassID == "illusive:suspicious")

+ | where Message !contains "hasForensics"

+ | where TimeGenerated > ago(30d)

+ | extend DeviceCustomNumber2 = coalesce(column_ifexists("FieldDeviceCustomNumber2", long(null)), DeviceCustomNumber2, long(null))

+ | summarize by DestinationServiceName, DeviceCustomNumber2

+ | summarize incident_count=count() by DestinationServiceName

+ ```

+**Top 10 alerting hosts in the last 30 days**

+ ```kusto

+union CommonSecurityLog

+ | where (DeviceEventClassID == "illusive:login" or DeviceEventClassID == "illusive:access" or DeviceEventClassID == "illusive:suspicious")

+ | where Message !contains "hasForensics"

+ | where TimeGenerated > ago(30d)

+ | extend DeviceCustomNumber2 = coalesce(column_ifexists("FieldDeviceCustomNumber2", long(null)), DeviceCustomNumber2, long(null))

+ | summarize by AlertingHost=iff(SourceHostName != "" and SourceHostName != "Failed to obtain", SourceHostName, SourceIP) ,DeviceCustomNumber2

+ | where AlertingHost != "" and AlertingHost != "Failed to obtain"

+ | summarize incident_count=count() by AlertingHost

+ | order by incident_count

+ | limit 10

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Illusive Common Event Format (CEF) logs to Syslog agent

+1. Set your security solution to send Syslog messages in CEF format to the proxy machine. Make sure you to send the logs to port 514 TCP on the machine's IP address.

+> 2. Log onto the Illusive Console, and navigate to Settings->Reporting.

+> 3. Find Syslog Servers

+> 4. Supply the following information:

+>> 1. Host name: Linux Syslog agent IP address or FQDN host name

+>> 2. Port: 514

+>> 3. Protocol: TCP

+>> 4. Audit messages: Send audit messages to server

+> 5. To add the syslog server, click Add.

+> 6. For more information about how to add a new syslog server in the Illusive platform, please find the Illusive Networks Admin Guide in here: https://support.illusivenetworks.com/hc/en-us/sections/360002292119-Documentation-by-Version

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/illusivenetworks.illusive_platform_mss?tab=Overview) in the Azure Marketplace.

+ Title: "Morphisec UTPP connector for Microsoft Sentinel"

+description: "Learn how to install the connector Morphisec UTPP to connect your data source to Microsoft Sentinel."

+# Morphisec UTPP connector for Microsoft Sentinel

+Integrate vital insights from your security products with the Morphisec Data Connector for Microsoft Sentinel and expand your analytical capabilities with search and correlation, threat intelligence, and customized alerts. Morphisec's Data Connector provides visibility into today's most advanced threats including sophisticated fileless attacks, in-memory exploits and zero days. With a single, cross-product view, you can make real-time, data-backed decisions to protect your most important assets

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Kusto function url** | https://aka.ms/sentinel-morphisecutpp-parser |

+| **Log Analytics table(s)** | CommonSecurityLog (Morphisec)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [Morphisec](https://support.morphisec.com/hc/en-us) |

+## Query samples

+**Threats count by host**

+ ```kusto

+Morphisec

+

+ | summarize Times_Attacked=count() by SourceHostName

+ ```

+**Threats count by username**

+ ```kusto

+Morphisec

+

+ | summarize Times_Attacked=count() by SourceUserName

+ ```

+**Threats with high severity**

+ ```kusto

+Morphisec

+

+ | where toint( LogSeverity) > 7

+ | order by TimeGenerated

+ ```

+## Vendor installation instructions

+These queries and workbooks are dependent on Kusto functions based on Kusto to work as expected. Follow the steps to use the Kusto functions alias "Morphisec"

+in queries and workbooks. [Follow steps to get this Kusto function.](https://aka.ms/sentinel-morphisecutpp-parser)

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-premises environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Set your security solution to send Syslog messages in CEF format to the proxy machine. Make sure you to send the logs to port 514 TCP on the machine's IP address.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/morphisec.morphisec_utpp_mss?tab=Overview) in the Azure Marketplace.

+The [Trend Micro Apex One](https://www.trendmicro.com/en_us/business/products/user-protection/sps/endpoint.html) data connector provides the capability to ingest [Trend Micro Apex One events](https://aka.ms/sentinel-TrendMicroApex-OneEvents) into Microsoft Sentinel. Refer to [Trend Micro Apex Central](https://aka.ms/sentinel-TrendMicroApex-OneCentral) for more information.

+ ThreatHunts_table_name

+ LogLevel

+ LogLevel

+Step 1 - Get the Function app endpoint

+1. Go to Azure function Overview page and Click on **"Functions"** tab.

+2. Click on the function called **"RubrikHttpStarter"**.

+3. Go to **"GetFunctionurl"** and copy the function url.

+Step 2 - Add a webhook in RubrikSecurityCloud to send data to Microsoft Sentinel.

+Follow the Rubrik User Guide instructions to [Add a Webhook](https://docs.rubrik.com/en-us/saas/saas/common/adding_webhook.html) to begin receiving event information related to Ransomware Anomalies

+ 2. Enter the URL part from copied Function-url as the webhook URL endpoint and replace **{functionname}** with **"RubrikAnomalyOrchestrator"**, for the Rubrik Microsoft Sentinel Solution

+ 3. Select the Advanced or Custom Authentication option

+ 5. Enter the Function access key(value of code parameter from copied function-url) as the HTTP value(Note: if you change this function access key in Microsoft Sentinel in the future you will need to update this webhook configuration)

+ 6. Select the EventType as Anomaly

+ 7. Select the following severity levels: Critical, Warning, Informational

+ 8. Repeat the same steps to add webhooks for Ransomware Investigation Analysis and Threat Hunt.

+ >[!NOTE]

+ > While adding webhooks for Ransomware Investigation Analysis and Threat Hunt, replace **{functionname}** with **"RubrikRansomwareOrchestrator"** and **"RubrikThreatHuntOrchestrator"** respectively in copied function-url.

+*Now we are done with the rubrik Webhook configuration. Once the webhook events triggered , you should be able to see the Anomaly, Ransomware Investigation Analysis, Threat Hunt events from the Rubrik into respective LogAnalytics workspace table called "Rubrik_Anomaly_Data_CL", "Rubrik_Ransomware_Data_CL", "Rubrik_ThreatHunt_Data_CL".*

+ Title: "SonicWall Firewall connector for Microsoft Sentinel"

+description: "Learn how to install the connector SonicWall Firewall to connect your data source to Microsoft Sentinel."

+# SonicWall Firewall connector for Microsoft Sentinel

+Common Event Format (CEF) is an industry standard format on top of Syslog messages, used by SonicWall to allow event interoperability among different platforms. By connecting your CEF logs to Microsoft Sentinel, you can take advantage of search & correlation, alerting, and threat intelligence enrichment for each log.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (SonicWall)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [SonicWall](https://www.sonicwall.com/support/) |

+## Query samples

+**All logs**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "SonicWall"

+ | sort by TimeGenerated desc

+ ```

+**Summarize by destination IP and port**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "SonicWall"

+ | summarize count() by DestinationIP, DestinationPort, TimeGenerated

+ | sort by TimeGenerated desc

+ ```

+**Show all dropped traffic from the SonicWall Firewall**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "SonicWall"

+ | where AdditionalExtensions contains "fw_action='drop'"

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward SonicWall Firewall Common Event Format (CEF) logs to Syslog agent

+Set your SonicWall Firewall to send Syslog messages in CEF format to the proxy machine. Make sure you send the logs to port 514 TCP on the machine's IP address.

+ Follow Instructions . Then Make sure you select local use 4 as the facility. Then select ArcSight as the Syslog format.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/sonicwall-inc.sonicwall-networksecurity-azure-sentinal?tab=Overview) in the Azure Marketplace.

+ > This data connector depends on a parser based on a Kusto Function to work as expected which is deployed as part of the solution. To view the function code in Log Analytics, open Log Analytics/Microsoft Sentinel Logs blade, click Functions and search for the alias Symantec Proxy SG and load the function code or click [here](https://aka.ms/sentinel-SymantecProxySG-parser), on the second line of the query, enter the hostname(s) of your Symantec Proxy SG device(s) and any other unique identifiers for the logstream. The function usually takes 10-15 minutes to activate after solution installation/update.

+ <p><code>1 $(date) $(time) $(time-taken) $(c-ip) $(cs-userdn) $(cs-auth-groups) $(x-exception-id) $(sc-filter-result) $(cs-categories) $(quot)$(cs(Referer))$(quot) $(sc-status) $(s-action) $(cs-method) $(quot)$(rs(Content-Type))$(quot) $(cs-uri-scheme) $(cs-host) $(cs-uri-port) $(cs-uri-path) $(cs-uri-query) $(cs-uri-extension) $(quot)$(cs(User-Agent))$(quot) $(s-ip) $(sr-bytes) $(rs-bytes) $(x-virus-id) $(x-bluecoat-application-name) $(x-bluecoat-application-operation) $(cs-uri-port) $(x-cs-client-ip-country) $(cs-threat-risk)</code></p>

+[Follow these instructions](https://help.symantec.com/cs/VIP_EG_INSTALL_CONFIG/VIP/v134652108_v128483142/Configuring-syslog) to configure the Symantec VIP Enterprise Gateway to forward syslog. Use the IP address or hostname for the Linux device with the Linux agent installed as the Destination IP address.

+| **Azure function app code** | https://aka.ms/sentinel-TenableIO-functionapp |

+ Title: "vArmour Application Controller connector for Microsoft Sentinel"

+description: "Learn how to install the connector vArmour Application Controller to connect your data source to Microsoft Sentinel."

+# vArmour Application Controller via Legacy Agent connector for Microsoft Sentinel

+vArmour reduces operational risk and increases cyber resiliency by visualizing and controlling application relationships across the enterprise. This vArmour connector enables streaming of Application Controller Violation Alerts into Microsoft Sentinel, so you can take advantage of search & correlation, alerting, & threat intelligence enrichment for each log.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (vArmour)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [vArmour Networks](https://www.varmour.com/contact-us/) |

+## Query samples

+**Top 10 App to App violations**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "vArmour"

+ | where DeviceProduct == "AC"

+ | where Activity == "POLICY_VIOLATION"

+ | extend AppNameSrcDstPair = extract_all("AppName=;(\\w+)", AdditionalExtensions)

+ | summarize count() by tostring(AppNameSrcDstPair)

+ | top 10 by count_

+ ```

+**Top 10 Policy names matching violations**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "vArmour"

+ | where DeviceProduct == "AC"

+ | where Activity == "POLICY_VIOLATION"

+ | summarize count() by DeviceCustomString1

+ | top 10 by count_ desc

+ ```

+**Top 10 Source IPs generating violations**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "vArmour"

+ | where DeviceProduct == "AC"

+ | where Activity == "POLICY_VIOLATION"

+ | summarize count() by SourceIP

+ | top 10 by count_

+ ```

+**Top 10 Destination IPs generating violations**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "vArmour"

+ | where DeviceProduct == "AC"

+ | where Activity == "POLICY_VIOLATION"

+ | summarize count() by DestinationIP

+ | top 10 by count_

+ ```

+**Top 10 Application Protocols matching violations**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "vArmour"

+ | where DeviceProduct == "AC"

+ | where Activity == "POLICY_VIOLATION"

+ | summarize count() by ApplicationProtocol

+ | top 10 by count_

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Configure the vArmour Application Controller to forward Common Event Format (CEF) logs to the Syslog agent

+Send Syslog messages in CEF format to the proxy machine. Make sure you to send the logs to port 514 TCP on the machine's IP address.

+2.1 Download the vArmour Application Controller user guide

+Download the user guide from https://support.varmour.com/hc/en-us/articles/360057444831-vArmour-Application-Controller-6-0-User-Guide.

+2.2 Configure the Application Controller to Send Policy Violations

+In the user guide - refer to "Configuring Syslog for Monitoring and Violations" and follow steps 1 to 3.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/varmournetworks.varmour_sentinel?tab=Overview) in the Azure Marketplace.

+ Title: "WireX Network Forensics Platform connector for Microsoft Sentinel"

+description: "Learn how to install the connector WireX Network Forensics Platform to connect your data source to Microsoft Sentinel."

+# WireX Network Forensics Platform connector for Microsoft Sentinel

+The WireX Systems data connector allows security professional to integrate with Microsoft Sentinel to allow you to further enrich your forensics investigations; to not only encompass the contextual content offered by WireX but to analyze data from other sources, and to create custom dashboards to give the most complete picture during a forensic investigation and to create custom workflows.

+## Connector attributes

+| Connector attribute | Description |

+| | |

+| **Log Analytics table(s)** | CommonSecurityLog (WireXNFPevents)<br/> |

+| **Data collection rules support** | [Workspace transform DCR](/azure/azure-monitor/logs/tutorial-workspace-transformations-portal) |

+| **Supported by** | [WireX Systems](https://wirexsystems.com/contact-us/) |

+## Query samples

+**All Imported Events from WireX**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "WireX"

+ ```

+**Imported DNS Events from WireX**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "WireX"

+ and ApplicationProtocol == "DNS"

+ ```

+**Imported DNS Events from WireX**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "WireX"

+ and ApplicationProtocol == "HTTP"

+ ```

+**Imported DNS Events from WireX**

+ ```kusto

+CommonSecurityLog

+ | where DeviceVendor == "WireX"

+ and ApplicationProtocol == "TDS"

+ ```

+## Vendor installation instructions

+1. Linux Syslog agent configuration

+Install and configure the Linux agent to collect your Common Event Format (CEF) Syslog messages and forward them to Microsoft Sentinel.

+> Notice that the data from all regions will be stored in the selected workspace

+1.1 Select or create a Linux machine

+Select or create a Linux machine that Microsoft Sentinel will use as the proxy between your security solution and Microsoft Sentinel this machine can be on your on-prem environment, Azure or other clouds.

+1.2 Install the CEF collector on the Linux machine

+Install the Microsoft Monitoring Agent on your Linux machine and configure the machine to listen on the necessary port and forward messages to your Microsoft Sentinel workspace. The CEF collector collects CEF messages on port 514 TCP.

+> 1. Make sure that you have Python on your machine using the following command: python -version.

+> 2. You must have elevated permissions (sudo) on your machine.

+ Run the following command to install and apply the CEF collector:

+ `sudo wget -O cef_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_installer.py&&sudo python cef_installer.py {0} {1}`

+2. Forward Common Event Format (CEF) logs to Syslog agent

+Contact WireX support (https://wirexsystems.com/contact-us/) in order to configure your NFP solution to send Syslog messages in CEF format to the proxy machine. Make sure that they central manager can send the logs to port 514 TCP on the machine's IP address.

+3. Validate connection

+Follow the instructions to validate your connectivity:

+Open Log Analytics to check if the logs are received using the CommonSecurityLog schema.

+>It may take about 20 minutes until the connection streams data to your workspace.

+If the logs are not received, run the following connectivity validation script:

+> 1. Make sure that you have Python on your machine using the following command: python -version

+>2. You must have elevated permissions (sudo) on your machine

+ Run the following command to validate your connectivity:

+ `sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python cef_troubleshoot.py {0}`

+4. Secure your machine

+Make sure to configure the machine's security according to your organization's security policy

+[Learn more >](https://aka.ms/SecureCEF)

+## Next steps

+For more information, go to the [related solution](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/wirexsystems1584682625009.wirex_network_forensics_platform_mss?tab=Overview) in the Azure Marketplace.

+To install Elements Connector follow [Elements Connector Docs](https://www.withsecure.com/userguides/product.html#business/connector/latest/en/).

+If api access has not been configured during installation follow [Configuring API access for Elements Connector](https://www.withsecure.com/userguides/product.html#business/connector/latest/en/task_F657F4D0F2144CD5913EE510E155E234-latest-en).

+- **REST API Credentials/permissions**: **AccountID**, **ClientID** and **ClientSecret** are required for Zoom API. [See the documentation to learn more about Zoom API](https://developers.zoom.us/docs/internal-apps/create/). [Follow the instructions for Zoom API configurations](https://aka.ms/sentinel-zoomreports-readme).

+>[!NOTE]

+> This data connector depends on a parser based on a Kusto Function to work as expected which is deployed as part of the solution. To view the function code in Log Analytics, open Log Analytics/Microsoft Sentinel Logs blade, click Functions and search for the alias Zoom and load the function code or click [here](https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/ZoomReports/Parsers/Zoom.yaml). The function usually takes 10-15 minutes to activate after solution installation/update.

+ [Follow the instructions](https://developers.zoom.us/docs/internal-apps/create/) to obtain the credentials.

+In other cases, you link them together to form an event and data pipeline. You use Event Grid to respond to events in the other services. For an example of using Event Grid with Event Hubs to migrate data to Azure Synapse Analytics, see [Stream big data into Azure Synapse Analytics](../event-grid/event-hubs-integration.md). The following image shows the workflow for streaming the data.

+Operations run on an Azure Service Bus namespace are of two kinds.

+With the peek functionality, you can use the Service Bus Explorer to view the top 100 messages in a queue, subscription, or dead-letter queue.

+ Switch to the **Message Properties** tab in the bottom pane to see the metadata.

+ Switch to the **Message Properties** tab in the bottom pane to see the metadata.

+

+IPv6 | Not supported | Mixed configurations that include both IPv4 and IPv6 are supported. However, Azure Site Recovery will use any free IPv4 address available, if there are no free IPv4 addresses in the subnet, then the configuration is not supported.

+param (

+ [parameter(Mandatory=$false)]

+ [Object]$RecoveryPlanContext

+)

+# Deploy hybrid Next.js websites on Azure Static Web Apps (Preview)

+>[!NOTE]

+> Next.js hybrid support is in preview.