+Popular third party applications, such as Dropbox, Snowflake, and Workplace by Facebook, are made available for customers through the Azure AD application gallery. New applications can easily be onboarded to the gallery using the [application network portal](../manage-apps/v2-howto-app-gallery-listing.md).

+> The sample app uses Azure Active Directory Authentication Library (ADAL). Read how to [add MSAL to your project](../develop/tutorial-v2-windows-desktop.md#add-msal-to-your-project). Remember to [add the reference to MSAL](../develop/tutorial-v2-windows-desktop.md#add-the-code-to-initialize-msal) to the class and remove the ADAL reference.

+- User Match ΓÇô Allows you to specify a different on-premises Active Directory attribute for matching Azure AD UPN instead of the default match to userPrincipalName

+For hybrid security, you can integrate Azure AD password protection with an on-premises Active Directory environment. A component installed in the on-premises environment receives the global banned password list and custom password protection policies from Azure AD, and domain controllers use them to process password change events. This hybrid approach makes sure that no matter how or where a user changes their credentials, you enforce the use of strong passwords.

+Azure AD provides ways to natively authenticate using passwordless methods to simplify the sign-in experience for users and reduce the risk of attacks.

+## Web browser cookies

+When authenticating against Azure Active Directory through a web browser, multiple cookies are involved in the process. Some of the cookies are common on all requests, other cookies are specific to some particular scenarios, i.e., specific authentication flows and/or specific client-side conditions.

+Persistent session tokens are stored as persistent cookies on the web browser's cookie jar, and non-persistent session tokens are stored as session cookies on the web browser and are destroyed when the browser session is closed.

+| Cookie Name | Type | Comments |

+|--|--|--|

+| ESTSAUTH | Common | Contains user's session information to facilitate SSO. Transient. |

+| ESTSAUTHPERSISTENT | Common | Contains user's session information to facilitate SSO. Persistent. |

+| ESTSAUTHLIGHT | Common | Contains Session GUID Information. Lite session state cookie used exclusively by client-side JavaScript in order to facilitate OIDC sign-out. Security feature. |

+| SignInStateCookie | Common | Contains list of services accessed to facilitate sign-out. No user information. Security feature. |

+| CCState | Common | Contains session information state to be used between Azure AD and the [Azure AD Backup Authentication Service](/azure/active-directory/conditional-access/resilience-defaults). |

+| buid | Common | Tracks browser related information. Used for service telemetry and protection mechanisms. |

+| fpc | Common | Tracks browser related information. Used for tracking requests and throttling. |

+| esctx | Common | Session context cookie information. For CSRF protection. Binds a request to a specific browser instance so the request can't be replayed outside the browser. No user information. |

+| ch | Common | ProofOfPossessionCookie. Stores the Proof of Possession cookie hash to the user agent. |

+| ESTSSC | Common | Legacy cookie containing session count information no longer used. |

+| ESTSSSOTILES | Common | Tracks session sign-out. When present and not expired, with value "ESTSSSOTILES=1", it will interrupt SSO, for specific SSO authentication model, and will present tiles for user account selection. |

+| AADSSOTILES | Common | Tracks session sign-out. Similar to ESTSSSOTILES but for other specific SSO authentication model. |

+| ESTSUSERLIST | Common | Tracks Browser SSO user's list. |

+| SSOCOOKIEPULLED | Common | Prevents looping on specific scenarios. No user information. |

+| cltm | Common | For telemetry purposes. Tracks AppVersion, ClientFlight and Network type. |

+| brcap | Common | Client-side cookie (set by JavaScript) to validate client/web browser's touch capabilities. |

+| clrc | Common | Client-side cookie (set by JavaScript) to control local cached sessions on the client. |

+| CkTst | Common | Client-side cookie (set by JavaScript). No longer in active use. |

+| wlidperf | Common | Client-side cookie (set by JavaScript) that tracks local time for performance purposes. |

+| x-ms-gateway-slice | Common | Azure AD Gateway cookie used for tracking and load balance purposes. |

+| stsservicecookie | Common | Azure AD Gateway cookie also used for tracking purposes. |

+| x-ms-refreshtokencredential | Specific | Available when [Primary Refresh Token (PRT)](/azure/active-directory/devices/concept-primary-refresh-token) is in use. |

+| estsStateTransient | Specific | Applicable to new session information model only. Transient. |

+| estsStatePersistent | Specific | Same as estsStateTransient, but persistent. |

+| ESTSNCLOGIN | Specific | National Cloud Login related Cookie. |

+| UsGovTraffic | Specific | US Gov Cloud Traffic Cookie. |

+| ESTSWCTXFLOWTOKEN | Specific | Saves flowToken information when redirecting to ADFS. |

+| CcsNtv | Specific | To control when Azure AD Gateway will send requests to [Azure AD Backup Authentication Service](/azure/active-directory/conditional-access/resilience-defaults). Native flows. |

+| CcsWeb | Specific | To control when Azure AD Gateway will send requests to [Azure AD Backup Authentication Service](/azure/active-directory/conditional-access/resilience-defaults). Web flows. |

+| Ccs* | Specific | Cookies with prefix Ccs*, have the same purpose as the ones without prefix, but only apply when [Azure AD Backup Authentication Service](/azure/active-directory/conditional-access/resilience-defaults) is in use. |

+| threxp | Specific | Used for throttling control. |

+| rrc | Specific | Cookie used to identify a recent B2B invitation redemption. |

+| debug | Specific | Cookie used to track if user's browser session is enabled for DebugMode. |

+| MSFPC | Specific | This cookie is not specific to any ESTS flow, but is sometimes present. It applies to all Microsoft Sites (when accepted by users). Identifies unique web browsers visiting Microsoft sites. It's used for advertising, site analytics, and other operational purposes. |

+> [!NOTE]

+> Cookies identified as client-side cookies are set locally on the client device by JavaScript, hence, will be marked with HttpOnly=false.

+>

+> Cookie definitions and respective names are subject to change at any moment in time according to Azure AD service requirements.

+ ]

+

+**Endpoints impacted**: Both v1.0 and [v2.0](./v2-oauth2-client-creds-grant-flow.md)

+**Protocol impacted**: Client Credentials (app-only tokens)

+The following list shows some examples of Microsoft web-hosted resources:

+The same is true for any third-party resources that have integrated with the Microsoft identity platform. Any of these resources can also define a set of permissions that can be used to divide the functionality of that resource into smaller chunks. As an example, [Microsoft Graph](https://graph.microsoft.com) has defined permissions to do the following tasks, among others:

+| Power BI Pro for GCC | POWERBI_PRO_GOV | f0612879-44ea-47fb-baf0-3d76d9235576 | EXCHANGE_S_FOUNDATION_GOV (922ba911-5694-4e99-a794-73aed9bfeec8)<br/>BI_AZURE_P_2_GOV (944e9726-f011-4353-b654-5f7d2663db76) | Exchange Foundation for Government (922ba911-5694-4e99-a794-73aed9bfeec8)<br/>Power BI Pro for Government (944e9726-f011-4353-b654-5f7d2663db76) |

+On July 26, 2019, we changed how we provide app-only tokens through the [client credentials grant](../develop/v2-oauth2-client-creds-grant-flow.md). Previously, apps could get tokens to call other apps, regardless of whether the client app was in the tenant. We've updated this behavior so single-tenant resources, sometimes called Web APIs, can only be called by client apps that exist in the resource tenant.

+6. Run a full sync cycle if group writeback was previously configured and will not be configured in the ΓüáAzure AD Connect wizard:

+ ``` PowerShell

+ Start-ADSyncSyncCycle -PolicyType Initial

+ ```

+> **The feature described in this document, Azure AD Integration (legacy), will be deprecated on June 1st, 2023.

+> KMS supports Konnectivity or [API Server Vnet Integration][api-server-vnet-integration].

+>

+Before creating the cluster, you need to [add the role assignment for control plane identity][add role assignment for control plane identity].

+[aad-pod-identity]: use-azure-ad-pod-identity.md

+[add role assignment for control plane identity]: use-managed-identity.md#add-role-assignment-for-control-plane-identity

+You can add a Mariner node pool into your existing cluster using the `az aks nodepool add` command and specifying `--os-sku CBLMariner`.

+ --os-sku CBLMariner

+1. Add a Mariner node pool into your existing cluster using the `az aks nodepool add` command and specifying `--os-sku CBLMariner`.

+## Scale:

+With the current limits set on Azure NPM for Linux, it can scale up to 500 Nodes and 40k Pods. You may see OOM kills beyond this scale. Please reach out to us on [aks-acn-github] if you'd like to increase your memory limit.

+[aks-acn-github]: https://github.com/Azure/azure-container-networking/issues

+> The feature described in this article, pod security policy (preview), will be deprecated starting with Kubernetes version 1.21, and it will be removed in version 1.25. AKS will mark the pod security policy as Deprecated with the AKS API on 06-01-2023 and remove it in version 1.25. You can migrate pod security policy to pod security admission controller before the deprecation deadline.

+If your plan uses the IP DDoS Protection SKU, see [Enable DDoS IP Protection Preview for a public IP address](../ddos-protection/manage-ddos-protection-powershell-ip.md#disable-ddos-ip-protection-preview-for-an-existing-public-ip-address).

+> The commandlet is restoring original slot to the target app's production slot.

+- **RestoreContentOnly**: By default `Restore-AzDeletedWebApp` will restore both your app configuration as well any content. If you want to only restore content, you can use the `-RestoreContentOnly` flag with this commandlet.

+- [mcr.microsoft.com/windows/servercore:ltsc2022](https://mcr.microsoft.com/product/windows/servercore/about)

+- [mcr.microsoft.com/windows/servercore:ltsc2019](https://mcr.microsoft.com/product/windows/servercore/about)

+- [mcr.microsoft.com/dotnet/framework/aspnet](https://mcr.microsoft.com/product/dotnet/framework/aspnet/tags):4.8-windowsservercore-ltsc2022

+- [mcr.microsoft.com/dotnet/framework/aspnet](https://mcr.microsoft.com/product/dotnet/framework/aspnet/tags):4.8-windowsservercore-ltsc2019

+- [mcr.microsoft.com/dotnet/runtime](https://mcr.microsoft.com/product/dotnet/runtime/tags):6.0-nanoserver-ltsc2022

+- [mcr.microsoft.com/dotnet/runtime](https://mcr.microsoft.com/product/dotnet/runtime/tags):6.0-nanoserver-1809

+- [mcr.microsoft.com/dotnet/aspnet](https://mcr.microsoft.com/product/dotnet/aspnet/tags):6.0-nanoserver-ltsc2022

+- [mcr.microsoft.com/dotnet/aspnet](https://mcr.microsoft.com/product/dotnet/aspnet/tags):6.0-nanoserver-1809

+### Azure Public:

+- Germany North

+- Korea South

+- Norway West

+- Switzerland West

+| Australia Central | | | ✅ |

+| Australia Central 2 | | | ✅ |

+| France South | | | ✅ |

+| Germany North | | | ✅ |

+| Korea South | | | ✅ |

+| Qatar Central | ✅ | ✅ | |

+| South India | | | ✅ |

+\* Limited availability and no support for dedicated host deployments

+To generate our database schema, set up a firewall rule on the SQL database server. This rule lets your local computer connect to Azure. For this step, you'll need to know your local computer's IP address. Azure will attempt to detect your IP automatically and presents the option to add it for you, as seen in the steps below. For more information about how to find your IP address manually, [see here](https://whatismyipaddress.com/).

+Next, update the name of the connection string in `appsettings.json` file to match the `AZURE_SQL_CONNECTION` name generated by the service connector. When the app is deployed to Azure, the `localdb` connection string value will be overridden by the connection string stored in Azure.

+"ConnectionStrings": {

+ "AZURE_SQL_CONNECTION": "Server=(localdb)\\mssqllocaldb;Trusted_Connection=True;MultipleActiveResultSets=true"

+ }

+Next, update the `Startup.cs` file the sample project by updating the existing connection string name `MyDbConnection` to `AZURE_SQL_CONNECTIONSTRING`. This change configures the `DbContext` to use the correct connection string in Azure and locally from the `appsettings.json` file.

+From a local terminal, run the following commands to install the necessary CLI tools for Entity Framework Core, create an initial database migration file, and apply those changes to update the database. Make sure to pass in the connection string value you copied from the Azure SQL database for the `connection` parameter. The `connection` parameter overrides the value of the connection string that is configured for the `DbContext` in the `startup.cs` file.

+dotnet ef database update --connection "<your-azure-sql-connection-string>"

+That we're able to create the schema in the database means that our .NET app can connect to the Azure database successfully with the new connection string. Remember that the service connector already configured the `AZURE_SQL_CONNECTIONSTRING` connection string in our App Service app. We're now ready to deploy our .NET app to the App Service. When the app is deployed, the `AZURE_SQL_CONNECTION` configuration applied to the App Service by the Service Connector will override the `localdb` connection string with the same name in the `appsettings.json` file.

+ | Prefix | Optional. A key prefix is the beginning part of a key-value's "key" property. Prefixes can be used to manage groups of key-values in a configuration store. The entered prefix will be appended to the front of the "key" property of every key-value you import from this file. | *TestApp:* |

+You've imported key-values from a JSON file, have assigned them the label "prod" and the prefix "TestApp". The separator ":" is used and all the key-values you've imported have content type set as "JSON".

+ | `--prefix` | Optional. A key prefix is the beginning part of a key. Prefixes can be used to manage groups of key-values in a configuration store. This prefix will be appended to the front of the "key" property of each imported key-value. | `TestApp:` |

+ Example: import all key-values and feature flags from a JSON file, apply the label "prod", and append the prefix "TestApp". Add the "application/json" content type.

+You've imported key-values from a JSON file, have assigned them the label "prod" and the prefix "TestApp:". The separator ";" is used and all key-values that you have imported have content type set as "JSON".

+ | From label | Select at least one label to import values with the corresponding labels. **Select all** will import key-values with any label, and **(No label)** will restrict the import to key-values with no label. | *prod* |

+ | At a specific time | Optional. Fill out to import key-values from a specific point in time. This is the point in time of the key-values in the selected configuration store. Format: "YYYY-MM-DDThh:mm:ssZ". This field defaults to the current point in time of the key-values when left empty. | *07/28/2022 12:00:00 AM* |

+You've imported key-values and feature flags with the "prod" label from an App Configuration store on January 28, 2021 at 12 AM, and have assigned them the label "new". All key-values that you have imported have content type set as "JSON".

+ | `--src-label`| Restrict your import to key-values with a specific label. If you don't use this parameter, only key-values with a null label will be imported. Supports star sign as filter: enter `*` for all labels; `abc*` for all labels with abc as prefix.| `prod` |

+ Example: import key-values and feature flags with the label "prod" from another App Configuration, and assign them the label "new". Add the "application/json" content type.

+You've imported key-values with the label "prod" from an App Configuration store and have assigned them the label "new". All key-values that you have imported have content type set as "JSON".

+ > A message is displayed, indicating the number of key-values that were successfully fetched from the source App Service resource.

+ | Prefix | Optional. A key prefix is the beginning part of a key-values's "key" property. Prefixes can be used to manage groups of key-values in a configuration store. This prefix will be appended to the front of the "key" property of each imported key-value. | *TestApp:* |

+You've imported all application settings from an App Service as key-values and assigned them the label "prod" and the prefix "TestApp". All key-values that you have imported have content type set as "JSON".

+ | `--prefix` | Optional. A key prefix is the beginning part of a key-value's "key" property. Prefixes can be used to manage groups of key-values in a configuration store. This prefix will be appended to the front of the "key" property of each imported key-value. | `TestApp:` |

+You've imported all application settings from your App Service as key-values, have assigned them the label "prod", and have added a "TestApp:" prefix. All key-values that you have imported have content type set as "JSON".

+ | Prefix | Optional. This prefix will be trimmed from each key-value's "key" property. A key prefix is the beginning part of a key. Prefixes can be used to manage groups of key-values in a configuration store. | *TestApp:* |

+ | At a specific time | Optional. Fill out to import key-values from a specific point in time. This is the point in time of the key-values in the selected configuration store. Format: "YYYY-MM-DDThh:mm:ssZ". This field defaults to the current point in time of the key-values when left empty. | *07/28/2022 12:00:00 AM* |

+ > If you don't select a *From label*, only key-values without labels will be exported. To export a key-value with a label, you must select its label. Note that you can only select one label per export in portal, in case you want to export the key-values with all labels specified please use CLI.

+ | `--label` | Enter a label to export key-values and feature flags with this label. If you don't specify a label, by default, you will only export key-values and feature flags with no label. You can enter one label, enter several labels by separating them with `,`, or use `*` to take all of the labels in account. | `prod` |

+ > If you don't select a label, only key-values without labels will be exported. To export a key-value with a label, you must select its label.

+ | `--prefix` | Optional. Prefix to be trimmed from each key-value's "key" property. A key prefix is the beginning part of a key. Prefixes can be used to manage groups of key-values in a configuration store. Prefix will be ignored for feature flags. | `TestApp:` |

+ Example: export all key-values and feature flags with label "prod" to a JSON file.

+ > A message is displayed on screen, indicating that the key-values were fetched successfully.

+ | From label | Select at least one label to export values with the corresponding labels. **Select all** will export key-values with any label, and **(No label)** will restrict the export to key-values with no label. | *prod* |

+ | At a specific time | Optional. Fill out to import key-values from a specific point in time. This is the point in time of the key-values in the selected configuration store. Format: "YYYY-MM-DDThh:mm:ssZ". This field defaults to the current point in time of the key-values when left empty. | *07/28/2022 12:00:00 AM* |

+ | `--label` | Enter a label to export key-values and feature flags with this label. If you don't specify a label, by default, you will only export key-values and feature flags with no label. You can enter one label, enter several labels by separating them with `,`, or use `*` to take all of the labels in account. | `prod` |

+ > If the key-values you want to export have labels, you must use the command `--label` and enter the corresponding labels. If you don't select a label, only key-values without labels will be exported. Use a comma sign (`,`) to select several labels or use `*` to include all labels, including the null label (no label).

+ Example: export key-values and feature flags with the label "prod" to another App Configuration store and add the destination label "new".

+1. The page now displays the selected **Target service** and resource ID. The **Select resource** action lets you switch to another target App Service resource.

+ | Prefix | Optional. This prefix will be trimmed from each exported key-value's "key" property. A key prefix is the beginning part of a key. Prefixes can be used to manage groups of key-values in a configuration store. Prefix will be ignored for feature flags. | *TestApp:* |

+ | Export as reference | Optional. Check to export key-values to App Service as App Configuration references. [Learn more](../app-service/app-service-configuration-references.md) |

+ | At a specific time | Optional. Fill out to export key-values from a specific point in time. This is the point in time of the key-values in the selected configuration store. Format: "YYYY-MM-DDThh:mm:ssZ". This field defaults to the current point in time of the key-values when left empty. | *07/28/2022 12:00:00 AM* |

+ | From label | Optional. Select an existing label to restrict your export to key-values with a specific label. If you don't select a label, only key-values with the "No label" label will be exported. | *prod* |

+You've exported key-values that have the "prod" label from an App Service resource, at their state from 07/28/2021 12:00:00 AM, and have trimmed the prefix "TestApp". The key-values have been exported with a content type in JSON format.

+If you checked the box to export key-values as references, the exported key-values will be indicated as App Configuration references in the "Source" column of your App Service resource configuration settings.

+ | `--label` | Optional. Enter a label to export key-values and feature flags with this label. If you don't specify a label, by default, you will only export key-values and feature flags with no label. | `prod` |

+ | `--prefix` | Optional. Prefix to be trimmed from an exported key-value's "key" property. A key prefix is the beginning part of a key. Prefixes can be used to manage groups of key-values in a configuration store. | `TestApp:` |

+ The command line displays a list of key-values getting exported to an App Service resource. Confirm the export by selecting `y`.

+ You've exported all key-values with the label "prod" to an Azure App Service resource and have trimmed the prefix "TestApp:".

+1. Optionally specify a flag to export as an App Configuration Reference.

+ | Parameter | Description |

+ ||-|

+ | `--export-as-reference` `-r` | Optional. Specify whether key-values are exported to App Service as App Configuration references. [Learn more](../app-service/app-service-configuration-references.md). |

+ Example: export all key-values with the label "prod" as app configuration references to an App Service application.

+ ```azurecli

+ az appconfig kv export --name my-app-config-store --destination appservice --appservice-account "/subscriptions/123/resourceGroups/my-resource-group/providers/Microsoft.Web/sites/my-app-service" --label prod --export-as-reference

+ ```

+ The command line displays a list of key-values getting exported as app configuration references to an App Service resource. Confirm the export by selecting `y`.

+ :::image type="content" source="./media/import-export/export-app-service-reference-cli-preview.png" alt-text="Screenshot of the CLI. Export App Configuration reference to App Service confirmation prompt.":::

+

+ You've exported all key-values with the label "prod" as app configuration references to an Azure App Service resource. In your App Service resource, the imported key-values will be indicated as App Configuration references in the "Source" column.

+ :::image type="content" source="./media/import-export/export-app-service-reference-value.png" alt-text="Screenshot of App Service configuration settings. Exported App Configuration reference in App Service.":::

+

+For more optional parameters and examples, go to [az appconfig kv export](/cli/azure/appconfig/kv#az-appconfig-kv-export).

+You may encounter the following error messages when importing or exporting App Configuration key-values:

+- **Public access is disabled for your store or you are accessing from a private endpoint that is not in the storeΓÇÖs private endpoint configurations**. To import key-values from an App Configuration store, you need to have access to that store. If necessary, enable public access for the source store or access it from an approved private endpoint. If you just enabled public access, wait up to 5 minutes for the cache to refresh.

+ name = "toDoItems",

+ name = "toDoItems",

+ name = "toDoItems",

+|**name** | Required. The unique name of the function binding. |

+* [HTTP trigger, write to two tables](#http-trigger-write-to-two-tables-java)

+ name = "toDoItem",

+ Id INT IDENTITY(1,1) PRIMARY KEY,

+ RequestTimeStamp DATETIME2 NOT NULL DEFAULT(GETDATE()),

+ ItemCount INT NOT NULL

+package com.function;

+import java.util.*;

+import com.microsoft.azure.functions.annotation.*;

+import com.microsoft.azure.functions.*;

+import com.microsoft.azure.functions.sql.annotation.SQLOutput;

+import com.fasterxml.jackson.core.JsonParseException;

+import com.fasterxml.jackson.core.JsonProcessingException;

+import com.fasterxml.jackson.databind.JsonMappingException;

+import com.fasterxml.jackson.databind.ObjectMapper;

+import java.util.Optional;

+public class PostToDoWithLog {

+ @FunctionName("PostToDoWithLog")

+ public HttpResponseMessage run(

+ @HttpTrigger(name = "req", methods = {HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,

+ @SQLOutput(

+ name = "toDoItem",

+ commandText = "dbo.ToDo",

+ connectionStringSetting = "SqlConnectionString")

+ OutputBinding<ToDoItem> output,

+ @SQLOutput(

+ name = "requestLog",

+ commandText = "dbo.RequestLog",

+ connectionStringSetting = "SqlConnectionString")

+ OutputBinding<RequestLog> outputLog,

+ final ExecutionContext context) throws JsonParseException, JsonMappingException, JsonProcessingException {

+ context.getLogger().info("Java HTTP trigger processed a request.");

+ String json = request.getBody().get();

+ ObjectMapper mapper = new ObjectMapper();

+ ToDoItem newToDo = mapper.readValue(json, ToDoItem.class);

+ newToDo.Id = UUID.randomUUID();

+ output.setValue(newToDo);

+ RequestLog newLog = new RequestLog();

+ newLog.ItemCount = 1;

+ outputLog.setValue(newLog);

+ return request.createResponseBuilder(HttpStatus.CREATED).header("Content-Type", "application/json").body(output).build();

+```

+|**name** | Required. The unique name of the function binding. |

+ <version>0.1.1</version>

+Extension bundles are a way to add a pre-defined set of compatible binding extensions to your function app. Extension bundles are versioned. Each version contains a specific set of binding extensions that are verified to work together. Select a bundle version based on the extensions that you need in your app.

+Use the Azure Tables input binding to read a table in [Azure Cosmos DB for Table](../cosmos-db/table/introduction.md) or [Azure Table Storage](../storage/tables/table-storage-overview.md).

+# [Azure Tables extension](#tab/table-api/in-process)

+ public class MyPoco : Azure.Data.Tables.ITableEntity

+ public string Text { get; set; }

+ public DateTimeOffset? Timestamp { get; set; }

+ public ETag ETag { get; set; }

+Use a `TableClient` method parameter to read the table by using the Azure SDK. Here's an example of a function that queries an Azure Functions log table:

+using Azure.Data.Tables;

+using Azure;

+ public class LogEntity : ITableEntity

+ public string PartitionKey { get; set; }

+ public string RowKey { get; set; }

+ public DateTimeOffset? Timestamp { get; set; }

+ public ETag ETag { get; set; }

+ [TimerTrigger("0 */1 * * * *")] TimerInfo myTimer,

+ [Table("AzureWebJobsHostLogscommon")] TableClient tableClient,

+ AsyncPageable<LogEntity> queryResults = tableClient.QueryAsync<LogEntity>(filter: $"PartitionKey eq 'FD2' and RowKey gt 't'");

+ await foreach (LogEntity entity in queryResults)

+ log.LogInformation($"{entity.PartitionKey}\t{entity.RowKey}\t{entity.Timestamp}\t{entity.OriginalName}");

+For more information about how to use `TableClient`, see the [Azure.Data.Tables API Reference](/dotnet/api/azure.data.tables.tableclient).

+# [Combined Azure Storage extension](#tab/storage-extension/in-process)

+ public class MyPoco

+ public string Text { get; set; }

+Use a `CloudTable` method parameter to read the table by using the Azure Storage SDK. Here's an example of a function that queries an Azure Functions log table:

+using Microsoft.Azure.WebJobs.Host;

+using Microsoft.Azure.Cosmos.Table;

+ public class LogEntity : TableEntity

+ [TimerTrigger("0 */1 * * * *")] TimerInfo myTimer,

+ [Table("AzureWebJobsHostLogscommon")] CloudTable cloudTable,

+ TableQuery<LogEntity> rangeQuery = new TableQuery<LogEntity>().Where(

+ TableQuery.CombineFilters(

+ TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal,

+ "FD2"),

+ TableOperators.And,

+ TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.GreaterThan,

+ "t")));

+ // Execute the query and loop through the results

+ foreach (LogEntity entity in

+ await cloudTable.ExecuteQuerySegmentedAsync(rangeQuery, null))

+ log.LogInformation(

+ $"{entity.PartitionKey}\t{entity.RowKey}\t{entity.Timestamp}\t{entity.OriginalName}");

+For more information about how to use CloudTable, see [Get started with Azure Table storage](../cosmos-db/tutorial-develop-table-dotnet.md).

+If you try to bind to `CloudTable` and get an error message, make sure that you have a reference to [the correct Storage SDK version](./functions-bindings-storage-table.md#azure-storage-sdk-version-in-functions-1x).

+# [Azure Tables extension](#tab/table-api/isolated-process)

+The following `MyTableData` class represents a row of data in the table:

+```csharp

+public class MyTableData : Azure.Data.Tables.ITableEntity

+{

+ public string Text { get; set; }

+ public string PartitionKey { get; set; }

+ public string RowKey { get; set; }

+ public DateTimeOffset? Timestamp { get; set; }

+ public ETag ETag { get; set; }

+}

+```

+The following function, which is started by a Queue Storage trigger, reads a row key from the queue, which is used to get the row from the input table. The expression `{queueTrigger}` binds the row key to the message metadata, which is the message string.

+```csharp

+[Function("TableFunction")]

+[TableOutput("OutputTable", Connection = "AzureWebJobsStorage")]

+public static MyTableData Run(

+ [QueueTrigger("table-items")] string input,

+ [TableInput("MyTable", "<PartitionKey>", "{queueTrigger}")] MyTableData tableInput,

+ FunctionContext context)

+{

+ var logger = context.GetLogger("TableFunction");

+ logger.LogInformation($"PK={tableInput.PartitionKey}, RK={tableInput.RowKey}, Text={tableInput.Text}");

+ return new MyTableData()

+ {

+ PartitionKey = "queue",

+ RowKey = Guid.NewGuid().ToString(),

+ Text = $"Output record with rowkey {input} created at {DateTime.Now}"

+ };

+}

+```

+The following Queue-triggered function returns the first 5 entities as an `IEnumerable<T>`, with the partition key value set as the queue message.

+```csharp

+[Function("TestFunction")]

+public static void Run([QueueTrigger("myqueue", Connection = "AzureWebJobsStorage")] string partition,

+ [TableInput("inTable", "{queueTrigger}", Take = 5, Filter = "Text eq 'test'",

+ Connection = "AzureWebJobsStorage")] IEnumerable<MyTableData> tableInputs,

+ FunctionContext context)

+{

+ var logger = context.GetLogger("TestFunction");

+ logger.LogInformation(partition);

+ foreach (MyTableData tableInput in tableInputs)

+ {

+ logger.LogInformation($"PK={tableInput.PartitionKey}, RK={tableInput.RowKey}, Text={tableInput.Text}");

+ }

+}

+```

+The `Filter` and `Take` properties are used to limit the number of entities returned.

+The `Filter` and `Take` properties are used to limit the number of entities returned.

+# [Azure Tables extension](#tab/table-api/csharp-script)

+The following example shows a table input binding in a *function.json* file and [C# script](./functions-reference-csharp.md) code that uses the binding. The function uses a queue trigger to read a single table row.

+The *function.json* file specifies a `partitionKey` and a `rowKey`. The `rowKey` value `{queueTrigger}` indicates that the row key comes from the queue message string.

+```json

+{

+ "bindings": [

+ {

+ "queueName": "myqueue-items",

+ "connection": "MyStorageConnectionAppSetting",

+ "name": "myQueueItem",

+ "type": "queueTrigger",

+ "direction": "in"

+ },

+ {

+ "name": "personEntity",

+ "type": "table",

+ "tableName": "Person",

+ "partitionKey": "Test",

+ "rowKey": "{queueTrigger}",

+ "connection": "MyStorageConnectionAppSetting",

+ "direction": "in"

+ }

+ ],

+ "disabled": false

+}

+```

+The [configuration](#configuration) section explains these properties.

+Here's the C# script code:

+```csharp

+#r "Microsoft.WindowsAzure.Storage"

+using Microsoft.Extensions.Logging;

+using Azure.Data.Tables;

+public static void Run(string myQueueItem, Person personEntity, ILogger log)

+{

+ log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");

+ log.LogInformation($"Name in Person entity: {personEntity.Name}");

+}

+public class Person : ITableEntity

+{

+ public string Name { get; set; }

+ public string PartitionKey { get; set; }

+ public string RowKey { get; set; }

+ public DateTimeOffset? Timestamp { get; set; }

+ public ETag ETag { get; set; }

+}

+```

+# [Azure Tables extension](#tab/table-api/in-process)

+To execute queries that return multiple entities, bind to a [TableClient] object. You can then use this object to create and execute queries against the bound table. Note that [TableClient] and related APIs belong to the [Azure.Data.Tables](/dotnet/api/azure.data.tables) namespace.

+# [Combined Azure Storage extension](#tab/storage-extension/in-process)

+To execute queries that return multiple entities, bind to a [CloudTable] object. You can then use this object to create and execute queries against the bound table. Note that [CloudTable] and related APIs belong to the [Microsoft.Azure.Cosmos.Table](/dotnet/api/microsoft.azure.cosmos.table) namespace.

+# [Azure Tables extension](#tab/table-api/isolated-process)

+To return a specific entity by key, use a binding parameter that derives from [TableEntity](/dotnet/api/azure.data.tables.tableentity).

+To execute queries that return multiple entities, bind to a [TableClient] object. You can then use this object to create and execute queries against the bound table. Note that [TableClient] and related APIs belong to the [Azure.Data.Tables](/dotnet/api/azure.data.tables) namespace.

+# [Azure Tables extension](#tab/table-api/csharp-script)

+To execute queries that return multiple entities, bind to a [TableClient] object. You can then use this object to create and execute queries against the bound table. Note that [TableClient] and related APIs belong to the [Azure.Data.Tables](/dotnet/api/azure.data.tables) namespace.

+# [Combined Azure Storage extension](#tab/storage-extension/csharp-script)

+To return a specific entity by key, use a binding parameter that derives from [TableEntity](/dotnet/api/azure.data.tables.tableentity).

+To execute queries that return multiple entities, bind to a [CloudTable] object. You can then use this object to create and execute queries against the bound table. Note that [CloudTable] and related APIs belong to the [Microsoft.Azure.Cosmos.Table](/dotnet/api/microsoft.azure.cosmos.table) namespace.

+Use an Azure Tables output binding to write entities to a table in [Azure Cosmos DB for Table](../cosmos-db/table/introduction.md) or [Azure Table Storage](../storage/tables/table-storage-overview.md).

+```csharp

+public class MyTableData : Azure.Data.Tables.ITableEntity

+{

+ public string Text { get; set; }

+ public string PartitionKey { get; set; }

+ public string RowKey { get; set; }

+ public DateTimeOffset? Timestamp { get; set; }

+ public ETag ETag { get; set; }

+}

+```

+```csharp

+[Function("TableFunction")]

+[TableOutput("OutputTable", Connection = "AzureWebJobsStorage")]

+public static MyTableData Run(

+ [QueueTrigger("table-items")] string input,

+ [TableInput("MyTable", "<PartitionKey>", "{queueTrigger}")] MyTableData tableInput,

+ FunctionContext context)

+{

+ var logger = context.GetLogger("TableFunction");

+ logger.LogInformation($"PK={tableInput.PartitionKey}, RK={tableInput.RowKey}, Text={tableInput.Text}");

+ return new MyTableData()

+ {

+ PartitionKey = "queue",

+ RowKey = Guid.NewGuid().ToString(),

+ Text = $"Output record with rowkey {input} created at {DateTime.Now}"

+ };

+}

+```

+# [Azure Tables extension](#tab/table-api/in-process)

+# [Azure Tables extension](#tab/table-api/isolated-process)

+The following types are supported for `out` parameters and return types:

+- A plain-old CLR object (POCO) that includes the `PartitionKey` and `RowKey` properties. You can accompany these properties by implementing `ITableEntity`.

+- `ICollector<T>` or `IAsyncCollector<T>` where `T` includes the `PartitionKey` and `RowKey` properties. You can accompany these properties by implementing `ITableEntity`.

+You can also bind to `TableClient` [from the Azure SDK](/dotnet/api/azure.data.tables.tableclient). You can then use that object to write to the table.

+# [Azure Tables extension](#tab/table-api/csharp-script)

+The following types are supported for `out` parameters and return types:

+- A plain-old CLR object (POCO) that includes the `PartitionKey` and `RowKey` properties. You can accompany these properties by implementing `ITableEntity`.

+- `ICollector<T>` or `IAsyncCollector<T>` where `T` includes the `PartitionKey` and `RowKey` properties. You can accompany these properties by implementing `ITableEntity`.

+You can also bind to `TableClient` [from the Azure SDK](/dotnet/api/azure.data.tables.tableclient). You can then use that object to write to the table.

+Azure Functions integrates with [Azure Tables](../cosmos-db/table/introduction.md) via [triggers and bindings](./functions-triggers-bindings.md). Integrating with Azure Tables allows you to build functions that read and write data using [Azure Cosmos DB for Table](../cosmos-db/table/introduction.md) and [Azure Table Storage](../storage/tables/table-storage-overview.md).

+# [Azure Tables extension](#tab/table-api/in-process)

+# Install the Azure Tables extension

+# Update the combined Azure Storage extension (to a version which no longer includes Azure Tables)

+> Tables have been moved out of this package starting in its 5.x version. You need to instead use version 4.x of the extension NuGet package or additionally include the [Azure Tables extension](#table-api-extension) when using version 5.x.

+# [Azure Tables extension](#tab/table-api/isolated-process)

+This version allows you to bind to types from [`Azure.Data.Tables`](/dotnet/api/azure.data.tables). It also introduces the ability to use Azure Cosmos DB for Table.

+This extension is available by installing the [Microsoft.Azure.Functions.Worker.Extensions.Tables NuGet package](https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.Tables) into a project using version 5.x or higher of the extensions for [blobs](./functions-bindings-storage-blob.md?tabs=isolated-process%2Cextensionv5) and [queues](./functions-bindings-storage-queue.md?tabs=isolated-process%2Cextensionv5).

+Using the .NET CLI:

+```dotnetcli

+# Install the Azure Tables extension

+dotnet add package Microsoft.Azure.Functions.Worker.Extensions.Tables --version 1.0.0

+# Update the combined Azure Storage extension (to a version which no longer includes Azure Tables)

+dotnet add package Microsoft.Azure.Functions.Worker.Extensions.Storage --version 5.0.0

+```

+> Tables have been moved out of this package starting in its 5.x version. You need to instead use version 4.x of the extension NuGet package or additionally include the [Azure Tables extension](#table-api-extension) when using version 5.x.

+# [Azure Tables extension (preview)](#tab/table-api/csharp-script)

+You can add this version of the extension from the extension bundle v3 by adding or replacing the following code in your `host.json` file:

+# [Combined Azure Storage extension](#tab/storage-extension/csharp-script)

+You can install this version of the extension in your function app by registering the [extension bundle], version 2.x.

+You can add this version of the extension from the extension bundle v3 by adding or replacing the following code in your `host.json` file:

+| Azure Blobs triggers and bindings | All | [Azure Blobs extension version 5.0.0 or later][blobv5],<br/>[Extension bundle 3.3.0 or later][blobv5] |

+| Azure Queues triggers and bindings | All | [Azure Queues extension version 5.0.0 or later][queuev5],<br/>[Extension bundle 3.3.0 or later][queuev5] |

+| Azure Tables (when using Azure Storage) | All | [Azure Tables extension version 1.0.0 or later](./functions-bindings-storage-table.md#table-api-extension),<br/>[Extension bundle 3.3.0 or later][tablesv1] |

+| Azure Event Hubs triggers and bindings | All | [Azure Event Hubs extension version 5.0.0 or later][eventhubv5],<br/>[Extension bundle 3.3.0 or later][eventhubv5] |

+| Azure Service Bus triggers and bindings | All | [Azure Service Bus extension version 5.0.0 or later][servicebusv5],<br/>[Extension bundle 3.3.0 or later][servicebusv5] |

+| Azure Cosmos DB triggers and bindings - Preview | Elastic Premium | [Azure Cosmos DB extension version 4.0.0-preview1 or later][cosmosv4],<br/> [Preview extension bundle 4.0.0 or later][cosmosv4]|

+| Durable Functions storage provider (Azure Storage) - Preview | All | [Durable Functions extension version 2.7.0 or later][durable-identity],<br/>[Extension bundle 3.3.0 or later][durable-identity] |

+[durable-identity]: ./durable/durable-functions-storage-providers.md#identity-based-connections-preview

+# [Azure Tables extension](#tab/table)

+[Host daemon]: /azure/azure-maps/how-to-secure-daemon-app#host-a-daemon-on-non-azure-resources

+1. ASP.NET codeless attach on VM/VMSS, and On-Premises emits standard metrics without dimensions. The same is true for App Service, but the collection level must be set to recommended. The SDK is required for all dimensions.

+- **Snapshot retention for all changes**: The Change Analysis data for resources is tracked by Azure Resource Graphs (ARG). ARG keeps snapshot history of tracked resources only for 14 days.

+| Criteria | Description |

+|:|:|

+| Azure tenants | If you have multiple Azure tenants, you'll usually create a workspace in each because several data sources can only send monitoring data to a workspace in the same Azure tenant. |

+| Azure regions | Each workspace resides in a particular Azure region, and you may have regulatory or compliance requirements to store data in particular locations. |

+| Data ownership | You may choose to create separate workspaces to define data ownership, for example by subsidiaries or affiliated companies. |

+| Multiple environments | You may have Azure Monitor workspaces supporting different environments such as test, pre-production, and production. |

+| Logical boundaries | You may choose to separate your data based on logical boundaries such as application team or company division. |

+| Workspace limits | See [Azure Monitor service limits](../service-limits.md#prometheus-metrics) for current capacity limits related to Azure Monitor workspaces. If your capacity reaches 80%, you should consider creating multiple workspaces according to logical boundaries that make sense for your organization. |

+##### Microsoft identity platform v2.0

+```

+ POST /YOUR_AAD_TENANT/oauth2/v2.0/token HTTP/1.1

+ Host: https://login.microsoftonline.com

+ Content-Type: application/x-www-form-urlencoded

+

+ grant_type=client_credentials

+ &client_id=YOUR_CLIENT_ID

+ &scope=https://management.azure.com/.default

+ &client_secret=YOUR_CLIENT_SECRET

+```

+By default, the workbook is auto-filled with the same settings as the LA workspace, with the same subscription and resource group. Workbooks are saved to 'My Reports' by default, and are only accessible by the individual user, but they can be saved directly to shared reports or shared later on. Workbooks are shared resources and they require write access to the parent resource group to be saved.

+## Share a workbook

+When you want to share a workbook or template, keep in mind that the person you want to share with must have permissions to access the workbook. They must have an Azure account, and **Azure Sentinel Workbook Reader** permissions.

+To share a workbook or workbook template:

+1. In the Azure portal, select the workbook or template you want to share.

+1. Select the **Share** icon from the top toolbar.

+1. The **Share workbook** or **Share template** window opens with a URL to use for sharing the workbook.

+1. Copy the link to share the workbook, or select **Share link via email** to open your default mail app.

+> With Dependency agent 9.10.15 and above, installation is not blocked for unsupported kernel versions, but the agent will run in degraded mode. In this mode, connection and port data stored in VMConnection and VMBoundport tables is not collected. The VMProcess table may have some data, but it will be minimal.

+>[!NOTE]

+> Dependency agent is not supported for Azure Virtual Machines with Ampere Altra ARMΓÇôbased processors.

+> [!IMPORTANT]

+> Once the volume is created using the availability zone volume placement feature, the volume has the same level of support as other volumes deployed in the subscription without this feature enabled. For example, if there is an issue with backup and restore on the volume, it will be supported because the problem is not with the availability zone volume placement feature itself.

+- Bicep loops only work with values that can be determined at the start of deployment.

+- To loop on multiple levels of properties, use [lambda map function](./bicep-functions-lambda.md#map).

+It's worth noting that the parameter file saves parameter values as plain text. For security reasons, this approach is not recommended for sensitive values such as passwords. If you must pass a parameter with a sensitive value, keep the value in a key vault. Instead of adding the sensitive value to your parameter file, use the [getSecret function](bicep-functions-resource.md#getsecret) to retrieve it. For more information, see [Use Azure Key Vault to pass secure parameter value during Bicep deployment](key-vault-parameter.md).

+It's worth noting that the parameter file saves parameter values as plain text. For security reasons, this approach is not recommended for sensitive values such as passwords. If you must pass a parameter with a sensitive value, keep the value in a key vault. Then, in your parameter file, include a reference to the key vault. During deployment, the sensitive value is securely retrieved. For more information, see [Use Azure Key Vault to pass secure parameter value during deployment](./key-vault-parameter.md).

+ } catch (err) {

+ Title: OData filter syntax in Azure Web PubSub service

+description: OData language reference and full syntax used for creating filter expressions in Azure Web PubSub service queries.

+# OData filter syntax in Azure Web PubSub service

+In Azure Web PubSub service, the **filter** parameter specifies inclusion or exclusion criteria for the connections to send messages to. This article describes the OData syntax of **filter** and provides examples.

+The complete syntax is described in the [formal grammar](#formal-grammar).

+There is also a browsable [syntax diagram](https://aka.ms/awps/filter-syntax-diagram) that allows you to interactively explore the grammar and the relationships between its rules.

+## Syntax

+A filter in the OData language is a Boolean expression, which in turn can be one of several types of expression, as shown by the following EBNF ([Extended Backus-Naur Form](https://en.wikipedia.org/wiki/Extended_BackusΓÇôNaur_form)):

+```

+/* Identifiers */

+string_identifier ::= 'connectionId' | 'userId'

+collection_identifier ::= 'groups'

+/* Rules for $filter */

+boolean_expression ::= logical_expression

+ | comparison_expression

+ | in_expression

+ | boolean_literal

+ | boolean_function_call

+ | '(' boolean_expression ')'

+```

+An interactive syntax diagram is also available:

+> [!div class="nextstepaction"]

+> [OData syntax diagram for Azure Web PubSub service](https://aka.ms/awps/filter-syntax-diagram)

+> [!NOTE]

+> See [formal grammar section](#formal-grammar) for the complete EBNF.

+### Identifiers

+The filter syntax is used to filter out the connections matching the filter expression to send messages to.

+Azure Web PubSub supports below identifiers:

+| Identifier | Description | Note | Examples

+| | | -- | --

+| `userId` | The userId of the connection. | Case insensitive. It can be used in [string operations](#supported-operations). | `userId eq 'user1'`

+| `connectionId` | The connectionId of the connection. | Case insensitive. It can be used in [string operations](#supported-operations). | `connectionId ne '123'`

+| `groups` | The collection of groups the connection is currently in. | Case insensitive. It can be used in [collection operations](#supported-operations). | `'group1' in groups`

+Identifiers are used to refer to the property value of a connection. Azure Web PubSub supports 3 identifiers matching the property name of the connection model. and supports identifiers `userId` and `connectionId` in string operations, supports identifier `groups` in [collection operations](#supported-operations). For example, to filter out connections with userId `user1`, we specify the filter as `userId eq 'user1'`. Read through the below sections for more samples using the filter.

+### Boolean expressions

+The expression for a filter is a boolean expression. When sending messages to connections, Azure Web PubSub sends messages to connections with filter expression evaluated to `true`.

+The types of Boolean expressions include:

+- Logical expressions that combine other Boolean expressions using the operators `and`, `or`, and `not`.

+- Comparison expressions, which compare fields or range variables to constant values using the operators `eq`, `ne`, `gt`, `lt`, `ge`, and `le`.

+- The Boolean literals `true` and `false`. These constants can be useful sometimes when programmatically generating filters, but otherwise don't tend to be used in practice.

+- Boolean expressions in parentheses. Using parentheses can help to explicitly determine the order of operations in a filter. For more information on the default precedence of the OData operators, see [operator precedence section](#operator-precedence).

+### Supported operations

+| Operator | Description | Example

+| | |

+| **Logical Operators**

+| `and` | Logical and | `length(userId) le 10 and length(userId) gt 3`

+| `or` | Logical or | `length(userId) gt 10 or length(userId) le 3`

+| `not` | Logical negation | `not endswith(userId, 'milk')`

+| **Comparison Operators**

+| `eq` | Equal | `userId eq 'user1'`, </br> `userId eq null`

+| `ne` | Not equal | `userId ne 'user1'`, </br> `userId ne null`

+| `gt` | Greater than | `length(userId) gt 10`

+| `ge` | Greater than or equal | `length(userId) ge 10`

+| `lt` | Less than | `length(userId) lt 3`

+| `le` | Less than or equal | `'group1' in groups`, </br> `user in ('user1','user2')`

+| **In Operator**

+| `in` | The right operand MUST be either a comma-separated list of primitive values, enclosed in parentheses, or a single expression that resolves to a collection.| `userId ne 'user1'`

+| **Grouping Operator**

+| `()` | Controls the evaluation order of an expression | `userId eq 'user1' or (not (startswith(userId,'user2'))`

+| **String Functions**

+| `string tolower(string p)` | Get the lower case for the string value | `tolower(userId) eq 'user1'` can match connections for user `USER1`

+| `string toupper(string p)` | Get the upper case for the string value | `toupper(userId) eq 'USER1'` can match connections for user `user1`

+| `string trim(string p)` | Trim the string value | `trim(userId) eq 'user1'` can match connections for user ` user1 `

+| `string substring(string p, int startIndex)`,</br>`string substring(string p, int startIndex, int length)` | Substring of the string | `substring(userId,5,2) eq 'ab'` can match connections for user `user-ab-de`

+| `bool endswith(string p0, string p1)` | Check if `p0` ends with `p1` | `endswith(userId,'de')` can match connections for user `user-ab-de`

+| `bool startswith(string p0, string p1)` | Check if `p0` starts with `p1` | `startswith(userId,'user')` can match connections for user `user-ab-de`

+| `int indexof(string p0, string p1)` | Get the index of `p1` in `p0`. Returns `-1` if `p0` does not contain `p1`. | `indexof(userId,'-ab-') ge 0` can match connections for user `user-ab-de`

+| `int length(string p)` | Get the length of the input string | `length(userId) gt 1` can match connections for user `user-ab-de`

+| **Collection Functions**

+| `int length(collection p)` | Get the length of the collection | `length(groups) gt 1` can match connections in 2 groups

+### Operator precedence

+If you write a filter expression with no parentheses around its sub-expressions, Azure Web PubSub service will evaluate it according to a set of operator precedence rules. These rules are based on which operators are used to combine sub-expressions. The following table lists groups of operators in order from highest to lowest precedence:

+| Group | Operator(s) |

+| | |

+| Logical operators | `not` |

+| Comparison operators | `eq`, `ne`, `gt`, `lt`, `ge`, `le` |

+| Logical operators | `and` |

+| Logical operators | `or` |

+An operator that is higher in the above table will "bind more tightly" to its operands than other operators. For example, `and` is of higher precedence than `or`, and comparison operators are of higher precedence than either of them, so the following two expressions are equivalent:

+```odata-filter-expr

+length(userId) gt 0 and length(userId) lt 3 or length(userId) gt 7 and length(userId) lt 10

+((length(userId) gt 0) and (length(userId) lt 3)) or ((length(userId) gt 7) and (length(userId) lt 10))

+```

+The `not` operator has the highest precedence of all -- even higher than the comparison operators. That's why if you try to write a filter like this:

+```odata-filter-expr

+not length(userId) gt 5

+```

+You'll get this error message:

+```text

+Invalid syntax for 'not length(userId)': Type 'null', expect 'bool'. (Parameter 'filter')

+```

+This error happens because the operator is associated with just the `length(userId)` expression, which is of type `null` when `userId` is `null`, and not with the entire comparison expression. The fix is to put the operand of `not` in parentheses:

+```odata-filter-expr

+not (length(userId) gt 5)

+```

+### Filter size limitations

+There are limits to the size and complexity of filter expressions that you can send to Azure Web PubSub service. The limits are based roughly on the number of clauses in your filter expression. A good guideline is that if you have over 100 clauses, you are at risk of exceeding the limit. We recommend designing your application in such a way that it doesn't generate filters of unbounded size.

+## Examples

+1. Send to multiple groups

+ ```odata-filter-expr

+ filter='group1' in groups or 'group2' in groups or 'group3' in groups

+ ```

+2. Send to multiple users in some specific group

+ ```odata-filter-expr

+ filter=userId in ('user1', 'user2', 'user3') and 'group1' in groups

+ ```

+3. Send to some user but not some specific connectionId

+ ```odata-filter-expr

+ filter=userId eq 'user1' and connectionId ne '123'

+ ```

+4. Send to some user not in some specific group

+ ```odata-filter-expr

+ filter=userId eq 'user1' and (not ('group1' in groups))

+ ```

+5. Escape `'` when userId contains `'`

+ ```odata-filter-expr

+ filter=userId eq 'user''1'

+ ```

+## Formal grammar

+We can describe the subset of the OData language supported by Azure Web PubSub service using an EBNF ([Extended Backus-Naur Form](https://en.wikipedia.org/wiki/Extended_BackusΓÇôNaur_form)) grammar. Rules are listed "top-down", starting with the most complex expressions, and breaking them down into more primitive expressions. At the top is the grammar rule for `$filter` that correspond to specific parameter `filter` of the Azure Azure Web PubSub service `Send*` REST APIs:

+```

+/* Top-level rule */

+filter_expression ::= boolean_expression

+/* Identifiers */

+string_identifier ::= 'connectionId' | 'userId'

+collection_identifier ::= 'groups'

+/* Rules for $filter */

+boolean_expression ::= logical_expression

+ | comparison_expression

+ | in_expression

+ | boolean_literal

+ | boolean_function_call

+ | '(' boolean_expression ')'

+logical_expression ::= boolean_expression ('and' | 'or') boolean_expression

+ | 'not' boolean_expression

+comparison_expression ::= primary_expression comparison_operator primary_expression

+in_expression ::= primary_expression 'in' ( '(' primary_expression (',' primary_expression)* ')' ) | collection_expression

+collection_expression ::= collection_variable

+ | '(' collection_expression ')'

+primary_expression ::= primary_variable

+ | function_call

+ | constant

+ | '(' primary_expression ')'

+string_expression ::= string_literal

+ | 'null'

+ | string_identifier

+ | string_function_call

+ | '(' string_expression ')'

+primary_variable ::= string_identifier

+collection_variable ::= collection_identifier

+comparison_operator ::= 'gt' | 'lt' | 'ge' | 'le' | 'eq' | 'ne'

+/* Rules for constants and literals */

+constant ::= string_literal

+ | integer_literal

+ | boolean_literal

+ | 'null'

+boolean_literal ::= 'true' | 'false'

+string_literal ::= "'"([^'] | "''")*"'"

+digit ::= [0-9]

+sign ::= '+' | '-'

+integer_literal ::= sign? digit+

+boolean_literal ::= 'true' | 'false'

+/* Rules for functions */

+function_call ::= indexof_function_call

+ | length_function_call

+ | string_function_call

+ | boolean_function_call

+boolean_function_call ::= endsWith_function_call

+ | startsWith_function_call

+ | contains_function_call

+string_function_call ::= tolower_function_call

+ | toupper_function_call

+ | trim_function_call

+ | substring_function_call

+ | concat_function_call

+/* Rules for string functions */

+indexof_function_call ::= "indexof" '(' string_expression ',' string_expression ')'

+concat_function_call ::= "concat" '(' string_expression ',' string_expression ')'

+contains_function_call ::= "contains" '(' string_expression ',' string_expression ')'

+endsWith_function_call ::= "endswith" '(' string_expression ',' string_expression ')'

+startsWith_function_call ::= "startswith" '(' string_expression ',' string_expression ')'

+substring_function_call ::= "substring" '(' string_expression ',' integer_literal (',' integer_literal)? ')'

+tolower_function_call ::= "tolower" '(' string_expression ')'

+toupper_function_call ::= "toupper" '(' string_expression ')'

+trim_function_call ::= "trim" '(' string_expression ')'

+/* Rules for string and collection functions */

+length_function_call ::= "length" '(' string_expression | collection_expression ')'

+```

+## Next steps

+<a name="tvm-backup">Trusted Launch VM</a> | Backup supported. <br><br> Backup of Trusted Launch VM is supported through [Enhanced policy](backup-azure-vms-enhanced-policy.md). You can enable backup through [Recovery Services vault](./backup-azure-arm-vms-prepare.md), [VM Manage blade](./backup-during-vm-creation.md#start-a-backup-after-creating-the-vm), and [Create VM blade](backup-during-vm-creation.md#create-a-vm-with-backup-configured). <br><br> **Feature details** <br><br> - Backup is supported in all regions where Trusted Launch VM is available. <br><br> - Configurations of Backup, Alerts, and Monitoring for Trusted Launch VM are currently not supported through Backup center. <br><br> - Migration of an existing [Generation 2](../virtual-machines/generation-2.md) VM (protected with Azure Backup) to Trusted Launch VM is currently not supported. Learn about how to [create a Trusted Launch VM](../virtual-machines/trusted-launch-portal.md?tabs=portal#deploy-a-trusted-launch-vm).

+[Confidential VM](../confidential-computing/confidential-vm-overview.md) | The backup support is in Limited Preview. <br><br> Backup is supported only for those Confidential VMs with no confidential disk encryption and for Confidential VMs with confidential OS disk encryption using Platform Managed Key (PMK). <br><br> Backup is currently not supported for Confidential VMs with confidential OS disk encryption using Customer Managed Key (CMK). <br><br> **Feature details** <br><br> - Backup is supported in [all regions where Confidential VM is available](../confidential-computing/confidential-vm-overview.md#regions). <br><br> - Backup is supported using [Enhanced Policy](backup-azure-vms-enhanced-policy.md) only. You can configure backup through [Create VM blade](backup-azure-arm-vms-prepare.md), [VM Manage blade](backup-during-vm-creation.md#start-a-backup-after-creating-the-vm), and [Recovery Services vault](backup-azure-arm-vms-prepare.md). <br><br> - [Cross Region Restore](backup-azure-arm-restore-vms.md#cross-region-restore) and File Recovery (Item level Restore) for Confidential VM are currently not supported.

+| **OS versions** | SLES 12 with SP2, SP3, SP4 and SP5; SLES 15 with SP0, SP1, SP2, SP3, and SP4 <br><br> RHEL 7.4, 7.6, 7.7, 7.9, 8.1, 8.2, 8.4, and 8.6 | |

+|Network (available bandwidth between nodes)|25 Gbps|25 Gbps|

+Applications that call the Azure Batch Management service authenticate with [Microsoft Authentication Library](../active-directory/develop/msal-overview.md) (Azure AD). Azure AD is Microsoft's multi-tenant cloud based directory and identity management service. Azure itself uses Azure AD for the authentication of its customers, service administrators, and organizational users.

+The [Microsoft Authentication Library](../active-directory/develop/msal-authentication-flows.md) (MSAL) provides a programmatic interface to Azure AD for use within your applications. To call MSAL from your application, you must register your application in an Azure AD tenant. When you register your application, you supply Azure AD with information about your application, including a name for it within the Azure AD tenant. Azure AD then provides an application ID that you use to associate your application with Azure AD at runtime. To learn more about the application ID, see [Application and service principal objects in Azure Active Directory](../active-directory/develop/app-objects-and-service-principals.md).

+The **Required Permissions** blade now shows that permissions to your application are granted to both the MSAL and Resource Manager APIs. Permissions are granted to MSAL by default when you first register your app with Azure AD.

+After you register the AccountManagement sample in the Azure AD tenant and update the sample source code with your values, the sample is ready to authenticate using Azure AD. When you run the sample, the MSAL attempts to acquire an authentication token. At this step, it prompts you for your Microsoft credentials:

+- In-depth examples showing how to use MSAL are available in the [Azure Code Samples](https://azure.microsoft.com/resources/samples/?service=active-directory) library.

+The first step in using Azure AD to authenticate is registering your application in an Azure AD tenant. Registering your application enables you to call the Azure [Microsoft Authentication Library](../active-directory/develop/msal-overview.md) (MSAL) from your code. The ADAL provides an API for authenticating with Azure AD from your application. Registering your application is required whether you plan to use integrated authentication or a service principal.

+To authenticate with integrated authentication from Batch .NET, reference the [Azure Batch .NET](https://www.nuget.org/packages/Microsoft.Azure.Batch/) package and the [MSAL](https://www.nuget.org/packages/Microsoft.Identity.Client/) package.

+using Microsoft.Identity.Client;

+Write a callback method to acquire the authentication token from Azure AD. The **GetAuthenticationTokenAsync** callback method shown here calls MSAL to authenticate a user who is interacting with the application. The **AcquireTokenAsync** method provided by MSAL prompts the user for their credentials, and the application proceeds once the user provides them (unless it has already cached credentials):

+To authenticate with a service principal from Batch .NET, reference the [Azure Batch .NET](https://www.nuget.org/packages/Azure.Batch/) package and the [MSAL](https://www.nuget.org/packages/Microsoft.Identity.Client/) package.

+using Microsoft.Identity.Client;

+Write a callback method to acquire the authentication token from Azure AD. The **GetAuthenticationTokenAsync** callback method shown here calls MSAL for unattended authentication:

+1. Acquire a security token from Azure AD by using [Acquire and cache tokens using the Microsoft Authentication Library (MSAL)](../active-directory/develop/msal-net-acquire-token-silently.md). If the user is not already signed in, they are prompted for their Azure credentials.

+BatchNodeManagement.*region* service tag. This rule is only created in `classic` pool communication mode.

+* Outbound any traffic on port 443 to Batch service IP addresses that correspond to the BatchNodeManagement.*region* service tag.

+> Batch service IP addresses can change over time. Therefore, we highly recommend that you use the

+> BatchNodeManagement.*region* service tag (or a regional variant) for the NSG rules indicated in the

+> following tables. Avoid populating NSG rules with specific Batch service IP addresses.

+| BatchNodeManagement.*region* [service tag](../../articles/virtual-network/network-security-groups-overview.md#service-tags) | 29876-29877 | TCP | Classic | Yes |

+| BatchNodeManagement.*region* [service tag](../../articles/virtual-network/network-security-groups-overview.md#service-tags) | 443 | * | Simplified | Yes |

+| Storage.*region* [service tag](../../articles/virtual-network/network-security-groups-overview.md#service-tags) | 443 | TCP | Classic | Yes |

+Outbound to BatchNodeManagement.*region* service tag is required in `classic` pool communication mode

+BatchNodeManagement.*region* in `simplified` pool communication mode, the Batch service currently only

+For more information about outbound security rules for the BatchNodeManagement.*region* service tag, see

+- The Batch service needs to communicate with nodes for scheduling tasks. To enable this communication, add a UDR corresponding to the BatchNodeManagement.*region* [service tag](../virtual-network/virtual-networks-udr-overview.md#service-tags-for-user-defined-routes) in the region where your Batch account exists. Set the **Next hop type** to **Internet**.

+- Ensure that outbound TCP/UDP traffic to the Azure Batch BatchNodeManagement.*region* service tag on destination port 443 isn't blocked by your on-premises network. Currently only TCP protocol is used, but UDP may be required for future compatibility.

+> Batch service IP addresses can change over time. To prevent outages due to Batch service IP address changes, do not directly specify IP addresses. Instead use the BatchNodeManagement.*region* [service tag](../virtual-network/virtual-networks-udr-overview.md#service-tags-for-user-defined-routes).

+When provisioning [Batch pools in a virtual network](batch-virtual-network.md), ensure that you're closely following the guidelines regarding the use of the BatchNodeManagement.*region* service tag, ports, protocols and direction of the rule. Use of the service tag is highly recommended; don't use underlying Batch service IP addresses as they can change over time. Using Batch service IP addresses directly can cause instability, interruptions, or outages for your Batch pools.

+For User Defined Routes (UDRs), it's recommended to use BatchNodeManagement.*region* [service tags](../virtual-network/virtual-networks-udr-overview.md#service-tags-for-user-defined-routes) instead of Batch service IP addresses as they can change over time.

+POST {batchURL}/pools?api-version=2022-10-01.16.0

+ "id": "pool-npip",

+ "vmSize": "standard_d2s_v3",

+ "offer": "0001-com-ubuntu-server-jammy",

+ "sku": "22_04-lts"

+ "nodeAgentSKUId": "batch.node.ubuntu 22.04"

+ "targetDedicatedNodes": 2,

+ "taskSlotsPerNode": 1,

+ "enableInterNodeCommunication": false,

+ "targetNodeCommunicationMode": "simplified"

+VNet injection allows Chaos resource provider to inject containerized workloads into your VNet. This means that resources without public endpoints can be accessed via a private IP address on the VNet. Below are the steps you can follow for vnet injection:

+- Virtual Networks (Azure Batch not supported)

+| Virtual networks with Azure Batch Deployments | Not supported |

+[4040966]: https://support.microsoft.com/topic/description-of-the-security-only-update-for-the-net-framework-3-5-1-for-windows-7-sp1-and-windows-server-2008-r2-sp1-september-12-2017-bd775b30-8761-a576-7c43-19ce534267f0

+you'll create a new application using the [Express](https://github.com/expressjs/express) module, which provides an MVC framework for creating Node.js web applications.

+Perform the following steps to create a new cloud service project named `expressapp`:

+2. Change directories to the **c:\\node** directory and then enter the following commands to create a new solution named `expressapp` and a web role named **WebRole1**:

+ You'll be prompted to overwrite your earlier application. Enter **y** or **yes** to continue. Express will generate the app.js file and a folder structure for building your application.

+ This change is required since we moved the file (formerly `bin/www`) to the same directory as the app file being required. After making this change, save the **server.js** file.

+ Jade is the default view engine used by Express applications.

+

+4. Refresh your browser and you'll see your changes.

+[http://jade-lang.com]: http://jade-lang.com

+ Write-Output "Checking if Python is installed...$nl"

+## Examples

+The generated bounding box can vary widely depending on what you specify for aspect ratio, as shown in the following images.

+| Aspect ratio | Bounding box |

+|-|--|

+| original | :::image type="content" source="Images/cropped-original.png" alt-text="Photo of a man with a dog at a table."::: |

+| 0.75 | :::image type="content" source="Images/cropped-075-bb.png" alt-text="Photo of a man with a dog at a table. A 0.75 ratio bounding box is drawn."::: |

+| 1.00 | :::image type="content" source="Images/cropped-1-0-bb.png" alt-text="Photo of a man with a dog at a table. A 1.00 ratio bounding box is drawn."::: |

+| 1.50 | :::image type="content" source="Images/cropped-150-bb.png" alt-text="Photo of a man with a dog at a table. A 1.50 ratio bounding box is drawn."::: |

+Custom Neural Voice (CNV) is a text-to-speech feature that lets you create a one-of-a-kind, customized, synthetic voice for your applications. With Custom Neural Voice, you can build a highly natural-sounding voice for your brand or characters by providing human speech samples as training data.

+Out of the box, [text-to-speech](text-to-speech.md) can be used with prebuilt neural voices for each [supported language](language-support.md?tabs=stt-tts). The prebuilt neural voices work very well in most text-to-speech scenarios if a unique voice isn't required.

+Here's an overview of the steps to create a custom neural voice in Speech Studio:

+1. [Create a project](how-to-custom-voice.md) to contain your data, voice models, tests, and endpoints. Each project is specific to a country and language. If you are going to create multiple voices, it's recommended that you create a project for each voice.

+- [Neural - cross lingual](?tabs=crosslingual#train-your-custom-neural-voice-model) (Preview): Create a secondary language for your voice model to speak a different language from your training data. For example, with the `zh-CN` training data, you can create a voice that speaks `en-US`. The language of the training data and the target language must both be one of the [languages that are supported](language-support.md?tabs=stt-tts) for cross lingual voice training. You don't need to prepare training data in the target language, but your test script must be in the target language.

+- [Neural - multi style](?tabs=multistyle#train-your-custom-neural-voice-model) (Preview): Create a custom neural voice that speaks in multiple styles and emotions, without adding new training data. Multi-style voices are particularly useful for video game characters, conversational chatbots, audiobooks, content readers, and more. To create a multi-style voice, you just need to prepare a set of general training data (at least 300 utterances), and select one or more of the preset target speaking styles. You can also create up to 10 custom styles by providing style samples as additional training data for the same voice.

+The language of the training data must be one of the [languages that are supported](language-support.md?tabs=stt-tts) for custom neural voice neural, cross-lingual, or multi-style training.

+ 1. Select style samples as training data. It's recommended that the style samples are all from the same voice talent profile.

+# Speech service release notes

+See below for information about new features and other changes to the Speech service.

+* Speech SDK 1.24.0 and Speech CLI 1.24.0 were released in October 2022.

+* Speech-to-text and text-to-speech container versions were updated in October 2022.

+* TTS Service November 2022, several voices for `es-MX`, `it-IT`, and `pr-BR` locales were made generally available.

+- **Participant end reasons**, which keep track of the reason why a participant left a call. End reasons are [SIP codes](https://en.wikipedia.org/wiki/List_of_SIP_response_codes), which are numeric codes that describe the specific status of a signaling request. SIP codes can be grouped into six categories: *Success*, *Client Failure*, *Server Failure*, *Global Failure*, *Redirection*, and *Provisional*. The distribution of SIP code categories is shown in the pie chart on the left hand side, while a list of the specific SIP codes for participant end reasons is provided on the right hand side

+The **Recording** tab displays data relevant to total recordings, recording format, recording channel types and number of recording per call:

+| Call Automation | - | [NuGet](https://www.nuget.org/packages/Azure.Communication.CallAutomation) | - | [Maven](https://search.maven.org/search?q=a:azure-communication-callautomation) | - | - | - |

+1. Clone the [sample Linux application](https://github.com/Azure/confidential-computing-cvm-guest-attestation/tree/main/cvm-platform-checker-exe/Linux).

+1. Clone the [sample Windows application](https://github.com/Azure/confidential-computing-cvm-guest-attestation/tree/main/cvm-platform-checker-exe/Windows).

+- [Azure Confidential Ledger write transaction receipts](write-transaction-receipts.md)

+resource_group = "<azure-resource-group>"

+ cert_file.write(network_identity['ledgerTlsCertificate'])

+We are prepared to write to the ledger. We will do so using the `create_ledger_entry` function.

+sample_entry = {"contents": "Hello world!"}

+append_result = ledger_client.create_ledger_entry(entry=sample_entry)

+print(append_result['transactionId'])

+entry = ledger_client.get_ledger_entry(transaction_id=append_result['transactionId'])['entry']

+print(f"Entry (transaction id = {entry['transactionId']}) in collection {entry['collectionId']}: {entry['contents']}")

+```

+If you just want the latest transaction that was committed to the ledger, you can use the `get_current_ledger_entry` function.

+```python

+latest_entry = ledger_client.get_current_ledger_entry()

+The print function will return "Hello world!", as that's the message in the ledger that corresponds to the transaction ID and is the latest transaction.

+resource_group = "<azure-resource-group>"

+ledger_name = "<your-unique-ledger-name>"

+confidential_ledger_mgmt.ledger.begin_create(resource_group, ledger_name, ledger_properties)

+print(f"{resource_group} / {ledger_name}")

+myledger = confidential_ledger_mgmt.ledger.get(resource_group, ledger_name)

+ cert_file.write(network_identity['ledgerTlsCertificate'])

+sample_entry = {"contents": "Hello world!"}

+ledger_client.create_ledger_entry(entry=sample_entry)

+latest_entry = ledger_client.get_current_ledger_entry()

+## Pollers

+If you'd like to wait for your write transaction to be committed to your ledger, you can use the `begin_create_ledger_entry` function. This will return a poller to wait until the entry is durably committed.

+```python

+sample_entry = {"contents": "Hello world!"}

+ledger_entry_poller = ledger_client.begin_create_ledger_entry(

+ entry=sample_entry

+)

+ledger_entry_result = ledger_entry_poller.result()

+```

+Querying an older ledger entry requires the ledger to read the entry from disk and validate it. You can use the `begin_get_ledger_entry` function to create a poller that will wait until the queried entry is in a ready state to view.

+```python

+get_entry_poller = ledger_client.begin_get_ledger_entry(

+ transaction_id=ledger_entry_result['transactionId']

+)

+entry = get_entry_poller.result()

+```

+- [Verify Azure Confidential Ledger write transaction receipts](verify-write-transaction-receipts.md)

+ Title: Verify Azure Confidential Ledger write transaction receipts

+description: Verify Azure Confidential Ledger write transaction receipts

+# Verify Azure Confidential Ledger write transaction receipts

+An Azure Confidential Ledger write transaction receipt represents a cryptographic Merkle proof that the corresponding write transaction has been globally committed by the CCF network. Azure Confidential Ledger users can get a receipt over a committed write transaction at any point in time to verify that the corresponding write operation was successfully recorded into the immutable ledger.

+For more information about Azure Confidential Ledger write transaction receipts, see the [dedicated article](write-transaction-receipts.md).

+## Receipt verification steps

+A write transaction receipt can be verified following a specific set of steps outlined in the following subsections. The same steps are outlined in the [CCF Documentation](https://microsoft.github.io/CCF/main/use_apps/verify_tx.html#receipt-verification).

+### Leaf node computation

+The first step is to compute the SHA-256 hash of the leaf node in the Merkle Tree corresponding to the committed transaction. A leaf node is composed of the ordered concatenation of the following fields that can be found in an Azure Confidential Ledger receipt, under `leaf_components`:

+1. `write_set_digest`

+2. SHA-256 digest of `commit_evidence`

+3. `claims_digest` fields

+These values need to be concatenated as arrays of bytes: both `write_set_digest` and `claims_digest` would need to be converted from strings of hexadecimal digits to arrays of bytes; on the other hand, the hash of `commit_evidence` (as an array of bytes) can be obtained by applying the SHA-256 hash function over the UTF-8 encoded `commit_evidence` string.

+Similarly, the leaf node hash digest can be computed by applying the SHA-256 hash function over the result concatenation of the resulting bytes.

+### Root node computation

+The second step is to compute the SHA-256 hash of the root of the Merkle Tree at the time the transaction was committed. The computation is done by iteratively concatenating and hashing the result of the previous iteration (starting from the leaf node hash computed in the previous step) with the ordered nodes' hashes provided in the `proof` field of a receipt. The `proof` list is provided as an ordered list and its elements need to be iterated in the given order.

+The concatenation needs to be done on the bytes representation with respect to the relative order indicated in the objects provided in the `proof` field (either `left` or `right`).

+* If the key of the current element in `proof` is `left`, then the result of the previous iteration should be appended to the current element value.

+* If the key of the current element in `proof` is `right`, then the result of the previous iteration should be prepended to the current element value.

+After each concatenation, the SHA-256 function needs to be applied in order to obtain the input for the next iteration. This process follows the standard steps to compute the root node of a [Merkle Tree](https://en.wikipedia.org/wiki/Merkle_tree) data structure given the required nodes for the computation.

+### Verify signature over root node

+The third step is to verify that the cryptographic signature produced over the root node hash is valid using the signing node certificate in the receipt. The verification process follows the standard steps for digital signature verification for messages signed using the [Elliptic Curve Digital Signature Algorithm (ECDSA)](https://wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm). More specifically, the steps are:

+1. Decode the base64 string `signature` into an array of bytes.

+2. Extract the ECDSA public key from the signing node certificate `cert`.

+3. Verify that the signature over the root of the Merkle Tree (computed using the instructions in the previous subsection) is authentic using the extracted public key from the previous step. This step effectively corresponds to a standard [digital signature](https://wikipedia.org/wiki/Digital_signature) verification process using ECDSA. There are many libraries in the most popular programming languages that allow verifying an ECDSA signature using a public key certificate over some data (for example, [ecdsa](https://pypi.org/project/ecdsa/) for Python).

+### Verify signing node certificate endorsement

+In addition to the above, it's also required to verify that the signing node certificate is endorsed (that is, signed) by the current ledger certificate. This step doesn't depend on the other three previous steps and can be carried out independently from the others.

+It's possible that the current service identity that issued the receipt is different from the one that endorsed the signing node (for example, due to a certificate renewal). In this case, it's required to verify the chain of certificates trust from the signing node certificate (that is, the `cert` field in the receipt) up to the trusted root Certificate Authority (CA) (that is, the current service identity certificate) through other previous service identities (that is, the `service_endorsements` list field in the receipt). The `service_endorsements` list is provided as an ordered list from the oldest to the latest service identity.

+Certificate endorsement need to be verified for the entire chain and follows the exact same digital signature verification process outlined in the previous subsection. There are popular open-source cryptographic libraries (for example, [OpenSSL](https://www.openssl.org/)) that can be typically used to carry out a certificate endorsement step.

+### More resources

+For more information about the content of an Azure Confidential Ledger write transaction receipt and explanation of each field, see the [dedicated article](write-transaction-receipts.md#write-transaction-receipt-content). The [CCF documentation](https://microsoft.github.io/CCF) also contains more information about receipt verification and other related resources at the following links:

+* [Receipt Verification](https://microsoft.github.io/CCF/main/use_apps/verify_tx.html#receipt-verification)

+* [CCF Glossary](https://microsoft.github.io/CCF/main/overview/glossary.html)

+* [Merkle Tree](https://microsoft.github.io/CCF/main/architecture/merkle_tree.html)

+* [Cryptography](https://microsoft.github.io/CCF/main/architecture/cryptography.html)

+* [Certificates](https://microsoft.github.io/CCF/main/operations/certificates.html)

+## Verify write transaction receipts

+### Setup and pre-requisites

+For reference purposes, we provide sample code in Python to fully verify Azure Confidential Ledger write transaction receipts following the steps outlined above.

+To run the full verification algorithm, the current service network certificate and a write transaction receipt from a running Confidential Ledger resource are required. Refer to [this article](write-transaction-receipts.md#get-write-transaction-receipts) for details on how to fetch a write transaction receipt and the service certificate from a Confidential Ledger instance.

+### Code walkthrough

+The following code can be used to initialize the required objects and run the receipt verification algorithm. A separate utility (`verify_receipt`) is used to run the full verification algorithm, and accepts input the content of the `receipt` field in a `GET_RECEIPT` response as a dictionary and the service certificate as a simple string. The function throws an exception if the receipt isn't valid or if any error was encountered during the processing.

+It's assumed that both the receipt and the service certificate can be loaded from files. Make sure to update both the `service_certificate_file_name` and `receipt_file_name` constants with the respective files names of the service certificate and receipt you would like to verify.

+```python

+import json

+# Constants

+service_certificate_file_name = "<your-service-certificate-file>"

+receipt_file_name = "<your-receipt-file>"

+# Use the receipt and the service identity to verify the receipt content

+with open(service_certificate_file_name, "r") as service_certificate_file, open(

+ΓÇ» ΓÇ» receipt_file_name, "r"

+) as receipt_file:

+ # Load relevant files content

+ΓÇ» ΓÇ» receipt = json.loads(receipt_file.read())["receipt"]

+ΓÇ» ΓÇ» service_certificate_cert = service_certificate_file.read()

+ΓÇ» ΓÇ» try:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» verify_receipt(receipt, service_certificate_cert)

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» print("Receipt verification succeeded")

+ΓÇ» ΓÇ» except Exception as e:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» print("Receipt verification failed")

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» # Raise caught exception to look at the error stack

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» raise e

+```

+As the verification process requires some cryptographic and hashing primitives, the following libraries are used to facilitate the computation.

+* The [CCF Python library](https://microsoft.github.io/CCF/main/audit/python_library.html): the module provides a set of tools for receipt verification.

+* The [Python cryptography library](https://cryptography.io/en/latest/): a widely used library that includes various cryptographic algorithms and primitives.

+* The [hashlib module](https://docs.python.org/3/library/hashlib.html), part of the Python standard library: a module that provides a common interface for popular hashing algorithms.

+```python

+from ccf.receipt import verify, check_endorsements, root

+from cryptography.x509 import load_pem_x509_certificate, Certificate

+from hashlib import sha256

+from typing import Dict, List, Any

+```

+Inside the `verify_receipt` function, we check that the given receipt is valid and contains all the required fields.

+```python

+# Check that all the fields are present in the receipt

+assert "cert" in receipt

+assert "is_signature_transaction" in receipt

+assert "leaf_components" in receipt

+assert "claims_digest" in receipt["leaf_components"]

+assert "commit_evidence" in receipt["leaf_components"]

+assert "write_set_digest" in receipt["leaf_components"]

+assert "proof" in receipt

+assert "service_endorsements" in receipt

+assert "signature" in receipt

+```

+We initialize the variables that are going to be used in the rest of the program.

+```python

+# Set the variables

+node_cert_pem = receipt["cert"]

+is_signature_transaction = receipt["is_signature_transaction"]

+claims_digest_hex = receipt["leaf_components"]["claims_digest"]

+commit_evidence_str = receipt["leaf_components"]["commit_evidence"]

+write_set_digest_hex = receipt["leaf_components"]["write_set_digest"]

+proof_list = receipt["proof"]

+service_endorsements_certs_pem = receipt["service_endorsements"]

+root_node_signature = receipt["signature"]

+```

+In the following snipper, we check that the receipt isn't related to a signature transaction. Receipts for signature transactions aren't allowed for Confidential Ledger.

+```python

+# Check that this receipt is not for a signature transaction

+assert not is_signature_transaction

+```

+We can load the PEM certificates for the service identity, the signing node, and the endorsements certificates from previous service identities using the cryptography library.

+```python

+# Load service and node PEM certificates

+service_cert = load_pem_x509_certificate(service_cert_pem.encode())

+node_cert = load_pem_x509_certificate(node_cert_pem.encode())

+# Load service endorsements PEM certificates

+service_endorsements_certs = [

+ΓÇ» ΓÇ» load_pem_x509_certificate(pem.encode())

+ΓÇ» ΓÇ» for pem in service_endorsements_certs_pem

+]

+```

+The first step of the verification process is to compute the digest of the leaf node.

+```python

+# Compute leaf of the Merkle Tree corresponding to our transaction

+leaf_node_hex = compute_leaf_node(

+ΓÇ» ΓÇ» claims_digest_hex, commit_evidence_str, write_set_digest_hex

+)

+```

+The `compute_leaf_node` function accepts as parameters the leaf components of the receipt (the `claims_digest`, the `commit_evidence`, and the `write_set_digest`) and returns the leaf node hash in hexadecimal form.

+As detailed above, we compute the digest of `commit_evidence` (using the SHA256 `hashlib` function). Then, we convert both `write_set_digest` and `claims_digest` into arrays of bytes. Finally, we concatenate the three arrays, and we digest the result using the SHA256 function.

+```python

+def compute_leaf_node(

+ΓÇ» ΓÇ» claims_digest_hex: str, commit_evidence_str: str, write_set_digest_hex: str

+) -> str:

+ΓÇ» ΓÇ» """Function to compute the leaf node associated to a transaction

+ΓÇ» ΓÇ» given its claims digest, commit evidence, and write set digest."""

+ΓÇ» ΓÇ» # Digest commit evidence string

+ΓÇ» ΓÇ» commit_evidence_digest = sha256(commit_evidence_str.encode()).digest()

+ΓÇ» ΓÇ» # Convert write set digest to bytes

+ΓÇ» ΓÇ» write_set_digest = bytes.fromhex(write_set_digest_hex)

+ΓÇ» ΓÇ» # Convert claims digest to bytes

+ΓÇ» ΓÇ» claims_digest = bytes.fromhex(claims_digest_hex)

+ΓÇ» ΓÇ» # Create leaf node by hashing the concatenation of its three components

+ΓÇ» ΓÇ» # as bytes objects in the following order:

+ΓÇ» ΓÇ» # 1. write_set_digest

+ΓÇ» ΓÇ» # 2. commit_evidence_digest

+ΓÇ» ΓÇ» # 3. claims_digest

+ΓÇ» ΓÇ» leaf_node_digest = sha256(

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» write_set_digest + commit_evidence_digest + claims_digest

+ΓÇ» ΓÇ» ).digest()

+ΓÇ» ΓÇ» # Convert the result into a string of hexadecimal digits

+ΓÇ» ΓÇ» return leaf_node_digest.hex()

+```

+After computing the leaf, we can compute the root of the Merkle tree.

+```python

+# Compute root of the Merkle Tree

+root_node = root(leaf_node_hex, proof_list)

+```

+We use the function `root` provided as part of the CCF Python library. The function successively concatenates the result of the previous iteration with a new element from `proof`, digests the concatenation, and then repeats the step for every element in `proof` with the previously computed digest. The concatenation needs to respect the order of the nodes in the Merkle Tree to make sure the root is recomputed correctly.

+```python

+def root(leaf: str, proof: List[dict]):

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» Recompute root of Merkle tree from a leaf and a proof of the form:

+ΓÇ» ΓÇ» [{"left": digest}, {"right": digest}, ...]

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» current = bytes.fromhex(leaf)

+ΓÇ» ΓÇ» for n in proof:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» if "left" in n:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» current = sha256(bytes.fromhex(n["left"]) + current).digest()

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» else:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» current = sha256(current + bytes.fromhex(n["right"])).digest()

+ΓÇ» ΓÇ» return current.hex()

+```

+After computing the root node hash, we can verify the signature contained in the receipt over the root to validate that the signature is correct.

+```python

+# Verify signature of the signing node over the root of the tree

+verify(root_node, root_node_signature, node_cert)

+```

+Similarly, the CCF library provides a function `verify` to do this verification. We use the ECDSA public key of the signing node certificate to verify the signature over the root of the tree.

+```python

+def verify(root: str, signature: str, cert: Certificate):

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» Verify signature over root of Merkle Tree

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» sig = base64.b64decode(signature)

+ΓÇ» ΓÇ» pk = cert.public_key()

+ΓÇ» ΓÇ» assert isinstance(pk, ec.EllipticCurvePublicKey)

+ΓÇ» ΓÇ» pk.verify(

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» sig,

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» bytes.fromhex(root),

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» ec.ECDSA(utils.Prehashed(hashes.SHA256())),

+ΓÇ» ΓÇ» )

+```

+The last step of receipt verification is validating the certificate that was used to sign the root of the Merkle tree.

+```python

+# Verify node certificate is endorsed by the service certificates through endorsements

+check_endorsements(node_cert, service_cert, service_endorsements_certs)

+```

+Likewise, we can use the CCF utility `check_endorsements` to validate that the certificate of the signing node is endorsed by the service identity. The certificate chain could be composed of previous service certificates, so we should validate that the endorsement is applied transitively if `service_endorsements` isn't an empty list.

+```python

+def check_endorsement(endorsee: Certificate, endorser: Certificate):

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» Check endorser has endorsed endorsee

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» digest_algo = endorsee.signature_hash_algorithm

+ΓÇ» ΓÇ» assert digest_algo

+ΓÇ» ΓÇ» digester = hashes.Hash(digest_algo)

+ΓÇ» ΓÇ» digester.update(endorsee.tbs_certificate_bytes)

+ΓÇ» ΓÇ» digest = digester.finalize()

+ΓÇ» ΓÇ» endorser_pk = endorser.public_key()

+ΓÇ» ΓÇ» assert isinstance(endorser_pk, ec.EllipticCurvePublicKey)

+ΓÇ» ΓÇ» endorser_pk.verify(

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» endorsee.signature, digest, ec.ECDSA(utils.Prehashed(digest_algo))

+ΓÇ» ΓÇ» )

+def check_endorsements(

+ΓÇ» ΓÇ» node_cert: Certificate, service_cert: Certificate, endorsements: List[Certificate]

+):

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» Check a node certificate is endorsed by a service certificate, transitively through a list of endorsements.

+ΓÇ» ΓÇ» """

+ΓÇ» ΓÇ» cert_i = node_cert

+ΓÇ» ΓÇ» for endorsement in endorsements:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» check_endorsement(cert_i, endorsement)

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» cert_i = endorsement

+ΓÇ» ΓÇ» check_endorsement(cert_i, service_cert)

+```

+As an alternative, we could also validate the certificate by using the OpenSSL library using a similar method.

+```python

+from OpenSSL.crypto import (

+ΓÇ» ΓÇ» X509,

+ΓÇ» ΓÇ» X509Store,

+ΓÇ» ΓÇ» X509StoreContext,

+)

+def verify_openssl_certificate(

+ΓÇ» ΓÇ» node_cert: Certificate,

+ΓÇ» ΓÇ» service_cert: Certificate,

+ΓÇ» ΓÇ» service_endorsements_certs: List[Certificate],

+) -> None:

+ΓÇ» ΓÇ» """Verify that the given node certificate is a valid OpenSSL certificate through

+ΓÇ» ΓÇ» the service certificate and a list of endorsements certificates."""

+ΓÇ» ΓÇ» store = X509Store()

+ΓÇ» ΓÇ» # pyopenssl does not support X509_V_FLAG_NO_CHECK_TIME. For recovery of expired

+ΓÇ» ΓÇ» # services and historical receipts, we want to ignore the validity time. 0x200000

+ΓÇ» ΓÇ» # is the bitmask for this option in more recent versions of OpenSSL.

+ΓÇ» ΓÇ» X509_V_FLAG_NO_CHECK_TIME = 0x200000

+ΓÇ» ΓÇ» store.set_flags(X509_V_FLAG_NO_CHECK_TIME)

+ΓÇ» ΓÇ» # Add service certificate to the X.509 store

+ΓÇ» ΓÇ» store.add_cert(X509.from_cryptography(service_cert))

+ΓÇ» ΓÇ» # Prepare X.509 endorsement certificates

+ΓÇ» ΓÇ» certs_chain = [X509.from_cryptography(cert) for cert in service_endorsements_certs]

+ΓÇ» ΓÇ» # Prepare X.509 node certificate

+ΓÇ» ΓÇ» node_cert_pem = X509.from_cryptography(node_cert)

+ΓÇ» ΓÇ» # Create X.509 store context and verify its certificate

+ΓÇ» ΓÇ» ctx = X509StoreContext(store, node_cert_pem, certs_chain)

+ΓÇ» ΓÇ» ctx.verify_certificate()

+```

+### Sample code

+The full sample code used in the code walkthrough can be found below.

+#### Main program

+```python

+import json

+# Use the receipt and the service identity to verify the receipt content

+with open("network_certificate.pem", "r") as service_certificate_file, open(

+ΓÇ» ΓÇ» "receipt.json", "r"

+) as receipt_file:

+ΓÇ» ΓÇ» # Load relevant files content

+ΓÇ» ΓÇ» receipt = json.loads(receipt_file.read())["receipt"]

+ΓÇ» ΓÇ» service_certificate_cert = service_certificate_file.read()

+ΓÇ» ΓÇ» try:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» verify_receipt(receipt, service_certificate_cert)

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» print("Receipt verification succeeded")

+ΓÇ» ΓÇ» except Exception as e:

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» print("Receipt verification failed")

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» # Raise caught exception to look at the error stack

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» raise e

+```

+#### Receipt verification

+```python

+from cryptography.x509 import load_pem_x509_certificate, Certificate

+from hashlib import sha256

+from typing import Dict, List, Any

+from OpenSSL.crypto import (

+ΓÇ» ΓÇ» X509,

+ΓÇ» ΓÇ» X509Store,

+ΓÇ» ΓÇ» X509StoreContext,

+)

+from ccf.receipt import root, verify, check_endorsements

+def verify_receipt(receipt: Dict[str, Any], service_cert_pem: str) -> None:

+ΓÇ» ΓÇ» """Function to verify that a given write transaction receipt is valid based

+ΓÇ» ΓÇ» on its content and the service certificate.

+ΓÇ» ΓÇ» Throws an exception if the verification fails."""

+ΓÇ» ΓÇ» # Check that all the fields are present in the receipt

+ΓÇ» ΓÇ» assert "cert" in receipt

+ΓÇ» ΓÇ» assert "is_signature_transaction" in receipt

+ΓÇ» ΓÇ» assert "leaf_components" in receipt

+ΓÇ» ΓÇ» assert "claims_digest" in receipt["leaf_components"]

+ΓÇ» ΓÇ» assert "commit_evidence" in receipt["leaf_components"]

+ΓÇ» ΓÇ» assert "write_set_digest" in receipt["leaf_components"]

+ΓÇ» ΓÇ» assert "proof" in receipt

+ΓÇ» ΓÇ» assert "service_endorsements" in receipt

+ΓÇ» ΓÇ» assert "signature" in receipt

+ΓÇ» ΓÇ» # Set the variables

+ΓÇ» ΓÇ» node_cert_pem = receipt["cert"]

+ΓÇ» ΓÇ» is_signature_transaction = receipt["is_signature_transaction"]

+ΓÇ» ΓÇ» claims_digest_hex = receipt["leaf_components"]["claims_digest"]

+ΓÇ» ΓÇ» commit_evidence_str = receipt["leaf_components"]["commit_evidence"]

+ΓÇ» ΓÇ» write_set_digest_hex = receipt["leaf_components"]["write_set_digest"]

+ΓÇ» ΓÇ» proof_list = receipt["proof"]

+ΓÇ» ΓÇ» service_endorsements_certs_pem = receipt["service_endorsements"]

+ΓÇ» ΓÇ» root_node_signature = receipt["signature"]

+ΓÇ» ΓÇ» # Check that this receipt is not for a signature transaction

+ΓÇ» ΓÇ» assert not is_signature_transaction

+ΓÇ» ΓÇ» # Load service and node PEM certificates

+ΓÇ» ΓÇ» service_cert = load_pem_x509_certificate(service_cert_pem.encode())

+ΓÇ» ΓÇ» node_cert = load_pem_x509_certificate(node_cert_pem.encode())

+ΓÇ» ΓÇ» # Load service endorsements PEM certificates

+ΓÇ» ΓÇ» service_endorsements_certs = [

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» load_pem_x509_certificate(pem.encode())

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» for pem in service_endorsements_certs_pem

+ΓÇ» ΓÇ» ]

+ΓÇ» ΓÇ» # Compute leaf of the Merkle Tree

+ΓÇ» ΓÇ» leaf_node_hex = compute_leaf_node(

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» claims_digest_hex, commit_evidence_str, write_set_digest_hex

+ΓÇ» ΓÇ» )

+ΓÇ» ΓÇ» # Compute root of the Merkle Tree

+ΓÇ» ΓÇ» root_node = root(leaf_node_hex, proof_list)

+ΓÇ» ΓÇ» # Verify signature of the signing node over the root of the tree

+ΓÇ» ΓÇ» verify(root_node, root_node_signature, node_cert)

+ΓÇ» ΓÇ» # Verify node certificate is endorsed by the service certificates through endorsements

+ΓÇ» ΓÇ» check_endorsements(node_cert, service_cert, service_endorsements_certs)

+ΓÇ» ΓÇ» # Alternative: Verify node certificate is endorsed by the service certificates through endorsements

+ΓÇ» ΓÇ» verify_openssl_certificate(node_cert, service_cert, service_endorsements_certs)

+def compute_leaf_node(

+ΓÇ» ΓÇ» claims_digest_hex: str, commit_evidence_str: str, write_set_digest_hex: str

+) -> str:

+ΓÇ» ΓÇ» """Function to compute the leaf node associated to a transaction

+ΓÇ» ΓÇ» given its claims digest, commit evidence, and write set digest."""

+ΓÇ» ΓÇ» # Digest commit evidence string

+ΓÇ» ΓÇ» commit_evidence_digest = sha256(commit_evidence_str.encode()).digest()

+ΓÇ» ΓÇ» # Convert write set digest to bytes

+ΓÇ» ΓÇ» write_set_digest = bytes.fromhex(write_set_digest_hex)

+ΓÇ» ΓÇ» # Convert claims digest to bytes

+ΓÇ» ΓÇ» claims_digest = bytes.fromhex(claims_digest_hex)

+ΓÇ» ΓÇ» # Create leaf node by hashing the concatenation of its three components

+ΓÇ» ΓÇ» # as bytes objects in the following order:

+ΓÇ» ΓÇ» # 1. write_set_digest

+ΓÇ» ΓÇ» # 2. commit_evidence_digest

+ΓÇ» ΓÇ» # 3. claims_digest

+ΓÇ» ΓÇ» leaf_node_digest = sha256(

+ΓÇ» ΓÇ» ΓÇ» ΓÇ» write_set_digest + commit_evidence_digest + claims_digest

+ΓÇ» ΓÇ» ).digest()

+ΓÇ» ΓÇ» # Convert the result into a string of hexadecimal digits

+ΓÇ» ΓÇ» return leaf_node_digest.hex()

+def verify_openssl_certificate(

+ΓÇ» ΓÇ» node_cert: Certificate,

+ΓÇ» ΓÇ» service_cert: Certificate,

+ΓÇ» ΓÇ» service_endorsements_certs: List[Certificate],

+) -> None:

+ΓÇ» ΓÇ» """Verify that the given node certificate is a valid OpenSSL certificate through

+ΓÇ» ΓÇ» the service certificate and a list of endorsements certificates."""

+ΓÇ» ΓÇ» store = X509Store()

+ΓÇ» ΓÇ» # pyopenssl does not support X509_V_FLAG_NO_CHECK_TIME. For recovery of expired

+ΓÇ» ΓÇ» # services and historical receipts, we want to ignore the validity time. 0x200000

+ΓÇ» ΓÇ» # is the bitmask for this option in more recent versions of OpenSSL.

+ΓÇ» ΓÇ» X509_V_FLAG_NO_CHECK_TIME = 0x200000

+ΓÇ» ΓÇ» store.set_flags(X509_V_FLAG_NO_CHECK_TIME)

+ΓÇ» ΓÇ» # Add service certificate to the X.509 store

+ΓÇ» ΓÇ» store.add_cert(X509.from_cryptography(service_cert))

+ΓÇ» ΓÇ» # Prepare X.509 endorsement certificates

+ΓÇ» ΓÇ» certs_chain = [X509.from_cryptography(cert) for cert in service_endorsements_certs]

+ΓÇ» ΓÇ» # Prepare X.509 node certificate

+ΓÇ» ΓÇ» node_cert_pem = X509.from_cryptography(node_cert)

+ΓÇ» ΓÇ» # Create X.509 store context and verify its certificate

+ΓÇ» ΓÇ» ctx = X509StoreContext(store, node_cert_pem, certs_chain)

+ΓÇ» ΓÇ» ctx.verify_certificate()

+```

+## Next steps

+* [Azure Confidential Ledger write transaction receipts](write-transaction-receipts.md)

+* [Overview of Microsoft Azure confidential ledger](overview.md)

+* [Azure confidential ledger architecture](architecture.md)

+ Title: Azure Confidential Ledger write transaction receipts

+description: Azure Confidential Ledger write transaction receipts

+# Azure Confidential Ledger write transaction receipts

+To enforce transaction integrity guarantees, an Azure Confidential Ledger uses a [Merkle tree](https://en.wikipedia.org/wiki/Merkle_tree) data structure to record the hash of all transactions blocks that are appended to the immutable ledger. After a write transaction is committed, Azure Confidential Ledger users can get a cryptographic Merkle proof, or receipt, over the entry produced in a Confidential Ledger to verify that the write operation was correctly saved. A write transaction receipt is proof that the system has committed the corresponding transaction and can be used to verify that the entry has been effectively appended to the ledger.

+More details about how a Merkle Tree is used in a Confidential Ledger can be found in the [CCF documentation](https://microsoft.github.io/CCF/main/architecture/merkle_tree.html).

+## Get write transaction receipts

+### Setup and pre-requisites

+Azure Confidential Ledger users can get a receipt for a specific transaction by using the [Azure Confidential Ledger client library](quickstart-python.md#use-the-data-plane-client-library). The following example shows how to get a write receipt using the [client library for Python](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/confidentialledger/azure-confidentialledger), but the steps are the same with any other supported SDK for Azure Confidential Ledger.

+We assume that a Confidential Ledger resource has already been created using the Azure Confidential Ledger Management library. If you don't have an existing ledger resource yet, create one using the [following instructions](quickstart-python.md#use-the-control-plane-client-library).

+### Code walkthrough

+We start by setting up the imports for our Python program.

+```python

+import json

+# Import the Azure authentication library

+from azure.identity import DefaultAzureCredential

+# Import the Confidential Ledger Data Plane SDK

+from azure.confidentialledger import ConfidentialLedgerClient

+from azure.confidentialledger.certificate import ConfidentialLedgerCertificateClient

+```

+The following are the constant values used to set up the Azure Confidential Ledger client. Make sure to update the `ledger_name` constant with the unique name of your Confidential Ledger resource.

+```python

+# Constants for our program

+ledger_name = "<your-unique-ledger-name>"

+identity_url = "https://identity.confidential-ledger.core.azure.com"

+ledger_url = "https://" + ledger_name + ".confidential-ledger.azure.com"

+```

+We authenticate using the [DefaultAzureCredential class](/python/api/azure-identity/azure.identity.defaultazurecredential).

+```python

+# Setup authentication

+credential = DefaultAzureCredential()

+```

+Then, we get and save the Confidential Ledger service certificate using the Certificate client from the [Confidential Ledger Identity URL](https://identity.confidential-ledger.core.azure.com/ledgerIdentity). The service certificate is a network identity public key certificate used as root of trust for [TLS](https://microsoft.github.io/CCF/main/overview/glossary.html#term-TLS) server authentication. In other words, it's used as the Certificate Authority (CA) for establishing a TLS connection with any of the nodes in the CCF network.

+```python

+# Create a Certificate client and use it to

+# get the service identity for our ledger

+identity_client = ConfidentialLedgerCertificateClient(identity_url)

+network_identity = identity_client.get_ledger_identity(

+     ledger_id=ledger_name

+)

+# Save network certificate into a file for later use

+ledger_tls_cert_file_name = "network_certificate.pem"

+with open(ledger_tls_cert_file_name, "w") as cert_file:

+ΓÇ» ΓÇ» cert_file.write(network_identity["ledgerTlsCertificate"])

+```

+Next, we can use our credentials, the fetched network certificate, and our unique ledger URL to create a Confidential Ledger client.

+```python

+# Create Confidential Ledger client

+ledger_client = ConfidentialLedgerClient(

+     endpoint=ledger_url,

+     credential=credential,

+     ledger_certificate_path=ledger_tls_cert_file_name

+)

+```

+Using the Confidential Ledger client, we can run any supported operations on an Azure Confidential Ledger instance. For example, we can append a new entry to the ledger and wait for corresponding write transaction to be committed.

+ ```python

+# The method begin_create_ledger_entry returns a poller that

+# we can use to wait for the transaction to be committed

+create_entry_poller = ledger_client.begin_create_ledger_entry(

+ΓÇ» ΓÇ» {"contents": "Hello World!"}

+)

+create_entry_result = create_entry_poller.result()

+```

+After the transaction is committed, we can use the client to get a receipt over the entry appended to the ledger in the previous step using the respective [transaction ID](https://microsoft.github.io/CCF/main/overview/glossary.html#term-Transaction-ID).

+```python

+# The method begin_get_receipt returns a poller that

+# we can use to wait for the receipt to be available by the system

+get_receipt_poller = ledger_client.begin_get_receipt(

+ΓÇ» ΓÇ» create_entry_result["transactionId"]

+)

+get_receipt_result = get_receipt_poller.result()

+```

+### Sample code

+The full sample code used in the code walkthrough can be found below.

+```python

+import json

+# Import the Azure authentication library

+from azure.identity import DefaultAzureCredential

+# Import the Confidential Ledger Data Plane SDK

+from azure.confidentialledger import ConfidentialLedgerClient

+from azure.confidentialledger.certificate import ConfidentialLedgerCertificateClient

+from receipt_verification import verify_receipt

+# Constants

+ledger_name = "<your-unique-ledger-name>"

+identity_url = "https://identity.confidential-ledger.core.azure.com"

+ledger_url = "https://" + ledger_name + ".confidential-ledger.azure.com"

+# Setup authentication

+credential = DefaultAzureCredential()

+# Create Ledger Certificate client and use it to

+# retrieve the service identity for our ledger

+identity_client = ConfidentialLedgerCertificateClient(identity_url)

+network_identity = identity_client.get_ledger_identity(ledger_id=ledger_name)

+# Save network certificate into a file for later use

+ledger_tls_cert_file_name = "network_certificate.pem"

+with open(ledger_tls_cert_file_name, "w") as cert_file:

+ΓÇ» ΓÇ» cert_file.write(network_identity["ledgerTlsCertificate"])

+# Create Confidential Ledger client

+ledger_client = ConfidentialLedgerClient(

+ΓÇ» ΓÇ» endpoint=ledger_url,

+ΓÇ» ΓÇ» credential=credential,

+ΓÇ» ΓÇ» ledger_certificate_path=ledger_tls_cert_file_name,

+)

+# The method begin_create_ledger_entry returns a poller that

+# we can use to wait for the transaction to be committed

+create_entry_poller = ledger_client.begin_create_ledger_entry(

+ΓÇ» ΓÇ» {"contents": "Hello World!"}

+)

+create_entry_result = create_entry_poller.result()

+# The method begin_get_receipt returns a poller that

+# we can use to wait for the receipt to be available by the system

+get_receipt_poller = ledger_client.begin_get_receipt(

+ΓÇ» ΓÇ» create_entry_result["transactionId"]

+)

+get_receipt_result = get_receipt_poller.result()

+# Save fetched receipt into a file

+with open("receipt.json", "w") as receipt_file:

+ΓÇ» ΓÇ» receipt_file.write(json.dumps(get_receipt_result, sort_keys=True, indent=2))

+```

+## Write transaction receipt content

+Here's an example of a JSON response payload returned by an Azure Confidential Ledger instance when calling the `GET_RECEIPT` endpoint.

+```json

+{

+ "receipt": {

+ "cert": "--BEGIN CERTIFICATE--\nMIIB0jCCAXmgAwIBAgIQPxdrEtGY+SggPHETin1XNzAKBggqhkjOPQQDAjAWMRQw\nEgYDVQQDDAtDQ0YgTmV0d29yazAeFw0yMjA3MjAxMzUzMDFaFw0yMjEwMTgxMzUz\nMDBaMBMxETAPBgNVBAMMCENDRiBOb2RlMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD\nQgAEWy81dFeEZ79gVJnfHiPKjZ54fZvDcFlntFwJN8Wf6RZa3PaV5EzwAKHNfojj\noXT4xNkJjURBN7q+1iE/vvc+rqOBqzCBqDAJBgNVHRMEAjAAMB0GA1UdDgQWBBQS\nwl7Hx2VkkznJNkVZUbZy+TOR/jAfBgNVHSMEGDAWgBTrz538MGI/SdV8k8EiJl5z\nfl3mBTBbBgNVHREEVDBShwQK8EBegjNhcGljY2lvbmUtdGVzdC1sZWRnZXIuY29u\nZmlkZW50aWFsLWxlZGdlci5henVyZS5jb22CFWFwaWNjaW9uZS10ZXN0LWxlZGdl\ncjAKBggqhkjOPQQDAgNHADBEAiAsGawDcYcH/KzF2iK9Ldx/yABUoYSNti2Cyxum\n9RRNKAIgPB/XGh/FQS3nmZLExgBVXkDYdghQu/NCY/hHjQ9AvWg=\n--END CERTIFICATE--\n",

+ "is_signature_transaction": false,

+ "leaf_components": {

+ "claims_digest": "0000000000000000000000000000000000000000000000000000000000000000",

+ "commit_evidence": "ce:2.40:f36ffe2930ec95d50ebaaec26e2bec56835abd051019eb270f538ab0744712a4",

+ "write_set_digest": "8452624d10bdd79c408c0f062a1917aa96711ea062c508c745469636ae1460be"

+ },

+ "node_id": "70e995887e3e6b73c80bc44f9fbb6e66b9f644acaddbc9c0483cfc17d77af24f",

+ "proof": [

+ {

+ "left": "b78230f9abb27b9b803a9cae4e4cec647a3be1000fc2241038867792d59d4bc1"

+ },

+ {

+ "left": "a2835d4505b8b6b25a0c06a9c8e96a5204533ceac1edf2b3e0e4dece78fbaf35"

+ }

+ ],

+ "service_endorsements": [],

+ "signature": "MEUCIQCjtMqk7wOtUTgqlHlCfWRqAco+38roVdUcRv7a1G6pBwIgWKpCSdBmhzgEdwguUW/Cj/Z5bAOA8YHSoLe8KzrlqK8="

+ },

+ "state": "Ready",

+ "transactionId": "2.40"

+}

+```

+The JSON response contains the following fields at the root level.

+* **receipt**: It contains the values that can be used to verify the validity of the receipt for the corresponding write transaction.

+* **state**: The status of the returned JSON response. The following are the possible values allowed:

+

+ * `Ready`: The receipt returned in the response is available

+ * `Loading`: The receipt isn't yet available to be retrieved and the request will have to be retried

+* **transactionId**: The transaction ID associated with the write transaction receipt.

+The `receipt` field contains the following fields.

+* **cert**: String with the [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) public key certificate of the CCF node that signed the write transaction. The certificate of the signing node should always be endorsed by the service identity certificate. See also more details about how transactions get regularly signed and how the signature transactions are appended to the ledger in CCF at the following [link](https://microsoft.github.io/CCF/main/architecture/merkle_tree.html).

+* **is_signature_transaction**: Boolean value indicating whether the receipt is related to a signature transaction or not. Receipts for signature transactions can't be retrieved for Confidential Ledgers.

+* **node_id**: Hexadecimal string representing the [SHA-256](https://en.wikipedia.org/wiki/SHA-2) hash digest of the public key of the signing CCF node.

+* **leaf_components**: The components of the leaf node hash in the [Merkle Tree](https://en.wikipedia.org/wiki/Merkle_tree) that are associated to the specified transaction. A Merkle Tree is a tree data structure that records the hash of every transaction and guarantees the integrity of the ledger. For more information on how a Merkle Tree is used in CCF, see the related [CCF documentation](https://microsoft.github.io/CCF/main/architecture/merkle_tree.html).

+* **proof**: List of key-value pairs representing the Merkle Tree nodes hashes that, when combined with the leaf node hash corresponding to the given transaction, allow the recomputation of the root hash of the tree. Thanks to the properties of a Merkle Tree, it's possible to recompute the root hash of the tree only a subset of nodes. The elements in this list are in the form of key-value pairs: keys indicate the relative position with respect to the parent node in the tree at a certain level; values are the SHA-256 hash digests of the node given, as hexadecimal strings.

+* **service_endorsements**: List of PEM-encoded certificates strings representing previous service identities certificates. It's possible that the service identity that endorsed the signing node isn't the same as the one that issued the receipt. For example, the service certificate is renewed after a disaster recovery of a Confidential Ledger. The list of past service certificates allows auditors to build the chain of trust from the CCF signing node to the current service certificate.

+* **signature**: Base64 string representing the signature of the root of the Merkle Tree at the given transaction, by the signing CCF node.

+The `leaf_components` field contains the following fields.

+* **claims_digest**: Hexadecimal string representing the SHA-256 hash digest of the [application claim](https://microsoft.github.io/CCF/main/use_apps/verify_tx.html#application-claims) attached by the Confidential Ledger application at the time the transaction was executed. Application claims are currently unsupported as the Confidential Ledger application doesn't attach any claim when executing a write transaction.

+* **commit_evidence**: A unique string produced per transaction, derived from the transaction ID and the ledger secrets. For more information about the commit evidence, see the related [CCF documentation](https://microsoft.github.io/CCF/main/use_apps/verify_tx.html#commit-evidence).

+* **write_set_digest**: Hexadecimal string representing the SHA-256 hash digest of the [Key-Value store](https://microsoft.github.io/CCF/main/build_apps/kv/https://docsupdatetracker.net/index.html), which contains all the keys and values written at the time the transaction was completed. For more information about the write set, see the related [CCF documentation](https://microsoft.github.io/CCF/main/overview/glossary.html#term-Write-Set).

+### More resources

+For more information about write transaction receipts and how CCF ensures the integrity of each transaction, see the following links:

+* [Write Receipts](https://microsoft.github.io/CCF/main/use_apps/verify_tx.html#write-receipts)

+* [Receipts](https://microsoft.github.io/CCF/main/audit/receipts.html)

+* [CCF Glossary](https://microsoft.github.io/CCF/main/overview/glossary.html)

+* [Merkle Tree](https://microsoft.github.io/CCF/main/architecture/merkle_tree.html)

+* [Cryptography](https://microsoft.github.io/CCF/main/architecture/cryptography.html)

+* [Certificates](https://microsoft.github.io/CCF/main/operations/certificates.html)

+## Next steps

+* [Verify Azure Confidential Ledger write transaction receipts](verify-write-transaction-receipts.md)

+* [Overview of Microsoft Azure confidential ledger](overview.md)

+* [Azure confidential ledger architecture](architecture.md)

+* **Registry REST API endpoint for certificates** - Azure container registry uses a wildcard SSL certificate for all subdomains. When connecting to the Azure container registry using SSL, the client must be able to download the certificate for the TLS handshake. In such cases, `azurecr.io` must also be accessible.

+[az-acr-show-endpoints]: /cli/azure/acr#az_acr_show_endpoints

+### Move data to new account

+If you desire, you can migrate your existing data from the free account to the newly created account.

+#### [NoSQL](#tab/nosql)

+1. Navigate back to the **Upgrade** page from the [Start upgrade](#start-upgrade) section of this guide. Select **Next** to move on to the third step and move your data.

+ :::image type="content" source="media/try-free/account-creation-options.png" lightbox="media/try-free/account-creation-options.png" alt-text="Screenshot of the sign-in/sign-up experience to upgrade your current account.":::

+#### [PostgreSQL](#tab/postgresql)

+1. Navigate back to the **Upgrade** page from the [Start upgrade](#start-upgrade) section of this guide. Select **Next** to move on to the third step and move your data.

+ :::image type="content" source="media/try-free/account-creation-options.png" lightbox="media/try-free/account-creation-options.png" alt-text="Screenshot of the sign-in/sign-up experience to upgrade your current account.":::

+#### [MongoDB / Cassandra / Gremlin / Table](#tab/mongodb+cassandra+gremlin+table)

+> [!IMPORTANT]

+> Data migration is not available for the APIs for MongoDB, Cassandra, Gremlin, or Table.

+Azure DDoS Protection is offered in two available SKUs, DDoS IP Protection Preview and DDoS Network Protection. For more information about the SKUs, see [SKU comparison](ddos-protection-sku-comparison.md).

+#### DDoS IP Protection Preview virtual machine architecture

+#### DDoS IP Protection Preview Windows N-tier architecture

+#### DDoS IP Protection Preview with PaaS web application architecture

+#### DDoS IP Protection Preview hub-and-spoke network

+## DDoS IP Protection Preview

+ Title: 'Quickstart: Create and configure Azure DDoS IP Protection Preview using PowerShell'

+description: Learn how to create Azure DDoS IP Protection Preview using PowerShell

+# Quickstart: Create and configure Azure DDoS IP Protection Preview using Azure PowerShell

+## Enable DDoS IP Protection Preview for a public IP address

+### Enable DDoS IP Protection Preview for an existing public IP address

+## Disable DDoS IP Protection Preview for an existing public IP address

+|Clouds:|:::image type="icon" source="./media/icons/yes-icon.png"::: Commercial clouds<br>:::image type="icon" source="./media/icons/yes-icon.png"::: National (Azure Government, Azure China 21Vianet) <br> <br> Software inventory is not currently supported in national clouds.|

+To access the software inventory, you'll need one of the following **paid** solutions:

+- [Agentless machine scanning](concept-agentless-data-collection.md) from [Defender Cloud Security Posture Management (CSPM)](concept-cloud-security-posture-management.md).

+- [Agentless machine scanning](concept-agentless-data-collection.md) from [Defender for Servers P2](defender-for-servers-introduction.md#defender-for-servers-plans).

+- [Microsoft Defender for Endpoint integration](integration-defender-for-endpoint.md) from [Defender for Servers](defender-for-servers-introduction.md).

+| Encryption: | **Azure**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Unencrypted<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Encrypted ΓÇô Azure Disk Encryption - platform-managed keys (PMK)<br>:::image type="icon" source="./media/icons/no-icon.png"::: Encrypted ΓÇô Azure Disk Encryption - customer-managed keys (CMK)<br>:::image type="icon" source="./media/icons/no-icon.png"::: Encrypted ΓÇô Disk Encryption Set - platform-managed keys (PMK)<br>:::image type="icon" source="./media/icons/no-icon.png"::: Encrypted ΓÇô Disk Encryption Set - customer-managed keys (CMK)<br><br>**AWS**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Unencrypted<br>:::image type="icon" source="./media/icons/no-icon.png"::: Encrypted |

+| Prerequisite: | Requires the [Defender Cloud Security Posture Management (CSPM) plan](concept-cloud-security-posture-management.md) to be enabled.|

+> [!NOTE]

+> Starting January 1, 2023, governance capabilities will require Defender Cloud Security Posture Management (CSPM) plan enablement.

+> Customers deciding to keep Defender CSPM plan off on scopes with governance content:

+> - Existing assignments remain as is and continue to work with no customization option or ability to create new ones.

+> - Existing rules will remain as is but wonΓÇÖt trigger new assignments creation.

+1. Navigate to **Environment settings** > **Governance rules**.

+1. Select **Create governance rule**.

+1. Select a scope to apply the rule to and use exclusions if needed. Rules for management scope (Azure management groups, AWS master accounts, GCP organizations) are applied prior to the rules on a single scope.

+1. Priority is assigned automatically after scope selection. You can override this field if needed.

+ - **By specific recommendations** - Select the specific recommendations that the rule applies to.

+1. If you don't want the resources to impact your secure score until they're overdue, select **Apply grace period**.

+> [!TIP]

+> Here are some sample use-cases for the at-scale experience:

+> - View and manage all governance rules effective in the organization using a single page.

+> - Create and apply rules on multiple scopes at once using management scopes cross cloud.

+> - Check effective rules on selected scope using the scope filter.

+| Clouds: | :::image type="icon" source="./media/icons/yes-icon.png"::: Commercial clouds<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/no-icon.png"::: Azure China 21Vianet <br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected AWS accounts <br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected GCP projects |

+- [Governance rules at scale (Preview)](#governance-rules-at-scale-preview)

+### Governance rules at scale (Preview)

+We're happy to announce the new ability to apply governance rules at scale (Preview) in Defender for Cloud.

+With this new experience, security teams are able to define governance rules in bulk for various scopes (subscriptions and connectors). Security teams can accomplish this task by using management scopes such as Azure management groups, AWS master accounts or GCP organizations.

+Additionally, the Governance rules (Preview) page presents all of the available governance rules that are effective in the organizationΓÇÖs environments.

+

+Learn more about the [new governance rules at-scale experience](governance-rules.md).

+

+> [!NOTE]

+> As of January 1, 2023, in order to experience the capabilities offered by Governance, you must have the [Defender CSPM plan](concept-cloud-security-posture-management.md) enabled on your subscription or connector.

+A DNS forwarding ruleset is a group of DNS forwarding rules (up to 25) that can be applied to one or more outbound endpoints, or linked to one or more virtual networks. This is a 1:N relationship. Rulesets are associated with a specific outbound endpoint. For more information, see [DNS forwarding rulesets](private-resolver-endpoints-rulesets.md#dns-forwarding-rulesets).

+ | **Microsoft.Graph.UserUpdated** | Triggered when a user in Azure AD is created and updated. |

+ | **Microsoft.Graph.UserDeleted** | Triggered when a user in Azure AD is permanently deleted. |

+ | **Microsoft.Graph.GroupUpdated** | Triggered when a group in Azure AD is created and updated. |

+ | **Microsoft.Graph.GroupDeleted** | Triggered when a group in Azure AD is permanently deleted. |

+> [!NOTE]

+> By default, deleting a user or a group is only a soft delete operation, which means that the user or group is marked as deleted but the user or group object still exists. Microsoft Graph sends an updated event when users are soft deleted. To permanently delete a user, navigate to the **Delete users** page in the Azure portal and select **Delete permanently**. Steps to permanently delete a group are similar.

+Azure [Event Grid](overview.md) is an intelligent event routing service that enables you to react to notifications or events from apps and services. For example, it can trigger an Azure function to process Event Hubs data that's captured to a Blob storage or Data Lake Storage. This [sample](https://github.com/Azure/azure-event-hubs/tree/master/samples/e2e/EventHubsCaptureEventGridDemo) shows you how to use Event Grid and Azure Functions to migrate captured Event Hubs data from blob storage to Azure Synapse Analytics, specifically a dedicated SQL pool.

+Replication, either through Azure Functions or Azure Stream Analytics, doesn't

+If you're using replication for disaster recovery purposes, to protect against

+For all failover scenarios, it's assumed that the required elements of the

+> Mind that Event Hubs doesn't allow for its endpoints to be

+> directly aliased with CNAME records, which means you'll use DNS as a

+`test.example.com`. For two alternate Event Hubs, you'll now create two

+[you will create a new checkpoint store](event-processor-balance-partition-load.md#checkpoint)

+The first two pattern variations are trivial and don't differ from plain

+ for whether they're in compliance with a set of rules before they may be

+ and drops the event if the event doesn't match the rule. Filtering out

+primary key looking for making the record unique.

+To scale your event processing application, you can run multiple instances of the application and have the load balanced among themselves. In the older versions, [EventProcessorHost](event-hubs-event-processor-host.md) allowed you to balance the load between multiple instances of your program and checkpoint events when receiving the events. In the newer versions (5.0 onwards), **EventProcessorClient** (.NET and Java), or **EventHubConsumerClient** (Python and JavaScript) allows you to do the same. The development model is made simpler by using events. You subscribe to the events that you're interested in by registering an event handler. If you're using the old version of the client library, see the following migration guides: [.NET](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/MigrationGuide.md), [Java](https://github.com/Azure/azure-sdk-for-jav).

+This article describes a sample scenario for using multiple instances of client `applications to read events from an event hub. It also gives you details about features of event processor client, which allows you to receive events from multiple partitions at once and load balance with other consumers that use the same event hub and consumer group.

+## Consumer application

+4. **Consume events:** While the previous three points deal with the management of the consumer, there must be code to consume events and do something useful with it. For example, aggregate it and upload it to blob storage.

+You don't need to build your own solution to meet these requirements. The Azure Event Hubs SDKs provide this functionality. In .NET or Java SDKs, you use an event processor client (`EventProcessorClient`), and in Python and JavaScript SDKs, you use `EventHubConsumerClient`. In the old version of SDK, it was the event processor host (`EventProcessorHost`) that supported these features.

+For most production scenarios, we recommend that you use the event processor client for reading and processing events. The processor client is intended to provide a robust experience for processing events across all partitions of an event hub in a performant and fault tolerant manner while providing a means to checkpoint its progress. Event processor clients can work cooperatively within the context of a consumer group for a given event hub. Clients will automatically manage distribution and balancing of work as instances become available or unavailable for the group.

+## Partition ownership

+Each event processor is given a unique identifier and claims ownership of partitions by adding or updating an entry in a checkpoint store. All event processor instances communicate with this store periodically to update its own processing state and to learn about other active instances. This data is then used to balance the load among the active processors. New instances can join the processing pool to scale up. When instances go down, either because of failures or to scale down, partition ownership is gracefully transferred to other active processors.

+| Event Hubs namespace | Event hub name | **Consumer group** | Owner | Partition ID | Last modified time |

+Each event processor instance acquires ownership of a partition and starts processing the partition from last known [checkpoint](#checkpoint). If a processor fails (VM shuts down), then other instances detect it by looking at the last modified time. Other instances try to get ownership of the partitions previously owned by the inactive instance. The checkpoint store guarantees that only one of the instances succeeds in claiming ownership of a partition. So, at any given point of time, there is at most one processor that receives events from a partition.

+When you create an event processor, you specify functions that will process events and errors. Each call to the function that processes events delivers a single event from a specific partition. It's your responsibility to handle this event. If you want to make sure the consumer processes every message at least once, you need to write your own code with retry logic. But be cautious about poisoned messages.

+## Checkpoint

+By default, the function that processes events is called sequentially for a given partition. Subsequent events and calls to this function from the same partition queue up behind the scenes as the event pump continues to run in the background on other threads. Events from different partitions can be processed concurrently and any shared state that is accessed across partitions have to be synchronized.

+> [!IMPORTANT]

+> Migration identity for Azure Front Door is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+> [!IMPORTANT]

+> Migration capability for Azure Front Door is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+## Overall decision flow

+The following diagram shows the overall decision flow:

+The decision steps are:

+1. **Available origins:** Select all origins that are enabled and returned healthy (200 OK) for the health probe.

+ - *Example: Suppose there are six origins A, B, C, D, E, and F, and among them C is unhealthy and E is disabled. The list of available origins is A, B, D, and F.*

+1. **Priority:** The top priority origins among the available ones are selected.

+ - *Example: Suppose origin A, B, and D have priority 1 and origin F has a priority of 2. Then, the selected origins will be A, B, and D.*

+1. **Latency signal (based on health probe):** Select the origins within the allowable latency range from the Front Door environment where the request arrived. This signal is based on the latency sensitivity setting on the origin group, as well as the latency of the closer origins.

+ - *Example: Suppose Front Door has measured the latency from the environment where the request arrived to origin A at 15 ms, while the latency for B is 30 ms and D is 60 ms away. If the origin group's latency sensitivity is set to 30 ms, then the lowest latency pool consists of origins A and B, because D is beyond 30 ms away from the closest origin that is A.*

+1. **Weights:** Lastly, Azure Front Door will round robin the traffic among the final selected group of origins in the ratio of weights specified.

+ - *Example: If origin A has a weight of 5 and origin B has a weight of 8, then the traffic will be distributed in the ratio of 5:8 among origins A and B.*

+If session affinity is enabled, then the first request in a session follows the flow listed above. Subsequent requests are sent to the origin selected in the first request.

+Each Front Door environment measures the origin latency separately. This means that different users in different locations are routed to the origin with the best performance for that environment.

+## <a name = "affinity"></a>Session affinity

+> [!IMPORTANT]

+> Migration capability for Azure Front Door is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+> [!IMPORTANT]

+> Upgrading Azure Front Door tier is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+[Customize HDInsight clusters with Script Actions](hdinsight-hadoop-customize-cluster-linux.md)

+description: This tutorial shows how to deploy and use and create an in-store analytics retail application in IoT Central.

+An IoT solution starts with a set of sensors capturing meaningful signals from within a retail store environment. It's reflected by different kinds of sensors on the far left of the architecture diagram above.

+1. Select **Customization > Appearance**.

+1. Use the **Change** button to choose an image to upload as the masthead logo. Optionally, specify a value for **Logo alt text**.

+1. Optionally, replace the default **Browser colors** by adding HTML hexadecimal color codes. For the **Header**, add *#008575*. For the **Accent**, add *#A1F3EA*.

+1. Select **Save**. After you save, the application updates the browser colors, the logo in the masthead, and the browser icon.

+1. Select **Application > Management.**

+1. Select **Change** to choose an image to upload as the application image. This image appears on the application tile in the **My Apps** page of the IoT Central application manager.

+1. Find and select the **RuuviTag Multisensor** device template in the Azure IoT device catalog.

+1. Select **Device Templates** in the left pane.

+1. Select the template for RuuviTag sensors.

+ :::image type="content" source="media/tutorial-in-store-analytics-create-app/ruuvitag-device-summary-view.png" alt-text="Screenshot showing the in-store analytics application RuuviTag device template." lightbox="media/tutorial-in-store-analytics-create-app/ruuvitag-device-summary-view.png":::

+1. Select **RuvviTag** model in the RuuviTag device template menu.

+1. Set **Minimum Length** to *2*.

+1. Select **Publish**.

+After you have created and customized device templates, it's time to add devices.

+In this section, you create a new rule that checks the maximum relative humidity level based on the RuuviTag sensor telemetry. You add an action to the rule so that if the humidity exceeds the maximum, the application sends an email.

+To create a rule:

+1. Enter *Humidity level* as the name of the rule.

+1. Choose the RuuviTag device template in **Device template**. The rule you define will apply to all sensors based on that template. Optionally, you could create a filter that would apply the rule only to a defined subset of the sensors.

+1. Choose `RelativeHumidity` as the **Telemetry**. It's the device capability that you customized in a previous step.

+1. Choose `Is greater than` as the **Operator**.

+1. Enter a typical upper range indoor humidity level for your environment as the **Value**. For example, enter *65*. You've set a condition for your rule that occurs when relative humidity in any RuuviTag real or simulated sensor exceeds this value. You may need to adjust the value up or down depending on the normal humidity range in your environment.

+1. Enter *High humidity notification* as the friendly **Display name** for the action.

+1. Select **Save** to save and activate the new rule.

+Now that you've created an Azure IoT Central condition monitoring application, here's the suggested next step:

+1. Select **Dashboard settings** and enter **Name** for your dashboard and select **Save**.

+An Azure IoT Central application dashboard consists of one or more tiles. A tile is a rectangular container for displaying content on a dashboard. You associate various types of content with tiles, and you drag, drop, and resize tiles to customize a dashboard layout. There are several types of tiles for displaying content. Image tiles contain images, and you can add a URL that enables users to click the image. Label tiles display plain text. Markdown tiles contain formatted content and let you set an image, a URL, a title, and markdown code that renders as HTML. Telemetry, property, or command tiles display device-specific data.

+1. Select **Edit** on the dashboard toolbar.

+1. Select **Image**. A dialog opens and enables you to upload a custom image.

+ :::image type="content" source="media/tutorial-in-store-analytics-customize-dashboard/brand-image-save.png" alt-text="Screenshot showing the in-store analytics application dashboard brand image tile." lightbox="media/tutorial-in-store-analytics-customize-dashboard/brand-image-save.png":::

+1. Optionally, select **Configure** on the tile titled **Documentation**, and specify a URL for support content.

+1. Select **Configure** on the image tile that displays the default store zone map.

+1. Select **Image**, and use the dialog to upload a custom image of a store zone map.

+ :::image type="content" source="media/tutorial-in-store-analytics-customize-dashboard/store-map-save.png" alt-text="Screenshot showing the in-store analytics application dashboard store map tile." lightbox="media/tutorial-in-store-analytics-customize-dashboard/store-map-save.png":::

+1. Select **Edit** on the dashboard toolbar.

+1. Select **ellipsis** and **Delete** to remove the following tiles: **Back to all zones**, **Visit store dashboard**, **Warm-up checkout zone**, **Cool-down checkout zone**, **Occupancy sensor settings**, **Thermostat settings**, **Wait time**, and **Environment conditions** and all three tiles associated with **Checkout 3**. The Contoso store dashboard doesn't use these tiles.

+1. View your layout changes.

+After you customize the dashboard layout, you're ready to add tiles to show telemetry. To create a telemetry tile, select a device template and device instance, then select device-specific telemetry to display in the tile. The **In-store analytics - checkout** application template includes several telemetry tiles in the dashboard. The four tiles in the two checkout zones display telemetry from the simulated occupancy sensor. The **People traffic** tile shows counts in the two checkout zones.

+In this section, you add two more telemetry tiles to show environmental telemetry from the RuuviTag sensors you added in the [Create an in-store analytics application in Azure IoT Central](./tutorial-in-store-analytics-create-app.md) tutorial.

+1. Select `RuuviTag` in the **Device template** list.

+1. Select a **Device instance** of one of the two RuuviTag sensors. In the example Contoso store, select `Zone 1 Ruuvi` to create a telemetry tile for Zone 1.

+1. Select `Relative humidity` and `Temperature` in the **Telemetry** list. These are the telemetry items that display for each zone on the tile.

+1. Select **Add tile**. A new tile appears to display combined humidity and temperature telemetry for the selected sensor.

+1. Select **Configure** on the new tile for the RuuviTag sensor.

+1. Change the **Title** to *Zone 1 environment*.

+1. Select **Update**.

+1. Drag the tile titled **Zone 2 environment** below the **Thermostat firmware** tile.

+1. Drag the tile titled **Zone 1 environment** below the **People traffic** tile.

+1. Select **Edit** on the **People traffic** tile.

+1. Remove the **count3** telemetry.

+1. Select **Update**.

+1. Select **Save**. The updated dashboard displays counts for only your two checkout zones, which are based on the simulated occupancy sensor.

+Application operators also use the dashboard to manage devices by running commands. You can add command tiles to the dashboard that will execute predefined commands on a device. In this section, you add a command tile to enable operators to reboot the Rigado gateway.

+1. Select **Edit**.

+1. Select `C500` in the **Device template** list. It is the template for the Rigado C500 gateway.

+1. Select the **Reboot** command.

+1. Select **Add tile**.

+1. Select **Save**.

+1. View your completed Contoso dashboard.

+ :::image type="content" source="media/tutorial-in-store-analytics-customize-dashboard/completed-dashboard.png" alt-text="Screenshot showing the completed in-store analytics application dashboard." lightbox="media/tutorial-in-store-analytics-customize-dashboard/completed-dashboard.png":::

+1. Make a note of the **Device IDs**. In the following screenshot, the IDs are **8r6vfyiv1x** and **1rvfk4ymk6z**:

+After you deploy the application, your default dashboard is a connected logistics operator focused portal. Northwind Trader is a fictitious logistics provider managing a cargo fleet at sea and on land. In this dashboard, you see two different gateways providing telemetry from shipments, along with associated commands, jobs, and actions.

+**Gateway Commands** - This interface organizes all the gateway command capabilities.

+Learn more about:

+The "cameras-as-sensors" and edge workloads are managed locally by Azure IoT Edge and the camera stream is processed by analytics pipeline. The video analytics processing pipeline at Azure IoT Edge brings many benefits, including decreased response time, low-bandwidth consumption, which results in low latency for rapid data processing. Only the most essential metadata, insights, or actions are sent to the cloud for further action or investigation.

+Azure IoT Central is a solution development platform that simplifies IoT device and Azure IoT Edge gateway connectivity, configuration, and management. The platform significantly reduces the burden and costs of IoT device management, operations, and related developments. Customers and partners can build an end-to-end enterprise solution to achieve a digital feedback loop in distribution centers.

+1. Select **Create app** under **Digital distribution center**.

+## Walk through the application

+The default dashboard is a distribution center operator focused portal. Northwind Trader is a fictitious distribution center solution provider managing conveyor systems.

+The dashboard is logically organized to show the device management capabilities of the Azure IoT gateway and IoT device. You can:

+### Device templates

+Navigate to **Device templates**. The application has two device templates:

+* **Camera** - Organizes all the camera-specific command capabilities.

+* **Digital Distribution Gateway** - Represents all the telemetry coming from camera, cloud defined device twin properties and gateway info.

+**Large package** - This rule will trigger if the camera detects huge package that can't be inspected for the quality.

+Learn more about:

+Azure IoT Central is a solution development platform that simplifies IoT device connectivity, configuration, and management. The platform significantly reduces the burden and costs of IoT device management, operations, and related developments. Customers and partners can build an end-to-end enterprise solution to achieve a digital feedback loop in inventory management.

+### Dashboard

+After you deploy the application, your default dashboard is a smart inventory management operator focused portal. Northwind Trader is a fictitious smart inventory provider managing warehouse with Bluetooth low energy (BLE) and retail store with Radio-frequency identification (RFID). In this dashboard, you'll see two different gateways providing telemetry about inventory along with associated commands, jobs, and actions that you can perform.

+Select the Device templates tab, and you'll see the gateway capability model. A capability model is structured around two different interfaces **Gateway Telemetry and Property** and **Gateway Commands**

+Learn more about:

+Many IoT sensors can feed raw signals directly to the cloud or to a gateway device located near them. The gateway device performs data aggregation at the edge before sending summary insights to an IoT Central application. The gateway devices are also responsible for relaying command and control operations to the sensor devices when applicable.

+## Walk through the application

+* **Structure Condition Monitoring**: This device template represents a device collection that allows you to monitor environment condition, as well as the gateway device hosting various edge workloads to power your fulfillment center. The device sends telemetry data, such as the temperature, the number of picks, and the number of orders. It also sends information about the state and health of the compute workloads running in your environment.

+Learn more about:

+Review the [REST API](/rest/api/load-balancer/loadbalancerbackendaddresspools/createorupdate) for IP based backend pool management.

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+Follow the steps below to view the scoring results in Azure Storage Explorer when the job is completed:

+ :::image type="content" source="../media/how-to-use-batch-endpoint/view-data-outputs.png" alt-text="Studio screenshot showing view data outputs location." lightbox="../media/how-to-use-batch-endpoint/view-data-outputs.png" :::

+ The scoring results in Storage Explorer are similar to the following sample page:

+ :::image type="content" source="../media/how-to-use-batch-endpoint/scoring-view.png" alt-text="Screenshot of the scoring output." lightbox="../media/how-to-use-batch-endpoint/scoring-view.png":::

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+ # [Azure CLI](#tab/azure-cli)

+ # [Python](#tab/python)

+ # [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Azure CLI](#tab/azure-cli)

+# [Python](#tab/python)

+# [Studio](#tab/azure-studio)

+ delimiter: ','

+ datastore = ml_client.datastores.get(name='your datastore name')

+ delimiter: ','

+This option is a simple listing of your application or service. Customers use the **Contact Me** button on your offer's listing page to ask that you connect with them about your offer. They can write notes to express their needs, so make sure to read them and to get back to valid leads. [See lead management] (~/partner-center-portal/commercial-marketplace-get-customer-leads.md)

+- A customer consents to sharing their information after they select **Contact me** from the commercial marketplace. This lead is an *initial interest* lead. We share information with you about the customer who has expressed interest in getting your product, including notes they wrote to express their needs. The lead is the top of the acquisition funnel.

+ 

+If youΓÇÖre not using Partner Center, or an existing CRM system integration, hereΓÇÖs how to understand the data in leads:

+- **Follow-up**: Acting quickly on leads yields the best results and customers submitting Contact me leads expect a swift response, so don't forget to follow up within 24 hours. You will get the lead in your CRM of choice immediately after the customer deploys a test drive; email them within while they are still warm. Request scheduling a phone call to better understand if your product is a good solution for their problem. Expect the typical transaction to require numerous follow-up calls.

+> [!IMPORTANT]

+> The marketo connector is not currently working due to a change in the Marketo platform. Use Leads from the Referrals workspace.

+ Title: Active Directory authentication - Azure Database for MySQL - Flexible Server

+# Active Directory authentication - Azure Database for MySQL - Flexible Server

+2. Select the user managed identity (UMI) with the following privileges to configure Azure AD authentication:

+ - [User.Read.All](/graph/permissions-reference#user-permissions): Allows access to Azure AD user information.

+ - [GroupMember.Read.All](/graph/permissions-reference#group-permissions): Allows access to Azure AD group information.

+ - [Application.Read.ALL](/graph/permissions-reference#application-resource-permissions): Allows access to Azure AD service principal (application) information.

+User-managed identities are required for Azure Active Directory authentication. When a User-Assigned Identity is linked to the flexible server, the Managed Identity Resource Provider (MSRP) issues a certificate internally to that identity, and when the managed identity is deleted, the corresponding service principal is automatically removed.

+The service then uses the managed identity to request access tokens for services that support Azure AD authentication. Only a User-assigned Managed Identity (UMI) is currently supported by Azure Database for MySQL-Flexible Server. For more information, see [Managed identity types](../../active-directory/managed-identities-azure-resources/overview.md#managed-identity-types) in Azure.

+When using Azure AD authentication, there are two Administrator accounts for the MySQL server; the original MySQL administrator and the Azure AD administrator.

+Only the administrator based on an Azure AD account can create the first Azure AD contained database user in a user database. The Azure AD administrator login can be an Azure AD user or an Azure AD group. When the administrator is a group account, it can be used by any group member, enabling multiple Azure AD administrators for the MySQL Flexible server. Using a group account as an administrator enhances manageability by allowing you to centrally add and remove group members in Azure AD without changing the users or permissions in the MySQL Flexible server. Only one Azure AD administrator (a user or group) can be configured at a time.

+ Title: Data encryption with customer managed keys ΓÇô Azure Database for MySQL ΓÇô Flexible Server

+# Customer managed keys data encryption ΓÇô Azure Database for MySQL ΓÇô Flexible Server

+Once Azure Database for MySQL flexible server is encrypted with a customer's managed key stored in Key Vault, any newly created copy of the server is also encrypted. When trying to encrypt Azure Database for MySQL flexible server with a customer managed key that already has a replica(s), we recommend configuring the replica(s) as well by adding the managed identity and key. If the flexible server is configured with geo-redundancy backup, the replica must be configured with the managed identity and key to which the identity has access and which resides in the server's geo-paired region.

+When attempting to restore an Azure Database for MySQL flexible server, you're given the option to select the User managed identity, and Key to encrypt the restore server. If the flexible server is configured with geo-redundancy backup, the restore server must be configured with the managed identity and key to which the identity has access and which resides in the server's geo-paired region.

+ Title: Set up Azure Active Directory authentication for Azure Database for MySQL flexible server

+# Set up Azure Active Directory authentication for Azure Database for MySQL - Flexible Server

+ Title: Set data encryption for Azure Database for MySQL flexible server by using the Azure CLI

+# Data encryption for Azure Database for MySQL - Flexible Server with Azure CLI

+Disable data encryption for flexible server:

+az mysql flexible-server update --resource-group testGroup --name testserver --disable-data-encryption

+## Create flexible server with geo redundant backup and data encryption enabled

+```azurecli-interactive

+az mysql flexible-server create -g testGroup -n testServer --location testLocation \\

+--geo-redundant-backup Enabled \\

+--key <key identifier of testKey> --identity testIdentity \\

+--backup-key <key identifier of testBackupKey> --backup-identity testBackupIdentity

+```

+Set or change key, identity, backup key and backup identity for data encryption with geo redundant backup:

+az mysql flexible-server update --resource-group testGroup --name testserver \\ --key \<key identifier of newKey\> --identity newIdentity \\ --backup-key \<key identifier of newBackupKey\> --backup-identity newBackupIdentity

+ Title: Set data encryption for Azure Database for MySQL flexible server by using the Azure portal

+# Data encryption for Azure Database for MySQL - Flexible Server by using the Azure portal

+9. If you do not see a **Server key** on the **Firebase Cloud Messaging** tab, follow these steps:

+ 1. Select the three-dots menu of the **Cloud Messaging API (Legacy) Disabled** heading.

+ 1. Follow the link to **Manage API in Google Cloud Console**.

+ 1. In the Google Cloud Console, select the button to enable the Google Cloud Messaging API.

+ 1. Wait a few minutes.

+ 1. Go back to your Firebase console project **Cloud Messaging** tab, and refresh the page.

+ 1. See that the Cloud Messaging API header has changed to **Cloud Messaging API (Legacy) Enabled** and now shows a server key.

+ :::image type="content" source="media/android-sdk/notification-hubs-enable-firebase-cloud-messaging-legacy-api.png" alt-text="Portal screenshot showing Enable Cloud Messaging API (Legacy).":::

+

+ NotificationHub.start(this.getApplication(), "Hub Name", "Connection-String");

+- [Connect to and manage SAP ECC source](register-scan-sapecc-source.md)

+- [Connect to and manage SAP S/4HANA source](register-scan-saps4hana-source.md)

+- [Connect to and manage SAP Business Warehouse (BW) source](register-scan-sap-bw.md)

+- [Connect to and manage a Power BI tenant](register-scan-power-bi-tenant.md)

+- [Connect to and manage Power BI across tenants](register-scan-power-bi-tenant-cross-tenant.md)

+- [Connect to and manage Power BI troubleshooting](register-scan-power-bi-tenant-troubleshoot.md)

+* [How to govern an Azure SQL Database](register-scan-azure-sql-database.md#creating-the-scan)

+Learn how to discover and govern various data sources:

+- [Discover and govern Azure Blob storage source](register-scan-azure-blob-storage-source.md)

+ Title: 'Connect to and manage Azure Data Lake Storage (ADLS) Gen1'

+description: This article outlines the process to register an Azure Data Lake Storage Gen1 data source in Microsoft Purview including instructions to authenticate and interact with the Azure Data Lake Storage Gen 1 source.

+ Title: 'Discover and govern Azure Data Lake Storage (ADLS) Gen2'

+description: This article outlines the process to register an Azure Data Lake Storage Gen2 data source in Microsoft Purview including instructions to authenticate and interact with the Azure Data Lake Storage Gen2 source.

+This article outlines the process to register and govern an Azure Data Lake Storage (ADLS Gen2) data source in Microsoft Purview including instructions to authenticate and interact with the ADLS Gen2 source.

+1. Select **SQL server on Azure Arc-enabled servers** and then **Continue**

+1. Select the Azure Arc-enabeld SQL Server source that you registered.

+ Title: 'Discover and govern Azure Blob Storage'

+This article outlines the process to register and govern Azure Blob Storage accounts in Microsoft Purview including instructions to authenticate and interact with the Azure Blob Storage source

+ Title: 'Connect to Azure Cosmos DB Database (SQL API)'

+This article outlines the process to register and scan Azure Cosmos DB for NoSQL instance in Microsoft Purview, including instructions to authenticate and interact with the Azure Cosmos DB database source

+ Title: Discover and govern multiple Azure sources

+# Discover and govern multiple Azure sources in Microsoft Purview

+# Connect to and manages an Azure Database for PostgreSQL in Microsoft Purview

+ Title: 'Discover and govern Azure SQL DB'

+# Discover and govern Azure SQL Database in Microsoft Purview

+|36 |Govern Data Sources in Microsoft Purview |*Data Source admin* <br> *Data Reader* or *Data Curator* | For more information, see [supported data sources and file types](azure-purview-connector-overview.md) |

+# Reliability terminology

+ Title: Stream CEF logs to Microsoft Sentinel with the AMA connector

+description: Stream and filter CEF-based logs from on-premises appliances to your Microsoft Sentinel workspace.

+#Customer intent: As a security operator, I want to stream and filter CEF-based logs from on-premises appliances to my Microsoft Sentinel workspace, so I can improve load time and easily analyze the data.

+# Stream CEF logs with the AMA connector

+This article describes how to use the **Common Event Format (CEF) via AMA** connector to quickly filter and upload logs in the Common Event Format (CEF) from multiple on-premises appliances over Syslog.

+The connector uses the Azure Monitor Agent (AMA), which uses Data Collection Rules (DCRs). With DCRs, you can filter the logs before they're ingested, for quicker upload, efficient analysis, and querying.

+The AMA is installed on a Linux machine that acts as a log forwarder, and the AMA collects the logs in the CEF format.

+- [Set up the connector](#set-up-the-common-event-format-cef-via-ama-connector)

+- [Learn more about the connector](#how-collection-works-with-the-common-event-format-cef-via-ama-connector)

+> [!IMPORTANT]

+> The CEF via AMA connector is currently in PREVIEW. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+> [!NOTE]

+> On February 28th 2023, we will introduce [changes to the CommonSecurityLog table schema](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/upcoming-changes-to-the-commonsecuritylog-table/ba-p/3643232). This means that custom queries will require being reviews and updates. Out-of-the-box content (detections, hunting queries, workbooks, parsers, etc.) will be updated by Microsoft Sentinel.

+## Overview

+### What is CEF collection?

+Many network, security appliances, and devices send their logs in the CEF format over Syslog. This format includes more structured information than Syslog, with information presented in a parsed key-value arrangement.

+If your appliance or system sends logs over Syslog using CEF, the integration with Microsoft Sentinel allows you to easily run analytics and queries across the data.

+CEF normalizes the data, making it more immediately useful for analysis with Microsoft Sentinel. Microsoft Sentinel also allows you to ingest unparsed Syslog events, and to analyze them with query time parsing.

+### How collection works with the Common Event Format (CEF) via AMA connector

+1. Your organization sets up a log forwarder (Linux VM), if one doesn't already exist. The forwarder can be on-premises or cloud-based.

+1. Your organization uploads CEF logs from your source devices to the forwarder.

+1. The AMA connector installed on the log forwarder collects and parses the logs.

+1. The connector streams the events to the Microsoft Sentinel workspace to be further analyzed.

+When you install a log forwarder, the originating device must be configured to send Syslog events to the Syslog daemon on this forwarder instead of the local daemon. The Syslog daemon on the forwarder sends events to the Log Analytics agent over UDP. If this Linux forwarder is expected to collect a high volume of Syslog events, its Syslog daemon sends events to the agent over TCP instead. In either case, the agent then sends the events from there to your Log Analytics workspace in Microsoft Sentinel.

+## Set up the Common Event Format (CEF) via AMA connector

+### Prerequisites

+Before you begin, verify that you have:

+- The Microsoft Sentinel solution enabled.

+- A defined Microsoft Sentinel workspace.

+- A Linux machine to collect logs.

+ - The Linux machine must have Python 2.7 or 3 installed on the Linux machine. Use the ``python --version`` or ``python3 --version`` command to check.

+- Either the `syslog-ng` or `rsyslog` daemon enabled.

+- To collect events from any system that isn't an Azure virtual machine, ensure that [Azure Arc](../azure-monitor/agents/azure-monitor-agent-manage.md) is installed.

+### Configure a log forwarder

+To ingest Syslog and CEF logs into Microsoft Sentinel, you need to designate and configure a Linux machine that collects the logs from your devices and forwards them to your Microsoft Sentinel workspace. This machine can be a physical or virtual machine in your on-premises environment, an Azure VM, or a VM in another cloud. This machine must have Azure Arc installed (see the [prerequisites](#prerequisites)).

+This machine has two components that take part in this process:

+- A Syslog daemon, either `rsyslog` or `syslog-ng`, which collects the logs.

+- The AMA, which forwards the logs to Microsoft Sentinel.

+When you set up the connector and the DCR, you [run a script](#run-the-installation-script) on the Linux machine, which configures the built-in Linux Syslog daemon (`rsyslog.d`/`syslog-ng`) to listen for Syslog messages from your security solutions on TCP/UDP port 514.

+The DCR installs the AMA to collect and parse the logs.

+#### Log forwarder - security considerations

+Make sure to configure the machine's security according to your organization's security policy. For example, you can configure your network to align with your corporate network security policy and change the ports and protocols in the daemon to align with your requirements. To improve your machine security configuration, [secure your VM in Azure](../virtual-machines/security-policy.md), or review these [best practices for network security](../security/fundamentals/network-best-practices.md).

+If your devices are sending Syslog and CEF logs over TLS (because, for example, your log forwarder is in the cloud), you need to configure the Syslog daemon (`rsyslog` or `syslog-ng`) to communicate in TLS:

+- [Encrypt Syslog traffic with TLS ΓÇô rsyslog](https://www.rsyslog.com/doc/v8-stable/tutorials/tls_cert_summary.html)

+- [Encrypt log messages with TLS ΓÇô syslog-ng](https://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.22/administration-guide/60#TOPIC-1209298)

+### Set up the connector

+You can set up the connector in two ways:

+- [Microsoft Sentinel portal](#set-up-the-connector-in-the-microsoft-sentinel-portal-ui). With this setup, you can create, manage, and delete DCRs per workspace.

+- [API](#set-up-the-connector-with-the-api). With this setup, you can create, manage, and delete DCRs. This option is more flexible than the UI. For example, with the API, you can filter by specific log levels, where with the UI, you can only select a minimum log level.

+#### Set up the connector in the Microsoft Sentinel portal (UI)

+1. [Open the connector page and create the DCR](#open-the-connector-page-and-create-the-dcr)

+1. [Define resources (VMs)](#define-resources-vms)

+1. [Select the data source type and create the DCR](#select-the-data-source-type-and-create-the-dcr)

+1. [Run the installation script](#run-the-installation-script)

+##### Open the connector page and create the DCR

+1. Open the [Azure portal](https://portal.azure.com/) and navigate to the **Microsoft Sentinel** service.

+1. Select **Data connectors**, and in the search bar, type *CEF*.

+1. Select the **Common Event Format (CEF) via AMA (Preview)** connector.

+1. Below the connector description, select **Open connector page**.

+1. In the **Configuration** area, select **Create data collection rule**.

+1. Under **Basics**:

+ - Type a DCR name

+ - Select your subscription

+ - Select the resource group where your collector is defined

+ :::image type="content" source="media/connect-cef-ama/dcr-basics-tab.png" alt-text="Screenshot showing the DCR details in the Basics tab." lightbox="media/connect-cef-ama/dcr-basics-tab.png":::

+##### Define resources (VMs)

+Select the machines on which you want to install the AMA. These machines are VMs or on-premises Linux machines with Arc installed.

+1. Select the **Resources** tab and select **Add Resource(s)**.

+1. Select the VMs on which you want to install the connector to collect logs.

+ :::image type="content" source="media/connect-cef-ama/dcr-select-resources.png" alt-text="Screenshot showing how to select resources when setting up the DCR." lightbox="media/connect-cef-ama/dcr-select-resources.png":::

+1. Review your changes and select **Collect**.

+##### Select the data source type and create the DCR

+> [!NOTE]

+> **Using the same machine to forward both plain Syslog *and* CEF messages**

+>

+> If you plan to use this log forwarder machine to forward Syslog messages as well as CEF, then in order to avoid the duplication of events to the Syslog and CommonSecurityLog tables:

+>

+> 1. On each source machine that sends logs to the forwarder in CEF format, you must edit the Syslog configuration file to remove the facilities that are being used to send CEF messages. This way, the facilities that are sent in CEF won't also be sent in Syslog. See [Configure Syslog on Linux agent](../azure-monitor/agents/data-sources-syslog.md#configure-syslog-on-linux-agent) for detailed instructions on how to do this.

+>

+> 1. You must run the following command on those machines to disable the synchronization of the agent with the Syslog configuration in Microsoft Sentinel. This ensures that the configuration change you made in the previous step does not get overwritten.<br>

+> `sudo su omsagent -c 'python /opt/microsoft/omsconfig/Scripts/OMS_MetaConfigHelper.py --disable'`

+1. Select the **Collect** tab and select **Linux syslog** as the data source type.

+1. Configure the minimum log level for each facility. When you select a log level, Microsoft Sentinel collects logs for the selected level and other levels with lower severity. For example, if you select **LOG_ERR**, Microsoft Sentinel collects logs for the **LOG_ERR**, **LOG_WARNING**, **LOG_NOTICE**, **LOG_INFO**, and **LOG_DEBUG** levels.

+ :::image type="content" source="media/connect-cef-ama/dcr-log-levels.png" alt-text="Screenshot showing how to select log levels when setting up the DCR.":::

+1. Review your changes and select **Next: Review and create**.

+1. In the **Review and create** tab, select **Create**.

+##### Run the installation script

+1. Log in to the Linux forwarder machine, where you want the AMA to be installed.

+1. Run this command to launch the installation script:

+

+ ```python

+ sudo wget -O Forwarder_AMA_installer.py https://raw.githubusercontent.com/Azure/Azure- Sentinel/master/DataConnectors/Syslog/Forwarder_AMA_installer.py&&sudo python Forwarder_AMA_installer.py

+ ```

+ The installation script configures the `rsyslog` or `syslog-ng` daemon to use the required protocol and restarts the daemon.

+ > [!NOTE]

+ > To avoid [Full Disk scenarios](../azure-monitor/agents/azure-monitor-agent-troubleshoot-linux-vm-rsyslog.md) where the agent can't function, we recommend that you set the `syslog-ng` or `rsyslog` configuration not to store unneeded logs. A Full Disk scenario disrupts the function of the installed AMA.

+ > Read more about [RSyslog](https://www.rsyslog.com/doc/master/configuration/actions.html) or [Syslog-ng](

+https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.26/administration-guide/34#TOPIC-1431029).

+### Set up the connector with the API

+You can create DCRs using the [API](/rest/api/monitor/data-collection-rules). Learn more about [DCRs](../azure-monitor/essentials/data-collection-rule-overview.md).

+Run this command to launch the installation script:

+

+```python

+sudo wget -O Forwarder_AMA_installer.py https://raw.githubusercontent.com/Azure/Azure- Sentinel/master/DataConnectors/Syslog/Forwarder_AMA_installer.py&&sudo python Forwarder_AMA_installer.py

+```

+The installation script configures the `rsyslog` or `syslog-ng` daemon to use the required protocol and restarts the daemon.

+#### Request URL and headerΓÇ»

+```rest

+GET

+https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}?api-version=2019-11-01-preview

+```

+

+#### Request body

+Edit the template:

+- Verify that the `streams` field is set to `Microsoft-CommonSecurityLog`.

+- Add the filter and facility log levels in the `facilityNames` and `logLevels` parameters.

+```rest

+{

+ "properties": {

+ "immutableId": "dcr-bcc4039c90f0489b80927bbdf1f26008",

+ "dataSources": {

+ "syslog": [

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "*"

+ ],

+ "logLevels": [ "*"

+ ],

+ "name": "sysLogsDataSource-1688419672"

+ }

+ ]

+ },

+ "destinations": {

+ "logAnalytics": [

+ {

+ "workspaceResourceId": "/subscriptions/{Your-Subscription-

+Id}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{SentinelWorkspaceName}", "workspaceId": "123x56xx-9123-xx4x-567x-89123xxx45",

+"name": "la-140366483"

+ }

+ ]

+ },

+ "dataFlows": [

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "destinations": [

+ "la-140366483"

+ ]

+ }

+ ],

+ "provisioningState": "Succeeded"

+ },

+ "location": "westeurope",

+ "tags": {},

+ "kind": "Linux",

+ "id": "/subscriptions/{Your-Subscription- Id}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{DCRName}",

+ "name": "{DCRName}",

+ "type": "Microsoft.Insights/dataCollectionRules",

+ "etag": "\"2401b6f3-0000-0d00-0000-618bbf430000\""

+}

+```

+After you finish editing the template, use `POST` or `PUT` to deploy it:

+```rest

+PUT

+https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}?api-version=2019-11-01-preview

+```

+#### Examples of facilities and log levels sections

+Review these examples of the facilities and log levels settings. The `name` field includes the filter name.

+This example collects events from the `cron`, `daemon`, `local0`, `local3` and `uucp` facilities, with the `Warning`, `Error`, `Critical`, `Alert`, and `Emergency` log levels:

+```rest

+ "syslog": [

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "cron",

+ "daemon",

+ "local0",

+ "local3",

+ "uucp"

+ ],

+

+ "logLevels": [

+ "Warning",

+ "Error",

+ "Critical",

+ "Alert",

+ "Emergency"

+ ],

+ "name": "sysLogsDataSource-1688419672"

+ }

+]

+```

+This example collects events for:

+- The `authpriv` and `mark` facilities with the `Info`, `Notice`, `Warning`, `Error`, `Critical`, `Alert`, and `Emergency` log levels

+- The `daemon` facility with the `Warning`, `Error`, `Critical`, `Alert`, and `Emergency` log levels

+- The `kern`, `local0`, `local5`, and `news` facilities with the `Critical`, `Alert`, and `Emergency` log levels

+- The `mail` and `uucp` facilities with the `Emergency` log level

+```rest

+ "syslog": [

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "authpriv",

+ "mark"

+ ],

+ "logLevels": [

+ "Info",

+ "Notice",

+ "Warning",

+ "Error",

+ "Critical",

+ "Alert",

+ "Emergency"

+ ],

+ "name": "sysLogsDataSource--1469397783"

+ },

+ {

+ "streams": [ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "daemon"

+ ],

+ "logLevels": [

+ "Warning",

+ "Error",

+ "Critical",

+ "Alert",

+ "Emergency"

+ ],

+

+ "name": "sysLogsDataSource--1343576735"

+ },

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "kern",

+ "local0",

+ "local5",

+ "news"

+ ],

+ "logLevels": [

+ "Critical",

+ "Alert",

+ "Emergency"

+ ],

+ "name": "sysLogsDataSource--1469572587"

+ },

+ {

+ "streams": [

+ "Microsoft-CommonSecurityLog"

+ ],

+ "facilityNames": [

+ "mail",

+ "uucp"

+ ],

+ "logLevels": [

+ "Emergency"

+ ],

+ "name": "sysLogsDataSource-1689584311"

+ }

+ ]

+}

+```

+### Test the connector

+1. To validate that the syslog daemon is running on the UDP port and that the AMA is listening, run this command:

+ ```

+ netstat -lnptv

+ ```

+ You should see the `rsyslog` or `syslog-ng` daemon listening on port 514.

+1. To capture messages sent from a logger or a connected device, run this command in the background:

+ ```

+ tcpdump -I any port 514 -A vv &

+ ```

+1. After you complete the validation, we recommend that you stop the `tcpdump`: Type `fg` and then select <kbd>Ctrl</kbd>+<kbd>C</kbd>.

+1. To send demo messages, do one of the following:

+ - Use the netcat utility. In this example, the utility reads data posted through the `echo` command with the newline switch turned off. The utility then writes the data to UDP port `514` on the localhost with no timeout. To execute the netcat utility, you might need to install an additional package.

+

+ ```

+ echo -n "<164>CEF:0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time" | nc -u -w0 localhost 514

+ ```

+ - Use the logger. This example writes the message to the `local 4` facility, at severity level `Warning`, to port `514`, on the local host, in the CEF RFC format. The `-t` and `--rfc3164` flags are used to comply with the expected RFC format.

+

+ ```

+ logger -p local4.warn -P 514 -n 127.0.0.1 --rfc3164 -t CEF "0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time"

+ ```

+1. To verify that the connector is installed correctly, run the troubleshooting script with this command:

+ ```

+ sudo wget -O cef_AMA_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_AMA_troubleshoot.py&&sudo python cef_AMA_troubleshoot.py

+ ```

+## Next steps

+In this article, you learned how to set up the Windows CEF via AMA connector to upload data from appliances that support CEF over Syslog. To learn more about Microsoft Sentinel, see the following articles:

+- Learn how to [get visibility into your data, and potential threats](get-visibility.md).

+- Get started [detecting threats with Microsoft Sentinel](detect-threats-built-in.md).

+- [Use workbooks](monitor-your-data.md) to monitor your data.

+#Customer intent: As a security operator, I want to proactively monitor Windows DNS activities so that I can prevent threats and attacks on DNS servers.

+## Common Event Format (CEF) via AMA

+| Connector attribute | Description |

+| | |

+| **Data ingestion method** | **[Azure monitor Agent-based connection](connect-cef-ama.md)** |

+| **Log Analytics table(s)** | [CommonSecurityLog](/azure/azure-monitor/reference/tables/commonsecuritylog) |

+| **DCR support** | Standard DCR |

+| **Supported by** | Microsoft |

+- [Common Event Format (CEF) via AMA](#common-event-format-cef-via-ama-preview)

+### Common Event Format (CEF) via AMA (Preview)

+The [Common Event Format (CEF) via AMA](connect-cef-ama.md) connector allows you to quickly filter and upload logs over CEF from multiple on-premises appliances to Microsoft Sentinel via the Azure Monitor Agent (AMA).

+The AMA supports Data Collection Rules (DCRs), which you can use to filter the logs before ingestion, for quicker upload, efficient analysis, and querying.

+Here are some benefits of using AMA for CEF log collection:

+- AMA is faster compared to the existing Log Analytics Agent (MMA/OMS).

+- AMA provides centralized configuration using Data Collection Rules (DCRs), and also supports multiple DCRs.

+- AMA is Syslog RFC compliant, a faster and a more resilient and reliant agent, more secure with lower footprint on the installed machine.

+> The other metric you could monitor is: **throttled requests**. It shouldn't be an issue though as long as the namespace stays within its memory, CPU, and brokered connections limits. For more information, see [Throttling in Azure Service Bus Premium tier](service-bus-throttling.md#throttling-in-premium-tier)

+Cloud native solutions give a notion of unlimited resources that can scale with your workload. While this notion is more true in the cloud than it is with on-premises systems, there are still limitations that exist in the cloud. These limitations may cause throttling of client application requests in both standard and premium tiers as discussed in this article.

+## Throttling in standard tier

+The standard tier of Service Bus operates as a multi-tenant setup with a pay-as-you-go pricing model. Here multiple namespaces in the same cluster share the allocated resources. Standard tier is the recommended choice for developer environments, QA environments, and low throughput production systems.

+In the past, Service Bus had coarse throttling limits strictly dependent on resource utilization. However, there's an opportunity to refine throttling logic and provide predictable throttling behavior to all namespaces that are sharing these resources.

+In an attempt to ensure fair usage and distribution of resources across all the Service Bus standard namespaces that use the same resources, the throttling logic has been modified to be credit-based.

+> It is important to note that throttling is **not new** to Azure Service Bus, or any cloud native service.

+> Credit based throttling is simply refining the way various namespaces share resources in a multi-tenant standard tier environment and thus enabling fair usage by all namespaces sharing the resources.

+Credit-based throttling limits the number of operations that can be performed on a given namespace in a specific time period. Here's the workflow for credit-based throttling.

+ * At the start of each time period, Service Bus provides a certain number of credits to each namespace.

+The credit limits are currently set to **1000 credits every second** (per namespace). Not all operations are created equal. Here are the credit costs of each of the operations

+| Data operations (`Send`, `SendAsync`, `Receive`, `ReceiveAsync`, `Peek`) | 1 credit per message |

+| Management operations (`Create`, `Read`, `Update`, `Delete` on queues, topics, subscriptions, filters) | 10 credits |

+> When sending to a topic, each message is evaluated against filters before being made available on the subscription. Each filter evaluation also counts against the credit limit (that is, 1 credit per filter evaluation).

+When the client application requests are being throttled, the client application receives the following server response.

+With shared resources, it's important to enforce some sort of fair usage across various Service Bus namespaces that share those resources. Throttling ensures that any spike in a single workload doesn't cause other workloads on the same resources to be throttled. As mentioned later in the article, there's no risk in being throttled because the client SDKs (and other Azure PaaS offerings) have the default retry policy built into them. Any throttled requests will be retried with exponential backoff and will eventually go through when the credits are replenished.

+Understandably, some applications may be sensitive to being throttled. In that case, it's recommended to [migrate your current Service Bus standard namespace to premium](service-bus-migrate-standard-premium.md). On migration, you can allocate dedicated resources to your Service Bus namespace and appropriately scale up the resources if there's a spike in your workload and reduce the likelihood of being throttled. Additionally, when your workload reduces to normal levels, you can scale down the resources allocated to your namespace.

+## Throttling in premium tier

+The [Service Bus premium](service-bus-premium-messaging.md) tier allocates dedicated resources, in terms of messaging units, to each namespace setup by the customer. These dedicated resources provide predictable throughput and latency and are recommended for high throughput or sensitive production grade systems. Additionally, the premium tier also enables customers to scale up their throughput capacity when they experience spikes in the workload. For more information, see [Automatically update messaging units of an Azure Service Bus namespace](automate-update-messaging-units.md).

+With exclusive resource allocation for the premium tier, throttling is purely driven by the limitations of the resources allocated to the namespace. If the number of requests are more than the current resources can service, then the requests will be throttled.

+There are various ways to identifying throttling in the Service Bus premium tier.

+As the Service Bus premium namespace already has dedicated resources, you can reduce the possibility of getting throttled by scaling up the number of messaging units allocated to your namespace in the event (or anticipation) of a spike in the workload. For more information, see [Automatically update messaging units of an Azure Service Bus namespace](automate-update-messaging-units.md).

+When a request is throttled, it implies that the service is busy because it's facing more requests than the resources allow. If the same operation is tried again after a few moments, once the service has worked through its current workload, then the request can be honored.

+As throttling is the expected behavior of any cloud native service, retry logic is built into the Service Bus SDK itself. The default is set to auto retry with an exponential back-off to ensure that we don't have the same request being throttled each time. The default retry logic will apply to every operation.

+Once the request is successfully acknowledged by Service Bus, it implies that Service Bus has successfully processed the request. If Service Bus returns a failure, then it implies that Service Bus hasn't been able to process the request and the client application must retry the request.

+However, when a request is throttled, the service is implying that it can't accept and process the request right now because of resource limitations. It **does not** imply any sort of data loss because Service Bus simply hasn't looked at the request. In this case, relying on the default retry policy of the Service Bus SDK ensures that the request is eventually processed.

+## Service Fabric 9.1

+We are excited to announce that the 9.1 release of the Service Fabric runtime has started rolling out to the various Azure regions along with tooling and SDK updates. The updates for .NET SDK, Java SDK, and Service Fabric runtimes can be downloaded from the links provided in Release Notes. The SDK, NuGet packages, and Maven repositories will be available in all regions within 7-10 days.

+### Key announcements

+- Azure Service Fabric will block deployments that do not meet Silver or Gold durability requirements starting on 11/10/2022 (The date is extended from 10/30/2022 to 11/10/2022). Five VMs or more will be enforced with this change for newer clusters created after 11/10/2022 to help avoid data loss from VM-level infrastructure requests for production workloads. VM count requirement is not changing for Bronze durability. Enforcement for existing clusters will be rolled out in the coming months.

+- Azure Service Fabric node types with Virtual Machine Scale Set durability of Silver or Gold should always have the property "virtualMachineProfile.osProfile.windowsConfiguration.enableAutomaticUpdates" set to false in the scale set model definition. Setting enableAutomaticUpdates to false will prevent unintended OS restarts due to the Windows updates like patching, which can impact the production workloads.

+Instead, you should enable Automatic OS upgrades through Virtual Machine Scale Set OS Image updates by setting "enableAutomaticOSUpgrade" to true. With automatic OS image upgrades enabled on your scale set, an extra patching process through Windows Update is not required.

+### Service Fabric 9.1 releases

+| Release date | Release | More info |

+||||

+| October 24, 2022 | [Azure Service Fabric 9.1](https://techcommunity.microsoft.com/t5/azure-service-fabric-blog/microsoft-azure-service-fabric-9-1-release/ba-p/3667628) | [Release notes](https://github.com/microsoft/service-fabric/blob/master/release_notes/Service_Fabric_ReleaseNotes_91.md)|

+- **General Availability** Support for Multi-AZ within a single Virtual Machine Scale Set

+- Azure Service Fabric node types with Virtual Machine Scale Set durability of Silver or Gold should always have Windows update explicitly disabled to avoid unintended OS restarts due to the Windows updates, which can impact the production workloads. This can be done by setting the "enableAutomaticUpdates": false, in the Virtual Machine Scale Set OSProfile. Consider enabling Automatic Virtual Machine Scale Set Image upgrades instead. The deployments will start failing from 09/30/2022 for new clusters, if the WindowsUpdates are not disabled on the Virtual Machine Scale Set. Enforcement for existing clusters will be rolled out in the coming months.

+| October 11, 2022 | [Azure Service Fabric 9.0 Fourth Refresh Release](https://techcommunity.microsoft.com/t5/azure-service-fabric-blog/microsoft-azure-service-fabric-9-0-fourth-refresh-update-release/ba-p/3658429) | [Release notes](https://github.com/microsoft/service-fabric/blob/master/release_notes/Service_Fabric_ReleaseNotes_90CU4.md)|

+- Azure Service Fabric will block deployments that do not meet Silver or Gold durability requirements starting on 11/10/2022 (The date is extended from 10/30/2022 to 11/10/2022). Five VMs or more will be enforced with this change for newer clusters created after 11/10/2022 to help avoid data loss from VM-level infrastructure requests for production workloads. VM count requirement is not changing for Bronze durability. Enforcement for existing clusters will be rolled out in the coming months.

+- Azure Service Fabric node types with Virtual Machine Scale Set durability of Silver or Gold should always have the property "virtualMachineProfile.osProfile.windowsConfiguration.enableAutomaticUpdates" set to false in the scale set model definition. Setting enableAutomaticUpdates to false will prevent unintended OS restarts due to the Windows updates like patching, which can impact the production workloads.

+Instead, you should enable Automatic OS upgrades through Virtual Machine Scale Set OS Image updates by setting "enableAutomaticOSUpgrade" to true. With automatic OS image upgrades enabled on your scale set, an extra patching process through Windows Update is not required.

+| October 11, 2022 | [Azure Service Fabric 8.2 Sixth Refresh Release](https://techcommunity.microsoft.com/t5/azure-service-fabric-blog/microsoft-azure-service-fabric-8-2-sixth-refresh-release/ba-p/3666852) | [Release notes](https://github.com/microsoft/service-fabric/blob/master/release_notes/Service_Fabric_ReleaseNotes_82CU6.md)|

+| October 24, 2022 | [Azure Service Fabric 8.2 Seventh Refresh Release](https://techcommunity.microsoft.com/t5/azure-service-fabric-blog/azure-service-fabric-8-2-seventh-refresh-is-now-available/ba-p/3666872) | [Release notes](https://github.com/microsoft/service-fabric/blob/master/release_notes/Service_Fabric_ReleaseNotes_82CU7.md)|

+ "pricingTier": "Standard",

+ "pricingTier": "Standard",

+The **AllowSharedKeyAccess** property of a storage account is not set by default and does not return a value until you explicitly set it. The storage account permits requests that are authorized with Shared Key when the property value is **null** or when it is **true**.

+This article describes how to use a DRAG (Detection-Remediation-Audit-Governance) framework to continuously manage Shared Key authorization for your storage account.

+## Prerequisites

+Before disallowing Shared Key access on any of your storage accounts:

+- [Understand how disallowing Shared Key affects SAS tokens](#understand-how-disallowing-shared-key-affects-sas-tokens)

+- [Consider compatibility with other Azure tools and services](#consider-compatibility-with-other-azure-tools-and-services)

+- Consider the need to [disallow Shared Key authorization to use Azure AD Conditional Access](#disallow-shared-key-authorization-to-use-azure-ad-conditional-access)

+- [Transition Azure Files workloads](#transition-azure-files-workloads)

+### Understand how disallowing Shared Key affects SAS tokens

+When Shared Key access is disallowed for the storage account, Azure Storage handles SAS tokens based on the type of SAS and the service that is targeted by the request. The following table shows how each type of SAS is authorized and how Azure Storage will handle that SAS when the **AllowSharedKeyAccess** property for the storage account is **false**.

+| Type of SAS | Type of authorization | Behavior when AllowSharedKeyAccess is false |

+|-|-|-|

+| User delegation SAS (Blob storage only) | Azure AD | Request is permitted. Microsoft recommends using a user delegation SAS when possible for superior security. |

+| Service SAS | Shared Key | Request is denied for all Azure Storage services. |

+| Account SAS | Shared Key | Request is denied for all Azure Storage services. |

+Azure metrics and logging in Azure Monitor do not distinguish between different types of shared access signatures. The **SAS** filter in Azure Metrics Explorer and the **SAS** field in Azure Storage logging in Azure Monitor both report requests that are authorized with any type of SAS. However, different types of shared access signatures are authorized differently, and behave differently when Shared Key access is disallowed:

+- A service SAS token or an account SAS token is authorized with Shared Key and will not be permitted on a request to Blob storage when the **AllowSharedKeyAccess** property is set to **false**.

+- A user delegation SAS is authorized with Azure AD and will be permitted on a request to Blob storage when the **AllowSharedKeyAccess** property is set to **false**.

+When you are evaluating traffic to your storage account, keep in mind that metrics and logs as described in [Detect the type of authorization used by client applications](#detect-the-type-of-authorization-used-by-client-applications) may include requests made with a user delegation SAS.

+For more information about shared access signatures, see [Grant limited access to Azure Storage resources using shared access signatures (SAS)](storage-sas-overview.md).

+### Consider compatibility with other Azure tools and services

+A number of Azure services use Shared Key authorization to communicate with Azure Storage. If you disallow Shared Key authorization for a storage account, these services will not be able to access data in that account, and your applications may be adversely affected.

+Some Azure tools offer the option to use Azure AD authorization to access Azure Storage. The following table lists some popular Azure tools and notes whether they can use Azure AD to authorize requests to Azure Storage.

+| Azure tool | Azure AD authorization to Azure Storage |

+|-|-|

+| Azure portal | Supported. For information about authorizing with your Azure AD account from the Azure portal, see [Choose how to authorize access to blob data in the Azure portal](../blobs/authorize-data-operations-portal.md). |

+| AzCopy | Supported for Blob storage. For information about authorizing AzCopy operations, see [Choose how you'll provide authorization credentials](storage-use-azcopy-v10.md#choose-how-youll-provide-authorization-credentials) in the AzCopy documentation. |

+| Azure Storage Explorer | Supported for Blob storage, Queue storage, Table storage and Azure Data Lake Storage Gen2. Azure AD access to File storage is not supported. Make sure to select the correct Azure AD tenant. For more information, see [Get started with Storage Explorer](../../vs-azure-tools-storage-manage-with-storage-explorer.md?tabs=windows#sign-in-to-azure) |

+| Azure PowerShell | Supported. For information about how to authorize PowerShell commands for blob or queue operations with Azure AD, see [Run PowerShell commands with Azure AD credentials to access blob data](../blobs/authorize-data-operations-powershell.md) or [Run PowerShell commands with Azure AD credentials to access queue data](../queues/authorize-data-operations-powershell.md). |

+| Azure CLI | Supported. For information about how to authorize Azure CLI commands with Azure AD for access to blob and queue data, see [Run Azure CLI commands with Azure AD credentials to access blob or queue data](../blobs/authorize-data-operations-cli.md). |

+| Azure IoT Hub | Supported. For more information, see [IoT Hub support for virtual networks](../../iot-hub/virtual-network-support.md). |

+| Azure Cloud Shell | Azure Cloud Shell is an integrated shell in the Azure portal. Azure Cloud Shell hosts files for persistence in an Azure file share in a storage account. These files will become inaccessible if Shared Key authorization is disallowed for that storage account. For more information, see [Connect your Microsoft Azure Files storage](../../cloud-shell/overview.md#connect-your-microsoft-azure-files-storage). <br /><br /> To run commands in Azure Cloud Shell to manage storage accounts for which Shared Key access is disallowed, first make sure that you have been granted the necessary permissions to these accounts via Azure RBAC. For more information, see [What is Azure role-based access control (Azure RBAC)?](../../role-based-access-control/overview.md). |

+### Disallow Shared Key authorization to use Azure AD Conditional Access

+To protect an Azure Storage account with Azure AD [Conditional Access](../../active-directory/conditional-access/overview.md) policies, you must disallow Shared Key authorization for the storage account.

+### Transition Azure Files workloads

+Azure Storage supports Azure AD authorization for requests to blob, table and queue storage only. If you disallow authorization with Shared Key for a storage account, requests to Azure Files that use Shared Key authorization will fail. Because the Azure portal always uses Shared Key authorization to access file data, if you disallow authorization with Shared Key for the storage account, you will not be able to access Azure Files data in the Azure portal.

+Microsoft recommends that you either migrate any Azure Files data to a separate storage account before you disallow access to an account via Shared Key, or do not apply this setting to storage accounts that support Azure Files workloads.

+Disallowing Shared Key access for a storage account does not affect SMB connections to Azure Files.

+## Identify storage accounts that allow Shared Key access

+There are two ways to identify storage accounts that allow Shared Key access:

+- [Check the Shared Key access setting for multiple accounts](#check-the-shared-key-access-setting-for-multiple-accounts)

+- [Configure the Azure Policy for Shared Key access in audit mode](#configure-the-azure-policy-for-shared-key-access-in-audit-mode)

+### Check the Shared Key access setting for multiple accounts

+To check the Shared Key access setting across a set of storage accounts with optimal performance, you can use the Azure Resource Graph Explorer in the Azure portal. To learn more about using the Resource Graph Explorer, see [Quickstart: Run your first Resource Graph query using Azure Resource Graph Explorer](../../governance/resource-graph/first-query-portal.md).

+Running the following query in the Resource Graph Explorer returns a list of storage accounts and displays the Shared Key access setting for each account:

+```kusto

+resources

+| where type =~ 'Microsoft.Storage/storageAccounts'

+| extend allowSharedKeyAccess = parse_json(properties).allowSharedKeyAccess

+| project subscriptionId, resourceGroup, name, allowSharedKeyAccess

+```

+### Configure the Azure Policy for Shared Key access in audit mode

+Azure Policy **Storage accounts should prevent shared key access** prevents users with appropriate permissions from configuring new or existing storage accounts to permit Shared Key authorization. Configure this policy in audit mode to identify storage accounts where Shared Key authorization is allowed. After you have changed applications to use Azure AD rather than Shared Key for authorization, you can [update the policy to prevent allowing Shared Key access](#update-the-azure-policy-to-prevent-allowing-shared-key-access).

+For more information about the built-in policy, see **Storage accounts should prevent shared key access** in [List of built-in policy definitions](../../governance/policy/samples/built-in-policies.md#storage).

+#### Assign the built-in policy for a resource scope

+Follow these steps to assign the built-in policy for the appropriate scope in the Azure portal:

+1. In the Azure portal, search for *Policy* to display the Azure Policy dashboard.

+1. In the **Authoring** section, select **Assignments**.

+1. Choose **Assign policy**.

+1. On the **Basics** tab of the **Assign policy** page, in the **Scope** section, specify the scope for the policy assignment. Select the **More** button (**...**) to choose the subscription and optional resource group.

+1. For the **Policy definition** field, select the **More** button (**...**), and enter *shared key access* in the **Search** field. Select the policy definition named **Storage accounts should prevent shared key access**.

+ :::image type="content" source="media/shared-key-authorization-prevent/policy-definition-select-portal.png" alt-text="Screenshot showing how to select the built-in policy to prevent allowing Shared Key access for your storage accounts" lightbox="media/shared-key-authorization-prevent/policy-definition-select-portal.png":::

+

+1. Select **Review + create**.

+

+1. On the **Review + create** tab, review the policy assignment then select **Create** to assign the policy definition to the specified scope.

+#### Monitor compliance with the policy

+To monitor your storage accounts for compliance with the Shared Key access policy, follow these steps:

+1. On the Azure Policy dashboard under **Authoring**, select **Assignments**.

+1. Locate and select the policy assignment you created in the previous section.

+1. Select the **View compliance** tab.

+1. Any storage accounts within the scope of the policy assignment that do not meet the policy requirements appear in the compliance report.

+ :::image type="content" source="media/shared-key-authorization-prevent/policy-compliance-report-portal.png" alt-text="Screenshot showing how to view the compliance report for the Shared Key access built-in policy." lightbox="media/shared-key-authorization-prevent/policy-compliance-report-portal.png":::

+To get more information about why a storage account is non-compliant, select **Details** under **Compliance reason**.

+To understand how disallowing Shared Key authorization may affect client applications before you make this change, enable logging and metrics for the storage account. You can then analyze patterns of requests to your account over a period of time to determine how requests are being authorized.

+### Determine the number and frequency of requests authorized with Shared Key

+1. The new metric box should appear:

+ :::image type="content" source="media/shared-key-authorization-prevent/metric-new-metric-portal.png" alt-text="Screenshot showing the new metric dialog." lightbox="media/shared-key-authorization-prevent/metric-new-metric-portal.png":::

+ If it doesn't, select **Add metric**.

+1. In the **Metric** dialog, specify the following values:

+ :::image type="content" source="media/shared-key-authorization-prevent/configure-metric-account-transactions.png" alt-text="Screenshot showing how to configure a metric to summarize transactions made with Shared Key or SAS." lightbox="media/shared-key-authorization-prevent/configure-metric-account-transactions.png":::

+1. In the Monitoring section, select **Diagnostic settings**.

+ :::image type="content" source="media/shared-key-authorization-prevent/create-diagnostic-setting-logs.png" alt-text="Screenshot showing how to create a diagnostic setting for logging requests." lightbox="media/shared-key-authorization-prevent/create-diagnostic-setting-logs.png":::

+For a reference of fields available in Azure Storage logs in Azure Monitor, see [Resource logs](../blobs/monitor-blob-storage-reference.md#resource-logs-preview).

+### Permissions for allowing or disallowing Shared Key access

+To set the **AllowSharedKeyAccess** property for the storage account, a user must have permissions to create and manage storage accounts. Azure role-based access control (Azure RBAC) roles that provide these permissions include the **Microsoft.Storage/storageAccounts/write** or **Microsoft.Storage/storageAccounts/\*** action. Built-in roles with this action include:

+- The Azure Resource Manager [Owner](../../role-based-access-control/built-in-roles.md#owner) role

+- The Azure Resource Manager [Contributor](../../role-based-access-control/built-in-roles.md#contributor) role

+- The [Storage Account Contributor](../../role-based-access-control/built-in-roles.md#storage-account-contributor) role

+These roles do not provide access to data in a storage account via Azure Active Directory (Azure AD). However, they include the **Microsoft.Storage/storageAccounts/listkeys/action**, which grants access to the account access keys. With this permission, a user can use the account access keys to access all data in a storage account.

+Role assignments must be scoped to the level of the storage account or higher to permit a user to allow or disallow Shared Key access for the storage account. For more information about role scope, see [Understand scope for Azure RBAC](../../role-based-access-control/scope-overview.md).

+Be careful to restrict assignment of these roles only to those who require the ability to create a storage account or update its properties. Use the principle of least privilege to ensure that users have the fewest permissions that they need to accomplish their tasks. For more information about managing access with Azure RBAC, see [Best practices for Azure RBAC](../../role-based-access-control/best-practices.md).

+> [!NOTE]

+> The classic subscription administrator roles Service Administrator and Co-Administrator include the equivalent of the Azure Resource Manager [Owner](../../role-based-access-control/built-in-roles.md#owner) role. The **Owner** role includes all actions, so a user with one of these administrative roles can also create and manage storage accounts. For more information, see [Classic subscription administrator roles, Azure roles, and Azure AD administrator roles](../../role-based-access-control/rbac-and-directory-admin-roles.md#classic-subscription-administrator-roles).

+### Disable Shared Key authorization

+Using an account that has the necessary permissions, disable Shared Key authorization in the Azure portal, with PowerShell or using the Azure CLI.

+ :::image type="content" source="media/shared-key-authorization-prevent/shared-key-access-portal.png" alt-text="Screenshot showing how to disallow Shared Key access for a storage account." lightbox="media/shared-key-authorization-prevent/shared-key-access-portal.png":::

+After you disallow Shared Key authorization, making a request to the storage account with Shared Key authorization will fail with error code 403 (Forbidden). Azure Storage returns an error indicating that key-based authorization is not permitted on the storage account.

+## Verify that Shared Key access is not allowed

+> Anonymous requests are not authorized and will proceed if you have configured the storage account and container for anonymous public read access. For more information, see [Configure anonymous public read access for containers and blobs](../blobs/anonymous-read-access-configure.md).

+## Monitor the Azure Policy for compliance

+After disallowing Shared Key access on the desired storage accounts, continue to [monitor the policy you created earlier](#monitor-compliance-with-the-policy) for ongoing compliance. Based on the monitoring results, take the appropriate action as needed, including changing the scope of the policy, disallowing Shared Key access on more accounts or allowing it for accounts where more time is needed for remediation.

+## Update the Azure Policy to prevent allowing Shared Key access

+To begin enforcing [the Azure Policy assignment you previously created](#configure-the-azure-policy-for-shared-key-access-in-audit-mode) for policy **Storage accounts should prevent shared key access**, change the **Effect** of the policy assignment to **Deny** to prevent authorized users from allowing Shared Key access on storage accounts. To change the effect of the policy, perform the following steps:

+1. On the Azure Policy dashboard, locate and select the policy assignment [you previously created](#configure-the-azure-policy-for-shared-key-access-in-audit-mode).

+1. Select **Edit assignment**.

+1. Go to the **Parameters** tab.

+1. Uncheck the **Only show parameters that need input or review** checkbox.

+1. In the **Effect** drop-down change **Audit** to **Deny**, then select **Review + save**.

+1. On the **Review + save** tab, review your changes, then select **Save**.

+> [!NOTE]

+> It might take up to 30 minutes for your policy change to take effect.

+- The container inside `storage1` that your Synapse workspace will use by default. This document calls it `container1`.

+### Manage dependencies for DEP-enabled Synapse Spark pools

+> [!NOTE]

+>

+> - Installing packages from public repo is not supported within [DEP-enabled workspaces](../security/workspace-data-exfiltration-protection.md), you should upload all your dependencies as workspace libraries and install to your Spark pool.

+>

+Please follow the steps below if you have trouble to identify the required dependencies:

+- **Step1: Run the following script to set up a local Python environment same with Synapse Spark environment**

+The setup script requires [Synapse-Python38-CPU.yml](https://github.com/Azure-Samples/Synapse/blob/main/Spark/Python/Synapse-Python38-CPU.yml) which is the list of libraries shipped in the default python env in Synapse spark.

+ ```powershell

+ # one-time synapse python setup

+ wget Synapse-Python38-CPU.yml

+ sudo bash Miniforge3-Linux-x86_64.sh -b -p /usr/lib/miniforge3

+ export PATH="/usr/lib/miniforge3/bin:$PATH"

+ sudo apt-get -yq install gcc g++

+ conda env create -n synapse-env -f Synapse-Python38-CPU.yml

+ source activate synapse-env

+```

+- **Step2: Run the following script to identify the required dependencies**

+The below snippet can be used to pass your requirement.txt which has all the packages and version you intend to install in the spark 3.1/spark3.2 spark pool. It will print the names of the *new* wheel files/dependencies needed for your input library requirements. Note this will list out only the dependencies that are not already present in the spark pool by default.

+ ```python

+ # command to list out wheels needed for your input libraries

+ # this command will list out only *new* dependencies that are

+ # not already part of the built-in synapse environment

+ pip install -r <input-user-req.txt> > pip_output.txt

+ cat pip_output.txt | grep "Using cached *"

+```

+## Apply a Windows license to a Windows client session host VM

+>[!NOTE]

+>The directions in this section apply to Windows client VMs, not Windows Server VMs.

+After you've created your session host virtual machines, [apply a Windows license to a session host VM](./apply-windows-license.md#apply-a-windows-license-to-a-windows-client-session-host-vm) to run your Windows or Windows Server virtual machines without paying for another license.

+>[The Log Analytics Agent is currently being deprecated](https://azure.microsoft.com/updates/were-retiring-the-log-analytics-agent-in-azure-monitor-on-31-august-2024/). While Azure Monitor currently uses the Log Analytics Agent for Azure Virtual Desktop support, you'll eventually need to migrate to Azure Monitor by August 31, 2024. We'll provide instructions for how to migrate when we release the update that allows Azure Virtual Desktop Insights to support the Azure Monitor Agent. Until then, continue to use the Log Analytics Agent.

+After you've created your session host virtual machines, [apply a Windows license to a session host VM](../apply-windows-license.md#apply-a-windows-license-to-a-windows-client-session-host-vm) to run your Windows or Windows Server virtual machines without paying for another license.

+If using [Azure Disk Encryption with Azure AD (previous version)](disk-encryption-overview-aad.md), the [Microsoft Authentication Library](../../active-directory/develop/msal-overview.md) will need to be installed manually for all distros (in addition to the packages appropriate for the distro, as [listed above](#package-management)).

+If you are looking for information about Maintenance Configurations for scale sets, see [Maintenance Control for Virtual Machine Scale Sets](virtual-machine-scale-sets-maintenance-control.md).

+Make sure `PowerShellGet` is up to date.

+```azurepowershell-interactive

+Install-Module -Name PowerShellGet -Repository PSGallery -Force

+```

+Install the `Az.Maintenance` PowerShell module.

+```azurepowershell-interactive

+```

+Check that you are running the latest version of `Az.Maintenance` PowerShell module (version 1.2.0)

+```azurepowershell-interactive

+Get-Module -ListAvailable -Name Az.Maintenance

+```

+Ensure that you are running the appropriate version of `Az.Maintenance` using

+```azurepowershell-interactive

+Import-Module -Name Az.Maintenance -RequiredVersion 1.2.0

+```

+You can declare a scheduled window when Azure will recurrently apply the updates on your resources. Once you create a scheduled window you no longer have to apply the updates manually. Maintenance **recurrence** can be expressed as daily, weekly or monthly. Some examples are:

+- **daily**- RecurEvery "Day" **or** "3Days"

+- **weekly**- RecurEvery "3Weeks" **or** "Week Saturday,Sunday"

+- **monthly**- RecurEvery "Month day23,day24" **or** "Month Last Sunday" **or** "Month Fourth Monday"

+### Host

+This example creates a maintenance configuration named myConfig scoped to **host** with a scheduled window of 5 hours on the fourth Monday of every month. It is important to note that the **duration** of the schedule for this scope should be at least two hours long. To begin, you will need to define all the parameters needed for `New-AzMaintenanceConfiguration`

+$RGName = "myMaintenanceRG"

+$configName = "myConfig"

+$scope = "Host"

+$location = "eastus"

+$timeZone = "Pacific Standard Time"

+$duration = "05:00"

+$startDateTime = "2022-11-01 00:00"

+$recurEvery = "Month Fourth Monday"

+After you have defined the parameters, you can now use the `New-AzMaintenanceConfiguration` cmdlet to create your configuration.

+```azurepowershell-interactive

+New-AzMaintenanceConfiguration

+ -ResourceGroup $RGName `

+ -Name $configName `

+ -MaintenanceScope $scope `

+ -Location $location `

+ -StartDateTime $startDateTime `

+ -TimeZone $timeZone `

+ -Duration $duration `

+ -RecurEvery $recurEvery

+```

+Using `$scope = "Host"` ensures that the maintenance configuration is used for controlling updates on host machines. You will need to ensure that you are creating a configuration for the specific scope of machines you are targeting. To read more about scopes check out [maintenance configuration scopes](maintenance-configurations.md#scopes).

+### OS Image

+In this example, we will create a maintenance configuration named myConfig scoped to **osimage** with a scheduled window of 8 hours every 5 days. It is important to note that the **duration** of the schedule for this scope should be at least 5 hours long. Another key difference to note is that this scope allows a max of 7 days for schedule recurrence.

+$RGName = "myMaintenanceRG"

+$configName = "myConfig"

+$scope = "osimage"

+$location = "eastus"

+$timeZone = "Pacific Standard Time"

+$duration = "08:00"

+$startDateTime = "2022-11-01 00:00"

+$recurEvery = "5days"

+After you have defined the parameters, you can now use the `New-AzMaintenanceConfiguration` cmdlet to create your configuration.

+New-AzMaintenanceConfiguration

+ -Name $configName `

+ -MaintenanceScope $scope `

+ -StartDateTime $startDateTime `

+ -TimeZone $timeZone `

+ -Duration $duration `

+ -RecurEvery $recurEvery

+```

+### Guest

+Our most recent addition to the maintenance configuration offering is the **InGuestPatch** scope. This example will show how to create a maintenance configuration for guest scope using PowerShell. To learn more about this scope see [Guest](maintenance-configurations.md#guest).

+```azurepowershell-interactive

+$RGName = "myMaintenanceRG"

+$configName = "myConfig"

+$scope = "InGuestPatch"

+$location = "eastus"

+$timeZone = "Pacific Standard Time"

+$duration = "04:00"

+$startDateTime = "2022-11-01 00:00"

+$recurEvery = "Week Saturday, Sunday"

+$WindowsParameterClassificationToInclude = "FeaturePack","ServicePack";

+$WindowParameterKbNumberToInclude = "KB123456","KB123466";

+$WindowParameterKbNumberToExclude = "KB123456","KB123466";

+$RebootOption = "IfRequired";

+$LinuxParameterClassificationToInclude = "Other";

+$LinuxParameterPackageNameMaskToInclude = "apt","httpd";

+$LinuxParameterPackageNameMaskToExclude = "ppt","userpk";

+After you have defined the parameters, you can now use the `New-AzMaintenanceConfiguration` cmdlet to create your configuration.

+```azurepowershell-interactive

+New-AzMaintenanceConfiguration

+ -ResourceGroup $RGName `

+ -Name $configName `

+ -MaintenanceScope $scope `

+ -Location $location `

+ -StartDateTime $startDateTime `

+ -TimeZone $timeZone `

+ -Duration $duration `

+ -RecurEvery $recurEvery `

+ -WindowParameterClassificationToInclude $WindowsParameterClassificationToInclude `

+ -WindowParameterKbNumberToInclude $WindowParameterKbNumberToInclude `

+ -WindowParameterKbNumberToExclude $WindowParameterKbNumberToExclude `

+ -InstallPatchRebootSetting $RebootOption `

+ -LinuxParameterPackageNameMaskToInclude $LinuxParameterPackageNameMaskToInclude `

+ -LinuxParameterClassificationToInclude $LinuxParameterClassificationToInclude `

+ -LinuxParameterPackageNameMaskToExclude $LinuxParameterPackageNameMaskToExclude `

+ -ExtensionProperty @{"InGuestPatchMode"="User"}

+```

+If you try to create a configuration with the same name, but in a different location, you will get an error. Configuration names must be unique to your resource group.

+You can check if you have successfully created the maintenance configurations by using [Get-AzMaintenanceConfiguration](/powershell/module/az.maintenance/get-azmaintenanceconfiguration).

+```azurepowershell-interactive

+Get-AzMaintenanceConfiguration | Format-Table -Property Name,Id

+```

+After you have created your configuration, you might want to also assign machines to it using PowerShell. To achieve this we will use [New-AzConfigurationAssignment](/powershell/module/az.maintenance/new-azconfigurationassignment).

+Apply the configuration to a VM using the ID of the configuration. Specify `-ResourceType VirtualMachines` and supply the name of the VM for `-ResourceName`, and the resource group of the VM for `-ResourceGroupName`.

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myVM" `

+ -ResourceType "VirtualMachines" `

+ -ProviderName "Microsoft.Compute" `

+ -ConfigurationAssignmentName "configName" `

+ -MaintenanceConfigurationId "configID"

+To apply a configuration to a dedicated host, you also need to include `-ResourceType hosts`, `-ResourceParentName` with the name of the host group, and `-ResourceParentType hostGroups`.

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myHost" `

+ -ResourceType "hosts" `

+ -ProviderName "Microsoft.Compute" `

+ -ConfigurationAssignmentName "configName" `

+ -MaintenanceConfigurationId "configID"

+```

+### Virtual Machine Scale Sets

+```azurepowershell-interactive

+New-AzConfigurationAssignment `

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myVMSS" `

+ -ResourceType "VirtualMachineScaleSets" `

+ -ProviderName "Microsoft.Compute" `

+ -ConfigurationAssignmentName "configName" `

+ -MaintenanceConfigurationId "configID"

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myVM" `

+ -ResourceType "VirtualMachines" `

+ -ProviderName "Microsoft.Compute" | Format-Table

+Check for pending updates for a dedicated host. In this example, the output is formatted as a table for readability.

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myHost" `

+ -ResourceType "hosts" `

+ -ResourceParentName "myHostGroup" `

+ -ResourceParentType "hostGroups" `

+ -ProviderName "Microsoft.Compute" | Format-Table

+### Virtual Machine Scale Sets

+```azurepowershell-interactive

+Get-AzMaintenanceUpdate `

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myVMSS" `

+ -ResourceType "VirtualMachineScaleSets" `

+ -ProviderName "Microsoft.Compute" | Format-Table

+```

+Use [New-AzApplyUpdate](/powershell/module/az.maintenance/new-azapplyupdate) to apply pending updates. Apply update calls can take up to 2 hours to complete. This cmdlet will only work for *Host* and *OSImage* scopes. It will NOT work for *Guest* scope.

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myVM" `

+ -ResourceType "VirtualMachines" `

+ -ProviderName "Microsoft.Compute"

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myHost" `

+ -ResourceType "hosts" `

+ -ResourceParentName "myHostGroup" `

+ -ResourceParentType "hostGroups" `

+### Virtual Machine Scale Sets

+```azurepowershell-interactive

+New-AzApplyUpdate `

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myVMSS" `

+ -ResourceType "VirtualMachineScaleSets" `

+ -ProviderName "Microsoft.Compute"

+```

+Use [Get-AzApplyUpdate](/powershell/module/az.maintenance/get-azapplyupdate) to check on the status of an update. The commands shown below show the status of the latest update by using `default` for the `-ApplyUpdateName` parameter. You can substitute the name of the update (returned by the [New-AzApplyUpdate](/powershell/module/az.maintenance/new-azapplyupdate) command) to get the status of a specific update. This cmdlet will only work for *Host* and *OSImage* scopes. It will NOT work for *Guest* scope.

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myVM" `

+ -ResourceType "VirtualMachines" `

+ -ProviderName "Microsoft.Compute" `

+ -ApplyUpdateName "applyUpdateName"

+ -ResourceGroupName "myResourceGroup" `

+ -ResourceName "myHost" `

+ -ResourceType "hosts" `

+ -ResourceParentName "myHostGroup" `

+ -ResourceParentType "hostGroups" `

+ -ProviderName "Microsoft.Compute" `

+ -ApplyUpdateName "applyUpdateName"

+### Virtual Machine Scale Sets

+```azurepowershell-interactive

+New-AzApplyUpdate `

+ -ResourceGroupName "myResourceGroup" `

+ -Location "eastus" `

+ -ResourceName "myVMSS" `

+ -ResourceType "VirtualMachineScaleSets" `

+ -ProviderName "Microsoft.Compute" `

+ -ApplyUpdateName "applyUpdateName"

+```

+## Delete a maintenance configuration

+ -ResourceGroupName "myResourceGroup" `

+ -Name "configName"

+After the move, consider deleting the moved maintenance configurations in the source region, [PowerShell](../virtual-machines/maintenance-configurations-powershell.md#delete-a-maintenance-configuration), or [CLI](../virtual-machines/maintenance-configurations-cli.md#delete-a-maintenance-configuration).

+After you create a virtual machine (VM), you can scale the VM up or down by changing the VM size. In some cases, you must deallocate the VM first. This can happen if the new size isn't available on the hardware cluster that is currently hosting the VM.

+If the virtual machine is currently running, changing its size will cause it to restart.

+ The VM restarts during this process. After the restart, your existing OS and data disks are kept. Anything on the temporary disk is lost.

+3. If the desired VM size isn't listed, you need to first deallocate the VM with [az vm deallocate](/cli/azure/vm). This process allows the VM to then be resized to any size available that the region supports and then started. The following steps deallocate, resize, and then start the VM named `myVM` in the resource group named `myResourceGroup`:

+If the size you want is listed, run the following commands to resize the VM. If the desired size isn't listed, go on to step 3.

+If the size you want isn't listed, run the following commands to deallocate the VM, resize it, and restart the VM. Replace **\<newVMsize>** with the size you want.

+If the new size for a VM in an availability set isn't available on the hardware cluster currently hosting the VM, then all VMs in the availability set will need to be deallocated to resize the VM. You also might need to update the size of other VMs in the availability set after one VM has been resized. To resize a VM in an availability set, perform the following steps.

+If the desired size is listed, run the following commands to resize the VM. If it isn't listed, go to the next section.

+If the size you want isn't listed, continue with the following steps to deallocate all VMs in the availability set, resize VMs, and restart them.

+If interested in a work-around, see [How do I migrate from a VM size with local temp disk to a VM size with no local temp disk?](azure-vms-no-temp-disk.yml#how-do-i-migrate-from-a-vm-size-with-local-temp-disk-to-a-vm-size-with-no-local-temp-disk)

+For more scalability, run multiple VM instances and scale out. For more information, see [Automatically scale machines in a Virtual Machine Scale Set](../virtual-machine-scale-sets/tutorial-autoscale-powershell.md).

+| `tf_version` | 1.3.0 | The Terraform version to use, see [Terraform download](https://www.terraform.io/downloads) |

+> | `resourcegroup_name` | Name of the resource group to be created | Optional |

+> | `resourcegroup_arm_id` | Azure resource identifier for an existing resource group | Optional |

+> | `resourcegroup_tags` | Tags to be associated to the resource group | Optional |

+[SAP Cloud Appliance Library](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) offers a quick and easy way to create SAP workloads in Azure. You can set up a fully configured demo environment from an Appliance Template or deploy a standardized system for an SAP product based on default or custom SAP software installation stacks.

+To deploy an appliance template, you'll need to authenticate with your S-User or P-User. You can create a P-User free of charge via the [SAP Community](https://community.sap.com/).

+[For details on Azure account creation, see the SAP learning video and description](https://www.youtube.com/watch?v=iORePziUMBk&list=PLWV533hWWvDmww3OX9YPhjjS1l1n6o-H2&index=18)

+The online library is continuously updated with Appliances for demo, proof of concept and exploration of new business cases. For the most recent ones, select ΓÇ£Create ApplianceΓÇ¥ here from the list ΓÇô or visit [cal.sap.com](https://cal.sap.com/catalog#/applianceTemplates) for further templates.

+| Appliance Template | Date | Description | Creation Link |

+| | - | -- | - |

+| [**SAP S/4HANA 2021 FPS02, Fully-Activated Appliance**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/3f4931de-b15b-47f1-b93d-a4267296b8bc) | July 19 2022 | This appliance contains SAP S/4HANA 2021 (FPS02) with pre-activated SAP Best Practices for SAP S/4HANA core functions, and further scenarios for Service, Master Data Governance (MDG), Portfolio Management (PPM), Human Capital Management (HCM), Analytics, Migration Cockpit, and more. User access happens via SAP Fiori, SAP GUI, SAP HANA Studio, Windows remote desktop, or the backend operating system for full administrative access. | [Create Appliance](https://cal.sap.com/registration?sguid=3f4931de-b15b-47f1-b93d-a4267296b8bc&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+| [**SAP S/4HANA 2021 FPS01, Fully-Activated Appliance**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/3f4931de-b15b-47f1-b93d-a4267296b8bc) | April 26 2022 | This appliance contains SAP S/4HANA 2021 (FPS01) with pre-activated SAP Best Practices for SAP S/4HANA core functions, and further scenarios for Service, Master Data Governance (MDG), Portfolio Management (PPM), Human Capital Management (HCM), Analytics, Migration Cockpit, and more. User access happens via SAP Fiori, SAP GUI, SAP HANA Studio, Windows remote desktop, or the backend operating system for full administrative access. | [Create Appliance](https://cal.sap.com/registration?sguid=3f4931de-b15b-47f1-b93d-a4267296b8bc&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+| [**SAP BW/4HANA 2021 including BW/4HANA Content 2.0 SP08 - Dev Edition**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/06725b24-b024-4757-860d-ac2db7b49577) | May 11 2022 | This solution offers you an insight of SAP BW/4HANA. SAP BW/4HANA is the next generation Data Warehouse optimized for HANA. Beside the basic BW/4HANA options, the solution offers a bunch of HANA optimized BW/4HANA Content and the next step of Hybrid Scenarios with SAP Data Warehouse Cloud. As the system is pre-configured, you can start directly implementing your scenarios. | [Create Appliance](https://cal.sap.com/registration?sguid=06725b24-b024-4757-860d-ac2db7b49577&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+| [**SAP ABAP Platform 1909, Developer Edition**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/7bd4548f-a95b-4ee9-910a-08c74b4f6c37) | June 21 2021 | The SAP ABAP Platform on SAP HANA gives you access to SAP ABAP Platform 1909 Developer Edition on SAP HANA. This solution is pre-configured with many other elements ΓÇô including: SAP ABAP RESTful Application Programming Model, SAP Fiori launchpad, SAP gCTS, SAP ABAP Test Cockpit, and pre-configured frontend / backend connections, etc. It also includes all the standard ABAP AS infrastructure: Transaction Management, database operations / persistence, Change and Transport System, SAP Gateway, interoperability with ABAP Development Toolkit and SAP WebIDE, and much more. | [Create Appliance](https://cal.sap.com/registration?sguid=7bd4548f-a95b-4ee9-910a-08c74b4f6c37&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+| [**SAP ERP 6.0 EhP 6 for Data Migration to SAP S/4HANA**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/applianceTemplates/56825489-df3a-4b6d-999c-329a63ef5e8a) | October 24 2022 | Update password of DDIC 100, SAP* 000. This system can be used as source system for the "direct transfer" data migration scenarios of the SAP S/4HANA *Fully-Activated Appliance*. It might also be useful as an "open playground" for SP ERP 6.0 EhP6 scenarios, however, the contained business processes and data structures aren't documented explicitly. | [Create Appliance](https://cal.sap.com/registration?sguid=56825489-df3a-4b6d-999c-329a63ef5e8a&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+| [**SAP NetWeaver 7.5 SP15 on SAP ASE**](https://cal.sap.com/catalog?provider=208b780d-282b-40ca-9590-5dd5ad1e52e8#/solutions/69efd5d1-04de-42d8-a279-813b7a54c1f6) | January 20 2020 | SAP NetWeaver 7.5 SP15 on SAP ASE | [Create Appliance](https://cal.sap.com/registration?sguid=69efd5d1-04de-42d8-a279-813b7a54c1f6&provider=208b780d-282b-40ca-9590-5dd5ad1e52e8) |

+You can now also deploy SAP S/4HANA systems with High Availability (HA), non-HA or single server architecture through SAP Cloud Appliance Library. The offering comprises default SAP S/4HANA software stacks including FPS levels and an integration into Maintenance Planner to enable creation and installation of custom SAP S/4HANA software stacks.

+To make sure that the restore or creation of databases isn't initializing the data files by zeroing the content of the files, you should make sure that the user context the SQL Server service is running in has a certain permission. Usually users in the Windows Administrator group have these permissions. If the SQL Server service is run in the user context of non-Windows Administrator user, you need to assign that user the User Right **Perform volume maintenance tasks**. For more information, see [Database instant file initialization](/sql/relational-databases/databases/database-instant-file-initialization).

+There is no charge to use Public IPv6 Addresses or Public IPv6 Prefixes. Associated resources and bandwidth are charged at the same rates as IPv4. You can find details about pricing for [public IP addresses](https://azure.microsoft.com/pricing/details/ip-addresses/), [network bandwidth](https://azure.microsoft.com/pricing/details/bandwidth/), or [Load Balancer](https://azure.microsoft.com/pricing/details/load-balancer/).

+Learn how to create and remove a NAT gateway resource from a virtual network subnet. A NAT gateway enables outbound connectivity for resources in an Azure Virtual Network. You may wish to change the IP address or prefix your resources use for outbound connectivity to the internet. The public IP address and public IP address prefixes associated with the NAT gateway can be changed after deployment.

+For information about public IP prefixes and a NAT gateway, see [Manage NAT gateway](/azure/virtual-network/nat-gateway/manage-nat-gateway?tabs=manage-nat-portal#add-or-remove-a-public-ip-prefix).

+### Bot protection rule set

+You can enable a managed bot protection rule set to take custom actions on requests from all bot categories.

+Three bot categories are supported:

+- **Bad**

+ Bad bots include bots from malicious IP addresses and bots that have falsified their identities. Bad bots with malicious IPs are sourced from the Microsoft Threat Intelligence feedΓÇÖs high confidence IP Indicators of Compromise.

+- **Good**

+ Good bots include validated search engines such as Googlebot, bingbot, and other trusted user agents.

+- **Unknown**

+ Unknown bots are classified via published user agents without additional validation. For example, market analyzer, feed fetchers, and data collection agents. Unknown bots also include malicious IP addresses that are sourced from Microsoft Threat Intelligence feedΓÇÖs medium confidence IP Indicators of Compromise.

+Bot signatures are managed and dynamically updated by the WAF platform.

+You may assign Microsoft_BotManagerRuleSet_1.0 by using the **Assign** option under **Managed Rulesets**:

+If Bot protection is enabled, incoming requests that match bot rules are blocked, allowed, or logged based on the configured action. Malicious bots are blocked, verified search engine crawlers are allowed, unknown search engine crawlers are blocked, and unknown bots are logged by default. You can set custom actions to block, allow, or log for different types of bots.

+You can access WAF logs from a storage account, event hub, log analytics, or send logs to a partner solution.

+### Bot rules

+You can enable a managed bot protection rule set to take custom actions on requests from all bot categories.

+|Rule group|Description|

+|||

+|**[BadBots](#bot100)**|Protect against bad bots|

+|**[GoodBots](#bot200)**|Identify good bots|

+|**[UnknownBots](#bot300)**|Identify unknown bots|

+# [Bot rules](#tab/bot)

+## <a name="bot"></a> Bot Manager rule sets

+### <a name="bot100"></a> Bad bots

+|RuleId|Description|

+|||

+|Bot100100|Malicious bots detected by threat intelligence|

+|Bot100200|Malicious bots that have falsified their identity|

+### <a name="bot200"></a> Good bots

+|RuleId|Description|

+|||

+|Bot200100|Search engine crawlers|

+|Bot200200|Unverified search engine crawlers|

+### <a name="bot300"></a> Unknown bots

+|RuleId|Description|

+|||

+|Bot300100|Unspecified identity|

+|Bot300200|Tools and frameworks for web crawling and attacks|

+|Bot300300|General purpose HTTP clients and SDKs|

+|Bot300400|Service agents|

+|Bot300500|Site health monitoring services|

+|Bot300600|Unknown bots detected by threat intelligence<br />(This rule also includes IP addresses matched to the Tor network.)|

+|Bot300700|Other bots|