+To help automate provisioning and deprovisioning, apps expose proprietary user and group APIs. User management in more than one app is a challenge because every app tries to perform the same actions. For example, creating or updating users, adding users to groups, or deprovisioning users. Yet, all these actions are implemented slightly differently by using different endpoint paths, different methods to specify user information, and a different schema to represent each element of information.

+The number of applications used in modern organizations continues to grow. IT admins are tasked with access management at scale. Admins use standards such as SAML or OIDC for single sign-on (SSO), but access also requires users to be provisioned into the app. To many admins, provisioning means manually creating every user account or uploading CSV files each week. These processes are time-consuming, expensive, and error prone. Solutions such as SAML just-in-time (JIT) have been adopted to automate provisioning. Enterprises also need a solution to deprovision users when they leave the organization or no longer require access to certain apps based on role change.

+>Number matching is a key security upgrade to traditional second factor notifications in Microsoft Authenticator. We will remove the admin controls and enforce the number match experience tenant-wide for all users of Microsoft Authenticator push notifications starting May 8, 2023.<br>

+>We highly recommend enabling number matching in the near term for improved sign-in security. Relevant services will begin deploying these changes after May 8, 2023 and users will start to see number match in approval requests. As services deploy, some may see number match while others don't. To ensure consistent behavior for all users, we highly recommend you enable number match for Microsoft Authenticator push notifications in advance.

+After May 8, 2023, when number matching is enabled for all users, anyone who performs a RADIUS connection with NPS extension version 1.2.2216.1 or later will be prompted to sign in with an OTP method instead.

+Prior to the release of NPS extension version 1.2.2216.1 after May 8, 2023, organizations that run any of these earlier versions of NPS extension can modify the registry to require users to enter an OTP:

+Number match will be enabled for all users of Microsoft Authenticator push notifications after May 8, 2023. We had previously announced that we will remove the admin controls and enforce the number match experience tenant-wide for all users of Microsoft Authenticator push notifications starting May 8, 2023. After listening to customers, we will extend the availability of the rollout controls for a few more weeks.

+Relevant services will begin deploying these changes after May 8, 2023 and users will start to see number match in approval requests. As services deploy, some may see number match while others don't. To ensure consistent behavior for all your users, we highly recommend you use the Azure portal or Graph API to roll out number match for all Microsoft Authenticator users.

+### Will the changes after May 8th, 2023, override number matching settings that are configured for a group in the Authentication methods policy?

+No, the changes after May 8th won't affect the **Enable and Target** tab for Microsoft Authenticator in the Authentication methods policy. Administrators can continue to target specific users and groups or **All Users** for Microsoft Authenticator **Push** or **Any** authentication mode.

+When Microsoft begins protecting all organizations by enabling number matching after May 8th, 2023, administrators will see the **Require number matching for push notifications** setting on the **Configure** tab of the Microsoft Authenticator policy is set to **Enabled** for **All users** and can't be disabled. In addition, the **Exclude** option for this setting will be removed.

+Users who are enabled for MFA push notifications in the legacy MFA policy will also see number match after May 8th, 2023. If the legacy MFA policy has enabled **Notifications through mobile app**, users will see number matching regardless of whether or not it's enabled on the **Enable and Target** tab for Microsoft Authenticator in the Authentication methods policy.

+Yes, currently you can disable number matching. We highly recommend that you enable number matching for all users in your tenant to protect yourself from MFA fatigue attacks. To protect the ecosystem and mitigate these threats, Microsoft will enable number matching for all tenants starting May 8, 2023. After protection is enabled by default, users can't opt out of number matching in Microsoft Authenticator push notifications.

+Relevant services will begin deploying these changes after May 8, 2023 and users will start to see number match in approval requests. As services deploy, some may see number match while others don't. To ensure consistent behavior for all users, we highly recommend you enable number match for Microsoft Authenticator push notifications in advance.

+If the user has a different default authentication method, there won't be any change to their default sign-in. If the default method is Microsoft Authenticator and the user is specified in either of the following policies, they'll start to receive number matching approval after May 8th, 2023:

+Regardless of their default method, any user who is prompted to sign-in with Authenticator push notifications will see number match after May 8th, 2023. If the user is prompted for another method, they won't see any change.

+| AADSTS76026 | RequestIssueTimeExpired - IssueTime in an SAML2 Authentication Request is expired. |

+* Have a question or can't find what you're looking for? Create a GitHub issue or see [Support and help options for developers](./developer-support-help-options.md) to learn about other ways you can get help and support.

+Now that you're set up, you can write your automated tests. The following are tests for:

+1. .NET example code uses [Microsoft Authentication Library (MSAL)](msal-overview.md) and [xUnit](https://xunit.net/), a common testing framework.

+1. JavaScript example code uses [Microsoft Authentication Library (MSAL)](msal-overview.md) and [Playwright](https://playwright.dev/), a common testing framework.

+## [.NET](#tab/dotnet)

+## [JavaScript](#tab/JavaScript)

+### Set up your authConfig.json file

+Add the client ID and the tenant ID of the test app you previously created, the key vault URI and the secret name to the authConfig.js file of your test project.

+```javascript

+export const msalConfig = {

+ auth: {

+ clientId: 'Enter_the_Application_Id_Here',

+ authority: 'https://login.microsoftonline.com/Enter_the_Tenant_Id_Here',

+ },

+};

+export const keyVaultConfig = {

+ keyVaultUri: 'https://<your-unique-keyvault-name>.vault.azure.net',

+ secretName: 'Enter_the_Secret_Name',

+};

+```

+### Initialize MSAL.js and fetch the user credentials from Key Vault

+Initialize the MSAL.js authentication context by instantiating a [PublicClientApplication](https://azuread.github.io/microsoft-authentication-library-for-js/ref/classes/_azure_msal_browser.publicclientapplication.html) with a [Configuration](https://azuread.github.io/microsoft-authentication-library-for-js/ref/modules/_azure_msal.html#configuration) object. The minimum required configuration property is the `clientID` of the application.

+Use [SecretClient()](/javascript/api/@azure/keyvault-secrets/secretclient) to get the test username and password secrets from Azure Key Vault.

+[DefaultAzureCredential()](/javascript/api/@azure/identity/defaultazurecredential) authenticates with Azure Key Vault by getting an access token from a service principal configured by environment variables or a managed identity (if the code is running on an Azure resource with a managed identity). If the code is running locally, `DefaultAzureCredential` uses the local user's credentials. Read more in the [Azure Identity client library](/javascript/api/@azure/identity/defaultazurecredential) content.

+Use Microsoft Authentication Library (MSAL) to authenticate using the ROPC flow and get an access token. The access token is passed along as a bearer token in the HTTP request.

+```javascript

+import { test, expect } from '@playwright/test';

+import { DefaultAzureCredential } from '@azure/identity';

+import { SecretClient } from '@azure/keyvault-secrets';

+import { PublicClientApplication, CacheKVStore } from '@azure/msal-node';

+import { msalConfig, keyVaultConfig } from '../authConfig';

+let tokenCache;

+const KVUri = keyVaultConfig.keyVaultUri;

+const secretName = keyVaultConfig.secretName;

+async function getCredentials() {

+ try {

+ const credential = new DefaultAzureCredential();

+ const secretClient = new SecretClient(KVUri, credential);

+ const secret = await secretClient.getSecret(keyVaultConfig.secretName);

+ const password = secret.value;

+ return [secretName, password];

+ } catch (error) {

+ console.log(error);

+ }

+}

+test.beforeAll(async () => {

+ const pca = new PublicClientApplication(msalConfig);

+ const [username, password] = await getCredentials();

+ const usernamePasswordRequest = {

+ scopes: ['user.read', 'User.ReadBasic.All'],

+ username: username,

+ password: password,

+ };

+ await pca.acquireTokenByUsernamePassword(usernamePasswordRequest);

+ tokenCache = pca.getTokenCache().getKVStore();

+});

+```

+### Run the test suite

+In the same file, add the tests as shown below:

+```javascript

+/**

+ * Stores the token in the session storage and reloads the page

+ */

+async function setSessionStorage(page, tokens) {

+ const cacheKeys = Object.keys(tokens);

+ for (let key of cacheKeys) {

+ const value = JSON.stringify(tokenCache[key]);

+ await page.context().addInitScript(

+ (arr) => {

+ window.sessionStorage.setItem(arr[0], arr[1]);

+ },

+ [key, value]

+ );

+ }

+ await page.reload();

+}

+test.describe('Testing Authentication with MSAL.js ', () => {

+ test('Test user has signed in successfully', async ({ page }) => {

+ await page.goto('http://localhost:<port>/');

+ let signInButton = page.getByRole('button', { name: /Sign In/i });

+ let signOutButton = page.getByRole('button', { name: /Sign Out/i });

+ let welcomeDev = page.getByTestId('WelcomeMessage');

+ expect(await signInButton.count()).toBeGreaterThan(0);

+ expect(await signOutButton.count()).toBeLessThanOrEqual(0);

+ expect(await welcomeDev.innerHTML()).toEqual('Please sign-in to see your profile and read your mails');

+ await setSessionStorage(page, tokenCache);

+ expect(await signInButton.count()).toBeLessThanOrEqual(0);

+ expect(await signOutButton.count()).toBeGreaterThan(0);

+ expect(await welcomeDev.innerHTML()).toContain(`Welcome`);

+ });

+});

+```

+For more information, please check the following code sample [MSAL.js Testing Example](https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/samples/msal-browser-samples/TestingSample).

+OpenID Connect (OIDC) extends the OAuth 2.0 authorization protocol for use as an additional authentication protocol. You can use OIDC to enable single sign-on (SSO) between your OAuth-enabled applications by using a security token called an *ID token*.

+The following diagram shows the basic OpenID Connect sign-in flow. The steps in the flow are described in more detail in later sections of the article.

+The *ID token* introduced by OpenID Connect is issued by the authorization server, the Microsoft identity platform, when the client application requests one during user authentication. The ID token enables a client application to verify the identity of the user and to get other information (claims) about them.

+ID tokens aren't issued by default for an application registered with the Microsoft identity platform. ID tokens for an application are enabled by using one of the following methods:

+1. Navigate to the [Azure portal](https://portal.azure.com) and select **Azure Active Directory** > **App registrations** > *\<your application\>* > **Authentication**.

+If ID tokens are not enabled for your app and one is requested, the Microsoft identity platform returns an `unsupported_response` error similar to:

+Requesting an ID token by specifying a `response_type` of `code` is explained in [Send the sign-in request](#send-the-sign-in-request) later in the article.

+OpenID providers like the Microsoft identity platform provide an [OpenID Provider Configuration Document](https://openid.net/specs/openid-connect-discovery-1_0.html) at a publicly accessible endpoint containing the provider's OIDC endpoints, supported claims, and other metadata. Client applications can use the metadata to discover the URLs to use for authentication and the authentication service's public signing keys.

+Authentication libraries are the most common consumers of the OpenID configuration document, which they use for discovery of authentication URLs, the provider's public signing keys, and other service metadata. If an authentication library is used in your app, you likely won't need to hand-code requests to and responses from the OpenID configuration document endpoint.

+To find the OIDC configuration document in the Azure portal, navigate to the [Azure portal](https://portal.azure.com) and then:

+The following request gets the OpenID configuration metadata from the `common` authority's OpenID configuration document endpoint on the Azure public cloud:

+* Specify `code` in the `response_type` parameter.

+| `response_type` | Required | Must include `code` for OpenID Connect sign-in. |

+ "issuer": "https://token.actions.githubusercontent.com",

+- For information about the required format of JWTs created by external identity providers, read about the [assertion format](active-directory-certificate-credentials.md#assertion-format).

+> [!Note]

+> Issuing this request can be performed directly in [Graph Explorer](https://aka.ms/ge).

+POST https://graph.microsoft.com/{tenant-id}/domains/foo.contoso.com/promote

+[Azure Active Directory B2C](../../active-directory-b2c/overview.md) provides business-to-customer identity as a service. Given that the integration with Azure AD B2C is similar to how you would allow enterprise users to sign in with Azure AD, the recommendations above still mostly apply when you want to use Azure AD B2C for your customers, consumers or citizens and allow them to use their preferred social, enterprise, or local account identities.

+There are a few important differences, however. Setting up Azure AD B2C as a corporate identity provider in IAS and configuring federation between both tenants is described in more detail in [this blog post](https://blogs.sap.com/2023/02/08/identity-federation-between-azure-ad-b2c-and-sap-cloud-identity-services-using-custom-policies/).

+Allows users to be added to Microsoft 365 and cloud-only security groups. Mail-enabled, distribution, dynamic and role-assignable groups are not supported. To control access to on-premises applications and resources, you need to enable group writeback. For more information, see [Azure AD Connect group writeback](../hybrid/how-to-connect-group-writeback-v2.md).

+Allows cloud-only user accounts to be enabled. Users with Azure AD role assignments are not supported, nor are users with membership or ownership of role-assignable groups. You're able to customize the task name and description for this task in the Azure portal.

+Allows cloud-only user accounts to be disabled. Users with Azure AD role assignments are not supported, nor are users with membership or ownership of role-assignable groups. You're able to customize the task name and description for this task in the Azure portal.

+Allows users to be removed from Microsoft 365 and cloud-only security groups. Mail-enabled, distribution, dynamic and role-assignable groups are not supported. To control access to on-premises applications and resources, you need to enable group writeback. For more information, see [Azure AD Connect group writeback](../hybrid/how-to-connect-group-writeback-v2.md).

+Allows users to be removed from every Microsoft 365 and cloud-only security group they're a member of. Mail-enabled, distribution, dynamic and role-assignable groups are not supported. To control access to on-premises applications and resources, you need to enable group writeback. For more information, see [Azure AD Connect group writeback](../hybrid/how-to-connect-group-writeback-v2.md).

+Allows cloud-only user accounts to be deleted. Users with Azure AD role assignments are not supported, nor are users with membership or ownership of role-assignable groups. You're able to customize the task name and description for this task in the Azure portal.

+Before you start making changes to filtering, make sure that you [disable the built-in scheduler](#disable-the-synchronization-scheduler) so you don't accidentally export changes that you haven't yet verified to be correct.

+```go

+You can see a sample request to the REST API:

+Notice a sample response from the REST API:

+The following request shows how to add the calculated value to the filter parameter of the request. At this moment only the filter=indexclaimhash eq format is supported.

+`GET /v1.0/verifiableCredentials/authorities/:authorityId/contracts/:contractId/credentials?filter=indexclaimhash eq {hashedsearchclaimvalue}`

+- [Entra Verified ID Network API](issuance-request-api.md)

+AKS follows a strict versioning window with regard to supportability. With properly selected auto-upgrade channels, you can avoid clusters falling into an unsupported version. For more on the AKS support window, see [Alias minor versions][supported-kubernetes-versions].

+> [!NOTE]

+> With AKS, you can create a cluster without specifying the exact patch version. When you create a cluster without designating a patch, the cluster will run the minor version's latest GA patch. To Learn more [AKS support window][supported-kubernetes-versions]

+ keyvaultName: ${KEYVAULT_NAME} # Set to the name of your key vault

+To disable local accounts on an existing Azure AD integration enabled AKS cluster, use the [`az aks update`][az-aks-update] command with the `disable-local-accounts` parameter.

+az aks update -g <resource-group> -n <cluster-name> --disable-local-accounts

+AKS supports enabling a disabled local account on an existing cluster with the `enable-local-accounts` parameter.

+az aks update -g <resource-group> -n <cluster-name> --enable-local-accounts

+If you configure an app setting with the same name in App Service and in *appsettings.json*, for example, the App Service value takes precedence over the *appsettings.json* value. The local *appsettings.json* value lets you debug the app locally, but the App Service value lets you run the app in production with production settings. Connection strings work in the same way. This way, you can keep your application secrets outside of your code repository and access the appropriate values without changing your code.

+az webapp config appsettings set --resource-group myResourceGroup --name --settings SCM_DO_BUILD_DURING_DEPLOYMENT=true

+3. Enter `certlm.msc` and select Enter. You can also search for Certificate Manager on the **Start** menu.

+4. Locate the certificate, typically in `Certificates - Local Computer\Personal\Certificates`, and open it.

+3. Enter **certlm.msc** and select Enter. You can also search for Certificate Manager on the **Start** menu.

+4. Locate the certificate (typically in `Certificates - Local Computer\Personal\Certificates`), and open the certificate.

+> [!NOTE]

+> Ensure that your **timeout value** is greater than your server-defined ping/pong interval to avoid experiencing timeout errors before a ping is sent from the client. A typical value for a WebSocket is 20 seconds, so, for example, a timeout value of 40 seconds will ensure that the gateway does not send a timeout error before the client sends a ping; otherwise, this would throw a 1006 error on the client side.

+ Title: What is Form Recognizer Studio?

+description: Learn how to set up and use Form Recognizer Studio to test features of Azure Form Recognizer on the web.

+monikerRange: 'form-recog-3.0.0'

+recommendations: false

+<!-- markdownlint-disable MD033 -->

+# What is Form Recognizer Studio?

+**This article applies to:**  **Form Recognizer v3.0**.

+Form Recognizer Studio is an online tool to visually explore, understand, train, and integrate features from the Form Recognizer service into your applications. The studio provides a platform for you to experiment with the different Form Recognizer models and sample their returned data in an interactive manner without the need to write code.

+The studio supports Form Recognizer v3.0 models and v3.0 model training. Previously trained v2.1 models with labeled data are supported, but not v2.1 model training. Refer to the [REST API migration guide](v3-migration-guide.md) for detailed information about migrating from v2.1 to v3.0.

+## Get started using Form Recognizer Studio

+1. To use Form Recognizer Studio, you need the following assets:

+ * **Azure subscription** - [Create one for free](https://azure.microsoft.com/free/cognitive-services/).

+ * **Cognitive Services or Form Recognizer resource**. Once you have your Azure subscription, create a [single-service](https://portal.azure.com/#create/Microsoft.CognitiveServicesFormRecognizer) or [multi-service](https://portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne) resource, in the Azure portal to get your key and endpoint. Use the free pricing tier (`F0`) to try the service, and upgrade later to a paid tier for production.

+1. Navigate to the [Form Recognizer Studio](https://formrecognizer.appliedai.azure.com/). If it's your first time logging in, a popup window appears prompting you to configure your service resource. You have two options:

+ **a. Access by Resource**.

+ * Choose your existing subscription.

+ * Select an existing resource group within your subscription or create a new one.

+ * Select your existing Form Recognizer or Cognitive services resource.

+ :::image type="content" source="media/studio/welcome-to-studio.png" alt-text="Screenshot of the configure service resource window.":::

+ **b. Access by API endpoint and key**.

+ * Retrieve your endpoint and key from the Azure portal.

+ * Go to the overview page for your resource and select **Keys and Endpoint** from the left navigation bar.

+ * Enter the values in the appropriate fields.

+ :::image type="content" source="media/containers/keys-and-endpoint.png" alt-text="Screenshot: keys and endpoint location in the Azure portal.":::

+1. Once you've completed configuring your resource, you're able to try the different models offered by Form Recognizer Studio. From the front page, select any Form Recognizer model to try using with a no-code approach.

+ :::image type="content" source="media/studio/form-recognizer-studio-front.png" alt-text="Screenshot of Form Recognizer Studio front page.":::

+1. After you've tried Form Recognizer Studio, use the [**C#**](quickstarts/get-started-sdks-rest-api.md?view=form-recog-3.0.0&preserve-view=true), [**Java**](quickstarts/get-started-sdks-rest-api.md?view=form-recog-3.0.0&preserve-view=true), [**JavaScript**](quickstarts/get-started-sdks-rest-api.md?view=form-recog-3.0.0&preserve-view=true) or [**Python**](quickstarts/get-started-sdks-rest-api.md?view=form-recog-3.0.0&preserve-view=true) client libraries or the [**REST API**](quickstarts/get-started-sdks-rest-api.md?view=form-recog-3.0.0&preserve-view=true) to get started incorporating Form Recognizer models into your own applications.

+ To learn more about each model, *see* concepts pages.

+ | Model type| Models |

+ |--|--|

+ |Document analysis models| <ul><li>[**Read model**](concept-read.md)</li><li>[**Layout model**](concept-layout.md)</li><li>[**General document model**](concept-general-document.md)</li></ul>.</br></br>

+ |**Prebuilt models**|<ul><li>[**W-2 form model**](concept-w2.md)</li><li>[**Invoice model**](concept-invoice.md)</li><li>[**Receipt model**](concept-receipt.md)</li><li>[**ID document model**](concept-id-document.md)</li><li>[**Business card model**](concept-business-card.md)</li></ul>

+ |Custom models|<ul><li>[**Custom model**](concept-custom.md)</li><ul><li>[**Template model**](concept-custom-template.md)</li><li>[**Neural model**](concept-custom-template.md)</li></ul><li>[**Composed model**](concept-model-overview.md)</li></ul>

+### Manage your resource

+ To view resource details such as name and pricing tier, select the **Settings** icon in the top-right corner of the Form Recognizer Studio home page and select the **Resource** tab. If you have access to other resources, you can switch resources as well.

+With Form Recognizer, you can quickly automate your data processing in applications and workflows, easily enhance data-driven strategies, and skillfully enrich document search capabilities.

+## Next steps

+* Visit [Form Recognizer Studio](https://formrecognizer.appliedai.azure.com/studio) to begin using the models presented by the service.

+* For more information on Form Recognizer capabilities, see [Azure Form Recognizer overview](overview.md).

+ Title: Azure Automation migration to managed identities FAQ

+# FAQ for migrating from a Run As account to managed identities

+The following FAQ can help you migrate from a Run As account to a Managed identity in Azure Automation. If you have any other questions about the capabilities, post them on the [discussion forum](https://aka.ms/retirement-announcement-automation-runbook-start-using-managed-identities). When a question is frequently asked, we add it to this article so that it benefits everyone.

+Automation Run As accounts will be supported until *30 September 2023*. Moreover, starting 01 April 2023, creation of **new** Run As accounts in Azure Automation will not be possible. Renewing of certificates for existing Run As accounts would be possible only till the end of support.

+Yes, they'll be able to authenticate. There will be no impact to existing runbooks that use a Run As account. After 30 September 2023, all runbook executions using RunAs accounts, including Classic Run As accounts wouldn't be supported. Hence, you must migrate all runbooks to use Managed identities before that date.

+## My Run as account will expire soon, how can I renew it?

+If your Run As account certificate is going to expire soon, it's a good time to start using Managed identities for authentication instead of renewing the certificate. However, if you still want to renew it, you would be able to do it through the portal only till 30 September 2023.

+## Can I create new Run As accounts?

+From 1 April 2023, creation of new Run As accounts wouldn't be possible. We strongly recommend that you start using Managed identities for authentication instead of creating new Run As accounts.

+

+Yes, the runbooks will be able to authenticate until the Run As account certificate expires. After 30 September 2023, all runbook executions using RunAs accounts wouldn't be supported.

+ Title: Migrate from a Run As account to Managed identities

+description: This article describes how to migrate from a Run As account to managed identities in Azure Automation.

+# Migrate from an existing Run As account to Managed identities

+> Azure Automation Run As accounts will retire on *30 September 2023* and completely move to [Managed Identities](automation-security-overview.md#managed-identities). All runbook executions using RunAs accounts, including Classic Run As accounts wouldn't be supported after this date. Starting 01 April 2023, the creation of **new** Run As accounts in Azure Automation will not be possible.

+For more information about migration cadence and the support timeline for Run As account creation and certificate renewal, see the [frequently asked questions](automation-managed-identity-faq.md).

+1. If you're using Classic Run As accounts, ensure that you have [migrated](../virtual-machines/classic-vm-deprecation.md) resources deployed through classic deployment model to Azure Resource Manager.

+1. Use [this script](https://github.com/azureautomation/runbooks/blob/master/Utility/AzRunAs/Check-AutomationRunAsAccountRoleAssignments.ps1) to find out which Automation accounts are using a Run As account. If your Azure Automation accounts contain a Run As account, it will have the built-in contributor role assigned to it by default. You can use the script to check the Azure Automation Run As accounts and determine if their role assignment is the default one or if it has been changed to a different role definition.

+ - If you're using `Az` modules, update to the latest version by following the steps in the [Update Azure PowerShell modules](./automation-update-azure-modules.md?branch=main#update-az-modules) article.

+The following examples of runbook scripts fetch the Resource Manager resources by using the Run As account (service principal) and the managed identity. You would notice the difference in runbook code at the beginning of the runbook, where it authenticates against the resource.

+# [Run As account](#tab/run-as-account)

+```powershell-interactive

+ $connectionName = "AzureRunAsConnection"

+ try

+ {

+ # Get the connection "AzureRunAsConnection"

+ $servicePrincipalConnection=Get-AutomationConnection -Name $connectionName

+ "Logging in to Azure..."

+ Add-AzureRmAccount `

+ -ServicePrincipal `

+ -TenantId $servicePrincipalConnection.TenantId `

+ -ApplicationId $servicePrincipalConnection.ApplicationId `

+ -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint

+ }

+ catch {

+ if (!$servicePrincipalConnection)

+ {

+ $ErrorMessage = "Connection $connectionName not found."

+ throw $ErrorMessage

+ } else{

+ Write-Error -Message $_.Exception

+ throw $_.Exception

+ }

+ }

+ #Get all Resource Manager resources from all resource groups

+ $ResourceGroups = Get-AzureRmResourceGroup

+ foreach ($ResourceGroup in $ResourceGroups)

+ {

+ Write-Output ("Showing resources in resource group " + $ResourceGroup.ResourceGroupName)

+ $Resources = Find-AzureRmResource -ResourceGroupNameContains $ResourceGroup.ResourceGroupName | Select ResourceName, ResourceType

+ ForEach ($Resource in $Resources)

+ {

+ Write-Output ($Resource.ResourceName + " of type " + $Resource.ResourceType)

+ }

+ Write-Output ("")

+ }

+ ```

+ For use with the Run As account, the cmdlet uses the `ServicePrinicipalCertificate` parameter set to `ApplicationId`. `CertificateThumbprint` will be from `RunAsAccountConnection`.

+* For integrating runbook source control with Visual Studio Codespaces, see [Azure Automation: Integrating Runbook Source Control using Visual Studio Codespaces](https://azure.microsoft.com/blog/azure-automation-integrating-runbook-source-control-using-visual-studio-online/).

+- `string: ProvisioningState`

+

+The following JSON document is an example of the Azure Arc Data Controller resource.

+```json

+{

+ "id": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.AzureArcData/dataControllers/contosodc",

+ "name": "contosodc",

+ "type": "microsoft.azurearcdata/datacontrollers",

+ "location": "eastus",

+ "extendedLocation": {

+ "name": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.ExtendedLocation/customLocations/contoso",

+ "type": "CustomLocation"

+ },

+ "tags": {},

+ "systemData": {

+ "createdBy": "contosouser@contoso.com",

+ "createdByType": "User",

+ "createdAt": "2023-01-03T21:35:36.8412132Z",

+ "lastModifiedBy": "319f651f-7ddb-4fc6-9857-7aef9250bd05",

+ "lastModifiedByType": "Application",

+ "lastModifiedAt": "2023-02-15T17:13:26.6429039Z"

+ },

+ "properties": {

+ "infrastructure": "azure",

+ "onPremiseProperty": {

+ "id": "4eb0a7a5-5ed6-4463-af71-12590b2fad5d",

+ "publicSigningKey": "MIIDWzCCAkOgAwIBAgIIA8OmTJKpD8AwDQYJKoZIhvcNAQELBQAwKDEmMCQGA1UEAxMdQ2x1c3RlciBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMjMwMTAzMjEzNzUxWhcNMjgwMTAyMjEzNzUxWjAaMRgwFgYDVQQDEw9iaWxsaW5nLXNpZ25pbmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3rAuXaXIeaipFiqGW5rtkdq/1+S58CRMEkANHvwFnimXEWIt8VnbG9foIm20r0RK+6XeRpn5r92jrOl/3R4Q9AAiF3Tgzy3NF9Dg9OsKo1bnrfWHMxmyX2w8TxyZSvWKEUVpVhjhqyhy/cqSJA5ASjEtthMx4Q1HTVcEDSTfnPHPz9EhfZqZ6ES3Yqun2D9MIatkSUpjHJbqYwRTzzrsPG84hJX7EGAWntvEzzCjmTUsouShEwUhi8c05CLBwzF5bxDNLhTdy+tj2ZyUzL7R+BmifwPR9jvOziYPlrbgIIs77sPbNlZjZvMeeBaJHktWZ0s8/UpUpV1W69m7hT2gbAgMBAAGjgZYwgZMwIAYDVR0lAQH/BBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMA4GA1UdDwEB/wQEAwIFoDBfBgNVHREEWDBWgg5jb250cm9sbGVyLXN2Y4IoY29udHJvbGxlci1zdmMuY29udG9zby5zdmMuY2x1c3Rlci5sb2NhbIIaY29udHJvbGxlci1zdmMuY29udG9zby5zdmMwDQYJKoZIhvcNAQELBQADggEBADcZNIZcDDUC79ElbRrXdbHo9bUUv/NJfY7Dx226jc8j0AdDq8MbHAnt+JiMH6+GDb88avleA448yZ9ujBP9zC8v8IyaWu4vQpPT7MagzlsAhb6VEWU0FQfM6R14WwbATWSOIwDlMn4I33mZULyJdZhk4TqzqTQ8F0I3TavHh8TWBbjnwg1IhR/8TQ9HfgceoI80SBE3BDI5at/CzYgoWcWS2pzfd3QYwD8DIPVLCdcx1LNSDjdlQCQTKal0yKMauGIzMuYpCF1M6Z0LunPU/Ns96T9mqLXJHu+wmAoJ2CwdXa4FruwTSgrQlY3pokjTMwGaP3uzpnCSI7ykvi5kp4Q=",

+ "signingCertificateThumbprint": "8FB48D0DD44DCFB25ECC13B9CB5F493F5438D38C"

+ },

+ "k8sRaw": {

+ "kind": "DataController",

+ "spec": {

+ "credentials": {

+ "dockerRegistry": "arc-private-registry",

+ "domainServiceAccount": "domain-service-account-secret",

+ "serviceAccount": "sa-arc-controller"

+ },

+ "security": {

+ "allowDumps": true,

+ "allowNodeMetricsCollection": true,

+ "allowPodMetricsCollection": true

+ },

+ "services": [

+ {

+ "name": "controller",

+ "port": 30080,

+ "serviceType": "LoadBalancer"

+ }

+ ],

+ "settings": {

+ "ElasticSearch": {

+ "vm.max_map_count": "-1"

+ },

+ "azure": {

+ "autoUploadMetrics": "true",

+ "autoUploadLogs": "false",

+ "subscription": "7894901a-dfga-rf4d-85r4-cc1234459df2",

+ "resourceGroup": "contoso-rg",

+ "location": "eastus",

+ "connectionMode": "direct"

+ },

+ "controller": {

+ "logs.rotation.days": "7",

+ "logs.rotation.size": "5000",

+ "displayName": "contosodc"

+ }

+ },

+ "storage": {

+ "data": {

+ "accessMode": "ReadWriteOnce",

+ "className": "managed-premium",

+ "size": "15Gi"

+ },

+ "logs": {

+ "accessMode": "ReadWriteOnce",

+ "className": "managed-premium",

+ "size": "10Gi"

+ }

+ },

+ "infrastructure": "azure",

+ "docker": {

+ "registry": "mcr.microsoft.com",

+ "imageTag": "v1.14.0_2022-12-13",

+ "repository": "arcdata",

+ "imagePullPolicy": "Always"

+ }

+ },

+ "metadata": {

+ "namespace": "contoso",

+ "name": "contosodc",

+ "annotations": {

+ "management.azure.com/apiVersion": "2022-03-01-preview",

+ "management.azure.com/cloudEnvironment": "AzureCloud",

+ "management.azure.com/correlationId": "aa531c88-6dfb-46c3-af5b-d93f7eaaf0f6",

+ "management.azure.com/customLocation": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.ExtendedLocation/customLocations/contoso",

+ "management.azure.com/location": "eastus",

+ "management.azure.com/operationId": "265b98a7-0fc2-4dce-9cef-26f9b6dd000c*705EDFCA81D01028EFA1C3E9CB3CEC2BF472F25894ACB2FFDF955711236F486D",

+ "management.azure.com/resourceId": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.AzureArcData/dataControllers/contosodc",

+ "management.azure.com/systemData": "{\"createdBy\":\"9c1a17be-338f-4b3c-90e9-55eb526c5aef\",\"createdByType\":\"User\",\"createdAt\":\"2023-01-03T21:35:36.8412132Z\",\"resourceUID\":\"74087467-4f98-4a23-bacf-a1e40404457f\"}",

+ "management.azure.com/tenantId": "123488bf-8asd-41wf-91ab-211kl345db47",

+ "traceparent": "00-197d885376f938d6138babf8ed4d809c-1a584b84b3c8f5df-01"

+ },

+ "creationTimestamp": "2023-01-03T21:35:42Z",

+ "generation": 2,

+ "resourceVersion": "15446366",

+ "uid": "4eb0a7a5-5ed6-4463-af71-12590b2fad5d"

+ },

+ "apiVersion": "arcdata.microsoft.com/v5",

+ "status": {

+ "observedGeneration": 2,

+ "state": "Ready",

+ "azure": {

+ "uploadStatus": {

+ "logs": {

+ "lastUploadTime": "0001-01-01T00:00:00Z",

+ "message": "Automatic upload of logs is disabled. Execution time: 02/15/2023 17:07:57"

+ },

+ "metrics": {

+ "lastUploadTime": "2023-02-15T17:00:57.047934Z",

+ "message": "Success"

+ },

+ "usage": {

+ "lastUploadTime": "2023-02-15T17:07:53.843439Z",

+ "message": "Success. Records uploaded: 1."

+ }

+ }

+ },

+ "lastUpdateTime": "2023-02-15T17:07:57.587925Z",

+ "runningVersion": "v1.14.0_2022-12-13",

+ "arcDataServicesK8sExtensionLatestVersion": "v1.16.0",

+ "registryVersions": {

+ "available": [

+ "v1.16.0_2023-02-14",

+ "v1.15.0_2023-01-10"

+ ],

+ "behind": 2,

+ "current": "v1.14.0_2022-12-13",

+ "latest": "v1.16.0_2023-02-14",

+ "next": "v1.15.0_2023-01-10",

+ "previous": "v1.13.0_2022-11-08"

+ }

+ }

+ },

+ "provisioningState": "Succeeded"

+ }

+}

+```

+- `object: K8sRaw` [Details](https://github.com/microsoft/azure_arc/tree/main/arc_data_services/crds)

+- `public string: ProvisioningState`

+

+The following JSON document is an example of the SQL Managed Instance - Azure Arc resource.

+```json

+{

+ "id": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.AzureArcData/sqlManagedInstances/sqlmi1",

+ "name": "sqlmi1",

+ "type": "microsoft.azurearcdata/sqlmanagedinstances",

+ "sku": {

+ "name": "vCore",

+ "tier": "BusinessCritical"

+ },

+ "location": "eastus",

+ "extendedLocation": {

+ "name": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourcegroups/contoso-rg/providers/microsoft.extendedlocation/customlocations/contoso",

+ "type": "CustomLocation"

+ },

+ "tags": {},

+ "systemData": {

+ "createdBy": "contosouser@contoso.com",

+ "createdByType": "User",

+ "createdAt": "2023-01-04T01:33:57.5232885Z",

+ "lastModifiedBy": "319f651f-7ddb-4fc6-9857-7aef9250bd05",

+ "lastModifiedByType": "Application",

+ "lastModifiedAt": "2023-02-15T01:39:11.6582399Z"

+ },

+ "properties": {

+ "dataControllerId": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.AzureArcData/dataControllers/contosodc",

+ "admin": "sqladmin",

+ "k8sRaw": {

+ "spec": {

+ "scheduling": {

+ "default": {

+ "resources": {

+ "requests": {

+ "cpu": "2",

+ "memory": "4Gi"

+ },

+ "limits": {

+ "cpu": "2",

+ "memory": "4Gi"

+ }

+ }

+ }

+ },

+ "replicas": 2,

+ "dev": true,

+ "services": {

+ "primary": {

+ "type": "LoadBalancer"

+ },

+ "readableSecondaries": {}

+ },

+ "readableSecondaries": 1,

+ "syncSecondaryToCommit": 0,

+ "storage": {

+ "data": {

+ "volumes": [

+ {

+ "size": "5Gi"

+ }

+ ]

+ },

+ "logs": {

+ "volumes": [

+ {

+ "size": "5Gi"

+ }

+ ]

+ },

+ "datalogs": {

+ "volumes": [

+ {

+ "size": "5Gi"

+ }

+ ]

+ },

+ "backups": {

+ "volumes": [

+ {

+ "className": "azurefile",

+ "size": "5Gi"

+ }

+ ]

+ }

+ },

+ "security": {

+ "adminLoginSecret": "sqlmi1-login-secret"

+ },

+ "tier": "BusinessCritical",

+ "update": {},

+ "backup": {

+ "retentionPeriodInDays": 7

+ },

+ "licenseType": "LicenseIncluded",

+ "orchestratorReplicas": 1,

+ "parentResource": {

+ "apiGroup": "arcdata.microsoft.com",

+ "kind": "DataController",

+ "name": "contosodc",

+ "namespace": "contoso"

+ },

+ "settings": {

+ "collation": "SQL_Latin1_General_CP1_CI_AS",

+ "language": {

+ "lcid": 1033

+ },

+ "network": {

+ "forceencryption": 0,

+ "tlsciphers": "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384",

+ "tlsprotocols": "1.2"

+ },

+ "sqlagent": {

+ "enabled": false

+ },

+ "timezone": "UTC"

+ }

+ },

+ "metadata": {

+ "annotations": {

+ "management.azure.com/apiVersion": "2022-03-01-preview",

+ "management.azure.com/cloudEnvironment": "AzureCloud",

+ "management.azure.com/correlationId": "3a49178d-a09f-48d3-9292-3133f6591743",

+ "management.azure.com/customLocation": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/microsoft.extendedlocation/customlocations/contoso",

+ "management.azure.com/location": "eastus",

+ "management.azure.com/operationId": "dbf2e708-78da-4762-8fd5-75ba43721b24*4C234309E6735F28E751F5734D64E8F98A910A88E54A1AD35C6469BCD0E6EA84",

+ "management.azure.com/resourceId": "/subscriptions/7894901a-dfga-rf4d-85r4-cc1234459df2/resourceGroups/contoso-rg/providers/Microsoft.AzureArcData/sqlManagedInstances/sqlmi1",

+ "management.azure.com/systemData": "{\"createdBy\":\"9c1a17be-338f-4b3c-90e9-55eb526c5aef\",\"createdByType\":\"User\",\"createdAt\":\"2023-01-04T01:33:57.5232885Z\",\"resourceUID\":\"40fa8b55-4b7d-4d6a-b783-043169d7fd03\"}",

+ "management.azure.com/tenantId": "123488bf-8asd-41wf-91ab-211kl345db47",

+ "traceparent": "00-3c07cf4caa8b4778591b02b1bf3979ef-f2ee2c890c21ea8a-01"

+ },

+ "creationTimestamp": "2023-01-04T01:34:03Z",

+ "generation": 1,

+ "labels": {

+ "management.azure.com/resourceProvider": "Microsoft.AzureArcData"

+ },

+ "name": "sqlmi1",

+ "namespace": "contoso",

+ "resourceVersion": "15215035",

+ "uid": "6d653cd8-f17e-437a-b0dc-48154164c1ad"

+ },

+ "status": {

+ "lastUpdateTime": "2023-02-15T01:39:07.691211Z",

+ "observedGeneration": 1,

+ "readyReplicas": "2/2",

+ "roles": {

+ "sql": {

+ "replicas": 2,

+ "lastUpdateTime": "2023-02-14T11:37:14.875705Z",

+ "readyReplicas": 2

+ }

+ },

+ "state": "Ready",

+ "endpoints": {

+ "logSearchDashboard": "https://230.41.13.18:5601/app/kibana#/discover?_a=(query:(language:kuery,query:'custom_resource_name:sqlmi1'))",

+ "metricsDashboard": "https://230.41.13.18:3000/d/40q72HnGk/sql-managed-instance-metrics?var-hostname=sqlmi1-0",

+ "mirroring": "230.41.13.18:5022",

+ "primary": "230.41.13.18,1433",

+ "secondary": "230.41.13.18,1433"

+ },

+ "highAvailability": {

+ "lastUpdateTime": "2023-02-14T11:47:42.208708Z",

+ "mirroringCertificate": "--BEGIN CERTIFICATE--\nMIIDQzCCAiugAwIBAgIISqqmfCPaolkwDQYJKoZIhvcNAQELBQAwKDEmMCQGA1UEAxMdQ2x1c3Rl\r\nciBDZXJ0aWZpDEzNDA2WhcNMjgwMTAzMDEzNDA2WjAO\r\nMQwwCgYDVQQDEwNkYm0wggEiMA0GCSqgEKAoIBAQDEXj2nm2cGkyfu\r\npXWQ4s6G//AI1rbH4JStZOAHwJNYmBuESSHz0i6znjnQQloFe+g2KM+1m4TN1T39Lz+/ufEYQQX9\r\nx9WuGP2IALgH1LXc/0DGuOB16QXqN7ZWULQ4ovW4Aaz5NxTSDXWYPK+zpb1c8adsQyamLHwmSPs4\r\nMpsgfOR9EUCqdnuKjSHbWCtkJTYogpAFyZb5HOgY1TMICrTkXG6VYoCPS/EDNmtPOyVuykdjjsxx\r\nIC5KkVgHWTaYIDjim7L44FPh4HUIVM/OFScRijCZTJogN/Fe94+kGDWfgWIG36Jlz127BbWV3HNJ\r\nkH2oLchIABvgTXsdKnjK3i2TAgMBAAGjgYowgYcwIAYDVR0lAQH/BBYwFAYIKwYBBQUHAwIGCCsG\r\nAQUFBwMBMA4GA1UdDwEB/wQEAwIFoDBTBgNVHREETDBKggpzcWxtaTEtc3ZjgiRzcWxtaTEtc3Zj\r\nLmNvbnRvc28uc3ZjLmNsdXN0ZXIubG9jYWyCFnNxbG1pMS1zdmMuY29udG9zby5zdmMwDQYJKoZI\r\nhvcNAQELBQADggEBAA+Wj6WK9NgX4szxT7zQxPVIn+0iviO/2dFxHmjmvj+lrAffsgNdfeX5095f\r\natxIO+no6VW2eoHze2f6AECh4/KefyAzd+GL9MIksJcMLqSqAemXju3pUfGBS1SAW8Rh361D8tmA\r\nEFpPMwZG3uMidYMso0GqO0tpejz2+5Q4NpweHBGoq6jk+9ApTLD+s5qetZHrxGD6tS1Z/Lvt24lE\r\nKtSKEDw5O2qnqbsOe6xxtPAuIfTmpwIzIv2WiGC3aGuXSr0bNyPHzh5RL1MCIpwLMrnruFwVzB25\r\nA0xRalcXVZRZ1H0zbznGsecyBRJiA+7uxNB7/V6i+SjB/qxj2xKh4s8=\n--END CERTIFICATE--\n",

+ "healthState": "Error",

+ "replicas": []

+ },

+ "logSearchDashboard": "https://230.41.13.18:5601/app/kibana#/discover?_a=(query:(language:kuery,query:'custom_resource_name:sqlmi1'))",

+ "metricsDashboard": "https://230.41.13.18:3000/d/40q72HnGk/sql-managed-instance-metrics?var-hostname=sqlmi1-0",

+ "primaryEndpoint": "230.41.13.18,1433",

+ "runningVersion": "v1.14.0_2022-12-13",

+ "registryVersions": {

+ "available": [],

+ "behind": 0,

+ "current": "v1.14.0_2022-12-13",

+ "latest": "v1.14.0_2022-12-13",

+ "previous": "v1.13.0_2022-11-08"

+ }

+ }

+ },

+ "provisioningState": "Succeeded",

+ "licenseType": "LicenseIncluded"

+ }

+}

+```

+az connectedk8s delete -n AzureArcTest1 -g AzureArcTest --force

+### Configuration file example

+The example below highlights a couple key requirements for Arc resource bridge when creating the configuration files. The IPs for `k8snodeippoolstart` and `k8snodeippoolend` reside in the subnet range designated in `ipaddressprefix`. The `ipaddressprefix` is provided in the format of the subnet's IP address range for the virtual network and subnet mask (IP Mask) in CIDR notation.

+```

+azurestackhciprovider:

+virtualnetwork:

+name: "mgmtvnet

+vswitchname: "Default Switch

+type: "Transparent"

+macpoolname

+vlanid: 0

+ipaddressprefix: 172.16.0.0/16

+gateway: 17.16.1.1

+dnsservers: 17.16.0.1

+vippoolstart: 172.16.250.0

+vippoolend: 172.16.250.254

+k8snodeippoolstart: 172.16.30.0

+k8snodeippoolend: 172.16.30.254

+```

+1. In the *web.config* file, you can set the `<appSettings>` element for running the application locally.

+- **Security**

+- **Performance**

+ - The AMA agent event throughput is 25% better than the MMA agent.

+- [Frequently asked questions for Azure Monitor Agent migration](/azure/azure-monitor/faq#azure-monitor-agent)

+Monitoring of your Java web applications running on [Azure App Services](../../app-service/index.yml) doesn't require any modifications to the code. This article walks you through enabling Azure Monitor Application Insights monitoring and provides preliminary guidance for automating the process for large-scale deployments.

+You can apply extra configurations, and then based on your specific scenario you [add your own custom telemetry](./opentelemetry-enable.md?tabs=java#modify-telemetry) if needed.

+You can turn on monitoring for your Java apps running in Azure App Service just with one selection, no code change required. The integration adds [Application Insights Java 3.x](./opentelemetry-enable.md?tabs=java) and auto-collects telemetry.

+3. This last step is optional. After specifying which resource to use, you can configure the Java agent. If you don't configure the Java agent, default configurations apply.

+ The full [set of configurations](./java-standalone-config.md) is available, you just need to paste a valid [json file](./java-standalone-config.md#an-example). **Exclude the connection string and any configurations that are in preview** - you're able to add the items that are currently in preview as they become generally available.

+ Once you modify the configurations through Azure portal, APPLICATIONINSIGHTS_CONFIGURATION_FILE environment variable are automatically populated and appear in App Service settings panel. This variable contains the full json content that you've pasted in Azure portal configuration text box for your Java app.

+Use our step-by-step guide to troubleshoot Java-based applications running on Azure App Services.

+1. After enabling application monitoring for your Java app, you can validate that the agent is working by looking at the live metrics - even before you deploy and app to App Service you'll see some requests from the environment. Remember that the full set of telemetry is only available when you have your app deployed and running.

+1. Set APPLICATIONINSIGHTS_SELF_DIAGNOSTICS_LEVEL environment variable to 'debug' if you don't see any errors and there's no telemetry

+## Manually deploy the latest Application Insights Java version

+The Application Insights Java version is updated automatically as part of App Services updates.

+If you encounter an issue that's fixed in the latest version of Application Insights Java, you can update it manually.

+To manually update, follow these steps:

+1. Upload the Java agent jar file to App Service

+ > a. First, get the latest version of Azure CLI by following the instructions [here](/cli/azure/install-azure-cli-windows?tabs=azure-cli).

+ > b. Next, get the latest version of the Application Insights Java agent by following the instructions [here](./opentelemetry-enable.md?tabs=java).

+ > c. Then, deploy the Java agent jar file to App Service using the following command: `az webapp deploy --src-path applicationinsights-agent-{VERSION_NUMBER}.jar --target-path jav?tabs=javase&pivots=platform-linux#3configure-the-maven-plugin) to deploy the agent through the Maven plugin.

+2. Disable Application Insights via the Application Insights tab in the Azure portal.

+3. Once the agent jar file is uploaded, go to App Service configurations and add a new environment variable, `JAVA_OPTS`, with the value `-javaagent:{PATH_TO_THE_AGENT_JAR}/applicationinsights-agent-{VERSION_NUMBER}.jar`.

+4. Restart the app, leaving the **Startup Command** field blank, to apply the changes.

+> [!NOTE]

+> If you set the JAVA_OPTS environment variable, you will have to disable Application Insights in the portal. Alternatively, if you prefer to enable Application Insights from the portal, make sure that you don't set the `JAVA_OPTS` variable in App Service configurations settings.

+|[Metrics Explorer](essentials/metrics-getting-started.md)|You can use Metrics Explorer to interactively work with metric data and create metric alerts. You need minimal training to use Metrics Explorer, but you must be familiar with the metrics you want to analyze. |- Once data collection is configured, no another configuration is required.<br>- Platform metrics for Azure resources are automatically available.<br>- Guest metrics for virtual machines are available after an Azure Monitor agent is deployed to the virtual machine.<br>- Application metrics are available after Application Insights is configured. |

+|[Log Analytics](logs/log-analytics-overview.md)|With Log Analytics, you can create log queries to interactively work with log data and create log query alerts.| Some training is required for you to become familiar with the query language, although you can use prebuilt queries for common requirements. You can also add [query packs](logs/query-packs.md) with queries that are unique to your organization. Then if you're familiar with the query language, you can build queries for others in your organization. |

+### Azure workbooks

+ [Azure Workbooks](./visualize/workbooks-overview.md) provide a flexible canvas for data analysis and the creation of rich visual reports. You can use workbooks to tap into multiple data sources from across Azure and combine them into unified interactive experiences. They're especially useful to prepare end-to-end monitoring views across multiple Azure resources. Insights use prebuilt workbooks to present you with critical health and performance information for a particular service. You can access a gallery of workbooks on the **Workbooks** tab of the Azure Monitor menu and create custom workbooks to meet the requirements of your different users.

+

+|[Azure Workbooks](./visualize/workbooks-overview.md)|- Native dashboarding platform in Azure.<br>- Designed for collaborating and troubleshooting.<br>- Out-of-the-box templates and reports.<br>- Fully customizable. |- Create an interactive report with parameters where selecting an element in a table dynamically updates associated charts and visualizations.<br>- Share a report with other users in your organization.<br>- Collaborate with other workbook authors in your organization by using a public GitHub-based template gallery. | |

+|[Azure dashboards](../azure-portal/azure-portal-dashboards.md)|- Native dashboarding platform in Azure.<br>- Supports at scale deployments.<br>- Supports RBAC.<br>- No added cost|- Create a dashboard that combines a metrics graph and the results of a log query with operational data for related services.<br>- Share a dashboard with service owners through integration with [Azure role-based access control](../role-based-access-control/overview.md). |Azure/Arc exclusive environments|

+|[Azure Managed Grafana](../managed-grafan)|- Multi-platform, multicloud single pane of glass visualizations.<br>- Out-of-the-box plugins from most monitoring tools and platforms.<br>- Dashboard templates with focus on operations.<br>- Supports portability, multi-tenancy, and flexible RBAC.<br>- Azure managed Grafana provides seamless integration with Azure. |- Combine time-series and event data in a single visualization panel.<br>- Create a dynamic dashboard based on user selection of dynamic variables.<br>- Create a dashboard from a community-created and community-supported template.<br>- Create a vendor-agnostic business continuity and disaster scenario that runs on any cloud provider or on-premises. |- Cloud Native CNCF monitoring.<br>- Best with Prometheus.<br>- Multicloud environments.<br>- Combining with 3rd party monitoring tools.|

+|[Power BI](https://powerbi.microsoft.com/documentation/powerbi-service-get-started/) |- Helps design business centric KPI dashboards for long term trends.<br>- Supports BI analytics with extensive slicing and dicing. <br>- Create rich visualizations.<br>- Benefit from extensive interactivity, including zoom-in and cross-filtering.<br>- Share easily throughout your organization.<br>- Integrate data from multiple data sources.<br>- Experience better performance with results cached in a cube. |Dashboarding for long term trends.|

+You can also build your own custom websites and applications using metric and log data in Azure Monitor using the REST API. The REST API gives you flexibility in UI, visualization, interactivity, and features.

+As you deploy resources and applications in Azure, start collecting telemetry to gain insights into their performance and health. Azure makes some metrics available to you out of the box. These metrics are called [standard or platform](./metrics-supported.md).

+Collect custom performance indicators or business-specific metrics to provide deeper insights. These *custom* metrics can be collected via your application telemetry, an agent that runs on your Azure resources, or even an outside-in monitoring system. They can then be submitted directly to Azure Monitor. Once custom metrics are published to Azure Monitor, you can browse, query, and alert on them for your Azure resources and applications along side the standard Azure metrics.

+- Install the Azure Diagnostics extension on your [Azure VM](../essentials/collect-custom-metrics-guestos-resource-manager-vm.md), [Virtual Machine Scale Set](../essentials/collect-custom-metrics-guestos-resource-manager-vmss.md), [classic VM](../essentials/collect-custom-metrics-guestos-vm-classic.md), or [classic cloud service](../essentials/collect-custom-metrics-guestos-vm-cloud-service-classic.md). Then send performance counters to Azure Monitor.

+For details on when billing is enabled for custom metrics and metrics queries, check the [Azure Monitor pricing page](https://azure.microsoft.com/pricing/details/monitor/). In summary, there's no cost to ingest standard metrics (platform metrics) into an Azure Monitor metrics store, but custom metrics incur costs when they enter general availability. Queries to the metrics API do incur costs.

+The subject property captures which Azure resource ID the custom metric is reported for. This information is encoded in the URL of the API call. Each API can submit metric values for only a single Azure resource.

+To limit the number of raw values that you have to emit and pay for in Azure Monitor, locally pre-aggregate and emit the aggregated values:

+Each metric data point published contains a namespace, name, and dimension information. The first time a custom metric is emitted to Azure Monitor, a metric definition is automatically created. This new metric definition is then discoverable on any resource that the metric is emitted from via the metric definitions. There's no need to predefine a custom metric in Azure Monitor before it's emitted.

+> Azure Monitor doesn't support defining **Units** for a custom metric.

+|Total active time series in a subscription per region|50,000|

+|The combined length of all custom metric names, using utf-8 encoding|64 KB|

+There is a limit of 64 KB on the combined length of all custom metrics names, assuming utf-8 or 1 byte per character. If the 64-KB limit is exceeded, metadata for additional metrics won't be available. The metric names for additional custom metrics won't appear in the Azure portal in selection fields, and won't be returned by the API in requests for metric definitions. The metric data is still available and can be queried.

+When the limit has been exceeded, reduce the number of metrics you're sending or shorten the length of their names. It then takes up to two days for the new metrics' names to appear.

+To avoid reaching the limit, don't include variable or dimensional aspects in your metric names.

+For example, the metrics for server CPU usage,`CPU_server_12345678-319d-4a50-b27e-1234567890ab` and `CPU_server_abcdef01-319d-4a50-b27e-abcdef012345` should be defined as metric `CPU` and with a `Server` dimension.

+ - [Virtual Machine Scale Set](../essentials/collect-custom-metrics-guestos-resource-manager-vmss.md)

+ Title: Azure networking analytics solution in Azure Monitor | Microsoft Docs

+description: You can use the Azure networking analytics solution in Azure Monitor to review Azure network security group logs and Azure Application Gateway logs.

+* Network Performance Monitor to:

+ * Monitor the health of your network.

+* Azure Application Gateway analytics to review:

+ * Application Gateway logs.

+ * Application Gateway metrics.

+* Solutions to monitor and audit network activity on your cloud network:

+ * [Traffic analytics](../../networking/network-monitoring-overview.md#traffic-analytics).

+## Network Performance Monitor

+The [Network Performance Monitor](../../networking/network-monitoring-overview.md) management solution is a network monitoring solution that monitors the health, availability, and reachability of networks. It's used to monitor connectivity between:

+* Public cloud and on-premises.

+* Datacenters and user locations like branch offices.

+* Subnets that host various tiers of a multi-tiered application.

+## Application Gateway analytics

+1. Consume the detailed summary for your resource by using the workbook template for Application Gateway.

+If diagnostic logs aren't enabled for Application Gateway, only the default metric data would be populated within the workbook.

+The Application Gateway analytics and the network security group analytics management solutions collect diagnostics logs directly from Application Gateway and network security groups. It isn't necessary to write the logs to Azure Blob Storage, and no agent is required for data collection.

+The following table shows data collection methods and other details about how data is collected for Application Gateway analytics and the network security group analytics.

+| Azure | | |&#8226; | | |When logged |

+### Enable Application Gateway diagnostics in the portal

+1. In the Azure portal, go to the Application Gateway resource to monitor.

+1. Select **Diagnostic settings** to open the following page.

+ 

+ [](media/azure-networking-analytics/application-gateway-diagnostics-2.png#lightbox)

+1. Select the **Send to Log Analytics workspace** checkbox.

+1. Select an existing Log Analytics workspace or create a workspace.

+1. Select the checkbox under **log** for each of the log types to collect.

+1. Select **Save** to enable the logging of diagnostics to Azure Monitor.

+#### Enable Azure network diagnostics by using PowerShell

+The following PowerShell script provides an example of how to enable resource logging for application gateways:

+#### Access Application Gateway analytics via Azure Monitor Network Insights

+Application insights can be accessed via the **Insights** tab in your Application Gateway resource.

+

+The **View detailed metrics** tab opens the pre-populated workbook that summarizes the data from your Application Gateway resource.

+[](media/azure-networking-analytics/application-gateway-workbook.png#lightbox)

+### New capabilities with an Azure Monitor Network Insights workbook

+> No other costs are associated with an Azure Monitor Network Insights workbook. The Log Analytics workspace will continue to be billed per usage.

+The Network Insights workbook allows you to take advantage of the latest capabilities of Azure Monitor and Log Analytics, including:

+* Flexible canvas to support creation of custom-rich [visualizations](../visualize/workbooks-overview.md#visualizations).

+* Ability to consume and [share workbook templates](../visualize/workbooks-templates.md) with a wider community.

+For more information about the capabilities of the new workbook solution, see [Workbooks overview](../visualize/workbooks-overview.md).

+## Migrate from the Azure Gateway analytics solution to Azure Monitor workbooks

+> We recommend the Azure Monitor Network Insights workbook solution for accessing metric and log analytics for your Application Gateway resources.

+1. Ensure that [diagnostics settings are enabled](#enable-application-gateway-diagnostics-in-the-portal) to store logs in a Log Analytics workspace. If it's already configured, the Azure Monitor Network Insights workbook will be able to consume data from the same location. No more changes are required.

+ > [!NOTE]

+ > All past data is already available within the workbook from the point when diagnostic settings were originally enabled. No data transfer is required.

+1. Access the [default insights workbook](#access-application-gateway-analytics-via-azure-monitor-network-insights) for your Application Gateway resource. All existing insights supported by the Application Gateway analytics solution will be already present in the workbook. You can add custom [visualizations](../visualize/workbooks-overview.md#visualizations) based on metric and log data.

+1. After you see all your metric and log insights, to clean up the Azure Gateway analytics solution from your workspace, delete the solution from the **Solution Resources** pane.

+ [](media/azure-networking-analytics/application-gateway-analytics-delete.png#lightbox)

+Follow the steps here to troubleshoot Azure Diagnostics.

+Use [log queries in Azure Monitor](../logs/log-query-overview.md) to view detailed Azure Diagnostics data.

+# Gather insights about your DNS infrastructure with the DNS Analytics preview solution

+

+> The Log Analytics agent will be **retired on August 31, 2024**. If you're using the Log Analytics agent in your Microsoft Sentinel deployment, we recommend that you start planning your migration to the Azure Monitor Agent. For more information, see [Azure Monitor Agent migration for Microsoft Sentinel](../..//sentinel/ama-migrate.md).

+| Connected source | Support | Description |

+| [Linux agents](../vm/monitor-virtual-machine.md) | No | The solution doesn't collect DNS information from direct Linux agents. |

+| [System Center Operations Manager management group](../agents/om-agents.md) | Yes | The solution collects DNS information from agents in a connected Operations Manager management group. A direct connection from the Operations Manager agent to Azure Monitor isn't required. Data is forwarded from the management group to the Log Analytics workspace. |

+| [Azure Storage account](../essentials/resource-logs.md#send-to-log-analytics-workspace) | No | Azure Storage isn't used by the solution. |

+- You can add the DNS Analytics solution to your Log Analytics workspace from [Azure Marketplace](https://aka.ms/dnsanalyticsazuremarketplace). You can also use the process described in [Add Azure Monitor solutions from the Solutions Gallery](solutions.md).

+From the Log Analytics workspace in the Azure portal, select **Workspace summary**. Then select the **DNS Analytics** tile. On the solution dashboard, select **Configuration** to open the **DNS Analytics Configuration** page. There are two types of configuration changes that you can make:

+- **Allowlisted Domain Names**: The solution doesn't process all the lookup queries. It maintains an allowlist of domain name suffixes. The lookup queries that resolve to the domain names that match domain name suffixes in this allowlist aren't processed by the solution. Not processing allowlisted domain names helps to optimize the data sent to Azure Monitor. The default allowlist includes popular public domain names, such as www.google.com and www.facebook.com. You can view the complete default list by scrolling.

+- **Talkative Client Threshold**: DNS clients that exceed the threshold for the number of lookup requests are highlighted in the **DNS Clients** pane. The default threshold is 1,000. You can edit the threshold.

+ 

+If you're using the Microsoft Monitoring Agent to connect to your Log Analytics workspace, the following management pack is installed:

+If your Operations Manager management group is connected to your Log Analytics workspace, the following management packs are installed in Operations Manager when you add this solution. There's no required configuration or maintenance of these management packs:

+The DNS tile includes the number of DNS servers where the data is being collected. It also includes the number of requests made by clients to resolve malicious domains in the past 24 hours. When you select a tile, the solution dashboard opens.

+

+

+**DNS Security**: Reports the DNS clients that are trying to communicate with malicious domains. By using Microsoft threat intelligence feeds, DNS Analytics can detect client IPs that are trying to access malicious domains. In many cases, malware-infected devices "dial out" to the "command and control" center of the malicious domain by resolving the malware domain name.

+

+When you select a client IP in the list, Log Search opens and shows the lookup details of the respective query. In the following example, DNS Analytics detected that the communication was done with an [IRCbot](https://www.microsoft.com/en-us/wdsi/threats/malware-encyclopedia-description?Name=Backdoor:Win32/IRCbot&threatId=2621):

+

+**Domains Queried**: Provides the most frequent domain names being queried by the DNS clients in your environment. You can view the list of all the domain names queried. You can also drill down into the lookup request details of a specific domain name in **Log Search**.

+

+**DNS Clients**: Reports the clients *breaching the threshold* for number of queries in the chosen time period. You can view the list of all the DNS clients and the details of the queries made by them in **Log Search**.

+

+**Dynamic DNS Registrations**: Reports name registration failures. All registration failures for address [resource records](https://en.wikipedia.org/wiki/List_of_DNS_record_types) (Type A and AAAA) are highlighted along with the client IPs that made the registration requests. You can then use this information to find the root cause of the registration failure by following these steps:

+1. Find the zone that's authoritative for the name that the client is trying to update.

+ 

+**Name registration requests**: The upper tile shows a trendline of successful and failed DNS dynamic update requests. The lower tile lists the top 10 clients that are sending failed DNS update requests to the DNS servers, sorted by the number of failures.

+

+**Sample DDI Analytics Queries**: Contains a list of the most common search queries that fetch raw analytics data directly.

+

+You can use these queries as a starting point for creating your own queries for customized reporting. The queries link to the **DNS Analytics Log Search** page where results are displayed:

+- **List of DNS Servers**: Shows a list of all DNS servers with their associated FQDN, domain name, forest name, and server IPs.

+- **List of DNS Zones**: Shows a list of all DNS zones with the associated zone name, dynamic update status, name servers, and DNSSEC signing status.

+- **Unused Resource Records**: Shows a list of all the unused/stale resource records. This list contains the resource record name, resource record type, the associated DNS server, record generation time, and zone name. You can use this list to identify the DNS resource records that are no longer in use. Based on this information, you can then remove those entries from the DNS servers.

+- **DNS Servers Query Load**: Shows information so that you can get a perspective of the DNS load on your DNS servers. This information can help you plan the capacity for the servers. You can go to the **Metrics** tab to change the view to a graphical visualization. This view helps you understand how the DNS load is distributed across your DNS servers. It shows DNS query rate trends for each server.

+ 

+- **DNS Zones Query Load**: Shows the DNS zone-query-per-second statistics of all the zones on the DNS servers being managed by the solution. Select the **Metrics** tab to change the view from detailed records to a graphical visualization of the results.

+- **Configuration Events**: Shows all the DNS configuration change events and associated messages. You can then filter these events based on time of the event, event ID, DNS server, or task category. The data can help you audit changes made to specific DNS servers at specific times.

+- **DNS Analytical Log**: Shows all the analytic events on all the DNS servers managed by the solution. You can then filter these events based on time of the event, event ID, DNS server, client IP that made the lookup query, and query type task category. DNS server analytic events enable activity tracking on the DNS server. An analytic event is logged each time the server sends or receives DNS information.

+On the **Log Search** page, you can create a query. You can filter your search results by using facet controls. You can also create advanced queries to transform, filter, and report on your results. Start by using the following queries:

+1. In the search query box, enter `DnsEvents` to view all the DNS events generated by the DNS servers managed by the solution. The results list the log data for all events related to lookup queries, dynamic registrations, and configuration changes.

+ 

+ 1. To view the log data for lookup queries, select **LookUpQuery** as the **Subtype** filter from the facet control on the left. A table that lists all the lookup query events for the selected time period appears.

+ 1. To view the log data for dynamic registrations, select **DynamicRegistration** as the **Subtype** filter from the facet control on the left. A table that lists all the dynamic registration events for the selected time period appears.

+ 1. To view the log data for configuration changes, select **ConfigurationChange** as the **Subtype** filter from the facet control on the left. A table that lists all the configuration change events for the selected time period appears.

+1. In the search query box, enter `DnsInventory` to view all the DNS inventory-related data for the DNS servers managed by the solution. The results list the log data for DNS servers, DNS zones, and resource records.

+ 

+* **Missing DNS Lookups Data**: To troubleshoot this issue, try resetting the config or loading the configuration page once in the portal. For resetting, change a setting to another value, change it back to the original value, and save the config.

+To provide feedback, see the [Log Analytics UserVoice page](https://aka.ms/dnsanalyticsuservoice) to post ideas for DNS Analytics features to work on.

+Review [Query logs](../logs/log-query-overview.md) to view detailed DNS log records.

+* Perform other queries to maintain awareness of the distribution of agents deployed in Azure, in other cloud environments, or on-premises.

+If your Operations Manager management group is connected to a Log Analytics workspace, the following management packs are installed in Operations Manager. These management packs are also installed on directly connected Windows computers after you add this solution:

+* Microsoft System Center Advisor HealthAssessment Direct Channel Intelligence Pack (Microsoft.IntelligencePacks.HealthAssessmentDirect)

+* Microsoft System Center Advisor HealthAssessment Server Channel Intelligence Pack (Microsoft.IntelligencePacks.HealthAssessmentViaServer)

+| System Center Operations Manager management group | Yes | Heartbeat events are collected from agents that report to the management group every 60 seconds and are then forwarded to Azure Monitor. A direct connection from Operations Manager agents to Azure Monitor isn't required. Heartbeat event data is forwarded from the management group to the Log Analytics workspace.|

+## Use the solution

+Select the **Agent Health** tile to open the **Agent Health** dashboard. The dashboard includes the columns in the following table. Each column lists the top 10 events by count that match that column's criteria for the specified time range. You can run a log search that provides the entire list. Select **See all** beneath each column or select the column heading.

+

+The solution creates one type of record in the Log Analytics workspace: heartbeat. Heartbeat records have the properties listed in the following table.

+Each agent that reports to an Operations Manager management server will send two heartbeats. The `SCAgentChannel` property's value will include both `Direct` and `SCManagementServer`, depending on what data sources and monitoring solutions you've enabled in your subscription.

+* Directly from an Operations Manager management server to Azure Monitor.

+* Directly from the agent to Azure Monitor, because of the volume of data collected on the agent.

+For heartbeat events that have the value `SCManagementServer`, the `ComputerIP` value is the IP address of the management server because it actually uploads the data. For heartbeats where `SCAgentChannel` is set to `Direct`, it's the public IP address of the agent.

+Learn about [generating alerts from log queries in Azure Monitor](../alerts/alerts-overview.md).

+ Title: Target monitoring solutions in Azure Monitor | Microsoft Docs

+description: Targeting monitoring solutions allows you to limit monitoring solutions to a specific set of agents. This article describes how to create a scope configuration and apply it to a solution.

+# Target monitoring solutions in Azure Monitor (preview)

+> This feature has been deprecated because the Log Analytics agent is being replaced with the Azure Monitor Agent. Solutions in Azure Monitor are being replaced with insights. You can continue to use it if you already have it configured, but it's being removed from regions where it isn't already being used. The feature will no longer be supported after August 31, 2024.

+When you add a monitoring solution to your subscription, it's automatically deployed by default to all Windows and Linux agents connected to your Log Analytics workspace. You might want to manage your costs and limit the amount of data collected for a solution by limiting it to a particular set of agents. This article describes how to use *solution targeting*, which is a feature that allows you to apply a scope to your solutions.

+## Target a solution

+There are three steps to targeting a solution, as described in the following sections.

+### Create a computer group

+You specify the computers that you want to include in a scope by creating a [computer group](../logs/computer-groups.md) in Azure Monitor. The computer group can be based on a log query or imported from other sources, such as Active Directory or Windows Server Update Services groups. As described in the section [Solutions and agents that can't be targeted](#solutions-and-agents-that-cant-be-targeted), only computers that are directly connected to Azure Monitor are included in the scope.

+After you have the computer group created in your workspace, you'll include it in a scope configuration that can be applied to one or more solutions.

+### Create a scope configuration

+ A *scope configuration* includes one or more computer groups and can be applied to one or more solutions.

+ To create a scope configuration:

+ 1. In the Azure portal, go to **Log Analytics workspaces** and select your workspace.

+ 1. In the properties for the workspace under **Workspace Data Sources**, select **Scope Configurations**.

+ 1. Select **Add** to create a new scope configuration.

+ 1. Enter a name for the scope configuration.

+ 1. Click **Select Computer Groups**.

+ 1. Select the computer group that you created and optionally any other groups to add to the configuration. Click **Select**.

+ 1. Select **OK** to create the scope configuration.

+### Apply the scope configuration to a solution.

+After you have a scope configuration, you can apply it to one or more solutions. Although a single scope configuration can be used with multiple solutions, each solution can only use one scope configuration.

+To apply a scope configuration:

+ 1. In the Azure portal, go to **Log Analytics workspaces** and select your workspace.

+ 1. In the properties for the workspace, select **Solutions**.

+ 1. Select the solution you want to scope.

+ 1. In the properties for the solution under **Workspace Data Sources**, select **Solution Targeting**. If the option isn't available, [this solution can't be targeted](#solutions-and-agents-that-cant-be-targeted).

+ 1. Select **Add scope configuration**. If you already have a configuration applied to this solution, this option is unavailable. You must remove the existing configuration before you add another one.

+ 1. Select the scope configuration that you created.

+ 1. Watch the **Status** of the configuration to ensure that it shows **Succeeded**. If the status indicates an error, select the ellipses to the right of the configuration and select **Edit scope configuration** to make changes.

+The following criteria are for agents and solutions that can't be used with solution targeting:

+- Solution targeting only applies to solutions provided by Microsoft. It doesn't apply to solutions [created by yourself or partners](./solutions.md).

+- You can only filter out agents that connect directly to Azure Monitor. Solutions automatically deploy to any agents that are part of a connected Operations Manager management group whether or not they're included in a scope configuration.

+Solution targeting can't be used with the following solution even though it fits the stated criteria:

+- Learn more about monitoring solutions, including the solutions that are available to install in your environment, in [Add Azure Log Analytics monitoring solutions to your workspace](solutions.md).

+- Learn more about creating computer groups in [Computer groups in Azure Monitor log queries](../logs/computer-groups.md).

+| Record created at data source | [TimeGenerated](./log-standard-columns.md#timegenerated) <br>If the data source doesn't set this value, it will be set to the same time as _TimeReceived. | If at processing time the Time Generated value is older than two days, the row will be dropped. |

+ Title: What is monitored by Azure Monitor

+description: Reference of all services and other resources monitored by Azure Monitor.

+# What is monitored by Azure Monitor?

+This article is a reference of the different applications and services that are monitored by Azure Monitor.

+Azure Monitor data is collected and stored based on resource provider namespaces. Each resource in Azure has a unique ID. The resource provider namespace is part of all unique IDs. For example, a key vault resource ID would be similar to `/subscriptions/d03b04c7-d1d4-eeee-aaaa-87b6fcb38b38/resourceGroups/KeyVaults/providers/Microsoft.KeyVault/vaults/mysafekeys ` . *Microsoft.KeyVault* is the resource provider namespace. *Microsoft.KeyVault/vaults/* is the resource provider.

+For a list of Azure resource provider namespaces, see [Resource providers for Azure services](../azure-resource-manager/management/azure-services-resource-providers.md).

+For a list of resource providers that support Azure Monitor

+- **Metrics** - See [Supported metrics in Azure Monitor](essentials/metrics-supported.md).

+- **Metric alerts** - See [Supported resources for metric alerts in Azure Monitor](alerts/alerts-metric-near-real-time.md).

+- **Prometheus metrics** - See [Prometheus metrics overview](essentials/prometheus-metrics-overview.md#enable).

+- **Resource logs** - See [Supported categories for Azure Monitor resource logs](essentials/resource-logs-categories.md).

+- **Activity log** - All entries in the activity log are available for query, alerting and routing to Azure Monitor Logs store regardless of resource provider.

+## Services that require agents

+Azure Monitor can't see inside a service running its own application, operating system or container. That type of service requires one or more agents to be installed. The agent then runs as well to collect metrics, logs, traces and changes and forward them to Azure Monitor. The following services require agents for this reason.

+- [Azure Cloud Services](../cloud-services-extended-support/index.yml)

+- [Azure Virtual Machines](../virtual-machines/index.yml)

+- [Azure Virtual Machine Scale Sets](../virtual-machine-scale-sets/index.yml)

+- [Azure Service Fabric](../service-fabric/index.yml)

+In addition, applications also require either the Application Insights SDK or auto-instrumentation (via an agent) to collect information and write it to the Azure Monitor data platform.

+## Services with Insights

+Some services have curated monitoring experiences call "insights". Insights are meant to be a starting point for monitoring a service or set of services. Some insights may also automatically pull additional data that's not captured or stored in Azure Monitor. For more information on monitoring insights, see [Insights Overview](insights/insights-overview.md).

+## Product integrations

+The services and [older monitoring solutions](insights/solutions.md) in the following table store their data in Azure Monitor Logs so that it can be analyzed with other log data collected by Azure Monitor.

+| Product/Service | Description |

+|:|:|

+| [Azure Automation](../automation/index.yml) | Manage operating system updates and track changes on Windows and Linux computers. See [Change tracking](../automation/change-tracking/overview.md) and [Update management](../automation/update-management/overview.md). |

+| [Azure Information Protection](/azure/information-protection/) | Classify and optionally protect documents and emails. See [Central reporting for Azure Information Protection](/azure/information-protection/reports-aip#configure-a-log-analytics-workspace-for-the-reports). |

+| [Defender for the Cloud](../defender-for-cloud/defender-for-cloud-introduction.md) | Collect and analyze security events and perform threat analysis. See [Data collection in Defender for the Cloud](../defender-for-cloud/monitoring-components.md). |

+| [Microsoft Sentinel](../sentinel/index.yml) | Connect to different sources including Office 365 and Amazon Web Services Cloud Trail. See [Connect data sources](../sentinel/connect-data-sources.md). |

+| [Microsoft Intune](/intune/) | Create a diagnostic setting to send logs to Azure Monitor. See [Send log data to storage, Event Hubs, or log analytics in Intune (preview)](/intune/fundamentals/review-logs-using-azure-monitor). |

+| Network [Traffic Analytics](../network-watcher/traffic-analytics.md) | Analyze Network Watcher network security group flow logs to provide insights into traffic flow in your Azure cloud. |

+| [System Center Operations Manager](/system-center/scom) | Collect data from Operations Manager agents by connecting their management group to Azure Monitor. See [Connect Operations Manager to Azure Monitor](agents/om-agents.md).<br> Assess the risk and health of your System Center Operations Manager management group with the [Operations Manager Assessment](insights/scom-assessment.md) solution. |

+| [Microsoft Teams Rooms](/microsoftteams/room-systems/azure-monitor-deploy) | Integrated, end-to-end management of Microsoft Teams Rooms devices. |

+| [Visual Studio App Center](/appcenter/) | Build, test, and distribute applications and then monitor their status and usage. See [Start analyzing your mobile app with App Center and Application Insights](https://github.com/Microsoft/appcenter). |

+| Windows | [Windows Update Compliance](/windows/deployment/update/update-compliance-get-started) - Assess your Windows desktop upgrades.<br>[Desktop Analytics](/configmgr/desktop-analytics/overview) - Integrates with Configuration Manager to provide insight and intelligence to make more informed decisions about the update readiness of your Windows clients. |

+| **The following solutions also integrate with parts of Azure Monitor. Note that solutions, which are based on Azure Monitor Logs and Log Analytics, are no longer under active development. Use [Insights](insights/insights-overview.md) instead.** | |

+| Network - [Network Performance Monitor solution](insights/network-performance-monitor.md) |

+| Network - [Azure Application Gateway solution](insights/azure-networking-analytics.md#application-gateway-analytics) |

+| [Office 365 solution](insights/solution-office-365.md) | Monitor your Office 365 environment. Updated version with improved onboarding available through Microsoft Sentinel. |

+| [SQL Analytics solution](insights/azure-sql.md) | Use SQL Insights instead. |

+| [Surface Hub solution](insights/surface-hubs.md) | |

+## Third-party integration

+| Integration | Description |

+|:|:|

+| [ITSM](alerts/itsmc-overview.md) | The IT Service Management (ITSM) Connector allows you to connect Azure and a supported ITSM product/service. |

+| [Azure Monitor Partners](./partners.md) | A list of partners that integrate with Azure Monitor in some form. |

+| [Azure Monitor Partner integrations](../partner-solutions/overview.md)| Specialized integrations between Azure Monitor and other non-Microsoft monitoring platforms if you've already built on them. Examples include Datadog and Elastic.|

+## Resources outside of Azure

+Azure Monitor can collect data from resources outside of Azure by using the methods listed in the following table.

+| Resource | Method |

+|:|:|

+| Applications | Monitor web applications outside of Azure by using Application Insights. See [What is Application Insights?](./app/app-insights-overview.md). |

+| Virtual machines | Use agents to collect data from the guest operating system of virtual machines in other cloud environments or on-premises. See [Overview of Azure Monitor agents](agents/agents-overview.md). |

+| REST API Client | Separate APIs are available to write data to Azure Monitor Logs and Metrics from any REST API client. See [Send log data to Azure Monitor with the HTTP Data Collector API](logs/data-collector-api.md) for Logs. See [Send custom metrics for an Azure resource to the Azure Monitor metric store by using a REST API](essentials/metrics-store-custom-rest-api.md) for Metrics. |

+## Next steps

+- Read more about the [Azure Monitor data platform that stores the logs and metrics collected by insights and solutions](data-platform.md).

+- Complete a [tutorial on monitoring an Azure resource](essentials/tutorial-resource-logs.md).

+- Complete a [tutorial on writing a log query to analyze data in Azure Monitor Logs](essentials/tutorial-resource-logs.md).

+- Complete a [tutorial on creating a metrics chart to analyze data in Azure Monitor Metrics](essentials/tutorial-metrics.md).

+2. On the left menu, select **Networking**.

+ 3. If you want to allow Microsoft services trusted by the Azure Relay service to bypass this firewall, select **Yes** for **Allow trusted Microsoft services to bypass this firewall?**.

+

+ :::image type="content" source="./media/ip-firewall/selected-networks-trusted-access-disabled.png" alt-text="Screenshot showing the Public access tab of the Networking page with the Firewall enabled.":::

+1. Select **Save** on the toolbar to save the settings. Wait for a few minutes for the confirmation to show up on the portal notifications.

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "namespaces_name": {

+ "defaultValue": "contosorelay0215",

+ "type": "String"

+ "variables": {},

+ {

+ "type": "Microsoft.Relay/namespaces",

+ "apiVersion": "2021-11-01",

+ "name": "[parameters('namespaces_name')]",

+ "location": "East US",

+ "sku": {

+ "name": "Standard",

+ "tier": "Standard"

+ "properties": {}

+ },

+ {

+ "type": "Microsoft.Relay/namespaces/authorizationrules",

+ "apiVersion": "2021-11-01",

+ "name": "[concat(parameters('namespaces_sprelayns0215_name'), '/RootManageSharedAccessKey')]",

+ "location": "eastus",

+ "dependsOn": [

+ "[resourceId('Microsoft.Relay/namespaces', parameters('namespaces_sprelayns0215_name'))]"

+ ],

+ "properties": {

+ "rights": [

+ "Listen",

+ "Manage",

+ "Send"

+ ]

+ }

+ },

+ {

+ "type": "Microsoft.Relay/namespaces/networkRuleSets",

+ "apiVersion": "2021-11-01",

+ "name": "[concat(parameters('namespaces_sprelayns0215_name'), '/default')]",

+ "location": "East US",

+ "dependsOn": [

+ "[resourceId('Microsoft.Relay/namespaces', parameters('namespaces_sprelayns0215_name'))]"

+ ],

+ "properties": {

+ "publicNetworkAccess": "Enabled",

+ "defaultAction": "Deny",

+ "ipRules": [

+ {

+ "ipMask": "172.72.157.204",

+ "action": "Allow"

+ },

+ {

+ "ipMask": "10.1.1.1",

+ "action": "Allow"

+ },

+ {

+ "ipMask": "11.0.0.0/24",

+ "action": "Allow"

+ }

+ ]

+ ]

+}

+To integrate an Azure Relay namespace with Azure Private Link, you need the following entities or permissions:

+4. On the left menu, select the **Networking** tab under **Settings**.

+ :::image type="content" source="./media/private-link-service/add-private-endpoint-button.png" alt-text="Screenshot showing the selection of the Add private endpoint button on the Private endpoint connections tab of the Networking page.":::

+ 3. Enter a **name** for the **private endpoint**.

+ 1. Enter a **name** for the **network interface**.

+ 1. Select a **region** for the private endpoint. Your private endpoint must be in the same region as your virtual network, but can be in a different region from the Azure Relay namespace that you're connecting to.

+ 1. Select **Next: Resource >** button at the bottom of the page.

+ :::image type="content" source="./media/private-link-service/create-private-endpoint-basics-page.png" alt-text="Screenshot showing the Basics page of the Create a private endpoint wizard.":::

+8. Review settings on the **Resource** page, and select **Next: Virtual Network**.

+ :::image type="content" source="./media/private-link-service/create-private-endpoint-resource-page.png" alt-text="Screenshot showing the Resource page of the Create a private endpoint wizard.":::

+9. On the **Virtual Network** page, select the **virtual network** and the **subnet** where you want to deploy the private endpoint. Only virtual networks in the currently selected subscription and location are listed in the drop-down list.

+ :::image type="content" source="./media/private-link-service/create-private-endpoint-virtual-network-page.png" alt-text="Screenshot showing the Virtual Network page of the Create a private endpoint wizard.":::

+ You can configure whether you want to **dynamically** allocate an IP address or **statically** allocate an **IP address** to the private endpoint

+ You can also associate a new or existing **application security group** to the private endpoint.

+3. Select **Next: DNS** to navigate to the **DNS** page of the wizard. On the **DNS** page, **Integrate with private DNZ zone** setting is enabled by default (recommended). You have an option to disable it.

+ :::image type="content" source="./media/private-link-service/create-private-endpoint-dns-page.png" alt-text="Screenshot showing the DNS page of the Create a private endpoint wizard.":::

+ To connect privately with your private endpoint, you need a DNS record. We recommend that you integrate your private endpoint with a **private DNS zone**. You can also utilize your own DNS servers or create DNS records using the host files on your virtual machines. For more information, see [Azure Private Endpoint DNS Configuration](../private-link/private-endpoint-dns.md).

+3. Select **Next: Tags >** button at the bottom of the page.

+ :::image type="content" source="./media/private-link-service/private-endpoint-page.png" alt-text="Screenshot showing the Private endpoint page in the Azure portal.":::

+ :::image type="content" source="./media/private-link-service/private-endpoint-created.png" alt-text="Screenshot showing the Private endpoint connections tab of the Networking page with the private endpoint you just created.":::

+1. If there are any connections that are pending, you see a connection listed with **Pending** in the provisioning state.

+ :::image type="content" source="./media/private-link-service/private-endpoint-approve.png" alt-text="Screenshot showing the Approve button on the command bar for the selected private endpoint.":::

+ :::image type="content" source="./media/private-link-service/approve-connection-page.png" alt-text="Screenshot showing the Approve connection page asking for your confirmation.":::

+1. If there are any private endpoint connections you want to reject, whether it's a pending request or existing connection that was approved earlier, select the endpoint connection and select the **Reject** button.

+ :::image type="content" source="./media/private-link-service/private-endpoint-reject.png" alt-text="Screenshot showing the Reject button on the command bar for the selected private endpoint.":::

+ :::image type="content" source="./media/private-link-service/reject-connection-page.png" alt-text="Screenshot showing the Reject connection page asking for your confirmation.":::

+ :::image type="content" source="./media/private-link-service/remove-endpoint.png" alt-text="Screenshot showing the Remove button on the command bar for the selected private endpoint.":::

+ :::image type="content" source="./media/private-link-service/delete-connection-page.png" alt-text="Screenshot showing the Delete connection page asking you for the confirmation.":::

+## What-if limits

+What-if expands nested templates until these limits are reached:

+- 500 nested templates.

+- 800 resource groups in a cross resource-group deployment.

+- 5 minutes taken for expanding the nested templates.

+When one of the limits is reached, the remaining resources' [change type](#change-types) is set to **Ignore**.

+- `New-AzResourceGroupDeployment -Whatif` for resource group deployments

+- `New-AzSubscriptionDeployment -Whatif` and `New-AzDeployment -Whatif` for subscription level deployments

+- `New-AzResourceGroupDeployment -Confirm` for resource group deployments

+- `New-AzSubscriptionDeployment -Confirm` and `New-AzDeployment -Confirm` for subscription level deployments

+- `$results = Get-AzResourceGroupDeploymentWhatIfResult` for resource group deployments

+- `$results = Get-AzSubscriptionDeploymentWhatIfResult` or `$results = Get-AzDeploymentWhatIfResult` for subscription level deployments

+- [az deployment group what-if](/cli/azure/deployment/group#az-deployment-group-what-if) for resource group deployments

+- [az deployment sub what-if](/cli/azure/deployment/sub#az-deployment-sub-what-if) for subscription level deployments

+- [az deployment mg what-if](/cli/azure/deployment/mg#az-deployment-mg-what-if) for management group deployments

+- [az deployment tenant what-if](/cli/azure/deployment/tenant#az-deployment-tenant-what-if) for tenant deployments

+- [az deployment group create](/cli/azure/deployment/group#az-deployment-group-create)

+- [az deployment sub create](/cli/azure/deployment/sub#az-deployment-sub-create).

+- [az deployment mg create](/cli/azure/deployment/mg#az-deployment-mg-create)

+- [az deployment tenant create](/cli/azure/deployment/tenant#az-deployment-tenant-create)

+- [Deployments - What If](/rest/api/resources/deployments/whatif) for resource group deployments

+- [Deployments - What If At Subscription Scope](/rest/api/resources/deployments/whatifatsubscriptionscope) for subscription deployments

+- [Deployments - What If At Management Group Scope](/rest/api/resources/deployments/whatifatmanagementgroupscope) for management group deployments

+- [Deployments - What If At Tenant Scope](/rest/api/resources/deployments/whatifattenantscope) for tenant deployments.

+- **Create**: The resource doesn't currently exist but is defined in the Bicep file. The resource will be created.

+- **Delete**: This change type only applies when using [complete mode](../templates/deployment-modes.md) for JSON template deployment. The resource exists, but isn't defined in the Bicep file. With complete mode, the resource will be deleted. Only resources that [support complete mode deletion](../templates/deployment-complete-mode-deletion.md) are included in this change type.

+- **Ignore**: The resource exists, but isn't defined in the Bicep file. The resource won't be deployed or modified. When you reach the limits for expanding nested templates, you will encounter this change type. See [What-if limits](#what-if-limits).

+- **NoChange**: The resource exists, and is defined in the Bicep file. The resource will be redeployed, but the properties of the resource won't change. This change type is returned when [ResultFormat](#result-format) is set to `FullResourcePayloads`, which is the default value.

+- **Modify**: The resource exists, and is defined in the Bicep file. The resource will be redeployed, and the properties of the resource will change. This change type is returned when [ResultFormat](#result-format) is set to `FullResourcePayloads`, which is the default value.

+- **Deploy**: The resource exists, and is defined in the Bicep file. The resource will be redeployed. The properties of the resource may or may not change. The operation returns this change type when it doesn't have enough information to determine if any properties will change. You only see this condition when [ResultFormat](#result-format) is set to `ResourceIdOnly`.

+- **FullResourcePayloads** - returns a list of resources that will change and details about the properties that will change

+- **ResourceIdOnly** - returns a list of resources that will change

+- Full resource payloads

+- Resource ID only

+- For Python, use [what-if](/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2019_10_01.operations.deploymentsoperations#what-if-resource-group-name--deployment-name--properties--location-none--custom-headers-none--raw-false--polling-true-operation-config-).

+- For Java, use [DeploymentWhatIf Class](/java/api/com.azure.resourcemanager.resources.models.deploymentwhatif).

+- For .NET, use [DeploymentWhatIf Class](/dotnet/api/microsoft.azure.management.resourcemanager.models.deploymentwhatif).

+- To use the what-if operation in a pipeline, see [Test ARM templates with What-If in a pipeline](https://4bes.nl/2021/03/06/test-arm-templates-with-what-if/).

+- If you notice incorrect results from the what-if operation, please report the issues at [https://aka.ms/whatifissues](https://aka.ms/whatifissues).

+- For a Learn module that demonstrates using what-if, see [Preview changes and validate Azure resources by using what-if and the ARM template test toolkit](/training/modules/arm-template-test/).

+## What-if limits

+What-if expands nested templates until these limits are reached:

+- 500 nested templates.

+- 800 resource groups in a cross resource-group deployment.

+- 5 minutes taken for expanding the nested templates.

+When one of the limits is reached, the remaining resources' [change type](#change-types) is set to **Ignore**.

+- `New-AzResourceGroupDeployment -Whatif` for resource group deployments

+- `New-AzSubscriptionDeployment -Whatif` and `New-AzDeployment -Whatif` for subscription level deployments

+- `New-AzResourceGroupDeployment -Confirm` for resource group deployments

+- `New-AzSubscriptionDeployment -Confirm` and `New-AzDeployment -Confirm` for subscription level deployments

+- `$results = Get-AzResourceGroupDeploymentWhatIfResult` for resource group deployments

+- `$results = Get-AzSubscriptionDeploymentWhatIfResult` or `$results = Get-AzDeploymentWhatIfResult` for subscription level deployments

+- [az deployment group what-if](/cli/azure/deployment/group#az-deployment-group-what-if) for resource group deployments

+- [az deployment sub what-if](/cli/azure/deployment/sub#az-deployment-sub-what-if) for subscription level deployments

+- [az deployment mg what-if](/cli/azure/deployment/mg#az-deployment-mg-what-if) for management group deployments

+- [az deployment tenant what-if](/cli/azure/deployment/tenant#az-deployment-tenant-what-if) for tenant deployments

+- [az deployment group create](/cli/azure/deployment/group#az-deployment-group-create)

+- [az deployment sub create](/cli/azure/deployment/sub#az-deployment-sub-create).

+- [az deployment mg create](/cli/azure/deployment/mg#az-deployment-mg-create)

+- [az deployment tenant create](/cli/azure/deployment/tenant#az-deployment-tenant-create)

+- [Deployments - What If](/rest/api/resources/deployments/whatif) for resource group deployments

+- [Deployments - What If At Subscription Scope](/rest/api/resources/deployments/whatifatsubscriptionscope) for subscription deployments

+- [Deployments - What If At Management Group Scope](/rest/api/resources/deployments/whatifatmanagementgroupscope) for management group deployments

+- [Deployments - What If At Tenant Scope](/rest/api/resources/deployments/whatifattenantscope) for tenant deployments.

+- **Ignore**: The resource exists, but isn't defined in the template. The resource won't be deployed or modified. When you reach the limits for expanding nested templates, you will encounter this change type. See [What-if limits](#what-if-limits).

+- **FullResourcePayloads** - returns a list of resources that will change and details about the properties that will change

+- **ResourceIdOnly** - returns a list of resources that will change

+- For Python, use [what-if](/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.v2019_10_01.operations.deploymentsoperations).

+- For Java, use [DeploymentWhatIf Class](/java/api/com.azure.resourcemanager.resources.models.deploymentwhatif).

+- For .NET, use [DeploymentWhatIf Class](/dotnet/api/microsoft.azure.management.resourcemanager.models.deploymentwhatif).

+ Title: Real-time apps with Azure SignalR Service and Azure Functions

+description: Learn about how Azure SignalR Service and Azure Functions together allow you to create real-time serverless web applications.

+# Real-time apps with Azure SignalR Service and Azure Functions

+Azure SignalR Services combined with Azure Functions allows you to run real-time messaging web apps in a serverless environment. This article provides an overview of how the services work together.

+Azure SignalR Service and Azure Functions are both fully managed, highly scalable services that allow you to focus on building applications instead of managing infrastructure. It's common to use the two services together to provide real-time communications in a [serverless](https://azure.microsoft.com/solutions/serverless/) environment.

+The Azure Functions service allows you to write code in [several languages](../azure-functions/supported-languages.md), including JavaScript, Python, C#, and Java that triggers whenever events occur in the cloud. Examples of these events include:

+ - Storage blobs and queues

+* Visualize IoT device telemetry on a real-time dashboard or map.

+* Update data in an application when documents update in Azure Cosmos DB.

+* Send in-app notifications when new orders are created in Salesforce.

+<!-- Are there more lanaguages now? -->

+1. A change is made in an Azure Cosmos DB collection.

+2. The change event is propagated to the Azure Cosmos DB change feed.

+3. An Azure Functions is triggered by the change event using the Azure Cosmos DB trigger.

+4. The SignalR Service output binding publishes a message to SignalR Service.

+5. The SignalR Service publishes the message to all connected clients.

+SignalR Service allows you to broadcast messages to all or a subset of clients, such as those belonging to a single user. You can combine the SignalR Service bindings for Azure Functions with App Service authentication to authenticate users with providers such as Azure Active Directory, Facebook, and Twitter. You can then send messages directly to these authenticated users.

+To try out the SignalR Service bindings for Azure Functions, see:

+* [Enable automatic updates in a web application using Azure Functions and SignalR Service](/training/modules/automatic-update-of-a-webapp-using-azure-functions-and-signalr).

+## Azure SQL Edge 1.0.7

+SQL engine build 15.0.2000.1574

+### What's new?

+- Security bug fixes

+ Title: Enable SQL Azure hybrid benefit for Azure VMware Solution

+# Enable SQL Azure hybrid benefit for Azure VMware Solution

+[Attach Azure NetApp Files datastores to Azure VMware Solution hosts](attach-azure-netapp-files-to-azure-vmware-solution-hosts.md)

+## February 2023 Guest OS

+>[!NOTE]

+>The February Guest OS is currently being rolled out to Cloud Service VMs that are configured for automatic updates. When the rollout is complete, this version will be made available for manual updates through the Azure portal and configuration files. The following patches are included in the February Guest OS. This list is subject to change.

+| Product Category | Parent KB Article | Vulnerability Description | Guest OS | Date First Introduced |

+| | | | | |

+| Rel 23-02 | [5022838] | Latest Cumulative Update(LCU) | 5.78 | Feb 14, 2023 |

+| Rel 23-02 | [5022835] | IE Cumulative Updates | 2.134, 3.121, 4.114 | Feb 14, 2023 |

+| Rel 23-02 | [5022842] | Latest Cumulative Update(LCU) | 7.22 | Feb 14, 2023 |

+| Rel 23-02 | [5022840] | Latest Cumulative Update(LCU) | 6.54 | Feb 14, 2023 |

+| Rel 23-02 | [5022523] | .NET Framework 3.5 Security and Quality Rollup  | 2.134 | Feb 14, 2023 |

+| Rel 23-02 | [5022515] | .NET Framework 4.6.2 Security and Quality Rollup  | 2.134 | Feb 14, 2023 |

+| Rel 23-02 | [5022525] | .NET Framework 3.5 Security and Quality Rollup  | 4.114 | Feb 14, 2023 |

+| Rel 23-02 | [5022513] | .NET Framework 4.6.2 Security and Quality Rollup  | 4.114 | Feb 14, 2023 |

+| Rel 23-02 | [5022574] | .NET Framework 3.5 Security and Quality Rollup  | 3.121 | Feb 14, 2023 |

+| Rel 23-02 | [5022512] | .NET Framework 4.6.2 Security and Quality Rollup  | 3.121 | Feb 14, 2023 |

+| Rel 23-02 | [5022511] | . NET Framework 4.7.2 Cumulative Update  | 6.54 | Feb 14, 2023 |

+| Rel 23-02 | [5022507] | .NET Framework 4.8 Security and Quality Rollup  | 7.22 | Feb 14, 2023 |

+| Rel 23-02 | [5022872] | Monthly Rollup  | 2.134 | Feb 14, 2023 |

+| Rel 23-02 | [5022903] | Monthly Rollup  | 3.121 | Feb 14, 2023 |

+| Rel 23-02 | [5022899] | Monthly Rollup  | 4.114 | Feb 14, 2023 |

+| Rel 23-02 | [5022923] | Servicing Stack Update  | 3.121 | Feb 14, 2023 |

+| Rel 23-02 | [5018922] | Servicing Stack update LKG  | 4.114 | Oct 11, 2022 |

+| Rel 23-02 | [4578013] | OOB Standalone Security Update  | 4.114 | Aug 19, 2020 |

+| Rel 23-02 | [5017396] | Servicing Stack Update LKG  | 5.78 | Sep 13, 2022 |

+| Rel 23-02 | [5017397] | Servicing Stack Update LKG  | 2.134 | Sep 13, 2022 |

+| Rel 23-02 | [4494175] | Microcode  | 5.78 | Sep 1, 2020 |

+| Rel 23-02 | [4494174] | Microcode  | 6.54 | Sep 1, 2020 |

+| Rel 23-02 | 5022947 | Servicing Stack Update  | 7.22 | |

+[5022838]: https://support.microsoft.com/kb/5022838

+[5022835]: https://support.microsoft.com/kb/5022835

+[5022842]: https://support.microsoft.com/kb/5022842

+[5022840]: https://support.microsoft.com/kb/5022840

+[5022523]: https://support.microsoft.com/kb/5022523

+[5022515]: https://support.microsoft.com/kb/5022515

+[5022525]: https://support.microsoft.com/kb/5022525

+[5022513]: https://support.microsoft.com/kb/5022513

+[5022574]: https://support.microsoft.com/kb/5022574

+[5022512]: https://support.microsoft.com/kb/5022512

+[5022511]: https://support.microsoft.com/kb/5022511

+[5022507]: https://support.microsoft.com/kb/5022507

+[5022872]: https://support.microsoft.com/kb/5022872

+[5022903]: https://support.microsoft.com/kb/5022903

+[5022899]: https://support.microsoft.com/kb/5022899

+[5022923]: https://support.microsoft.com/kb/5022923

+[5018922]: https://support.microsoft.com/kb/5018922

+[4578013]: https://support.microsoft.com/kb/4578013

+[5017396]: https://support.microsoft.com/kb/5017396

+[5017397]: https://support.microsoft.com/kb/5017397

+[4494175]: https://support.microsoft.com/kb/4494175

+[4494174]: https://support.microsoft.com/kb/4494174

+ Title: Create a Translator resource

+description: This article shows you how to create an Azure Cognitive Services Translator resource and get a key and endpoint URL.

+# Create a Translator resource

+In this article, you learn how to create a Translator resource in the Azure portal. [Azure Translator](translator-overview.md) is a cloud-based machine translation service that is part of the [Azure Cognitive Services](../what-are-cognitive-services.md) family of REST APIs. Azure resources are instances of services that you create. All API requests to Azure services require an **endpoint** URL and a read-only **key** for authenticating access.

+## Prerequisites

+To get started, you need an active [**Azure account**](https://azure.microsoft.com/free/cognitive-services/). If you don't have one, you can [**create a free 12-month subscription**](https://azure.microsoft.com/free/).

+## Create your resource

+The Translator service can be accessed through two different resource types:

+* [**Single-service**](https://portal.azure.com/#create/Microsoft.CognitiveServicesTextTranslation) resource types enable access to a single service API key and endpoint.

+* [**Multi-service**](https://portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne) resource types enable access to multiple Cognitive Services using a single API key and endpoint.

+## Complete your project and instance details

+1. **Subscription**. Select one of your available Azure subscriptions.

+1. **Resource Group**. You can create a new resource group or add your resource to a pre-existing resource group that shares the same lifecycle, permissions, and policies.

+1. **Resource Region**. Choose **Global** unless your business or application requires a specific region. If you're planning on using the Document Translation feature with managed identity authentication, choose a non-global region.

+1. **Name**. Enter the name you have chosen for your resource. The name you choose must be unique within Azure.

+ > [!NOTE]

+ > If you are using a Translator feature that requires a custom domain endpoint, such as Document Translation, the value that you enter in the Name field will be the custom domain name parameter for the endpoint.

+1. **Pricing tier**. Select a [pricing tier](https://azure.microsoft.com/pricing/details/cognitive-services/translator) that meets your needs:

+ * Each subscription has a free tier.

+ * The free tier has the same features and functionality as the paid plans and doesn't expire.

+ * Only one free tier is available per subscription.

+ * Document Translation isn't supported in the free tier. Select Standard S1 to try that feature.

+1. If you've created a multi-service resource, you need to confirm more usage details via the check boxes.

+1. Select **Review + Create**.

+1. Review the service terms and select **Create** to deploy your resource.

+1. After your resource has successfully deployed, select **Go to resource**.

+### Authentication keys and endpoint URL

+All Cognitive Services API requests require an endpoint URL and a read-only key for authentication.

+* **Authentication keys**. Your key is a unique string that is passed on every request to the Translation service. You can pass your key through a query-string parameter or by specifying it in the HTTP request header.

+* **Endpoint URL**. Use the Global endpoint in your API request unless you need a specific Azure region or custom endpoint. *See* [Base URLs](reference/v3-0-reference.md#base-urls). The Global endpoint URL is `api.cognitive.microsofttranslator.com`.

+## Get your authentication keys and endpoint

+1. After your new resource deploys, select **Go to resource** or navigate directly to your resource page.

+1. In the left rail, under *Resource Management*, select **Keys and Endpoint**.

+1. Copy and paste your keys and endpoint URL in a convenient location, such as *Microsoft Notepad*.

+## How to delete a resource or resource group

+> [!Warning]

+> Deleting a resource group also deletes all resources contained in the group.

+To remove a Cognitive Services or Translator resource, you can **delete the resource** or **delete the resource group**.

+To delete the resource:

+1. Navigate to your Resource Group in the Azure portal.

+1. Select the resources to be deleted by selecting the adjacent check box.

+1. Select **Delete** from the top menu near the right edge.

+1. Type *yes* in the **Deleted Resources** dialog box.

+1. Select **Delete**.

+To delete the resource group:

+1. Navigate to your Resource Group in the Azure portal.

+1. Select the **Delete resource group** from the top menu bar near the left edge.

+1. Confirm the deletion request by entering the resource group name and selecting **Delete**.

+## How to get started with Translator

+In our quickstart, you learn how to use the Translator service with REST APIs.

+> [!div class="nextstepaction"]

+> [Get Started with Translator](quickstart-translator.md)

+## More resources

+* [Microsoft Translator code samples](https://github.com/MicrosoftTranslator). Multi-language Translator code samples are available on GitHub.

+* [Microsoft Translator Support Forum](https://www.aka.ms/TranslatorForum)

+* [Get Started with Azure (3-minute video)](https://azure.microsoft.com/get-started/?b=16.24)

+ Document Translation is a cloud-based feature of the [Azure Translator](../../translator-overview.md) service that asynchronously translates whole documents in [supported languages](../../language-support.md) and various [file formats](../overview.md#supported-document-formats). In this quickstart, learn to use Document Translation with a programming language of your choice to translate a source document into a target language while preserving structure and text formatting.

+To get started, you need:

+* An [**Azure blob storage account**](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM). You also need to create containers to store and organize your blob data within your storage account.

+ > [!div class="nextstepaction"]

+ > [I ran into an issue with the prerequisites.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?Pillar=Language&Product=Document-translation&Page=quickstart&Section=Prerequisites)

+1. You **`key`** and **`document translation endpoint`** into the code samples to authenticate your request to the Document Translation service.

+ > [!div class="nextstepaction"]

+ > [I ran into an issue retrieving my key and endpoint.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?Pillar=Language&Product=Document-translation&Page=quickstart&Section=Retrieve-your-keys-and-endpoint)

+You need to [**create containers**](../../../../storage/blobs/storage-quickstart-blobs-portal.md#create-a-container) in your [**Azure blob storage account**](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM) for source and target files.

+* **Target container**. This container is where your translated files are stored (required).

+ > [!div class="nextstepaction"]

+ > [I ran into an issue creating blob storage containers with authentication.](https://microsoft.qualtrics.com/jfe/form/SV_0Cl5zkG3CnDjq6O?Pillar=Language&Product=Document-translation&Page=quickstart&Section=Create-blob-storage-containers)

+For this project, you need a **source document** uploaded to your **source container**. You can download our [document translation sample document](https://raw.githubusercontent.com/Azure-Samples/cognitive-services-REST-api-samples/master/curl/Translator/document-translation-sample.docx) for this quickstart.

+A batch Document Translation request is submitted to your Translator service endpoint via a POST request. If successful, the POST method returns a `202 Accepted` response code and the batch request is created by the service. The translated documents are listed in your target container.

+>[Document Translation REST API guide](../reference/rest-api-guide.md) </br></br>[Language support](../../language-support.md)

+In this how-to guide, you learn to use the [Translator service REST APIs](reference/rest-api-guide.md). You start with basic examples and move onto some core configuration options that are commonly used during development, including:

+* You need the key and endpoint from the resource to connect your application to the Translator service. Later, you paste your key and endpoint into the code samples. You can find these values on the Azure portal **Keys and Endpoint** page:

+To call the Translator service via the [REST API](reference/rest-api-guide.md), you need to make sure the following headers are included with each request. Don't worry, we include the headers in the sample code in the following sections.

+1. Delete the pre-existing code, including the line `Console.Writeline("Hello World!")`. Copy and paste the code samples into your application's Program.cs file. For each code sample, make sure you update the key and endpoint variables with values from your Azure portal Translator instance.

+1. Copy and paste the code samples into your **text-translator.go** file. Make sure you update the key variable with the value from your Azure portal Translator instance.

+* If you aren't using Visual Studio Code, make sure you have the following installed in your development environment:

+1. Run the `gradle init` command from the translator-text-app directory. This command creates essential build files for Gradle, including *build.gradle.kts*, which is used at runtime to create and configure your application.

+ You create the following directory structure:

+1. Copy and paste the code samples `TranslatorText.java` file. **Make sure you update the key with one of the key values from your Azure portal Translator instance**.

+1. Copy and paste the code samples into your `index.js` file. **Make sure you update the key variable with the value from your Azure portal Translator instance**.

+The core operation of the Translator service is to translate text. In this section, you build a request that takes a single source (`from`) and provides two outputs (`to`). Then we review some parameters that can be used to adjust both the request and the response.

+You can check the consumption (the number of characters charged) for each request in the [**response headers: x-metered-usage**](reference/v3-0-translate.md#response-headers) field.

+If you need translation, but don't know the language of the text, you can use the language detection operation. There's more than one way to identify the source text language. In this section, you learn how to use language detection using the `translate` endpoint, and the `detect` endpoint.

+If you don't include the `from` parameter in your translation request, the Translator service attempts to detect the source text's language. In the response, you get the detected language (`language`) and a confidence score (`score`). The closer the `score` is to `1.0`, means that there's increased confidence that the detection is correct.

+It's possible to use the Translator service to detect the language of source text without performing a translation. To do so, you use the [`/detect`](./reference/v3-0-detect.md) endpoint.

+The `/detect` endpoint response includes alternate detections, and indicates if the translation and transliteration are supported for all of the detected languages. After a successful call, you should see the following response:

+Transliteration is the process of converting a word or phrase from the script (alphabet) of one language to another based on phonetic similarity. For example, you could use transliteration to convert "สวัสดี" (`thai`) to "sawatdi" (`latn`). There's more than one way to perform transliteration. In this section, you learn how to use language detection using the `translate` endpoint, and the `transliterate` endpoint.

+If you're translating into a language that uses a different alphabet (or phonemes) than your source, you might need a transliteration. In this example, we translate "Hello" from English to Thai. In addition to getting the translation in Thai, you get a transliteration of the translated phrase using the Latin alphabet.

+After a successful call, you should see the following response. In addition to the detected source language and translation, you get character counts for each detected sentence for both the source (`srcSentLen`) and translation (`transSentLen`).

+With the endpoint, you can get alternate translations for a word or phrase. For example, when translating the word "sunshine" from `en` to `es`, this endpoint returns "`luz solar`", "`rayos solares`", and "`soleamiento`", "`sol`", and "`insolaci├│n`".

+After you've performed a dictionary lookup, pass the source and translation text to the `dictionary/examples` endpoint, to get a list of examples that show both terms in the context of a sentence or phrase. Building on the previous example, you use the `normalizedText` and `normalizedTarget` from the dictionary lookup response as `text` and `translation` respectively. The source language (`from`) and output target (`to`) parameters are required.

+> To configure a custom DNS suffix for all container apps in an environment, see [Custom environment DNS suffix in Azure Container Apps](environment-custom-dns-suffix.md). If you configure a custom environment DNS suffix, you cannot add a custom domain that contains this suffix to your Container App.

+registryId=$(az acr show \

+There are default limits that require quota increases. Not all quota increases may be approved: [Resource availability & quota limits for ACI - Azure Container Instances | Microsoft Learn](./container-instances-resource-and-quota-limits.md)

+> Container group deployment to a virtual network is generally available for Linux and Windows containers, in most regions where Azure Container Instances is available. For details, see [available-regions][available-regions].

+[available-regions]: https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?products=container-instances

+| `ALTER MATERIALIZED VIEW` | Yes |

+| `CREATE MATERIALIZED VIEW` | Yes |

+| `DROP MATERIALIZED VIEW` | Yes |

+> [!NOTE]

+> Serverless accounts for Azure Cosmos DB can only run in a single Azure region. For more information, see [using serverless resources](serverless.md).

+- [Python 3.7+](https://github.com/Azure/azure-sdk-for-python/wiki/Azure-SDKs-Python-version-support-policy) including [pip](https://pip.pypa.io/en/stable/installing/) package installer.

+Now let's switch to working with code. Let's clone a Gremlin API app from GitHub, set the connection string, and run it. You'll see how easy it's to work with data programmatically.

+1. Run the following command to clone the sample repository to your local machine. This command creates a copy of the sample app on your computer. Start at in the root of the folder where you typically store GitHub repositories.

+ git clone https://github.com/Azure-Samples/azure-cosmos-db-graph-python-getting-started.git

+1. Change to the directory where the sample app is located.

+ cd azure-cosmos-db-graph-python-getting-started

+This step is optional. If you're interested in learning how the database resources are created in the code, you can review the following snippets. The snippets are all taken from the *connect.py* file in the repo [git-samples\azure-cosmos-db-graph-python-getting-started](https://github.com/Azure-Samples/azure-cosmos-db-graph-python-getting-started). Otherwise, you can skip ahead to [Update your connection string](#update-your-connection-information).

+* The Gremlin `client` is initialized in *connect.py* with `client.Client()`. Make sure to replace `<YOUR_DATABASE>` and `<YOUR_CONTAINER_OR_GRAPH>` with the values of your account's database name and graph name:

+* A series of Gremlin steps (queries) are declared at the beginning of the *connect.py* file. They're then executed using the `client.submitAsync()` method. For example, to run the cleanup graph step, you'd use the following code:

+ Copy the first portion of the **URI** value.

+2. Open the *connect.py* file, find the `client.Client()` definition, and paste the URI value over `<YOUR_ENDPOINT>` in here:

+4. On the **Keys** page, use the copy button to copy the **PRIMARY KEY** and paste it over `<YOUR_PASSWORD>` in the `password=<YOUR_PASSWORD>` parameter.

+ The `client` object definition should now look similar to the following:

+1. Start in a terminal window in the root of the folder where you cloned the sample app. If you are using Visual Studio Code, you can open a terminal window by selecting **Terminal** > **New Terminal**. Typically, you'll create a virtual environment to run the code. For more information, see [Python virtual environments](https://docs.python.org/3/tutorial/venv.html).

+ ```bash

+ cd azure-cosmos-db-graph-python-getting-started

+1. Install the required Python packages.

+1. Start the Python application.

+After the vertices and edges are inserted, you can now go back to Data Explorer and see the vertices added to the graph, and add more data points.

+1. In your Azure Cosmos DB account in the Azure portal, select **Data Explorer**, expand **sample-database**, expand **sample-graph**, select **Graph**, and then select **Execute Gremlin Query**.

+ :::image type="content" source="./media/quickstart-python/azure-cosmosdb-data-explorer-expanded.png" alt-text="Screenshot shows Graph selected from the A P I with the option to Execute Gremlin Query.":::

+5. Select **Add property** to add each of the following properties. Notice that you can create unique properties for each person in your graph. Only the ID key is required.

+ id|ashley|The unique identifier for the vertex. If you don't specify an ID, one is generated for you.

+7. Select **New Vertex** again and add another new user.

+ id|rakesh|The unique identifier for the vertex. If you don't specify an ID, one is generated for you.

+11. Select the **Execute Gremlin Query** button with the default `g.V()` filter to display all the values in the graph. All of the users now show in the **Results** list.

+ As you add more data, you can use filters to limit your results. By default, Data Explorer uses `g.V()` to retrieve all vertices in a graph. You can change it to a different [graph query](tutorial-query.md), such as `g.V().count()`, to return a count of all the vertices in the graph in JSON format. If you changed the filter, change the filter back to `g.V()` and select **Execute Gremlin Query** to display all the results again.

+12. Now we can connect **rakesh** and **ashley**. Ensure **ashley** is selected in the **Results** list, then select the edit button next to **Targets** on lower right side. You may need to widen your window to see the **Properties** area.

+ Title: Measure performance with a benchmarking framework

+description: Use YCSB to benchmark Azure Cosmos DB for NoSQL with recipes to measure read, write, scan, and update performance.

+# Measure Azure Cosmos DB for NoSQL performance with a benchmarking framework

+There are more choices, now than ever, on the type of database to use with your data workload. One of the key factors to picking a database is the performance of the database or service, but benchmarking performance can be cumbersome and error-prone. The [benchmarking framework for Azure Databases](https://github.com/Azure/azure-db-benchmarking) simplifies the process of measuring performance with popular open-source benchmarking tools with low-friction recipes that implement common best practices. In Azure Cosmos DB for NoSQL, the framework implements [best practices for the Java SDK](performance-tips-java-sdk-v4.md) and uses the open-source [YCSB](https://ycsb.site) tool. In this guide, you use this benchmarking framework to implement a read workload to familiarize yourself with the framework.

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free).

+- Azure Cosmos DB for NoSQL account. [Create a API for NoSQL account](how-to-create-account.md).

+ - Make sure you note the endpoint URI and primary key for the account. [API for NoSQL primary keys](../database-security.md?tabs=sql-api#primary-keys).

+- Azure Storage account. [Create an Azure Storage account](../../storage/common/storage-account-create.md).

+ - Make sure you note the connection string for the storage account. [Vies Azure Storage connection string](../../storage/common/storage-account-keys-manage.md?tabs=azure-portal#view-account-access-keys).

+- Second empty resource group. [Create a resource group](../../azure-resource-manager/management/manage-resource-groups-portal.md#create-resource-groups).

+- [Azure Command-Line Interface (CLI)](/cli/azure/).

+## Create Azure Cosmos DB account resources

+First, you create a database and container in the existing API for NoSQL account.

+### [Azure portal](#tab/azure-portal)

+1. Navigate to your existing API for NoSQL account in the [Azure portal](https://portal.azure.com/).

+1. In the resource menu, select **Data Explorer**.

+ :::image type="content" source="media/benchmarking-framework/resource-menu-data-explorer.png" lightbox="media/benchmarking-framework/resource-menu-data-explorer.png" alt-text="Screenshot of the Data Explorer option highlighted in the resource menu.":::

+1. On the **Data Explorer** page, select the **New Container** option in the command bar.

+ :::image type="content" source="media/benchmarking-framework/page-data-explorer-new-container.png" alt-text="Screenshot of the New Container option in the Data Explorer command bar.":::

+1. In the **New Container** dialog, create a new container with the following settings:

+ | Setting | Value |

+ | | |

+ | **Database id** | `ycsb` |

+ | **Database throughput type** | **Manual** |

+ | **Database throughput amount** | `400` |

+ | **Container id** | `usertable` |

+ | **Partition key** | `/id` |

+ :::image type="content" source="media/benchmarking-framework/dialog-new-container.png" alt-text="Screenshot of the New Container dialog on the Data Explorer page.":::

+### [Azure CLI](#tab/azure-cli)

+1. If you haven't already, sign in to the Azure CLI using the [`az login`](/cli/azure/reference-index#az-login) command.

+1. Create shell variables for the following values:

+ - Name of your existing Azure Cosmos DB for NoSQL account named `cosmosAccountName`.

+ - Name of your first resource group with resources named `sourceResourceGroupName`.

+ - Name of your second empty resource group named `targetResourceGroupName`.

+ - Existing Azure Cosmos DB for NoSQL account endpoint URI named `cosmosEndpoint`

+ - Existing Azure Cosmos DB for NoSQL account primary key named `cosmosPrimaryKey`

+ ```azurecli-interactive

+ # Variable for Azure Cosmos DB for NoSQL account name

+ cosmosAccountName="<cosmos-db-nosql-account-name>"

+ # Variable for resource group with Azure Cosmos DB and Azure Storage accounts

+ sourceResourceGroupName="<first-resource-group-name>"

+ # Variable for empty resource group

+ targetResourceGroupName="<second-resource-group-name>"

+ # Variable for API for NoSQL endpoint URI

+ cosmosEndpoint="<cosmos-db-nosql-endpoint-uri>"

+ # Variable for API for NoSQL primary key

+ cosmosPrimaryKey="<cosmos-db-nosql-primary-key>"

+ # Variable for Azure Storage account name

+ storageAccountName="<storage-account-name>"

+ # Variable for storage account connection string

+ storageConnectionString="<storage-connection-string>"

+ ```

+1. Using the [`az cosmosdb sql database create`](/cli/azure/cosmosdb/sql/database#az-cosmosdb-sql-database-create) command, create a new database with the following settings:

+ | Setting | Value |

+ | | |

+ | **Database id** | `ycsb` |

+ | **Database throughput type** | **Manual** |

+ | **Database throughput amount** | `400` |

+ ```azurecli-interactive

+ az cosmosdb sql database create \

+ --resource-group $sourceResourceGroupName \

+ --account-name $cosmosAccountName \

+ --name "ycsb" \

+ --throughput 400

+ ```

+1. Using the [`az cosmosdb sql container create`](/cli/azure/cosmosdb/sql/container#az-cosmosdb-sql-container-create) command, create a new container with the following settings:

+ | Setting | Value |

+ | | |

+ | **Database id** | `ycsb` |

+ | **Container id** | `usertable` |

+ | **Partition key** | `/id` |

+ ```azurecli-interactive

+ az cosmosdb sql container create \

+ --resource-group $sourceResourceGroupName \

+ --account-name $cosmosAccountName \

+ --database-name "ycsb" \

+ --name "usertable" \

+ --partition-key-path "/id"

+ ```

+## Deploy benchmarking framework to Azure

+Now, you use an [Azure Resource Manager template](../../azure-resource-manager/templates/overview.md) to deploy the benchmarking framework to Azure with the default read recipe.

+### [Azure portal](#tab/azure-portal)

+1. Deploy the benchmarking framework using an Azure Resource Manager template available at this link.

+ :::image type="content" source="https://aka.ms/deploytoazurebutton" alt-text="Deploy to Azure button." link="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-db-benchmarking%2Fmain%2Fcosmos%2Fsql%2Ftools%2Fjava%2Fycsb%2Frecipes%2Fread%2Ftry-it-read%2Fazuredeploy.json":::

+1. On the Custom Deployment page, the following parameters

+ :::image type="content" source="media/benchmarking-framework/page-custom-deployment.png" lightbox="media/benchmarking-framework/page-custom-deployment.png" alt-text="Screenshot of the Custom Deployment page with parameters values filled out.":::

+1. Select **Review + create** and then **Create** to deploy the template.

+1. Wait for the deployment to complete.

+ > [!TIP]

+ > The deployment can take 5-10 minutes to complete.

+### [Azure CLI](#tab/azure-cli)

+1. Use [`az deployment group create`](/cli/azure/deployment/group#az-deployment-group-create) to deploy the benchmarking framework using an Azure Resource Manager template.

+ ```azurecli-interactive

+ # Variable for raw template JSON on GitHub

+ templateUri="https://raw.githubusercontent.com/Azure/azure-db-benchmarking/main/cosmos/sql/tools/java/ycsb/recipes/read/try-it-read/azuredeploy.json"

+ az deployment group create \

+ --resource-group $targetResourceGroupName \

+ --name "benchmarking-framework" \

+ --template-uri $templateUri \

+ --parameters \

+ adminPassword='P@ssw.rd' \

+ resultsStorageConnectionString=$storageConnectionString \

+ cosmosURI=$cosmosEndpoint \

+ cosmosKey=$cosmosPrimaryKey

+ ```

+1. Wait for the deployment to complete.

+ > [!TIP]

+ > The deployment can take 5-10 minutes to complete.

+## View results of the benchmark

+Now, you can use the existing Azure Storage account to check the status of the benchmark job and view the aggregated results. The status is stored using a storage table and the results are aggregated into a storage blob using the CSV format.

+### [Azure portal](#tab/azure-portal)

+1. Navigate to your existing Azure Storage account in the [Azure portal](https://portal.azure.com/).

+1. Navigate to a storage table named **ycsbbenchmarkingmetadata** and locate the entity with a partition key of `ycsb_sql`.

+ :::image type="content" source="media/benchmarking-framework/storage-table-data.png" alt-text="Screenshot of the ycsbbenchmarkingMetadata table in a storage account.":::

+1. Observe the `JobStatus` field of the table entity. Initially, the status of the job is `Started` and it includes a timestamp in the `JobStartTime` property but not the `JobFinishTime` property.

+1. Wait until the job has a status of `Finished` and includes a timestamp in the `JobFinishTime` property.

+ > [!TIP]

+ > It can take approximately 20-30 minutes for the job to finish.

+1. Navigate to the storage container in the same account with a prefix of **ycsbbenchmarking-***. Observe the output and diagnostic blobs for the tool.

+ :::image type="content" source="media/benchmarking-framework/storage-blob-output.png" alt-text="Screenshot of the container and output blobs from the benchmarking tool.":::

+1. Open the **aggregation.csv** blob and observe the content. You should now have a CSV dataset with aggregated results from all the benchmark clients.

+ :::image type="content" source="media/benchmarking-framework/storage-blob-aggregation-results.png" alt-text="Screenshot of the content of the aggregation results blob.":::

+ ```output

+ Operation,Count,Throughput,Min(microsecond),Max(microsecond),Avg(microsecond),P9S(microsecond),P99(microsecond)

+ READ,180000,299,706,448255,1079,1159,2867

+ ```

+### [Azure CLI](#tab/azure-cli)

+1. Query the job record in a storage table named `ycsbbenchmarkingmetadata` using [`az storage entity query`](/cli/azure/storage/entity#az-storage-entity-query).

+ ```azurecli-interactive

+ az storage entity query \

+ --account-name $storageAccountName \

+ --connection-string $storageConnectionString \

+ --table-name ycsbbenchmarkingmetadata

+ ```

+1. Observe the results of this query. The results should return a single job with `JobStartTime`, `JobStatus`, and `JobFinishTime` properties. Initially, the status of the job is `Started` and it includes a timestamp in the `JobStartTime` property but not the `JobFinishTime` property.

+ ```output

+ {

+ "items": [

+ {

+ "JobFinishTime": "",

+ "JobStartTime": "2023-02-02T13:59:42Z",

+ "JobStatus": "Started",

+ "NoOfClientsCompleted": "0",

+ "NoOfClientsStarted": {

+ "edm_type": "Edm.Int64",

+ "value": 1

+ },

+ "PartitionKey": "ycsb_sql",

+ ...

+ }

+ ],

+ ...

+ }

+ ```

+1. If necessary, run `az storage entity query` multiple times until the job has a status of `Finished` and includes a timestamp in the `JobFinishTime` property.

+ ```output

+ {

+ "items": [

+ {

+ "JobFinishTime": "2023-02-02T14:21:12Z",

+ "JobStartTime": "2023-02-02T13:59:42Z",

+ "JobStatus": "Finished",

+ ...

+ }

+ ],

+ ...

+ }

+ ```

+ > [!TIP]

+ > It can take approximately 20-30 minutes for the job to finish.

+1. Find the name of the most recently modified storage container with a prefix of `ycsbbenchmarking-*` using [`az storage container list`](/cli/azure/storage/container#az-storage-container-list) and a [JMESPath query](/cli/azure/query-azure-cli).

+ ```azurecli-interactive

+ az storage container list \

+ --account-name $storageAccountName \

+ --connection-string $storageConnectionString \

+ --query "sort_by([?starts_with(name, 'ycsbbenchmarking-')], &properties.lastModified)[-1].name" \

+ --output tsv

+ ```

+1. Store the container string in a variable named `storageConnectionString`.

+ ```azurecli-interactive

+ storageContainerName=$( \

+ az storage container list \

+ --account-name $storageAccountName \

+ --connection-string $storageConnectionString \

+ --query "sort_by([?starts_with(name, 'ycsbbenchmarking-')], &properties.lastModified)[-1].name" \

+ --output tsv \

+ )

+ ```

+1. Use [`az storage blob query`]/cli/azure/storage/blob#az-storage-blob-query) to query the job results in a storage blob stored in the previously located container.

+ ```azurecli-interactive

+ az storage blob query \

+ --account-name $storageAccountName \

+ --connection-string $storageConnectionString \

+ --container-name $storageContainerName \

+ --name aggregation.csv \

+ --query-expression "SELECT * FROM BlobStorage"

+ ```

+1. Observe the results of this query. You should now have a CSV dataset with aggregated results from all the benchmark clients.

+ ```output

+ Operation,Count,Throughput,Min(microsecond),Max(microsecond),Avg(microsecond),P9S(microsecond),P99(microsecond)

+ READ,180000,299,706,448255,1079,1159,2867

+ ```

+## Recipes

+The [benchmarking framework for Azure Databases](https://github.com/Azure/azure-db-benchmarking) includes recipes to encapsulate the workload definitions that are passed to the underlying benchmarking tool for a "1-Click" experience. The workload definitions were designed based on the best practices published by the Azure Cosmos DB team and the benchmarking tool's team. The recipes have been tested and validated for consistent results.

+You can expect to see the following latencies for all the read and write recipes in the [GitHub repository](https://github.com/Azure/azure-db-benchmarking/tree/main/cosmos/sql/tools/java/ycsb/recipes).

+- **Read latency**

+ :::image type="content" source="media\benchmarking-framework\typical-read-latency.png" lightbox="media\benchmarking-framework\typical-read-latency.png" alt-text="Diagram of the typical read latency averaging around 1 millisecond to 2 milliseconds.":::

+- **Write latency**

+ :::image type="content" source="media\benchmarking-framework\typical-write-latency.png" lightbox="media\benchmarking-framework\typical-write-latency.png" alt-text="Diagram of the typical write latency averaging around 4 milliseconds.":::

+## Common issues

+This section includes the common errors that may occur when running the benchmarking tool. The error logs for the tool are typically available in a container within the Azure Storage account.

+- If the logs aren't available in the storage account, this issue is typically caused by an incorrect or missing storage connection string. In this case, this error is listed in the **agent.out** file within the **/home/benchmarking** folder of the client virtual machine.

+ ```output

+ Error while accessing storage account, exiting from this machine in agent.out on the VM

+ ```

+- This error is listed in the **agent.out** file both in the client VM and the storage account if the Azure Cosmos DB endpoint URI is incorrect or unreachable.

+ ```output

+ Caused by: java.net.UnknownHostException: rtcosmosdbsss.documents.azure.com: Name or service not known

+ ```

+- This error is listed in the **agent.out** file both in the client VM and the storage account if the Azure Cosmos DB key is incorrect.

+ ```output

+ The input authorization token can't serve the request. The wrong key is being used….

+ ```

+## Next steps

+- Learn more about the benchmarking tool with the [Getting Started guide](https://github.com/Azure/azure-db-benchmarking/tree/main/cosmos/sql/tools/java/ycsb/recipes).

+> [!NOTE]

+> Server-side JavaScript features including stored procedures, triggers, and user-defined functions do not support importing modules.

+> [!TIP]

+> [!NOTE]

+> Server-side JavaScript features including user-defined functions do not support importing modules.

+> [!NOTE]

+> Server-side JavaScript features including stored procedures, triggers, and user-defined functions do not support importing modules.

+> [!TIP]

+> For transaction support in Azure Cosmos DB for NoSQL, you can also implement a transactional batch using your preferred client SDK. For more information, see [Transactional batch operations in Azure Cosmos DB for NoSQL](transactional-batch.md).

+### Request level timeouts

+The `CosmosClientOptions.RequestTimeout` (or `ConnectionPolicy.RequestTimeout` for SDK v2) configuration allows you to set a timeout for the network request after the request left the SDK and is on the network, until a response is received.

+The `CosmosClientOptions.OpenTcpConnectionTimeout` (or `ConnectionPolicy.OpenTcpConnectionTimeout` for SDK v2) configuration allows you to set a timeout for the time spent opening an initial connection. Once a connection is opened, subsequent requests will use the connection.

+An operation started by a user can span multiple network requests, for example, retries. These two configurations are per-request, not end-to-end for an operation.

+All the async operations in the SDK have an optional CancellationToken parameter. This [CancellationToken](/dotnet/standard/threading/how-to-listen-for-cancellation-requests-by-polling) parameter is used throughout the entire operation, across all network requests and retries. In between network requests, the cancellation token might be checked and an operation canceled if the related token is expired. The cancellation token should be used to define an approximate expected timeout on the operation scope.

+Verify the configured time in your `CancellationToken`, make sure that it's greater than your [RequestTimeout](#request-level-timeouts) and the [CosmosClientOptions.OpenTcpConnectionTimeout](/dotnet/api/microsoft.azure.cosmos.cosmosclientoptions.opentcpconnectiontimeout) (if you're using [Direct mode](sdk-connection-modes.md)).

+## Can't view subscription

+If you created a subscription but can't find it in the Subscriptions list view, a view filter might be applied.

+To clear the filter and view all subscriptions:

+1. In the Azure portal, navigate to **Subscriptions**.

+2. At the top of the list, select the Subscriptions filter item.

+3. At the top of the subscriptions filter box, select **All**. At the bottom of the subscriptions filter box, clear **Show only subscriptions selected in the global subscriptions filter**.

+ :::image type="content" source="./media/create-enterprise-subscription/subscriptions-filter-item.png" alt-text="Screenshot showing the Subscriptions filter box with options." lightbox="./media/create-enterprise-subscription/subscriptions-filter-item.png" :::

+4. Select **Apply** to close the box and refresh the list of subscriptions.

+## Can't view subscription

+If you created a subscription but can't find it in the Subscriptions list view, a view filter might be applied.

+To clear the filter and view all subscriptions:

+1. In the Azure portal, navigate to **Subscriptions**.

+2. At the top of the list, select the Subscriptions filter item.

+3. At the top of the subscriptions filter box, select **All**. At the bottom of the subscriptions filter box, clear **Show only subscriptions selected in the global subscriptions filter**.

+ :::image type="content" source="./media/create-subscription/subscriptions-filter-item.png" alt-text="Screenshot showing the Subscriptions filter box with options." lightbox="./media/create-subscription/subscriptions-filter-item.png" :::

+4. Select **Apply** to close the box and refresh the list of subscriptions.

+Users with an individual subscription can get the amortized cost data from their usage file. When a resource gets a reservation discount, the *AdditionalInfo* section in the usage file contains the reservation details. For more information, see [View and download your Azure usage and charges](../understand/download-azure-daily-usage.md).

+Users with an individual subscription can get the amortized cost data from their usage file. When a resource gets a savings plan discount, the _AdditionalInfo_ section in the usage file contains the savings plan details. For more information, see [View and download your Azure usage and charges](../understand/download-azure-daily-usage.md).

+## Download usage for MOSP billing accounts

+Use the following information to download usage for billed charges. The same steps are used to download open and pending charges, which is the month-to-date usage for the current billing period. Open and pending charges haven't been billed yet.

+### Download usage file

+1. Search for *Cost Management + Billing*.

+1. Select a billing profile. Depending on your access, you might need to select a billing account first.

+1. In the left menu, select **Invoices**.

+1. In the invoice grid, find the row of the invoice corresponding to the usage file that you want to download.

+1. Select the ellipsis symbol (`...`) at the end of the row.

+1. In the context menu, select **Prepare Azure usage file**. A notification message appears stating that the usage file is being prepared.

+1. When the file is ready to download, select the **Click here to download** link in the notification. If you missed the notification, you can view it from **Notifications** area in top right of the Azure portal (the bell symbol).

+This article applies to customers with a Microsoft Customer Agreement (MCA) and to customers who signed up for Azure through the Azure website, Azure.com (for a Microsoft Online Services Program account also called pay-as-you-go account).

+Here's a table summarizing payment methods for different agreement types

+|Agreement type| Credit card | Wire transfer┬╣ | Check┬▓ |

+| | - | | |

+| Microsoft Customer Agreement<br>purchased through a Microsoft representative | Γ£ö (with a $50,000.00 USD limit) | Γ£ö | Γ£ö |

+| Enterprise Agreement | Γ£ÿ | Γ£ö | Γ£ö |

+| Azure.com | Γ£ö | Γ£ö if approved to pay by invoice | Γ£ÿ |

+┬╣ If supported by your bank, an ACH credit transaction can be made automatically.

+┬▓ As noted previously, on April 1, 2023, Microsoft will stop accepting checks as a payment method for subscriptions that are paid by invoice.

+Azure Data Factory offers [Pipelines](concepts-pipelines-activities.md) to visually orchestrate data processes (UI-based authoring). While Managed Airflow, offers Airflow based python DAGs (python code-centric authoring) for defining the data orchestration process. If you have the Airflow background, or are currently using Apache Airflow, you may prefer to use the Managed Airflow instead of the pipelines. On the contrary, if you wouldn't like to write/ manage python-based DAGs for data process orchestration, you may prefer to use pipelines.

+For pricing, check out the [pricing page](https://azure.microsoft.com/pricing/details/defender-for-cloud/).

+## Prerequisites

+**Required roles and permissions**: To save workbooks, you must have at least [Workbook Contributor](../role-based-access-control/built-in-roles.md#workbook-contributor) permissions on the target resource group

+**Cloud availability**: :::image type="icon" source="./media/icons/yes-icon.png"::: Commercial clouds :::image type="icon" source="./media/icons/yes-icon.png"::: National (Azure Government, Azure China 21Vianet)

+- Price Estimation workbook - View monthly consolidated price estimations for Microsoft Defender for Cloud plans based on the resource telemetry in your own environment. These numbers are estimates based on retail prices and don't provide actual billing data.

+- ['DevOps Security (Preview)' workbook](#use-the-devops-security-preview-workbook) - View a customizable foundation that helps you visualize the state of your DevOps posture for the connectors you've configured.

+In addition to the built-in workbooks, you can also find other useful workbooks found under the ΓÇ£Community" category, which is provided as is with no SLA or support. Choose one of the supplied workbooks or create your own.

+|**Scores for specific security controls**<br>Defender for Cloud's security controls is logical groupings of recommendations. This chart shows you, at a glance, the weekly scores for all of your controls.|:::image type="content" source="media/custom-dashboards-azure-workbooks/secure-score-over-time-table-4.png" alt-text="Scores for your security controls over the selected time period.":::|

+The MITRE ATT&CK tactics display by the order of the kill-chain, and the number of alerts the subscription has at each stage.

+You can see all of the active alerts in a table with the ability to filter by columns. Select an alert to view button appears.

+Select a location on the map to view all of the alerts for that location.

+### Use the 'DevOps Security (Preview)' workbook

+This workbook provides a customizable data analysis and gives you the ability to create visual reports. You can use this workbook to view insights into your DevOps security posture in coordination with Defender for DevOps. This workbook allows you to visualize the state of your DevOps posture for the connectors you've configured in Defender for Cloud, code, dependencies, and hardening. You can then investigate credential exposure, including types of credentials and repository locations.

+> [!NOTE]

+> You must have a [Github connector](quickstart-onboard-github.md) or a [DevOps connector](quickstart-onboard-devops.md), connected to your environment in order to utilize this workbook

+**To deploy the workbook**:

+1. Sign in to the [Azure portal](https://portal.azure.com/).

+1. Navigate to **Microsoft Defender for Cloud** > **Workbooks**.

+1. Select the **DevOps Security (Preview)** workbook.

+The workbook will load and show you the Overview tab where you can see the number of exposed secrets, code security and DevOps security. All of these findings are broken down by total for each repository and the severity.

+Select the Secrets tab to view the count by secret type.

+The Code tab displays your count findings by tool and repository and your code scanning by severity.

+The Open Source Security (OSS) Vulnerabilities tab displays your OSS vulnerabilities by severity and the count of findings by repository.

+The Infrastructure as Code tab displays your findings by tool and repository.

+The Posture tab displays your security posture by severity and repository.

+The Threats and Tactics tab displays the total count of threats and tactics and by repository.

+To move workbooks that you've built in other Azure services into your Microsoft Defender for Cloud workbooks gallery:

+> [Governance capability improvements in Defender for Cloud](episode-twenty-six.md)

+ Title: Governance capability improvements in Defender for Cloud | Defender for Cloud in the field

+description: Learn about the need for governance and new at scale governance capability

+# Governance capability improvements in Defender for Cloud | Defender for Cloud in the field

+**Episode description**: In this episode of Defender for Cloud in the Field, Lior Arviv joins Yuri Diogenes to talk about the Governance capability improvements in Defender for Cloud. Lior gives a quick recap of the business need for governance and covers the new at scale governance capability. Lior demonstrates how to deploy governance at scale and how to monitor rules assignments and define priorities.

+<br>

+<br>

+<iframe src="https://aka.ms/docs/player?id=b1581d03-6575-4f13-b2ed-5b0c22d80c63" width="1080" height="530" allowFullScreen="true" frameBorder="0"></iframe>

+- [01:13](/shows/mdc-in-the-field/governance-improvements#time=01m13s) - Reviewing the need for cloud security governance

+- [04:10](/shows/mdc-in-the-field/governance-improvements#time=04m10s) - Governance at scale

+- [07:03](/shows/mdc-in-the-field/governance-improvements#time=07m03s) - Deployment options

+- [07:45](/shows/mdc-in-the-field/governance-improvements#time=07m45s) - Demonstration

+- [19:00](/shows/mdc-in-the-field/governance-improvements#time=19m00s) - Learn more about governance

+## Recommended resources

+ - Learn how to [drive your organization to remediate security recommendations with governance](governance-rules.md)

+ - Subscribe to [Microsoft Security on YouTube](https://www.youtube.com/playlist?list=PL3ZTgFEc7LysiX4PfHhdJPR7S8mGO14YS)

+ - Join our [Tech Community](https://aka.ms/SecurityTechCommunity)

+ - For more about [Microsoft Security](https://msft.it/6002T9HQY)

+- Follow us on social media:

+ - [LinkedIn](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbFk5TXZuQld2NlpBRV9BQlJqMktYSm95WWhCZ3xBQ3Jtc0tsQU13MkNPWGNFZzVuem5zc05wcnp0VGxybHprVTkwS2todWw0b0VCWUl4a2ZKYVktNGM1TVFHTXpmajVLcjRKX0cwVFNJaDlzTld4MnhyenBuUGRCVmdoYzRZTjFmYXRTVlhpZGc4MHhoa3N6ZDhFMA&q=https%3A%2F%2Fwww.linkedin.com%2Fshowcase%2Fmicrosoft-security%2F)

+ - [Twitter](https://twitter.com/msftsecurity)

+- Join our [Tech Community](https://aka.ms/SecurityTechCommunity)

+- Learn more about [Microsoft Security](https://msft.it/6002T9HQY)

+## Next steps

+> [!div class="nextstepaction"]

+> [New AWS Connector in Microsoft Defender for Cloud](episode-one.md)

+ on:

+ push:

+ branches: [ main ]

+ pull_request:

+ branches: [ main ]

+ workflow_dispatch:

+

+ jobs:

+ sample:

+

+ # MSDO runs on windows-latest and ubuntu-latest.

+ # macos-latest supporting coming soon

+ runs-on: windows-latest

+

+ steps:

+ - uses: actions/checkout@v3

+

+ - uses: actions/setup-dotnet@v3

+ with:

+ dotnet-version: |

+ 5.0.x

+ 6.0.x

+

+ # Run analyzers

+ - name: Run Microsoft Security DevOps Analysis

+ uses: microsoft/security-devops-action@preview

+ id: msdo

+

+ # Upload alerts to the Security tab

+ - name: Upload alerts to Security tab

+ uses: github/codeql-action/upload-sarif@v1

+ with:

+ sarif_file: ${{ steps.msdo.outputs.sarifFile }}

+

+| [Multiple changes to identity recommendations](#multiple-changes-to-identity-recommendations) | August 2023 |

+As we continue to improve the quality of our alerts, the following three alerts from the Defender for Azure Resource Manager plan will be deprecated:

+In the scenario where an activity from a suspicious IP address is detected, one of the following Defender for Azure Resource Manager plan alerts `Azure Resource Manager operation from suspicious IP address` or `Azure Resource Manager operation from suspicious proxy IP address` will be present.

+Currently, Defenders for Cloud security alerts are automatically exported to a default Log Analytics workspace on the resource level. This causes an indeterministic behavior and therefore, this feature is set to be deprecated.

+### Multiple changes to identity recommendations

+**Estimated date for change: August 2023**

+We announced previously the [availability of identity recommendations V2 (preview)](release-notes.md#extra-recommendations-added-to-identity), which included enhanced capabilities.

+As part of these changes, the following recommendations will be released as General Availability (GA) and replace the V1 recommendations that are set to be deprecated.

+#### General Availability (GA) release of identity recommendations V2

+The following security recommendations will be released as GA and replace the V1 recommendations:

+

+|Recommendation | Assessment Key|

+|--|--|

+|Accounts with owner permissions on Azure resources should be MFA enabled | 6240402e-f77c-46fa-9060-a7ce53997754 |

+|Accounts with write permissions on Azure resources should be MFA enabled | c0cb17b2-0607-48a7-b0e0-903ed22de39b |

+| Accounts with read permissions on Azure resources should be MFA enabled | dabc9bc4-b8a8-45bd-9a5a-43000df8aa1c |

+| Guest accounts with owner permissions on Azure resources should be removed | 20606e75-05c4-48c0-9d97-add6daa2109a |

+| Guest accounts with write permissions on Azure resources should be removed | 0354476c-a12a-4fcc-a79d-f0ab7ffffdbb |

+| Guest accounts with read permissions on Azure resources should be removed | fde1c0c9-0fd2-4ecc-87b5-98956cbc1095 |

+| Blocked accounts with owner permissions on Azure resources should be removed | 050ac097-3dda-4d24-ab6d-82568e7a50cf |

+| Blocked accounts with read and write permissions on Azure resources should be removed | 1ff0b4c9-ed56-4de6-be9c-d7ab39645926 |

+#### Deprecation of identity recommendations V1

+The following security recommendations will be deprecated as part of this change:

+The following security recommendations will be deprecated as part of this change:

+

+| Recommendation | Assessment Key |

+|--|--|

+| MFA should be enabled on accounts with owner permissions on subscriptions | 94290b00-4d0c-d7b4-7cea-064a9554e681 |

+| MFA should be enabled on accounts with write permissions on subscriptions | 57e98606-6b1e-6193-0e3d-fe621387c16b |

+| MFA should be enabled on accounts with read permissions on subscriptions | 151e82c5-5341-a74b-1eb0-bc38d2c84bb5 |

+| External accounts with owner permissions should be removed from subscriptions | c3b6ae71-f1f0-31b4-e6c1-d5951285d03d |

+| External accounts with write permissions should be removed from subscriptions | 04e7147b-0deb-9796-2e5c-0336343ceb3d |

+| External accounts with read permissions should be removed from subscriptions | a8c6a4ad-d51e-88fe-2979-d3ee3c864f8b |

+| Deprecated accounts with owner permissions should be removed from subscriptions | e52064aa-6853-e252-a11e-dffc675689c2 |

+| Deprecated accounts should be removed from subscriptions | 00c6d40b-e990-6acf-d4f3-471e747a27c4 |

+We recommend updating custom scripts, workflows, and governance rules to correspond with the V2 recommendations.

+We've improved the coverage of the V2 identity recommendations by scanning all Azure resources (rather than just subscriptions) which allows security administrators to view role assignments per account. These changes may result in changes to your Secure Score throughout the GA process.

+|**Status** | Not available pre-configured|

+| Temperature |0┬░C ~ 60┬░C |

+# [Restore from the sensor console](#tab/restore-from-sensor-console)

+# [Restore the latest backup file by using the CLI](#tab/restore-using-cli)

+You can filter the types of model references that appear in the Model Graph. Turning off one of the reference types via the switches in this menu will prevent that reference type from displaying in the graph.

+You can also filter the models and references that appear in the graph by text, by selecting this **Filter** icon:

+You can highlight the models and references that appear in the graph by text, by selecting this **Highlight** icon:

+1. Database Migration Service (classic) - via Azure portal, PowerShell and Azure CLI.

+**Database Migration Service (classic)** via Azure portal, PowerShell and Azure CLI is an older version of the Azure Database Migration Service. It offers database modernization to Azure and support scenarios like – SQL Server, PostgreSQL, MySQL, and MongoDB. 

+|Feature |DMS (classic) |Azure SQL extension for Azure Data Studio |Notes|

+description: Learn to perform an online migration from SQL Server to an Azure SQL Managed Instance by using Azure Database Migration Service (classic)

+# Tutorial: Migrate SQL Server to an Azure SQL Managed Instance online using DMS (classic)

+description: Learn to migrate from SQL Server to Azure SQL Database offline by using Azure Database Migration Service (classic).

+# Tutorial: Migrate SQL Server to Azure SQL Database using DMS (classic)

+description: Learn to migrate from SQL Server to an Azure SQL Managed Instance by using Azure Database Migration Service (classic).

+# Tutorial: Migrate SQL Server to an Azure SQL Managed Instance offline using DMS (classic)

+ Title: Scenarios for Azure Private DNS zones

+description: In this article, learn about common scenarios for using Azure Private DNS zones.

+# Azure Private DNS zones scenarios

+Azure Private DNS zones provide name resolution within a virtual network and between virtual networks. In this article, we'll look at some common scenarios that can benefit using this feature.

+To learn more about Private DNS zones, see [Using Azure DNS for private domains](private-dns-overview.md).

+Learn how to [create a Private DNS zone](./private-dns-getstarted-powershell.md) in Azure DNS.

+ # You don't need to modify this id

+ # But Azure Event Grid Azure AD Application Id is different for different clouds

+

+ $eventGridAppId = "4962773b-9cdb-44cf-a8bf-237846a00ab7" # Azure Public Cloud

+ # $eventGridAppId = "54316b56-3481-47f9-8f30-0300f5542a7b" # Azure Government Cloud

+> [!IMPORTANT]

+> Microsoft Graph API's ability to send events to Azure Event Grid is currently in **private preview**.

+> In the following steps, you will follow instructions from [Node.js](https://github.com/microsoftgraph/nodejs-webhooks-sample), [Java](https://github.com/microsoftgraph/java-spring-webhooks-sample), and[.NET Core](https://github.com/microsoftgraph/aspnetcore-webhooks-sample) Webhook samples to enable flow of events from Microsoft Graph API. At some point in the sample, you will have an application registered with Azure AD. Email your application ID to <a href="mailto:ask-graph-and-grid@microsoft.com?subject=Please allow my application ID">mailto:ask-graph-and-grid@service.microsoft.com?subject=Please allow my Azure AD application with ID to send events through Graph API</a> so that the Microsoft Graph API team can add your application ID to allow list to use this new capability.

+An idle timer is in place to recycle idle sessions. The default value is four minutes for east-west connections and can't be changed. Applications that maintain keepalives don't idle out.

+For north-south connections that need more than 4 minutes (typical of IOT devices), you can contact support to extent the time for inbound connections to 30 minutes in the backend.

+For TLS 1.2 the following cipher suites are supported:

+When using custom domains with TLS 1.0 and 1.1 enabled, the following cipher suites are supported:

+Azure Front Door doesnΓÇÖt support disabling or configuring specific cipher suites for your profile.

+Azure Front Door delivers large files without a cap on file size. If caching is enabled, Front Door uses a technique called *object chunking*. When a large file is requested, Front Door retrieves smaller pieces of the file from the origin. After receiving a full file request or byte-range file request, the Azure Front Door environment requests the file from the origin in chunks of 8 MB.

+After the chunk arrives at the Azure Front Door environment, it's cached and immediately served to the user. Front Door then pre-fetches the next chunk in parallel. This pre-fetch ensures that the content stays one chunk ahead of the user, which reduces latency. This process continues until the entire file gets downloaded (if requested) or the client closes the connection. For more information on the byte-range request, read [RFC 7233](https://www.rfc-editor.org/info/rfc7233).

+Front Door caches any chunks as they're received so the entire file doesn't need to be cached on the Front Door cache. Subsequent requests for the file or byte ranges are served from the cache. If the chunks aren't all cached, pre-fetching is used to request chunks from the origin.

+This optimization relies on the origin's ability to support byte-range requests. If the origin doesn't support byte-range requests, or if it doesn't handle range requests correctly, then this optimization isn't effective.

+When your origin responds to a request with a `Range` header, it must respond in one of the following ways:

+- **Return a ranged response.** The response must use HTTP status code 206. Also, the `Content-Range` response header must be present, and must match the actual length of the content that your origin returns. If your origin doesn't send the correct response headers with valid values, Azure Front Door doesn't cache the response, and you might see inconsistent behavior.

+ > [!TIP]

+ > If your origin compresses the response, ensure that the `Content-Range` header value matches the actual length of the compressed response.

+- **Return a non-ranged response.** If your origin can't handle range requests, it can ignore the `Range` header and return a non-ranged response. Ensure that the origin returns a response status code other than 206. For example, the origin might return a 200 OK response.

+Cache purges on the Front Door are case-insensitive. Additionally, they're query string agnostic, which means that purging a URL purges all query string variations of it.

+- `TCP_HIT` or `TCP_REMOTE_HIT`: The first 8 MB chunk of the response is a cache hit, and the content is served from the Front Door cache.

+- `TCP_MISS`: The first 8 MB chunk of the response is a cache miss, and the content is fetched from the origin.

+* **When caching is enabled**, the cache behavior differs based on the cache behavior value applied by the Rules Engine:

+ Title: Apache Kafka TLS encryption & authentication for ESP Kafka Clusters - Azure HDInsight

+description: Set up TLS encryption for communication between Kafka clients and Kafka brokers, Set up SSL authentication of clients for ESP Kafka clusters

+# Set up TLS encryption and authentication for ESP Apache Kafka cluster in Azure HDInsight

+This article shows you how to set up Transport Layer Security (TLS) encryption, previously known as Secure Sockets Layer (SSL) encryption, between Apache Kafka clients and Apache Kafka brokers. It also shows you how to set up authentication of clients (sometimes referred to as two-way TLS).

+> [!Important]

+> There are two clients which you can use for Kafka applications: a Java client and a console client. Only the Java client `ProducerConsumer.java` can use TLS for both producing and consuming. The console producer client `console-producer.sh` does not work with TLS.

+## Apache Kafka broker setup

+The Kafka TLS broker setup uses four HDInsight cluster VMs in the following way:

+* headnode 0 - Certificate Authority (CA)

+* worker node 0, 1, and 2 - brokers

+> [!Note]

+> This guide uses self-signed certificates, but the most secure solution is to use certificates issued by trusted CAs.

+The summary of the broker setup process is as follows:

+1. The following steps are repeated on each of the three worker nodes:

+ 1. Generate a certificate.

+ 1. Create a cert signing request.

+ 1. Send the cert signing request to the Certificate Authority (CA).

+ 1. Sign in to the CA and sign the request.

+ 1. SCP the signed certificate back to the worker node.

+ 1. SCP the public certificate of the CA to the worker node.

+1. Once you have all of the certificates, put the certs into the cert store.

+1. Go to Ambari and change the configurations.

+Use the following detailed instructions to complete the broker setup:

+> [!Important]

+> In the following code snippets wnX is an abbreviation for one of the three worker nodes and should be substituted with `wn0`, `wn1` or `wn2` as appropriate. `WorkerNode0_Name` and `HeadNode0_Name` should be substituted with the names of the respective machines.

+1. Perform initial setup on head node 0, which for HDInsight fills the role of the Certificate Authority (CA).

+ ```bash

+ # Create a new directory 'ssl' and change into it

+ mkdir ssl

+ cd ssl

+ ```

+1. Perform the same initial setup on each of the brokers (worker nodes 0, 1 and 2).

+ ```bash

+ # Create a new directory 'ssl' and change into it

+ mkdir ssl

+ cd ssl

+ ```

+1. On each of the worker nodes, execute the following steps using the code snippet.

+ 1. Create a keystore and populate it with a new private certificate.

+ 1. Create a certificate signing request.

+ 1. SCP the certificate signing request to the CA (headnode0)

+ ```bash

+ keytool -genkey -keystore kafka.server.keystore.jks -validity 365 -storepass "MyServerPassword123" -keypass "MyServerPassword123" -dname "CN=FQDN_WORKER_NODE" -storetype pkcs12

+ keytool -keystore kafka.server.keystore.jks -certreq -file cert-file -storepass "MyServerPassword123" -keypass "MyServerPassword123"

+ scp cert-file sshuser@HeadNode0_Name:~/ssl/wnX-cert-sign-request

+ ```

+ > [!Note]

+ > FQDN_WORKER_NODE is Fully Qualified Domain Name of worker node machine.You can get that details from /etc/hosts file in head node

+

+ For example,

+ ```

+ wn0-espkaf.securehadooprc.onmicrosoft.com

+ wn0-kafka2.zbxwnwsmpcsuvbjqbmespcm1zg.bx.internal.cloudapp.net

+ ```

+ :::image type="content" source="./media/apache-esp-kafka-ssl-encryption-authentication/etc-hosts.png" alt-text="Screenshot showing host file output." border="true":::

+

+1. On the CA machine, run the following command to create ca-cert and ca-key files:

+ ```bash

+ openssl req -new -newkey rsa:4096 -days 365 -x509 -subj "/CN=Kafka-Security-CA" -keyout ca-key -out ca-cert -nodes

+ ```

+1. Change to the CA machine and sign all of the received cert signing requests:

+ ```bash

+ openssl x509 -req -CA ca-cert -CAkey ca-key -in wn0-cert-sign-request -out wn0-cert-signed -days 365 -CAcreateserial -passin pass:"MyServerPassword123"

+ openssl x509 -req -CA ca-cert -CAkey ca-key -in wn1-cert-sign-request -out wn1-cert-signed -days 365 -CAcreateserial -passin pass:"MyServerPassword123"

+ openssl x509 -req -CA ca-cert -CAkey ca-key -in wn2-cert-sign-request -out wn2-cert-signed -days 365 -CAcreateserial -passin pass:"MyServerPassword123"

+ ```

+1. Send the signed certificates back to the worker nodes from the CA (headnode0).

+ ```bash

+ scp wn0-cert-signed sshuser@WorkerNode0_Name:~/ssl/cert-signed

+ scp wn1-cert-signed sshuser@WorkerNode1_Name:~/ssl/cert-signed

+ scp wn2-cert-signed sshuser@WorkerNode2_Name:~/ssl/cert-signed

+ ```

+1. Send the public certificate of the CA to each worker node.

+ ```bash

+ scp ca-cert sshuser@WorkerNode0_Name:~/ssl/ca-cert

+ scp ca-cert sshuser@WorkerNode1_Name:~/ssl/ca-cert

+ scp ca-cert sshuser@WorkerNode2_Name:~/ssl/ca-cert

+ ```

+1. On each worker node, add the CAs public certificate to the truststore and keystore. Then add the worker node's own signed certificate to the keystore

+ ```bash

+ keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert -storepass "MyServerPassword123" -keypass "MyServerPassword123" -noprompt

+ keytool -keystore kafka.server.keystore.jks -alias CARoot -import -file ca-cert -storepass "MyServerPassword123" -keypass "MyServerPassword123" -noprompt

+ keytool -keystore kafka.server.keystore.jks -import -file cert-signed -storepass "MyServerPassword123" -keypass "MyServerPassword123" -noprompt

+ ```

+## Update Kafka configuration to use TLS and restart brokers

+You have now set up each Kafka broker with a keystore and truststore, and imported the correct certificates. Next, modify related Kafka configuration properties using Ambari and then restart the Kafka brokers.

+To complete the configuration modification, do the following steps:

+1. Sign in to the Azure portal and select your Azure HDInsight Apache Kafka cluster.

+1. Go to the Ambari UI by clicking **Ambari home** under **Cluster dashboards**.

+1. Under **Kafka Broker** set the **listeners** property to `PLAINTEXT://localhost:9092,SASL_SSL://localhost:9093`

+1. Under **Advanced kafka-broker** set the **security.inter.broker.protocol** property to `SASL_SSL`

+ :::image type="content" source="./media/apache-esp-kafka-ssl-encryption-authentication/properties-file-with-sasl.png" alt-text="Screenshot showing how to edit Kafka sasl configuration properties in Ambari." border="true":::

+1. Under **Custom kafka-broker** set the **ssl.client.auth** property to `required`.

+

+ > [!Note]

+ > This step is only required if you're setting up authentication and encryption.

+

+ :::image type="content" source="./media/apache-esp-kafka-ssl-encryption-authentication/editing-configuration-ambari2.png" alt-text="Screenshot showing how to edit Kafka ssl configuration properties in Ambari." border="true":::

+1. Here's the screenshot that shows Ambari configuration UI with these changes.

+

+ > [!Note]

+ > 1. ssl.keystore.location and ssl.truststore.location is the complete path of your keystore, truststore location in Certificate Authority (hn0)

+ > 1. ssl.keystore.password and ssl.truststore.password is the password set for the keystore and truststore. In this case as an example,` MyServerPassword123`

+ > 1. ssl.key.password is the key set for the keystore and trust store. In this case as an example, `MyServerPassword123`

+

+ For HDI version 4.0 or 5.0

+

+ a. If you're setting up authentication and encryption, then the screenshot looks like

+ :::image type="content" source="./media/apache-esp-kafka-ssl-encryption-authentication/properties-file-authentication-as-required.png" alt-text="Screenshot showing how to edit Kafka-env template property in Ambari authentication as required." border="true":::

+

+ b. If you are setting up encryption only, then the screenshot looks like

+

+ :::image type="content" source="./media/apache-esp-kafka-ssl-encryption-authentication/properties-file-authentication-as-none.png" alt-text="Screenshot showing how to edit Kafka-env template property in Ambari authentication as none." border="true":::

+1. Restart all Kafka brokers.

+## Client setup (without authentication)

+If you don't need authentication, the summary of the steps to set up only TLS encryption are:

+1. Sign in to the CA (active head node).

+1. Copy the CA cert to client machine from the CA machine (wn0).

+1. Sign in to the client machine (hn1) and navigate to the `~/ssl` folder.

+1. Import the CA cert to the truststore.

+1. Import the CA cert to the keystore.

+These steps are detailed in the following code snippets.

+1. Sign in to the CA node.

+ ```bash

+ ssh sshuser@HeadNode0_Name

+ cd ssl

+ ```

+1. Copy the ca-cert to the client machine

+ ```bash

+ scp ca-cert sshuser@HeadNode1_Name:~/ssl/ca-cert

+ ```

+1. Sign in to the client machine (standby head node).

+ ```bash

+ ssh sshuser@HeadNode1_Name

+ cd ssl

+ ```

+1. Import the CA certificate to the truststore.

+ ```bash

+ keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert -storepass "MyClientPassword123" -keypass "MyClientPassword123" -noprompt

+ ```

+1. Import the CA cert to keystore.

+

+ ```bash

+ keytool -keystore kafka.client.keystore.jks -alias CARoot -import -file ca-cert -storepass "MyClientPassword123" -keypass "MyClientPassword123" -noprompt

+ ```

+1. Create the file `client-ssl-auth.properties` on client machine (hn1) . It should have the following lines:

+ ```config

+ security.protocol=SASL_SSL

+ sasl.mechanism=GSSAPI

+ sasl.kerberos.service.name=kafka

+ ssl.truststore.location=/home/sshuser/ssl/kafka.client.truststore.jks

+ ssl.truststore.password=MyClientPassword123

+ ```

+1. Start the admin client with producer and consumer options to verify that both producers and consumers are working on port 9093. Refer to [Verification](apache-kafka-ssl-encryption-authentication.md#verification) section for steps needed to verify the setup using console producer/consumer.

+## Client setup (with authentication)

+> [!Note]

+> The following steps are required only if you are setting up both TLS encryption **and** authentication. If you are only setting up encryption, then see [Client setup without authentication](apache-kafka-ssl-encryption-authentication.md#client-setup-without-authentication).

+The following four steps summarize the tasks needed to complete the client setup:

+1. Sign in to the client machine (standby head node).

+1. Create a Java keystore and get a signed certificate for the broker. Then copy the certificate to the VM where the CA is running.

+1. Switch to the CA machine (active head node) to sign the client certificate.

+1. Go to the client machine (standby head node) and navigate to the `~/ssl` folder. Copy the signed cert to client machine.

+The details of each step are given.

+1. Sign in to the client machine (standby head node).

+ ```bash

+ ssh sshuser@HeadNode1_Name

+ ```

+1. Remove any existing ssl directory.

+ ```bash

+ rm -R ~/ssl

+ mkdir ssl

+ cd ssl

+ ```

+1. Create a Java keystore and create a certificate signing request.

+ ```bash

+ keytool -genkey -keystore kafka.client.keystore.jks -validity 365 -storepass "MyClientPassword123" -keypass "MyClientPassword123" -dname "CN=HEADNODE1_FQDN" -storetype pkcs12

+

+ keytool -keystore kafka.client.keystore.jks -certreq -file client-cert-sign-request -storepass "MyClientPassword123" -keypass "MyClientPassword123"

+ ```

+1. Copy the certificate signing request to the CA

+ ```bash

+ scp client-cert-sign-request sshuser@HeadNode0_Name:~/ssl/client-cert-sign-request

+ ```

+1. Switch to the CA machine (active head node) and sign the client certificate.

+ ```bash

+ ssh sshuser@HeadNode0_Name

+ cd ssl

+ openssl x509 -req -CA ca-cert -CAkey ca-key -in ~/ssl/client-cert-sign-request -out ~/ssl/client-cert-signed -days 365 -CAcreateserial -passin pass:MyClientPassword123

+ ```

+1. Copy signed client cert from the CA (active head node) to client machine.

+ ```bash

+ scp client-cert-signed sshuser@HeadNode1_Name:~/ssl/client-signed-cert

+ ```

+1. Copy the ca-cert to the client machine

+ ```bash

+ scp ca-cert sshuser@HeadNode1_Name:~/ssl/ca-cert

+ ```

+ 1. Sign in to the client machine (standby head node) and navigate to ssl directory.

+ ```bash

+ ssh sshuser@HeadNode1_Name

+ cd ssl

+ ```

+1. Create client store with signed cert, and import CA certificate into the keystore and truststore on client machine (hn1):

+ ```bash

+ keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert -storepass "MyClientPassword123" -keypass "MyClientPassword123" -noprompt

+

+ keytool -keystore kafka.client.keystore.jks -alias CARoot -import -file ca-cert -storepass "MyClientPassword123" -keypass "MyClientPassword123" -noprompt

+

+ keytool -keystore kafka.client.keystore.jks -import -file client-signed-cert -storepass "MyClientPassword123" -keypass "MyClientPassword123" -noprompt

+ ```

+1. Create a file `client-ssl-auth.properties` on client machine (hn1). It should have the following lines:

+ ```bash

+ security.protocol=SASL_SSL

+ sasl.mechanism=GSSAPI

+ sasl.kerberos.service.name=kafka

+ ssl.truststore.location=/home/sshuser/ssl/kafka.client.truststore.jks

+ ssl.truststore.password=MyClientPassword123

+ ssl.keystore.location=/home/sshuser/ssl/kafka.client.keystore.jks

+ ssl.keystore.password=MyClientPassword123

+ ssl.key.password=MyClientPassword123

+ ```

+## Verification

+Run these steps on the client machine.

+> [!Note]

+> If HDInsight 4.0 and Kafka 2.1 is installed, you can use the console producer/consumers to verify your setup. If not, run the Kafka producer on port 9092 and send messages to the topic, and then use the Kafka consumer on port 9093 which uses TLS.

+### Kafka 2.1 or above

+1. Create a topic if it doesn't exist already.

+ ```bash

+ /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper <ZOOKEEPER_NODE>:2181 --create --topic topic1 --partitions 2 --replication-factor 2

+ ```

+1. Start console producer and provide the path to `client-ssl-auth.properties` as a configuration file for the producer.

+ ```bash

+ /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list <FQDN_WORKER_NODE>:9093 --topic topic1 --producer.config ~/ssl/client-ssl-auth.properties

+ ```

+1. Open another ssh connection to client machine and start console consumer and provide the path to `client-ssl-auth.properties` as a configuration file for the consumer.

+ ```bash

+ /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --bootstrap-server <FQDN_WORKER_NODE>:9093 --topic topic1 --consumer.config ~/ssl/client-ssl-auth.properties --from-beginning

+ ```

+## Next steps

+* [What is Apache Kafka on HDInsight?](apache-kafka-introduction.md)

+# Set up TLS encryption and authentication for Non ESP Apache Kafka cluster in Azure HDInsight

+The Kafka TLS broker setup uses four HDInsight cluster VMs in the following way:

+> This guide uses self-signed certificates, but the most secure solution is to use certificates issued by trusted CAs.

+1. Perform initial setup on head node 0, which for HDInsight fills the role of the Certificate Authority (CA).

+ :::image type="content" source="./media/apache-kafka-ssl-encryption-authentication/etc-hosts.png" alt-text="Screenshot showing hosts file output." border="true":::

+ :::image type="content" source="./media/apache-kafka-ssl-encryption-authentication/editing-configuration-ambari.png" alt-text="Editing Kafka ssl configuration properties in Ambari." border="true":::

+ > Note: This step is only required if you're setting up authentication and encryption.

+ :::image type="content" source="./media/apache-kafka-ssl-encryption-authentication/editing-configuration-ambari2.png" alt-text="Editing kafka ssl configuration properties in Ambari." border="true":::

+

+ > [!Note]

+ > 1. ssl.keystore.location and ssl.truststore.location is the complete path of your keystore, truststore location in Certificate Authority (hn0)

+ > 1. ssl.keystore.password and ssl.truststore.password is the password set for the keystore and truststore. In this case as an example, `MyServerPassword123`

+ > 1. ssl.key.password is the key set for the keystore and trust store. In this case as an example, `MyServerPassword123`

+

+ 1. If you're setting up authentication and encryption, then the screenshot looks like

+ :::image type="content" source="./media/apache-kafka-ssl-encryption-authentication/editing-configuration-kafka-env-four.png" alt-text="Editing kafka-env template property in Ambari four." border="true":::

+

+ 1. If you are setting up encryption only, then the screenshot looks like

+

+ :::image type="content" source="./media/apache-kafka-ssl-encryption-authentication/editing-configuration-kafka-env-four-encryption-only.png" alt-text="Screenshot showing how to edit kafka-env template property field in Ambari for encryption only." border="true":::

+

+1. Copy the CA certificate to client machine from the CA machine (wn0).

+1. Import the CA certificate to the truststore.

+1. Import the CA certificate to the keystore.

+1. Import the CA certificate to keystore.

+ Title: Connect Apache Kafka cluster with VM in different VNet on Azure HDInsight - Azure HDInsight

+description: Learn how to connect Apache Kafka cluster with VM in different VNet on Azure HDInsight

+# How to connect Kafka cluster with VM in different VNet

+This Document lists steps that must be followed to set up connectivity between VM and HDI Kafka residing in two different VNet.

+1. Create two different VNets where HDInsight Kafka cluster and VM will be hosted respectively. For more information, see [Create a virtual network using the Azure portal](https://learn.microsoft.com/azure/virtual-network/quick-create-portal)

+ > [!Note]

+ > These two VNets must be peered, so that IP addresses of their subnets must not overlap with each other. For more information, see [Create a virtual network using the Azure portal](https://learn.microsoft.com/azure/virtual-network/tutorial-connect-virtual-networks-portal)

+1. Make sure that the peering status shows as connected.

+

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/kakfa-event-peering-window.png" alt-text="Screenshot showing Kafka event peering." border="true":::

+

+1. After the above steps are completed, we can create HDInsight Kafka cluster in one VNet. For more information, see [Create an Apache Kafka cluster](./apache-kafka-get-started.md#create-an-apache-kafka-cluster)

+1. Create a Virtual Machine in the second VNet. While creating the VM, specify the second VNet name where this virtual machine must be deployed. For more information, see [Create a Linux virtual machine in the Azure portal](https://learn.microsoft.com/azure/virtual-machines/linux/quick-create-portal)

+1. After this step, we can copy the entries of the file /etc/host from Kafka headnode to VM.

+

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/etc-host-output.png" alt-text="Screenshot showing host file output." border="true":::

+

+1. Remove the `headnodehost` string entries from the file. For example, the above image has `headnodehost` entry for the ip 10.0.0.16. After removal, it will be as

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/modified-etc-hosts-output.png" alt-text="Screenshot showing modified host file output." border="true":::

+1. After these entries are made, try to reach the Kafka Ambari dashboard using the curl command using the hn0 or hn1 FQDN as

+

+ From Linux VM

+

+ ```

+ curl hn0-vnetka.glarbkztnoqubmzjylls33qnse.bx.internal.cloudapp.net:8080

+ ```

+

+ Output:

+ ```

+ ΓÇ£<!--

+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

+ -->

+ <!DOCTYPE html>

+ <html lang="en">

+

+ <head>

+ <meta charset="utf-8">

+ <meta http-equiv="X-UA-Compatible" content="IE=edge">

+ <meta name="viewport" content="width=device-width, initial-scale=1.0">

+ <link rel="stylesheet" href="stylesheets/vendor.css">

+ <link rel="stylesheet" href="stylesheets/app.css">

+ <script src="javascripts/vendor.js"></script>

+ <script src="javascripts/app.js"></script>

+

+ <script>

+ $(document).ready(function() {

+ require('initialize');

+ // make favicon work in firefox

+ $('link[type*=icon]').detach().appendTo('head');

+ $('#loading').remove();

+ });

+ </script>

+ <title>Ambari</title>

+ <link rel="shortcut icon" href="/img/logo.png" type="image/x-icon">

+ </head>

+ <body>

+ <div id="loading">...Loading...</div>

+ <div id="wrapper">

+ <!-- ApplicationView -->

+ </div>

+ <footer>

+ <div class="container footer-links">

+ <a data-qa="license-link" href="http://www.apache.org/licenses/LICENSE-2.0" target="_blank">Licensed under the Apache License, Version 2.0</a>. <br>

+ <a data-qa="third-party-link" href="/licenses/NOTICE.txt" target="_blank">See third-party tools/resources that Ambari uses and their respective authors</a>

+ </div>

+ </footer>

+ </body>

+ </html>ΓÇ¥

+ ```

+ From Windows VM

+

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/windows-vm.png" alt-text="Screenshot showing windows VM output." border="true":::

+

+ > [!Note]

+ > 1. In Windows VM , static hostnames are added in the file hosts present in the path `C:\Windows\System32\drivers\etc\`

+ > 1. This document assumes that the Ambari server is active on hn0. If the Ambari server is active on hn1 use the FQDN of hn1 to access the Ambari UI.

+

+1. You can also send messages to kafka topic and read the topics from the VM. For that you can try to use this sample java application, https://github.com/Azure-Samples/hdinsight-kafka-java-get-started

+

+ Make sure to create the topic inside the Kafka cluster using the command

+

+ ```

+ java -jar kafka-producer-consumer.jar create <topic_name> $KAFKABROKERS

+ ```

+1. After creating the topic, we can use the below commands to produce and consume. The $KAFKABROKERS must be replaced appropriately with the broker worker node FQDN and port as mentioned in the documentation.

+

+ ```

+ java -jar kafka-producer-consumer.jar producer test $KAFKABROKERS `

+ java -jar kafka-producer-consumer.jar consumer test $KAFKABROKERS

+ ```

+

+1. After this step you get an output as

+

+ **Producer output:**

+

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/kafka-producer-output.png" alt-text="Screenshot showing Kafka producer output VM." border="true":::

+

+ **Consumer output:**

+

+ :::image type="content" source="./media/connect-kafka-cluster-with-different-vnet/kafka-consumer-output.png" alt-text="Screenshot showing Kafka producer output." border="true":::

+

+description: This article provides an overview of the MedTech service device messaging data transformation to FHIR Observation resources. The MedTech service ingests, normalizes, groups, transforms, and persists device message data in the FHIR service.

+This article provides an overview of the device message data processing stages within the [MedTech service](overview.md). The MedTech service transforms device message data into FHIR [Observation](https://www.hl7.org/fhir/observation.html) resources for persistence in the [FHIR service](../fhir/overview.md).

+If no Device resource for a given device identifier exists in the FHIR service, the outcome depends upon the value of [Resolution Type](deploy-new-config.md#configure-the-destination-tab) set at the time of the MedTech service deployment. When set to `Lookup`, the specific message is ignored, and the pipeline continues to process other incoming device messages. If set to `Create`, the MedTech service creates minimal Device and Patient resources in the FHIR service.

+FHIR&#174; is a registered trademark of Health Level Seven International, registered in the U.S. Trademark Office and is used with their permission.

+To configure monitoring on your IoT Edge device, follow the [Tutorial: Monitor IoT Edge devices](tutorial-monitor-with-workbooks.md). You learn how to add the metrics-collector module to your device. This article gives you an overview of the monitoring architecture and explains your options on configuring metrics on your device.

+All configuration for the metrics-collector is done using environment variables. Minimally, the variables noted in this table marked as **Required** need to be specified.

+| `AllowedMetrics` | List of metrics to collect, all other metrics are ignored. Set to an empty string to disable. For more information, see [allow and disallow lists](#allow-and-disallow-lists). <br><br>Example: *metricToScrape{quantile=0.99}[endpoint=http://MetricsSpewer:9417/metrics]*<br><br> **Not required** <br><br> Default value: *empty* |

+| `BlockedMetrics` | List of metrics to ignore. Overrides *AllowedMetrics*, so a metric isn't reported if it's included in both lists. For more information, see [allow and disallow lists](#allow-and-disallow-lists). <br><br> Example: *metricToIgnore{quantile=0.5}[endpoint=http://VeryNoisyModule:9001/metrics], docker_container_disk_write_bytes*<br><br> **Not required** <br><br>Default value: *empty* |

+| `AllowedMetrics` | List of metrics to collect, all other metrics are ignored. Set to an empty string to disable. For more information, see [allow and disallow lists](#allow-and-disallow-lists). <br><br>Example: *metricToScrape{quantile=0.99}[endpoint=http://MetricsSpewer:9417/metrics]*<br><br> **Not required** <br><br> Default value: *empty* |

+| `BlockedMetrics` | List of metrics to ignore. Overrides *AllowedMetrics*, so a metric is reported if it's included in both lists. For more information, see [allow and disallow lists](#allow-and-disallow-lists). <br><br> Example: *metricToIgnore{quantile=0.5}[endpoint=http://VeryNoisyModule:9001/metrics], docker_container_disk_write_bytes*<br><br> **Not required** <br><br>Default value: *empty* |

+The `AllowedMetrics` and `BlockedMetrics` configuration options take space- or comma-separated lists of metric selectors. A metric matches the list and is included or excluded if it matches one or more metrics in either list.

+* A metric is matched if at least all labels in the selector are present and also match.

+ * Regex is fully anchored (A `^` and `$` are automatically added to the start and end of each regex)

+Sometimes it's necessary to ingest metrics through IoT Hub instead of sending them directly to Log Analytics. For example, when monitoring [IoT Edge devices in a nested configuration](tutorial-nested-iot-edge.md) where child devices have access only to the IoT Edge hub of their parent device. Another example is when deploying an IoT Edge device with *outbound network access only* to IoT Hub.

+This option does require extra setup, a cloud workflow, to deliver metrics messages arriving at IoT Hub to the Log Analytics workspace. Without this set up, the other portions of the integration such as [curated visualizations](how-to-explore-curated-visualizations.md) and [alerts](how-to-create-alerts.md) don't work.

+Sometimes it's necessary to ingest metrics through IoT Central instead of sending them directly to Log Analytics. For example, when monitoring [IoT Edge devices in a nested configuration](tutorial-nested-iot-edge.md) where child devices have access only to the IoT Edge hub of their parent device. Another example is when deploying an IoT Edge device with *outbound network access only* to IoT Central.

+ :::image type="content" source="media/how-to-collect-and-transport-metrics/add-metrics-interface.png" alt-text="Screenshot that shows how to add the IoT Edge Metrics standard interface." lightbox="media/how-to-collect-and-transport-metrics/add-metrics-interface.png":::

+ :::image type="content" source="media/how-to-collect-and-transport-metrics/iot-edge-metrics-telemetry.png" alt-text="Screenshot that shows the IoT Edge metrics available as telemetry." lightbox="media/how-to-collect-and-transport-metrics/iot-edge-metrics-telemetry.png":::

+This article shows how to autoprovision one or more [IoT Edge for Linux on Windows](iot-edge-for-linux-on-windows.md) devices using symmetric keys. You can automatically provision Azure IoT Edge devices with the [Azure IoT Hub device provisioning service](../iot-dps/index.yml) (DPS). If you're unfamiliar with the process of autoprovisioning, review the [provisioning overview](../iot-dps/about-iot-dps.md#provisioning-process) before continuing.

+The **deviceToCloudUpload** feature is a configurable functionality. This function automatically uploads the data from your local blob storage to Azure with intermittent internet connectivity support. It allows you to:

+If an unexpected process termination (like power failure) happens during a blob upload, all blocks due for the upload are uploaded again once the module comes back online.

+* Specify the time in minutes (deleteAfterMinutes) after which the blobs are automatically deleted.

+The name of this setting is `deviceToCloudUploadProperties`. If you're using the IoT Edge simulator, set the values to the related environment variables for these properties, which you can find in the explanation section.

+| uploadOn | true, false | Set to `false` by default. If you want to turn on the feature, set this field to `true`. <br><br> Environment variable: `deviceToCloudUploadProperties__uploadOn={false,true}` |

+| cloudStorageConnectionString | | `"DefaultEndpointsProtocol=https;AccountName=<your Azure Storage Account Name>;AccountKey=<your Azure Storage Account Key>;EndpointSuffix=<your end point suffix>"` is a connection string that allows you to specify the storage account to which you want your data uploaded. Specify `Azure Storage Account Name`, `Azure Storage Account Key`, `End point suffix`. Add appropriate EndpointSuffix of Azure where data is uploaded, it varies for Global Azure, Government Azure, and Microsoft Azure Stack. <br><br> You can choose to specify Azure Storage SAS connection string here. But you have to update this property when it expires. SAS permissions may include create access for containers and create, write, and add access for blobs. <br><br> Environment variable: `deviceToCloudUploadProperties__cloudStorageConnectionString=<connection string>` |

+| storageContainersForUpload | `"<source container name1>": {"target": "<target container name>"}`,<br><br> `"<source container name1>": {"target": "%h-%d-%m-%c"}`, <br><br> `"<source container name1>": {"target": "%d-%c"}` | Allows you to specify the container names you want to upload to Azure. This module allows you to specify both source and target container names. If you don't specify the target container name, it's automatically assigned a container name such as `<IoTHubName>-<IotEdgeDeviceID>-<ModuleName>-<SourceContainerName>`. You can create template strings for target container name, check out the possible values column. <br>* %h -> IoT Hub Name (3-50 characters). <br>* %d -> IoT Edge Device ID (1 to 129 characters). <br>* %m -> Module Name (1 to 64 characters). <br>* %c -> Source Container Name (3 to 63 characters). <br><br>Maximum size of the container name is 63 characters. The name is automatically assigned the target container name if the size of container exceeds 63 characters. In this case, name is trimmed in each section (IoTHubName, IotEdgeDeviceID, ModuleName, SourceContainerName) to 15 characters. <br><br> Environment variable: `deviceToCloudUploadProperties__storageContainersForUpload__<sourceName>__target=<targetName>` |

+| deleteAfterUpload | true, false | Set to `false` by default. When set to `true`, the data automatically deletes when the upload to cloud storage is finished. <br><br> **CAUTION**: If you're using append blobs, this setting deletes append blobs from local storage after a successful upload, and any future Append Block operations to those blobs will fail. Use this setting with caution. Don't enable this setting if your application does infrequent append operations or doesn't support continuous append operations<br><br> Environment variable: `deviceToCloudUploadProperties__deleteAfterUpload={false,true}`. |

+The name of this setting is `deviceAutoDeleteProperties`. If you're using the IoT Edge simulator, set the values to the related environment variables for these properties, which you can find in the explanation section.

+| deleteOn | true, false | Set to `false` by default. If you want to turn on the feature, set this field to `true`. <br><br> Environment variable: `deviceAutoDeleteProperties__deleteOn={false,true}` |

+| deleteAfterMinutes | `<minutes>` | Specify the time in minutes. The module automatically deletes your blobs from local storage when this value expires. Current maximum minutes allowed are 35791. <br><br> Environment variable: `deviceAutoDeleteProperties__ deleteAfterMinutes=<minutes>` |

+| retainWhileUploading | true, false | By default it's set to `true`, and retains the blob while it's uploading to cloud storage if `deleteAfterMinutes` expire. You can set it to `false` and it deletes the data as soon as `deleteAfterMinutes` expires. Note: For this property to work uploadOn should be set to true. <br><br> **CAUTION**: If you use append blobs, this setting deletes append blobs from local storage when the value expires, and any future Append Block operations to those blobs fail. Make sure the expiry value is large enough for the expected frequency of append operations performed by your application.<br><br> Environment variable: `deviceAutoDeleteProperties__retainWhileUploading={false,true}`|

+The configuration steps:

+This command uses the credentials to authenticate with the remote SMB server. Then, map the remote share path to G: drive letter (can be any other available drive letter). The IoT device now has the data volume mapped to a path on the G: drive.

+If you use [volume mount](https://docs.docker.com/storage/volumes/) for storage in your create options for Linux containers, then you don't have to do any extra steps, but if you use [bind mount](https://docs.docker.com/storage/bind-mounts/), then these steps are required to run the service correctly.

+Following the principle of least privilege to limit the access rights for users to bare minimum permissions they need to perform their work, this module includes a user (name: absie, ID: 11000) and a user group (name: absie, ID: 11000). If the container is started as **root** (default user is **root**), our service is started as the low-privilege **absie** user.

+This behavior makes configuration of the permissions on host path binds crucial for the service to work correctly, otherwise the service crashes with access denied errors. The path that is used in directory binding needs to be accessible by the container user (example: absie 11000). You can grant the container user access to the directory by executing these commands on the host:

+If you need to run the service as a user other than **absie**, you can specify your custom user ID in createOptions under "User" property in your deployment manifest. In such a case, use default or root group ID `0`.

+ * The Azure Blob Storage on IoT Edge module v1.4.0 and earlier are compatible with WindowsAzure.Storage 9.3.3 SDK and v1.4.1 also supports Azure.Storage.Blobs 12.8.0 SDK.

+ * Versions before V2.1 of the Python SDK have a known issue where the module doesn't return the blob creation time. Because of that issue, some methods like list blobs don't work. As a workaround, explicitly set the API version on the blob client to '2017-04-17'. Example: `block_blob_service._X_MS_VERSION = '2017-04-17'`

+* Put and get blocklist

+Stay up-to-date with recent updates and announcement on the [Azure Blob Storage on IoT Edge release notes](https://hub.docker.com/_/microsoft-azure-blob-storage) page.

+- Install [Java SE Development Kit 10](/azure/developer/java/fundamentals/java-support-on-azure) and [Maven](https://maven.apache.org/). You need to [set the `JAVA_HOME` environment variable](https://docs.oracle.com/cd/E19182-01/820-7851/inst_cli_jdk_javahome_t/) to point to your JDK installation.

+- Install [Node.js](https://nodejs.org). Install [Yeoman](https://www.npmjs.com/package/yo) and the [Azure IoT Edge Node.js Module Generator](https://www.npmjs.com/package/generator-azure-iot-edge-module).

+To test your module on a device:

+- An **.env** file lists your environment variables. The environment variable for the container registry is *localhost* by default. If Azure Container Registry is your registry, set an Azure Container Registry username and password. For example,

+1. Provide the name of the module's image repository. Visual Studio Code autopopulates the module name with **localhost:5000/<your module name\>**. Replace it with your own registry information. Use **localhost** if you use a local Docker registry for testing. If you use Azure Container Registry, then use sign in server from your registry's settings. The sign in server looks like **_\<registry name\>_.azurecr.io**. Only replace the **localhost:5000** part of the string so that the final result looks like **\<*registry name*\>.azurecr.io/_\<your module name\>_**.

+- A **modules** folder has subfolders for each module. Within the folder for each module, there's a file called **module.json** that controls how modules are built and deployed. This file would need to be modified to change the module deployment container registry from localhost to a remote registry. At this point, you only have one module. But you can add more if needed

+- An **.env** file lists your environment variables. The environment variable for the container registry is *localhost* by default. If Azure Container Registry is your registry, set an Azure Container Registry username and password. For example,

+## Add more modules

+The sample modules are designed so that you can build the solution, push to your container registry, and deploy to a device. This process lets you start testing without modifying any code. The sample module takes input from a source (in this case, the *SimulatedTemperatureSensor* module that simulates data) and pipes it to IoT Hub.

+- C# modules, including modules for Azure Functions, support debugging in Linux amd64 containers

+1. In Visual Studio Code Debug view, you see the variables in the left panel.

+1. If you're using an Azure Container Registry to store your module image, add your credentials to **deployment.debug.template.json** in the *edgeAgent* settings. For example:

+ For example, the *filtermodule* configuration should be similar to:

+Select **Start Debugging** or select **F5**. Select the process to attach to. In the Visual Studio Code Debug view, you see variables in the left panel.

+ The output should list the containers running on the remote device similar:

+1. In the *.Visual Studio Code* directory, add a new configuration to **launch.json** by opening the file in Visual Studio Code. Select **Add configuration** then choose the matching remote attach template for your module. For example, the following configuration is for .NET Core. Change the value for the *-H* parameter in *PipeArgs* to your device DNS name or IP address.

+1. In the Visual Studio Code Debug view, you see the variables in the left panel.

+Use [Visual Studio Code](https://code.visualstudio.com/) to develop and deploy code to devices running IoT Edge.

+* Use your own computer or a virtual machine.

+* Your development machine must support [nested virtualization](/virtualization/hyper-v-on-windows/user-guide/nested-virtualization) for running a container engine, which you'll install in the next section.

+* We recommend not to run IoT Edge on your development machine, but instead use a separate device. This distinction between development machine and IoT Edge device simulates a true deployment scenario and helps keep the different concepts straight.

+> For guidance on interactive debugging in Visual Studio Code or Visual Studio 2022:

+>* [Use Visual Studio 2022 to develop and debug modules for Azure IoT Edge](how-to-visual-studio-develop-module.md)

+| | Visual Studio Code | Visual Studio 2019/2022 |

+| **More information** | [Azure IoT Edge for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-edge) <br> [Azure IoT Hub](https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-toolkit)| [Azure IoT Edge Tools for Visual Studio 2019](https://marketplace.visualstudio.com/items?itemName=vsc-iot.vs16iotedgetools) <br> [Azure IoT Edge Tools for Visual Studio 2022](https://marketplace.visualstudio.com/items?itemName=vsc-iot.vs17iotedgetools) |

+2. Once the installation is finished, open Visual Studio Code and select **View** > **Extensions**.

+4. On each extension, select **Install**.

+In the Visual Studio Code command palette, search for and select **Azure IoT Edge: New IoT Edge Solution**. Follow the prompts to create your solution:

+1. Select folder: choose the location on your development machine for Visual Studio Code to create the solution files.

+1. Provide a solution name: enter a descriptive name for your solution or accept the default **EdgeSolution**.

+1. Select a module template: choose **C# Module**.

+1. Provide a module name: accept the default **SampleModule**.

+1. Provide Docker image repository for the module: an image repository includes the name of your container registry and the name of your container image. Your container image is prepopulated from the name you provided in the last step. Replace **localhost:5000** with the **Login server** value from your Azure container registry. You can retrieve the Login server value from the Overview page of your container registry in the Azure portal.

+ The final image repository looks like:

+

+ \<registry name\>.azurecr.io/samplemodule.

+ :::image type="content" source="./media/tutorial-develop-for-linux/image-repository.png" alt-text="Screenshot showing where to provide a Docker image repository in the command palette.":::

+* The **deployment.debug.template.json** file and **deployment.template.json** file are templates that help you create a deployment manifest. A *deployment manifest* is a file that defines exactly which modules you want deployed on a device, how they should be configured, and how they can communicate with each other and the cloud. The template files use pointers for some values. When you transform the template into a true deployment manifest, the pointers are replaced with values taken from other solution files.

+* Open the **deployment.template.json** file and locate two common placeholders:

+ * In the `registryCredentials` section, the address is auto-filled from the information you provided when you created the solution. However, the username and password reference the variables stored in the .env file. This configuration is for security, as the .env file is git ignored, but the deployment template is not.

+ * In the `SampleModule` section, the container image isn't filled in even though you provided the image repository when you created the solution. This placeholder points to the **module.json** file inside the SampleModule folder. If you go to that file, you'll see that the image field does contain the repository, but also a tag value that is made up of the version and the platform of the container. You can iterate the version manually as part of your development cycle, and you select the container platform using a switcher that we introduce later in this section.

+>If you didn't replace the **localhost:5000** value with the login server value from your Azure container registry, in the [**Create a project template**](#create-a-project-template) step, the **.env** file and the `registryCredentials` section of the deployment manifest will be missing. If that section is missing, return to the **Provide Docker image repository for the module** step in the **Create a project template** section to see how to replace the **localhost:5000** value.

+1. Open the command palette and search for **Azure IoT Edge: Set Default Target Platform for Edge Solution**, or select the shortcut icon at the bottom of the window.

+ :::image type="content" source="./media/tutorial-develop-for-linux/select-architecture.png" alt-text="Screenshot showing the location of the architecture icon at the bottom of the Visual Studio Code window." lightbox="./media/tutorial-develop-for-linux/select-architecture.png":::

+ :::image type="content" source="./media/tutorial-develop-for-linux/declare-input-queue.png" alt-text="Screenshot showing where to find the input name in the SetInputMessageCallback constructor." lightbox="./media/tutorial-develop-for-linux/declare-input-queue.png":::

+ The [SendEventAsync](/dotnet/api/microsoft.azure.devices.client.moduleclient.sendeventasync) method processes received messages and sets up an output queue to pass them along. Review this method and see that it initializes an output queue called **output1**.

+ :::image type="content" source="./media/tutorial-develop-for-linux/declare-output-queue.png" alt-text="Screenshot showing where to find the output name in SendEventAsync method." lightbox="./media/tutorial-develop-for-linux/declare-output-queue.png":::

+7. Find the **modules** property nested in **$edgeAgent**.

+8. At the bottom of the file, find **properties.desired** within the **$edgeHub** module.

+ :::image type="content" source="./media/tutorial-develop-for-linux/deployment-routes.png" alt-text="Screenshot showing routes in the deployment.template.json file." lightbox="./media/tutorial-develop-for-linux/deployment-routes.png":::

+2. Sign in to Docker with the Azure Container Registry (ACR) credentials that you saved after creating the registry.

+ :::image type="content" source="./media/tutorial-develop-for-linux/build-and-push-modules.png" alt-text="Screenshot showing the right-click menu option Build and Push IoT Edge Solution." lightbox="./media/tutorial-develop-for-linux/build-and-push-modules.png":::

+> [!TIP]

+> If you're working with Azure Key Vault Certificates resources in a Spring application, we recommend that you consider [Spring Cloud Azure](/azure/developer/java/spring-framework/) as an alternative. Spring Cloud Azure is an open-source project that provides seamless Spring integration with Azure services. To learn more about Spring Cloud Azure, and to see an example using Key Vault Certificates, see [Enable HTTPS in Spring Boot with Azure Key Vault certificates](/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-key-vault-certificates).

+key_vault_name = "<your-unique-keyvault-name>"

+key_vault_uri = f"https://{key_vault_name}.vault.azure.net"

+secret_name = "mySecret"

+client = SecretClient(vault_url=key_vault_uri, credential=credential)

+retrieved_secret = client.get_secret(secret_name)

+print(f"The value of secret '{secret_name}' in '{key_vault_name}' is: '{retrieved_secret.value}'")

+[Azure Key Vault REST API](/rest/api/keyvault/)

+> [!TIP]

+> If you're working with Azure Key Vault Secrets resources in a Spring application, we recommend that you consider [Spring Cloud Azure](/azure/developer/java/spring-framework/) as an alternative. Spring Cloud Azure is an open-source project that provides seamless Spring integration with Azure services. To learn more about Spring Cloud Azure, and to see an example using Key Vault Secrets, see [Load a secret from Azure Key Vault in a Spring Boot application](/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-key-vault).

+While the test script runs, Azure Load Testing collects and aggregates the Apache JMeter worker logs from all test engine instances. You can [download the logs for analyzing errors during the load test](./how-to-troubleshoot-failing-test.md).

+- Learn more about [troubleshooting load test execution errors](./how-to-troubleshoot-failing-test.md).

+You can [download the JMeter errors logs](./how-to-troubleshoot-failing-test.md) to troubleshoot errors during the load test.

+- Learn more about [troubleshooting load test execution errors](./how-to-troubleshoot-failing-test.md).

+In this article, you learn how to download the test results from Azure Load Testing in the Azure portal. You might use these results for reporting in third-party tools or for diagnosing test failures. Azure Load Testing generates the test results in comma-separated values (CSV) file format, and provides details of each application request for the load test.

+You can also use the test results to diagnose errors during a load test. The `responseCode` and `responseMessage` fields give you more information about failed requests. For more information about investigating errors, see [Troubleshoot test execution errors](./how-to-troubleshoot-failing-test.md).

+## Test results file

+Azure Load Testing generates a test results CSV file for each [test engine instance](./concept-load-testing-concepts.md#test-engine). Learn how you can [scale out your load test](./how-to-high-scale-load.md).

+Azure Load Testing uses the [Apache JMeter CSV log format](https://jmeter.apache.org/usermanual/listeners.html#csvlogformat). For more information about the different fields, see the [JMeter Glossary in the Apache JMeter documentation](https://jmeter.apache.org/usermanual/glossary.html).

+You can find the details of each application request for the load test run in the test results file. The following snippet shows a sample test result:

+```output

+timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,URL,Latency,IdleTime,Connect

+1676040230680,104,Homepage,200,OK,172.18.33.7-Thread Group 1-5,text,true,,1607,133,5,5,https://www.example.com/,104,0,100

+1676040230681,101,Homepage,200,OK,172.18.33.7-Thread Group 1-3,text,true,,1591,133,5,5,https://www.example.com/,101,0,93

+1676040230680,101,Homepage,200,OK,172.18.33.7-Thread Group 1-1,text,true,,1591,133,5,5,https://www.example.com/,98,0,94

+```

+# [Azure portal](#tab/portal)

+To download the test results for a test run in the Azure portal:

+# [GitHub Actions](#tab/github)

+When you run a load test as part of your CI/CD pipeline, Azure Load Testing generates a test results file. Follow these steps to publish these test results and attach them to your CI/CD pipeline run:

+1. Go to your GitHub repository, and select **Code**.

+1. In the **Code** window, select your GitHub Actions workflow YAML file in the `.github/workflow` folder.

+ :::image type="content" source="./media/how-to-export-test-results/github-repository-workflow-definition-file.png" alt-text="Screenshot that shows the folder that contains the GitHub Actions workflow definition file." lightbox="./media/how-to-export-test-results/github-repository-workflow-definition-file.png":::

+1. Edit the workflow file and add the `actions/upload-artifact` action after the `azure/load-testing` action in the workflow file.

+ Azure Load Testing places the test results in the `loadTest` folder of the GitHub Actions workspace.

+ ```yml

+ - name: 'Azure Load Testing'

+ uses: azure/load-testing@v1

+ with:

+ loadTestConfigFile: 'SampleApp.yaml'

+ loadTestResource: ${{ env.LOAD_TEST_RESOURCE }}

+ resourceGroup: ${{ env.LOAD_TEST_RESOURCE_GROUP }}

+

+ - uses: actions/upload-artifact@v2

+ with:

+ name: loadTestResults

+ path: ${{ github.workspace }}/loadTest

+ ```

+1. After your GitHub Actions workflow completes, you can select the test results from the **Artifacts** section on the **Summary** page of the workflow run.

+ :::image type="content" source="./media/how-to-export-test-results/github-actions-run-summary.png" alt-text="Screenshot that shows the GitHub Actions workflow summary page, highlighting the test results in the Artifacts section." lightbox="./media/how-to-export-test-results/github-actions-run-summary.png":::

+# [Azure Pipelines](#tab/pipelines)

+When you run a load test as part of your CI/CD pipeline, Azure Load Testing generates a test results file. Follow these steps to publish these test results and attach them to your CI/CD pipeline run:

+1. In your Azure DevOps project, select **Pipelines** in the left navigation, and select your pipeline from the list.

+1. On the pipeline details page, select **Edit** to edit the workflow definition.

+1. Edit the workflow file and add the `publish` task after the `AzureLoadTest` task in the workflow file.

+ Azure Load Testing places the test results in the `loadTest` folder of the Azure Pipelines default working directory.

+ ```yml

+ - task: AzureLoadTest@1

+ inputs:

+ azureSubscription: $(serviceConnection)

+ loadTestConfigFile: 'SampleApp.yaml'

+ resourceGroup: $(loadTestResourceGroup)

+ loadTestResource: $(loadTestResource)

+

+ - publish: $(System.DefaultWorkingDirectory)/loadTest

+ artifact: results

+ ```

+1. After your Azure Pipelines workflow completes, you can select the test results from the **Stages** section on the **Summary** page of the workflow run.

+ You can find and download the test results in the **Results** folder.

+ :::image type="content" source="./media/how-to-export-test-results/azure-pipelines-run-summary.png" alt-text="Screenshot that shows the Azure Pipelines workflow summary page, highlighting the test results in the Stages section." lightbox="./media/how-to-export-test-results/azure-pipelines-run-summary.png":::

+- Learn more about [Troubleshooting test execution errors](./how-to-troubleshoot-failing-test.md).

+> Apache JMeter only reports requests that made it to the server and back, either successful or not. If Apache JMeter is unable to connect to your application, the actual number of requests per second will be lower than the maximum value. Possible causes might be that the server is too busy to handle the request, or that an TLS/SSL certificate is missing. To diagnose connection problems, you can check the **Errors** chart in the load testing dashboard and [download the load test log files](./how-to-troubleshoot-failing-test.md).

+ Title: Troubleshoot load test errors

+description: Learn how you can diagnose and troubleshoot errors in Azure Load Testing. Download and analyze the Apache JMeter worker logs in the Azure portal.

+# Troubleshoot failing load tests in Azure Load Testing

+Learn how to diagnose and troubleshoot errors while running a load test with Azure Load Testing. Download the Apache JMeter worker logs or load test results for detailed logging information. Alternately, you can configure server-side metrics to identify issues in specific Azure application components.

+Azure Load Testing runs your Apache JMeter script on the [test engine instances](./concept-load-testing-concepts.md#test-engine). During a load test run, errors might occur at different stages. For example, the JMeter test script could have an error that prevents the test from starting. Or there might be a problem to connect to the application endpoint, which results in the load test to have a large number of failed requests.

+Azure Load Testing provides different sources of information to diagnose these errors:

+- [Download the Apache JMeter worker logs](#download-apache-jmeter-worker-logs) to investigate issues with JMeter and the test script execution.

+- [Diagnose failing tests using test results](#diagnose-failing-tests-using-test-results) and analyze the response code and response message of each HTTP request.

+- [Diagnose failing tests using server-side metrics](#diagnose-failing-tests-using-server-side-metrics) to identify issues with specific Azure application components.

+## Prerequisites

+- An Azure account with an active subscription. If you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

+- An Azure load testing resource that has a completed test run. If you need to create an Azure load testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).

+## Identify load test errors

+You can identify errors in your load test in the following ways:

+# [Azure portal](#tab/portal)

+- The test run status is failed.

+ You can view the test run status in list of test runs for your load test, or in **Test details** in the load test dashboard for your test run.

+ :::image type="content" source="media/how-to-find-download-logs/dashboard-test-failed.png" alt-text="Screenshot that shows the load test dashboard, highlighting status information for a failed test." lightbox="media/how-to-find-download-logs/dashboard-test-failed.png":::

+- The test run has a non-zero error percentage value.

+ If the test error percentage is below the default threshold, your test run shows as succeeded, even though there are errors. You can add [test fail criteria](./how-to-define-test-criteria.md) based on the error percentage.

+ You can view the error percentage in the **Statistics** in the load test dashboard for your test run.

+- The errors chart in the client-side metrics in the load test dashboard shows errors.

+ :::image type="content" source="media/how-to-find-download-logs/dashboard-errors.png" alt-text="Screenshot that shows the load test dashboard, highlighting the error information." lightbox="media/how-to-find-download-logs/dashboard-errors.png":::

+# [GitHub Actions](#tab/github)

+- The test run status is failed.

+ You can view the test run status in GitHub Actions for your repository, on the **Summary** page, or drill down into the workflow run details.

+ :::image type="content" source="media/how-to-find-download-logs/github-actions-summary-failed-test.png" alt-text="Screenshot that shows the summary page for an Azure Pipelines run, highlighting the failed load test stage." lightbox="media/how-to-find-download-logs/github-actions-summary-failed-test.png":::

+- The test run has a non-zero error percentage value.

+ If the test error percentage is below the default threshold, your test run shows as succeeded, even though there are errors. You can add [test fail criteria](./how-to-define-test-criteria.md) based on the error percentage.

+ You can view the error percentage in GitHub Actions, in the workflow run logging information.

+ :::image type="content" source="media/how-to-find-download-logs/github-actions-log-error-percentage.png" alt-text="Screenshot that shows the GitHub Actions workflow logs, highlighting the error statistics information for a load test run." lightbox="media/how-to-find-download-logs/github-actions-log-error-percentage.png":::

+- The test run log contains errors.

+ When there's a problem running the load test, the test run log might contain details about the root cause.

+ You can view the list of errors in GitHub Actions, on the workflow run **Summary** page, in the **Annotations** section. From this section, you can drill down into the workflow run details to view the error details.

+# [Azure Pipelines](#tab/pipelines)

+- The test run status is failed.

+ You can view the test run status in Azure Pipelines, on the pipeline run **Summary** page, or drill down into the pipeline run details.

+ :::image type="content" source="media/how-to-find-download-logs/azure-pipelines-summary-failed-test.png" alt-text="Screenshot that shows the summary page for an Azure Pipelines run, highlighting the failed load test stage.":::

+- The test run has a non-zero error percentage value.

+ If the test error percentage is below the default threshold, your test run shows as succeeded, even though there are errors. You can add [test fail criteria](./how-to-define-test-criteria.md) based on the error percentage.

+ You can view the error percentage in Azure Pipelines, in the pipeline run logging information.

+ :::image type="content" source="media/how-to-find-download-logs/azure-pipelines-log-error-percentage.png" alt-text="Screenshot that shows the Azure Pipelines run logs, highlighting the error statistics information for a load test run." lightbox="media/how-to-find-download-logs/azure-pipelines-log-error-percentage.png":::

+- The test run log contains errors.

+ When there's a problem running the load test, the test run log might contain details about the root cause.

+ You can view the list of errors in Azure Pipelines, on the pipeline run **Summary** page, in the **Errors** section. From this section, you can drill down into the pipeline run details to view the error details.

+## Download Apache JMeter worker logs

+When you run a load test, the Azure Load Testing test engines execute your Apache JMeter test script. During the load test, Apache JMeter stores detailed logging in the worker node logs. You can download these JMeter worker logs for each test run in the Azure portal. Azure Load Testing generates a worker log for each [test engine instance](./concept-load-testing-concepts.md#test-engine).

+For example, if there's a problem with your JMeter script, the load test status is **Failed**. In the worker logs you might find additional information about the cause of the problem.

+To download the worker logs for an Azure Load Testing test run, follow these steps:

+1. In the [Azure portal](https://portal.azure.com), go to your Azure Load Testing resource.

+1. Select **Tests** to view the list of tests, and then select your load test from the list.

+ :::image type="content" source="media/how-to-find-download-logs/test-list.png" alt-text="Screenshot that shows the list of load tests for an Azure Load Test resource.":::

+1. Select a test run from the list to view the test run dashboard.

+1. On the dashboard, select **Download**, and then select **Logs**.

+ The browser should now start downloading a zipped folder that contains the JMeter worker node log file for each [test engine instance](./concept-load-testing-concepts.md#test-engine).

+ :::image type="content" source="media/how-to-find-download-logs/logs.png" alt-text="Screenshot that shows how to download the test log files from the test run details page.":::

+1. You can use any zip tool to extract the folder and access the log files.

+ The *worker.log* file can help you diagnose the root cause of a failing load test. In the screenshot, you can see that the test failed because of a missing file.

+ :::image type="content" source="media/how-to-find-download-logs/jmeter-log.png" alt-text="Screenshot that shows the JMeter log file content.":::

+## Diagnose failing tests using test results

+To diagnose load tests that have failed requests, for example because the application endpoint is not available, the worker logs don't provide request details. You can use the test results to get detailed information about the individual application requests.

+1. Follow these steps to [download the test results for a load test run](./how-to-export-test-results.md).

+1. Open the test results `.csv` file in an editor of your choice.

+1. Use the information in the `responseCode` and `responseMessage` fields to determine the root cause of failing application requests.

+ In the following example, the test run failed because the application endpoint was not available (`java.net.UnknownHostException`):

+ ```output

+ timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,URL,Latency,IdleTime,Connect

+ 1676471293632,13,Home page,Non HTTP response code: java.net.UnknownHostException,Non HTTP response message: backend.contoso.com: Name does not resolve,172.18.44.4-Thread Group 1-1,text,false,,2470,0,1,1,https://backend.contoso.com/blabla,0,0,13

+ 1676471294339,0,Home page,Non HTTP response code: java.net.UnknownHostException,Non HTTP response message: backend.contoso.com,172.18.44.4-Thread Group 1-1,text,false,,2201,0,1,1,https://backend.contoso.com/blabla,0,0,0

+ 1676471294346,0,Home page,Non HTTP response code: java.net.UnknownHostException,Non HTTP response message: backend.contoso.com,172.18.44.4-Thread Group 1-1,text,false,,2201,0,1,1,https://backend.contoso.com/blabla,0,0,0

+ 1676471294350,0,Home page,Non HTTP response code: java.net.UnknownHostException,Non HTTP response message: backend.contoso.com,172.18.44.4-Thread Group 1-1,text,false,,2201,0,1,1,https://backend.contoso.com/blabla,0,0,0

+ 1676471294354,0,Home page,Non HTTP response code: java.net.UnknownHostException,Non HTTP response message: backend.contoso.com,172.18.44.4-Thread Group 1-1,text,false,,2201,0,1,1,https://backend.contoso.com/blabla,0,0,0

+ ```

+## Diagnose failing tests using server-side metrics

+For Azure-hosted applications, you can configure your load test to monitor resource metrics for your Azure application components. For example, a load test run might produce failed requests because an application component, such as a database, is throttling requests.

+Learn how you can [monitor server-side application metrics in Azure Load Testing](./how-to-monitor-server-side-metrics.md).

+For application endpoints that you host on Azure App Service, you can [use App Service Insights to get additional insights](./how-to-appservice-insights.md) about the application behavior.

+## Next steps

+- Learn how to [Export the load test result](./how-to-export-test-results.md).

+- Learn how to [Monitor server-side application metrics](./how-to-monitor-server-side-metrics.md).

+- Learn how to [Get detailed insights for Azure App Service based applications](./how-to-appservice-insights.md).

+- Learn how to [Compare multiple load test runs](./how-to-compare-multiple-test-runs.md).

+> [!NOTE]

+> We recommend that you build the executable JAR using Java 17.

+- Learn how to [Download JMeter logs to troubleshoot a load test](./how-to-troubleshoot-failing-test.md).

+Azure Load Testing uses Apache JMeter version 5.5 for running load tests. You can use Apache JMeter plugins from https://jmeter-plugins.org or [upload your own plugin code](./how-to-use-jmeter-plugins.md).

+| Test log files | 6 months | Learn how to [download the logs for troubleshooting tests](./how-to-troubleshoot-failing-test.md). |

+| `AzureMachineLearning` | Any | `VirtualNetwork` | 44224 | Inbound to compute instance/cluster. __Only needed if the instance/cluster is configured to use a public IP address__. |

+ code: mlflow/sklearn-diabetes/src

+## How often are offers badged? 

+Badges are updated periodically every 30 days. Please allow a minimum of 45 days before reaching out to file a ticket with the support team.  

+| Single to Flexible Server (Azure portal) | Database Migration Service (classic) and the Azure portal | [Tutorial: DMS (classic) with the Azure portal (offline)](../../dms/tutorial-mysql-azure-single-to-flex-offline-portal.md) | Recommended |

+| MySQL databases (< 1 TB) to Azure Database for MySQL | Database Migration Service (classic) and the Azure portal | [Migrate MySQL databases to Azure Database for MySQL using DMS (classic)](../../dms/tutorial-mysql-azure-mysql-offline-portal.md) | If network bandwidth between source and target is good (e.g: High-speed express route), use Azure DMS (database migration service) |

+| Single to Flexible Server (Azure portal) | Database Migration Service (classic) | [Tutorial: DMS (classic) with the Azure portal (online)](../../dms/tutorial-mysql-Azure-single-to-flex-online-portal.md) | Recommended |

+In the Azure portal, go to your Azure Database for MySQL server and open **Settings** > **Connection strings**. Copy the **ADO.NET** connection string. Replace the placeholder values for `your_database` and `your_password`. The connection string looks similar to the following.

+4. Rename your workflow `MySQL for GitHub Actions` and add the checkout and login actions. These actions check out your site code and authenticate with Azure using the `AZURE_CREDENTIALS` GitHub secret you created earlier.

+6. Complete your workflow by adding an action to sign out of Azure. Here's the completed workflow. The file appears in the `.github/workflows` folder of your repository.

+If you currently have an Azure Database for MySQL - Single Server service hosting production servers, we're glad to let you know that you can migrate your Azure Database for MySQL - Single Server servers to the Azure Database for MySQL - Flexible Server service free of cost using Azure Database Migration Service (classic) . Review the different ways to migrate using Azure Data Migration Service (DMS) in the section below.

+Learn how to migrate from Azure Database for MySQL - Single Server to Azure Database for MySQL - Flexible Server using the Azure Database Migration Service (Classic).

+| Offline | Database Migration Service (classic) and the Azure portal | [Tutorial: DMS (classic) with the Azure portal (offline)](../../dms/tutorial-mysql-azure-single-to-flex-offline-portal.md) |

+| Online | Database Migration Service (classic) and the Azure portal | [Tutorial: DMS (classic) with the Azure portal (online)](../../dms/tutorial-mysql-Azure-single-to-flex-online-portal.md) |

+| Single servers with Cross-Region Read Replicas enabled | Cross-Region Read Replicas for flexible server (for paired region) is in private preview, and you can start migrating your single server. Cross-Region Read Replicas for flexible server (for any cross-region) is on the road map for later this year, post, which you can migrate your single server. |

+| Single server deployed in regions where flexible server isn't supported (Learn more about regions [here](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?regions=all&products=mysql)). | Azure Database Migration Service (classic) supports cross-region migration. Deploy your target flexible server in a suitable region and migrate using DMS (classic). |

+A. When running the migration, you pay for the target flexible server and the source single server. The configuration and compute of the target flexible server determines the additional costs incurred. For more information, see, [Pricing](https://azure.microsoft.com/pricing/details/mysql/flexible-server/). Once you've decommissioned the source single server post successful migration, you only pay for your running flexible server. There are no costs incurred while running the migration through the Azure Database Migration Service (classic) migration tooling.

+A. You can use Database Migration Service (classic) to run [online](../../dms/tutorial-mysql-Azure-single-to-flex-online-portal.md) or [offline](../../dms/tutorial-mysql-azure-single-to-flex-offline-portal.md) migrations (recommended). In addition, you can use community tools such as m[ydumper/myloader together with Data-in replication](../migrate/how-to-migrate-single-flexible-minimum-downtime.md) to perform migrations.

+A. Azure Database Migration Service (classic) supports cross-region migration, so you can select a suitable region for your target flexible server and then proceed with DMS (classic) migration.

+A. Flexible Server support for cross-region read replicas is on our roadmap as our highest priority. Cross-Region Read Replicas for flexible server (for paired region) is in private preview, and you can start migrating your single server. Cross-Region Read Replicas for flexible server (for any cross-region) is on the road map for later this year, post, which you can migrate your single server.

+Visit the **[FAQ](../../dms/faq-mysql-single-to-flex.md)** for information about using the Azure Database Migration Service (classic) for Azure Database for MySQL - Single Server to Flexible Server migrations.

+- [Frequently Asked Questions about DMS (classic) migrations](../../dms/faq-mysql-single-to-flex.md)

+ Title: VPN troubleshoot overview

+description: Learn about Azure Network Watcher VPN troubleshoot capability.

+# VPN troubleshoot overview

+Virtual network gateways provide connectivity between on-premises resources and Azure Virtual Networks. Monitoring virtual network gateways and their connections are critical to ensure communication isn't broken. Azure Network Watcher provides the capability to troubleshoot virtual network gateways and their connections. The capability can be called through the Azure portal, Azure PowerShell, Azure CLI, or REST API. When called, Network Watcher diagnoses the health of the gateway, or connection, and returns the appropriate results. The request is a long running transaction. The results are returned once the diagnosis is complete.

+## Supported Gateway types

+The following table lists which gateways and connections are supported with Network Watcher troubleshooting:

+| Gateway or connection | Supported |

+|||

+|**Gateway types** | |

+|VPN | Supported |

+|ExpressRoute | Not Supported |

+|**VPN types** | |

+|Route Based | Supported|

+|Policy Based | Not Supported|

+|**Connection types**||

+|IPSec| Supported|

+|VNet2VNet| Supported|

+|ExpressRoute| Not Supported|

+|VPNClient| Not Supported|

+* **code** - This value is UnHealthy, if there's a single diagnosis failure.

+The following tables show the different fault types (ID under results from the preceding list) that are available and if the fault creates logs.

+| GatewayNotFound | Can't find gateway or gateway isn't provisioned |No|

+| PlatformInActive | There's an issue with the platform. | No|

+| ServiceNotRunning | The underlying service isn't running. | No|

+| ConnectionsNotConnected | Connections aren't connected. This fault is only a warning.| Yes|

+| GatewayNotFound | Can't find gateway or gateway isn't provisioned |No|

+| ConnectionNotConfiguredOnGateway | The underlying service doesn't have the connection configured. | Yes |

+| PeerReachability | The peer gateway isn't reachable. | Yes|

+| IkePolicyMismatch | The peer gateway has IKE policies that aren't supported by Azure. | Yes|

+> 2. For newer gateway versions, the IkeErrors.txt, Scrubbed-wfpdiag.txt and wfpdiag.txt.sum have been replaced by an IkeLogs.txt file that contains the whole IKE activity (not just errors).

+For instructions on downloading files from Azure storage accounts, see [Download a block blob](../storage/blobs/storage-quickstart-blobs-portal.md#download-a-block-blob). Another tool that can be used is Storage Explorer. For information about Azure Storage Explorer, see [Use Azure Storage Explorer to download blobs](../storage/blobs/quickstart-storage-explorer.md#download-blobs)

+The following example shows the contents of the Scrubbed-wfpdiag.txt file. In this example, the pre-shared key of a Connection wasn't correct as can be seen from the third line from the bottom. The following example is just a snippet of the entire log, as the log can be lengthy depending on the issue.

+* Only one VPN troubleshoot operation can be run at a time per subscription. To run another VPN troubleshoot operation, wait for the previous one to complete. Triggering a new operation while a previous one hasn't completed causes the subsequent operations to fail.

+* CLI Bug: If you're using Azure CLI to run the command, the VPN Gateway and the Storage account need to be in same resource group. Customers with the resources in different resource groups can use PowerShell or the Azure portal instead.

+To learn how to diagnose a problem with a virtual network gateway or gateway connection, see [Diagnose communication problems between networks](diagnose-communication-problem-between-networks.md).

+> CLI examples below are based on 2.45.0 version of Azure Database for PostgreSQL - Flexible Server CLI libraries

+## pg_buffercache

+`Pg_buffercache` can be used to study the contents of *shared_buffers* . Using [this extension](https://www.postgresql.org/docs/current/pgbuffercache.html) you can tell if a particular relation is cached or not(in *shared_buffers*) . This extension can help you in troubleshooting performance issues (caching related performance issues)

+This is part of contrib and it is very easy to install this extension.

+```sql

+CREATE EXTENSION pg_buffercache;

+```

+* Learn about [Cross-region replication with VNET](concepts-networking.md#replication-across-azure-regions-and-virtual-networks-with-private-networking).

+The Azure Database for PostgreSQL server is created with the 3 default roles defined. You can see these roles by running the command:

+```sql

+SELECT rolname FROM pg_roles;

+```

+* azure_pg_admin.

+* azuresu.

+* administrator role.

+While you're creating the Azure Database for PostgreSQL server, you provide credentials for an **administrator role**. This administrator role can be used to create more [PostgreSQL roles](https://www.postgresql.org/docs/current/user-manag.html).

+The **administrator role** should never be used by the application.

+In cloud-based PaaS environments access to a PostgreSQL superuser account is restricted to control plane operations only by cloud operators. Therefore, the **azure_pg_admin** account is added to the database as a pseudo-superuser account. Your administrator role is a member of the **azure_pg_admin** role.

+However, the server admin account is not part of the **azuresu** role, which has superuser privileges and is used to perform control pane operations. Since this service is a managed PaaS service, only Microsoft is part of the superuser role.

+> CLI examples below are based on 2.45.0 version of Azure Database for PostgreSQL - Flexible Server CLI libraries

+[//]: # (* Learn how to [create and manage read replicas in the Azure CLI and REST API]&#40;how-to-read-replicas-cli.md&#41;.)

+You'll need:

+In the Azure portal, go to your Azure Database for PostgreSQL server and open **Settings** > **Connection strings**. Copy the **ADO.NET** connection string. Replace the placeholder values for `your_database` and `your_password`. The connection string looks similar to this.

+You'll use the connection string as a GitHub secret.

+1. Rename your workflow `PostgreSQL for GitHub Actions` and add the checkout and login actions. These actions check out your site code and authenticate with Azure using the GitHub secret(s) you created earlier.

+3. Complete your workflow by adding an action to logout of Azure. Here's the completed workflow. The file appears in the `.github/workflows` folder of your repository.

+- Fetching relationship between external tables and Azure Data Lake Storage Gen2/Azure Blob assets (external locations).

+- Fetching static lineage between tables and views based on the view definition.

+This connector brings metadata from Databricks metastore. Comparing to scan via [Hive Metastore connector](register-scan-hive-metastore-source.md) in case you use it to scan Azure Databricks earlier:

+Go to the Hive table/view asset -> lineage tab, you can see the asset relationship when applicable. For relationship between table and external storage assets, you'll see Hive table asset and the storage asset are directly connected bi-directionally, as they mutually impact each other. If you use mount point in create table statement, you need to provide the mount point information in [scan settings](#scan) to extract such relationship.

+[Azure Virtual Machines](../virtual-machines/virtual-machines-reliability.md?toc=/azure/reliability/toc.json&bc=/azure/reliability/breadcrumb/toc.json)|

+- Azure VMs and associated disks (Azure Spot VMs are not currently supported)

+| - [Microsoft 365 Defender incident integration](../../sentinel/microsoft-365-defender-sentinel-integration.md) | GA | GA |

+- [AWS CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) - [CloudWatch logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html)

+- You must have an **S3 bucket** to which you will ship the logs from your AWS services - VPC, GuardDuty, CloudTrail, or CloudWatch.

+- [Export your CloudWatch log data to an S3 bucket](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Export.html).

+- [New CloudWatch data type for the AWS S3 connector (Preview)](#new-cloudwatch-data-type-for-the-aws-s3-connector)

+### New CloudWatch data type for the AWS S3 connector

+The Microsoft Sentinel AWS S3 connector now supports [CloudWatch logs](connect-aws.md) in addition to the supported CloudTrail, VPC Flow, and Guard Duty logs. Logs from AWS CloudWatch provide operational information from different AWS sources, which enables Microsoft Sentinel customers with AWS footprints to better understand and operate their AWS systems and applications.

+The CloudWatch data type has the ability to perform the same data transformation functions as the other data types within the AWS S3 connector. Learn how to [transform your data for CloudWatch](../azure-monitor/logs/tutorial-workspace-transformations-portal.md).

+> Sequence number on its own guarantees the queuing order and the extractor order of messages, but not the processing order, which requires [sessions](message-sessions.md).

+>

+> Say, there are 5 messages in the queue and 2 consumers. Consumer 1 picks up message 1. Consumer 2 picks up message 2. Consumer 2 finishes processing message 2 and picks up message 3 while Consumer 1 is not done with processing message 1 yet. Consumer 2 finishes processing message 3 but consumer 1 is still not done with processing message 1 yet. Finally, consumer 1 completes processing message 1. So, the messages are processed in this order: message 2, message 3, and message 1. If you need message 1, 2, and 3 to be processed in order, you need to use sessions.

+>

+> So, if messages just need to be retrieved in order, you don't need to use sessions. If messages need to be processed in order, use sessions. The same session ID should be set on messages that belong together, which could be message 1, 4, and 8 in one set, and 2, 3, and 6 in another set.

+>

+> For more information, see [Service Bus message sessions](message-sessions.md).

+To realize a FIFO guarantee in processing messages in Service Bus queues or subscriptions, use sessions. Service Bus isn't prescriptive about the nature of the relationship between messages, and also doesn't define a particular model for determining where a message sequence starts or ends.

+The session state held in a queue or in a subscription counts towards that entity's storage quota. When the application is finished with a session, it's therefore recommended for the application to clean up its retained state to avoid external management cost.

+The definition of delivery count per message in the context of sessions varies slightly from the definition in the absence of sessions. Here's a table summarizing when the delivery count is incremented.

+| Session is accepted, the messages within the session aren't completed (even if they're locked), and the session is closed | No |

+## Sequencing vs. sessions

+[Sequence number](message-sequencing.md) on its own guarantees the queuing order and the extractor order of messages, but not the processing order, which requires sessions.

+

+Say, there are three messages in the queue and two consumers. Consumer 1 picks up message 1. Consumer 2 picks up message 2. Consumer 2 finishes processing message 2 and picks up message 3 while Consumer 1 isn't done with processing message 1 yet. Consumer 2 finishes processing message 3 but consumer 1 is still not done with processing message 1 yet. Finally, consumer 1 completes processing message 1. So, the messages are processed in this order: message 2, message 3, and message 1. If you need message 1, 2, and 3 to be processed in order, you need to use sessions.

+So, if messages just need to be retrieved in order, you don't need to use sessions. If messages need to be processed in order, use sessions. The same session ID should be set on messages that belong together, which could be message 1, 4, and 8 in a set, and 2, 3, and 6 in another set.

+- .NET

+ - [Sending and receiving session messages](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Azure.Messaging.ServiceBus/samples/Sample03_SendReceiveSessions.md)

+ - [Using the session processor](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Azure.Messaging.ServiceBus/samples/Sample05_SessionProcessor.md)

+- Java

+ - [Send messages to a session](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/servicebus/azure-messaging-servicebus/src/samples/java/com/azure/messaging/servicebus/SendSessionMessageAsyncSample.java)

+ - [Receive messages from the first available session](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/servicebus/azure-messaging-servicebus/src/samples/java/com/azure/messaging/servicebus/ReceiveSingleSessionAsyncSample.java)

+ - [Receive messages from a specific session](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/servicebus/azure-messaging-servicebus/src/samples/java/com/azure/messaging/servicebus/ReceiveNamedSessionAsyncSample.java)

+ - [Process all session messages using a processor](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/servicebus/azure-messaging-servicebus/src/samples/java/com/azure/messaging/servicebus/ServiceBusSessionProcessorSample.java)

+- Python

+ - [Send and receive messages from a session-enabled queue](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/servicebus/azure-servicebus/samples/sync_samples/session_send_receive.py)

+ - [Receive messages from multiple available sessions in parallel with a thread pool](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/servicebus/azure-servicebus/samples/sync_samples/session_pool_receive.py)

+- JavaScript

+ - [Send to and receive messages from session enabled queues or subscriptions](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/servicebus/service-bus/samples/v7/javascript/session.js)

+ - [Continually read through all available sessions](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/servicebus/service-bus/samples/v7/javascript/advanced/sessionRoundRobin.js)

+ - [Use session state](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/servicebus/service-bus/samples/v7/javascript/advanced/sessionState.js)

+

+- [Abstract away infrastructure concerns with higher-level frameworks like NServiceBus](/azure/service-bus-messaging/build-message-driven-apps-nservicebus)

+## Enable FastZonalUpdate on Service Fabric managed clusters (preview)

+Service Fabric managed clusters support faster cluster and application upgrades by reducing the max upgrade domains per availability zone. The default configuration right now can have at most 15 UDs in multiple AZ nodetype. This huge number of UDs reduced the upgrade velocity. Using the new configuration, the max UDs are reduced, which results in faster updates, keeping the safety of the upgrades intact.

+The update should be done via ARM template by setting the zonalUpdateMode property to ΓÇ£fastΓÇ¥ and then modifying a node type attribute, such as adding a node and then removing the node to each nodetype (see required steps 2 and 3 below). The Service Fabric managed cluster resource apiVersion should be 2022-10-01-preview or later.

+1. Modify the ARM template with the new property mentioned above.

+```json

+ "resources": [

+ {

+ "type": "Microsoft.ServiceFabric/managedClusters",

+ "apiVersion": "2022-10-01-preview",

+ '''

+ "properties": {

+ '''

+ "zonalResiliency": true,

+ "zonalUpdateMode": ΓÇ£fastΓÇ¥,

+ ...

+ }

+ }]

+```

+2. Add a node to the node type from a cluster by following the procedure to [modify node type](how-to-managed-cluster-modify-node-type.md).

+3. Remove a node to the node type from a cluster by following the procedure to [modify node type](how-to-managed-cluster-modify-node-type.md).

+|ForwardClientCertificate|bool, default is FALSE|Dynamic|When set to false, reverse proxy will not request for the client certificate. When set to true, reverse proxy will request for the client certificate during the TLS handshake and forward the base64 encoded PEM format string to the service in a header named X-Client-Certificate. The service can fail the request with appropriate status code after inspecting the certificate data. If this is true and client does not present a certificate, reverse proxy will forward an empty header and let the service handle the case. Reverse proxy will act as a transparent layer. To learn more, see [Set up client certificate authentication](service-fabric-reverseproxy-configure-secure-communication.md#setting-up-client-certificate-authentication-through-the-reverse-proxy). |

+|ClusterId |String | Dynamic |The unique ID of the cluster. This is generated when the cluster is created. |

+|PartitionPrefix|string, default is "--"|Static|Controls the partition prefix string value in DNS queries for partitioned services. The value: <ul><li>Should be RFC-compliant as it will be part of a DNS query.</li><li>Should not contain a dot, '.', as dot interferes with DNS suffix behavior.</li><li>Should not be longer than 5 characters.</li><li>Cannot be an empty string.</li><li>If the PartitionPrefix setting is overridden, then PartitionSuffix must be overridden, and vice-versa.</li></ul>For more information, see [Service Fabric DNS Service.](service-fabric-dnsservice.md).|

+|PartitionSuffix|string, default is ""|Static|Controls the partition suffix string value in DNS queries for partitioned services. The value: <ul><li>Should be RFC-compliant as it will be part of a DNS query.</li><li>Should not contain a dot, '.', as dot interferes with DNS suffix behavior.</li><li>Should not be longer than 5 characters.</li><li>If the PartitionPrefix setting is overridden, then PartitionSuffix must be overridden, and vice-versa.</li></ul>For more information, see [Service Fabric DNS Service.](service-fabric-dnsservice.md). |

+|RecursiveQuerySerialMaxAttempts|Int, default is 2|Static|The number of serial queries that will be attempted, at most. If this number is higher than the number of forwarding DNS servers, querying will stop once all the servers have been attempted exactly once.|

+|IgnoreReplicaRestartWaitDurationWhenBelowMinReplicaSetSize|bool, default is FALSE|Dynamic|If IgnoreReplicaRestartWaitDurationWhenBelowMinReplicaSetSize is set to:<br>- false: Windows Fabric will wait for fixed time specified in ReplicaRestartWaitDuration for a replica to come back up.<br>- true: Windows Fabric will wait for fixed time specified in ReplicaRestartWaitDuration for a replica to come back up if partition is above or at Min Replica Set Size. If partition is below Min Replica Set Size new replica will be created right away.|

+|CompletedActionKeepDurationInSeconds | Int, default is 604800 |Static| This is approximately how long to keep actions that are in a terminal state. This also depends on StoredActionCleanupIntervalInSeconds; since the work to clean up is only done on that interval. 604800 is 7 days. |

+|FreeDiskSpaceNotificationSizeInKB|int64, default is 25\*1024 |Dynamic|The size of free disk space below which health warning may occur. Minimum values of this config and FreeDiskSpaceNotificationThresholdPercentage config are used to determine sending of the health warning. |

+|DisableDockerRequestRetry|bool, default is FALSE |Dynamic| By default SF communicates with DD (docker dameon) with a timeout of 'DockerRequestTimeout' for each http request sent to it. If DD does not responds within this time period; SF resends the request if top level operation still has remaining time. With Hyper-V container; DD sometimes take much more time to bring up the container or deactivate it. In such cases DD request times out from SF perspective and SF retries the operation. Sometimes this seems to adds more pressure on DD. This config allows to disable this retry and wait for DD to respond. |

+|AzureStorageMaxConnections | Int, default is 5000 |Dynamic|The maximum number of concurrent connections to Azure storage. |

+|PreventTransientOvercommit | Bool, default is false | Dynamic|Determines should PLB immediately count on resources that will be freed up by the initiated moves. By default; PLB can initiate move out and move in on the same node which can create transient overcommit. Setting this parameter to true will prevent those kinds of overcommits and on-demand defrag (also known as placementWithMove) will be disabled. |

+|AADCertEndpointFormat|string, default is ""|Static|Azure Active Directory Cert Endpoint Format, default Azure Commercial, specified for non-default environment such as Azure Government "https:\//login.microsoftonline.us/{0}/federationmetadata/2007-06/federationmetadata.xml" |

+|AADLoginEndpoint|string, default is ""|Static|Azure Active Directory Login Endpoint, default Azure Commercial, specified for non-default environment such as Azure Government "https:\//login.microsoftonline.us" |

+|AADTokenEndpointFormat|string, default is ""|Static|Azure Active Directory Token Endpoint, default Azure Commercial, specified for non-default environment such as Azure Government "https:\//login.microsoftonline.us/{0}" |

+|BlockAccessToWireServer|bool, default is FALSE| Static |Blocks access to ports of the WireServer endpoint from Docker containers deployed as Service Fabric applications. This parameter is supported for Service Fabric clusters deployed on Azure Virtual Machines, Windows and Linux, and defaults to 'false' (access is permitted).|

+|Providers |string, default is "DSTS" |Static|Comma separated list of token validation providers to enable (valid providers are: DSTS; Azure Active Directory). Currently only a single provider can be enabled at any time. |

+## Service Fabric community Q & A schedule

+Join the community call on the following days to hear about new feature releases and key updates and get answers to the questions from the Service Fabric team.

+| Schedule |

+| |

+| March 30, 2023 |

+| May 25, 2023 |

+| July 27, 2023|

+| September 28, 2023|

+| January 25, 2024 |

+| March 28, 2024 |

+- The secondary (target) region is Azure Southeast Asia:

+> Adding a custom domain to a [preview environment](preview-environments.md) is not supported. Unicode domains, including Punycode domains and the `xn--` prefix are also not supported.

+You can access resource logs either as a blob in a storage account, as event data, or through Log Analytics queries. For information about how to find those logs, see [Azure resource logs](../../azure-monitor/essentials/resource-logs.md).

+> [!IMPORTANT]

+> The NFS 3.0 protocol uses ports 111 and 2048. If you're connecting from an on-premises network, make sure that your client allows outgoing communication through those ports. If you have granted access to specific VNets, make sure that any network security groups associated with those VNets don't contain security rules that block incoming communication through those ports.

+> The NFS 3.0 protocol uses ports 111 and 2048. If you're connecting from an on-premises network, make sure that your client allows outgoing communication through those ports. If you have granted access to specific VNets, make sure that any network security groups associated with those VNets don't contain security rules that block incoming communication through those ports.

+When blob soft delete is enabled, deleting a blob marks that blob as soft-deleted. No snapshot is created. When the retention period expires, the soft-deleted blob is permanently deleted. In accounts that have a hierarchical namespace, the access control list of a blob is unaffected and will remain in tact if the blob is restored.

+You can restore soft-deleted blobs or directories (in a hierarchical namespace) by calling the [Undelete Blob](/rest/api/storageservices/undelete-blob) operation within the retention period. The **Undelete Blob** operation restores a blob and any soft-deleted snapshots associated with it. Any snapshots that were deleted during the retention period are restored. In accounts that have a hierarchical namespace, the access control list of a blob is restored along with the blob.

+- A complete list of Blob storage events and how each event is triggered.

+- An example of the data the Event Grid would send for each of these events.

+- The purpose of each key value pair that appears in the data.

+- As multiple subscriptions can be configured to route events to the same event handler, it is important not to assume events are from a particular source, but to check the topic of the message to ensure that it comes from the storage account you are expecting.

+- Similarly, check that the eventType is one you are prepared to process, and do not assume that all events you receive will be the types you expect.

+- There is no service level agreement around the time it takes for a message to arrive. It's not uncommon for messages to arrive anywhere from 30 minutes to two hours. As messages can arrive after some delay, use the etag fields to understand if your information about objects is still up-to-date. To learn how to use the etag field, see [Managing concurrency in Blob storage](./concurrency-manage.md?toc=/azure/storage/blobs/toc.json#managing-concurrency-in-blob-storage).

+- As messages can arrive out of order, use the sequencer fields to understand the order of events on any particular object. The sequencer field is a string value that represents the logical sequence of events for any particular blob name. You can use standard string comparison to understand the relative sequence of two events on the same blob name.

+- Storage events guarantees at-least-once delivery to subscribers, which ensures that all messages are outputted. However due to retries between backend nodes and services or availability of subscriptions, duplicate messages may occur. To learn more about message delivery and retry, see [Event Grid message delivery and retry](../../event-grid/delivery-and-retry.md).

+- Use the blobType field to understand what type of operations are allowed on the blob, and which client library types you should use to access the blob. Valid values are either `BlockBlob` or `PageBlob`.

+- Use the url field with the `CloudBlockBlob` and `CloudAppendBlob` constructors to access the blob.

+- Ignore fields you don't understand. This practice will help keep you resilient to new features that might be added in the future.

+- If you want to ensure that the **Microsoft.Storage.BlobCreated** event is triggered only when a Block Blob is completely committed, filter the event for the `CopyBlob`, `PutBlob`, `PutBlockList` or `FlushWithClose` REST API calls. These API calls trigger the **Microsoft.Storage.BlobCreated** event only after data is fully committed to a Block Blob. To learn how to create a filter, see [Filter events for Event Grid](../../event-grid/how-to-filter-events.md).

+A container name must be a valid DNS name, as it forms part of the unique URI (Uniform resource identifier) used to address the container or its blobs. Follow these rules when naming a container:

+To list blob versions or snapshots, specify the [BlobStates](/dotnet/api/azure.storage.blobs.models.blobstates) parameter with the **Version** or **Snapshot** field. Versions and snapshots are listed from oldest to newest.

+1. Toggle the **Show deleted versions** button to display soft-deleted versions. If blob soft delete is enabled for the storage account, then any soft-deleted versions that are still within the soft-delete retention interval will appear in the list.

+ :::image type="content" source="media/versioning-enable/portal-list-deleted-versions.png" alt-text="Screenshot showing how to list soft-deleted versions in Azure portal.":::

+description: Blob storage versioning automatically maintains previous versions of an object and identifies them with timestamps. You can restore a previous version of a blob to recover your data if it's erroneously modified or deleted.

+You can enable Blob storage versioning to automatically maintain previous versions of an object. When blob versioning is enabled, you can access earlier versions of a blob to recover your data if it's modified or deleted.

+- Blob versioning, to automatically maintain previous versions of a blob. When blob versioning is enabled, you can restore an earlier version of a blob to recover your data if it's erroneously modified or deleted. To learn how to enable blob versioning, see [Enable and manage blob versioning](versioning-enable.md).

+When you create a new blob, a single version exists, and that version is the current version. When you modify an existing blob, the current version becomes a previous version. A new version is created to capture the updated state, and that new version is the current version. When you delete a blob, the current version of the blob becomes a previous version, and there's no longer a current version. Any previous versions of the blob persist.

+Blob versions are immutable. You can't modify the content or metadata of an existing blob version.

+Blob versioning is available for standard general-purpose v2, premium block blob, and legacy Blob storage accounts. Storage accounts with a hierarchical namespace enabled for use with Azure Data Lake Storage Gen2 aren't currently supported.

+When blob versioning is enabled for a storage account, all write operations on block blobs trigger the creation of a new version, except for the [Put Block](/rest/api/storageservices/put-block) operation.

+For page blobs and append blobs, only a subset of write operations triggers the creation of a version. These operations include:

+The following operations don't trigger the creation of a new version. To capture changes from those operations, take a manual snapshot:

+All versions of a blob must be of the same blob type. If a blob has previous versions, you can't overwrite a blob of one type with another type unless you first delete the blob and all of its versions.

+When you call the [Delete Blob](/rest/api/storageservices/delete-blob) operation without specifying a version ID, the current version becomes a previous version, and there's no longer a current version. All existing previous versions of the blob are preserved.

+Disabling blob versioning doesn't delete existing blobs, versions, or snapshots. When you turn off blob versioning, any existing versions remain accessible in your storage account. No new versions are subsequently created.

+After versioning is disabled, modifying the current version creates a blob that isn't a version. All subsequent updates to the blob overwrite its data without saving the previous state. All existing versions persist as previous versions.

+The following diagram shows how modifying a blob after versioning is disabled creates a blob that isn't versioned. Any existing versions associated with the blob persist.

+Blob versioning and blob soft delete are part of the recommended data protection configuration for storage accounts. For more information about Microsoft's recommendations for data protection, see [Recommended data protection configuration](#recommended-data-protection-configuration) in this article, and [Data protection overview](data-protection-overview.md).

+If blob versioning and blob soft delete are both enabled for a storage account, then overwriting a blob automatically creates a new version. The new version isn't soft-deleted and isn't removed when the soft-delete retention period expires. No soft-deleted snapshots are created.

+If versioning and soft delete are both enabled for a storage account, then when you delete a blob, the current version of the blob becomes a previous version. No new version is created and no soft-deleted snapshots are created. The soft delete retention period isn't in effect for the deleted blob.

+Soft delete offers additional protection for deleting blob versions. When you delete a previous version of the blob, that version is soft-deleted. The soft-deleted version is preserved until the soft delete retention period elapses, at which point it's permanently deleted.

+You can use the [Undelete Blob](/rest/api/storageservices/undelete-blob) operation to restore soft-deleted versions during the soft delete retention period. The **Undelete Blob** operation always restores all soft-deleted versions of the blob. It isn't possible to restore only a single soft-deleted version.

+Restoring soft-deleted versions with the **Undelete Blob** operation doesn't promote any version to be the current version. To restore the current version, first restore all soft-deleted versions, and then use the [Copy Blob](/rest/api/storageservices/copy-blob) operation to copy a previous version to a new current version.

+Although it isn't recommended, you can take a snapshot of a blob that is also versioned. If you can't update your application to stop taking snapshots of blobs when you enable versioning, your application can support both snapshots and versions.

+Enabling blob versioning can result in additional data storage charges to your account. When designing your application, it's important to be aware of how these charges might accrue so that you can minimize costs.

+If you haven't changed a blob or version's tier, then you're billed for unique blocks of data across that blob, its versions, and any snapshots it may have. For more information, see [Billing when the blob tier has not been explicitly set](#billing-when-the-blob-tier-has-not-been-explicitly-set).

+If you've changed a blob or version's tier, then you're billed for the entire object, regardless of whether the blob and version are eventually in the same tier again. For more information, see [Billing when the blob tier has been explicitly set](#billing-when-the-blob-tier-has-been-explicitly-set).

+If you have not explicitly set the blob tier for any versions of a blob, then you're charged for unique blocks or pages across all versions, and any snapshots it may have. Data that is shared across blob versions is charged only once. When a blob is updated, then data in the new current version diverges from the data stored in previous versions, and the unique data is charged per block or page.

+When you replace a block within a block blob, that block is subsequently charged as a unique block. This is true even if the block has the same block ID and the same data as it has in the previous version. After the block is committed again, it diverges from its counterpart in the previous version, and you'll be charged for its data. The same holds true for a page in a page blob that's updated with identical data.

+Blob storage doesn't have a means to determine whether two blocks contain identical data. Each block that is uploaded and committed is treated as unique, even if it has the same data and the same block ID. Because charges accrue for unique blocks, it's important to keep in mind that updating a blob when versioning is enabled will result in additional unique blocks and additional charges.

+The following scenarios demonstrate how charges accrue for a block blob and its versions when the blob tier hasn't been explicitly set.

+In scenario 1, the blob has a previous version. The blob hasn't been updated since the version was created, so charges are incurred only for unique blocks 1, 2, and 3.

+In scenario 2, one block (block 3 in the diagram) in the blob has been updated. Even though the updated block contains the same data and the same ID, it isn't the same as block 3 in the previous version. As a result, the account is charged for four blocks.

+In scenario 3, the blob has been updated, but the version hasn't. Block 3 was replaced with block 4 in the current blob, but the previous version still reflects block 3. As a result, the account is charged for four blocks.

+In scenario 4, the current version has been completely updated and contains none of its original blocks. As a result, the account is charged for all eight unique blocksΓÇöfour in the current version, and four combined in the two previous versions. This scenario can occur if you're writing to a blob with the [Put Blob](/rest/api/storageservices/put-blob) operation, because it replaces the entire contents of the blob.

+If you have explicitly set the blob tier for a blob or version (or snapshot), then you're charged for the full content length of the object in the new tier, regardless of whether it shares blocks with an object in the original tier. You're also charged for the full content length of the oldest version in the original tier. Any other previous versions or snapshots that remain in the original tier are charged for unique blocks that they may share, as described in [Billing when the blob tier has not been explicitly set](#billing-when-the-blob-tier-has-not-been-explicitly-set).

+The following table describes the billing behavior for a blob or version when it's moved to a new tier.

+| When blob tier is set… | Then you're billed for... |

+¹If there are other previous versions or snapshots that haven't been moved from their original tier, those versions or snapshots are charged based on the number of unique blocks they contain, as described in [Billing when the blob tier has not been explicitly set](#billing-when-the-blob-tier-has-not-been-explicitly-set).

+Explicitly setting the tier for a blob, version, or snapshot can't be undone. If you move a blob to a new tier and then move it back to its original tier, you're charged for the full content length of the object even if it shares blocks with other objects in the original tier.

+ Title: Create and list blob versions in .NET

+description: Learn how to use the .NET client library to create a previous version of a blob.

+ms.devlang: csharp

+# Create and list blob versions in .NET

+Blob versioning automatically creates a previous version of a blob anytime it is modified or deleted. When blob versioning is enabled, then you can restore an earlier version of a blob to recover your data if it is erroneously modified or deleted.

+For optimal data protection, Microsoft recommends enabling both blob versioning and blob soft delete for your storage account. For more information, see [Blob versioning](versioning-overview.md) and [Soft delete for blobs](soft-delete-blob-overview.md).

+## Modify a blob to trigger a new version

+The following code example shows how to trigger the creation of a new version with the Azure Storage client library for .NET, version [12.5.1](https://www.nuget.org/packages/Azure.Storage.Blobs/12.5.1) or later. Before running this example, make sure you have enabled versioning for your storage account.

+The example creates a block blob, and then updates the blob's metadata. Updating the blob's metadata triggers the creation of a new version. The example retrieves the initial version and the current version, and shows that only the current version includes the metadata.

+## List blob versions

+To list blob versions, specify the [BlobStates](/dotnet/api/azure.storage.blobs.models.blobstates) parameter with the **Version** field. Versions are listed from oldest to newest.

+The following code example shows how to list blob versions.

+## See also

+- [Blob versioning](versioning-overview.md)

+- [Enable and manage blob versioning](versioning-enable.md)

+ Title: Configure autoscale settings for a Stream Analytics job by using the CI/CD tool

+description: This article shows how to configure autoscale settings for a Stream Analytics job by using the CI/CD tool.

+# Configure autoscale settings for a Stream Analytics job by using the CI/CD tool

+Streaming units (SUs) represent the computing resources that are allocated to run an Azure Stream Analytics job. The higher the number of SUs, the more CPU and memory resources are allocated to your job.

+The autoscale feature dynamically adjusts SUs based on your rule definitions. You can configure autoscale settings for your Stream Analytics job in the Azure portal or by using the Stream Analytics continuous integration and continuous delivery (CI/CD) tool on your local machine.

+This article explains how you can use the Stream Analytics CI/CD tool to configure autoscale settings for Stream Analytics jobs. If you want to learn more about autoscaling jobs in the Azure portal, see [Autoscale streaming units (preview)](stream-analytics-autoscale.md).

+The Stream Analytics CI/CD tool allows you to specify the maximum number of streaming units and configure a set of rules for autoscaling your jobs. Then it determines whether to add SUs (to handle increases in load) or reduce the number of SUs (when computing resources are sitting idle).

+Here's an example of an autoscale setting:

+- If the maximum number of SUs is set to 12, increase SUs when the average SU utilization of the job over the last 2 minutes goes above 75 percent.

+To complete the steps in this article, you need either:

+- A Stream Analytics project on the local machine. If don't have one, follow [this guide](quick-create-visual-studio-code.md) to create one.

+- A running Stream Analytics job in Azure.

+## Configure autoscale settings

+### Scenario 1: Configure settings for a local Stream Analytics project

+If you have a working Stream Analytics project on the local machine, follow these steps to configure autoscale settings:

+1. Open your Stream Analytics project in Visual Studio Code.

+2. On the **Terminal** panel, run the following command to install the Stream Analytics CI/CD tool:

+ Here's the list of supported commands for `azure-streamanalytics-cicd`:

+ |`build` |Generate a standard Azure Resource Manager template (ARM template) for a Stream Analytics project in Visual Studio Code.|

+ |`localrun` |Run locally for a Stream Analytics project in Visual Studio Code.|

+ |`test` |Test for a Stream Analytics project in Visual Studio Code.|

+ |`addtestcase` |Add test cases for a Stream Analytics project in Visual Studio Code.|

+ |`autoscale` |Generate an ARM template file for an autoscale setting.|

+ |`help` |Display more information on a specific command.|

+3. Build the project:

+ If you build the project successfully, two JSON files are created under the *Deploy* folder. One is the ARM template file, and the other is the parameter file.

+ 

+ > We highly recommend that you use the `--v2` option for the updated ARM template schema. The updated schema has fewer parameters yet retains the same functionality as the previous version.

+ >

+ > The old ARM template will be deprecated in the future. After that, only templates that were created via `build --v2` will receive updates or bug fixes.

+4. Configure the autoscale setting. Add parameter keys and values by using the `azure-streamanalytics-cicd autoscale` command.

+ |`capacity`| Maximum SUs (1, 3, 6, or multiples of 6 up to 396)|`12`|

+ |`metrics` | Metrics used for autoscale rules | `ProcessCPUUsagePercentage` `ResourceUtilization`|

+ |`targetJobName`| Project name| `ClickStream-Filter`|

+ |`outputPath`| Output path for ARM templates | `./Deploy`|

+ Here's an example:

+ If you configure the autoscale setting successfully, two JSON files are created under the *Deploy* folder. One is the ARM template file, and the other is the parameter file.

+ 

+ Here's the list of metrics that you can use to define autoscale rules:

+ |Metric | Description |

+ ||-|

+ |`ProcessCPUUsagePercentage` | CPU utilization percentage |

+ |`ResourceUtilization` | SU or memory utilization percentage |

+ |`OutputWatermarkDelaySeconds` | Watermark delay |

+ |`InputEventsSourcesBacklogged` | Backlogged input events |

+ |`DroppedOrAdjustedEvents` | Out-of-order events |

+ |`Errors` | Runtime errors |

+ |`InputEventBytes` | Input event bytes |

+ |`LateInputEvents` | Late input events |

+ |`InputEvents` | Input events |

+ |`EarlyInputEvents` | Early input events |

+ |`InputEventsSourcesPerSecond` | Input sources received |

+ |`OutputEvents` | Output events |

+ |`AMLCalloutRequests` | Function requests |

+ |`AMLCalloutFailedRequests` | Failed function requests |

+ |`AMLCalloutInputEvents` | Function events |

+ |`ConversionErrors` | Data conversion errors |

+ |`DeserializationError` | Input deserialization error |

+ The default value for all metric thresholds is `70`. If you want to set the metric threshold to another number, open the *\*.AutoscaleSettingTemplate.parameters.json* file and change the `Threshold` value.

+ 

+ To learn more about defining autoscale rules, see [Understand autoscale settings](/azure/azure-monitor/autoscale/autoscale-understanding-settings).

+5. Deploy to Azure.

+ 1. Connect to your Azure account:

+ # Set the Azure subscription

+ 1. Deploy your Stream Analytics project:

+ 1. Deploy your autoscale settings:

+After you deploy your project successfully, you can view the autoscale settings in the Azure portal.

+### Scenario 2: Configure settings for a running Stream Analytics job in Azure

+If you have a Stream Analytics job running in Azure, you can use the Stream Analytics CI/CD tool in PowerShell to configure autoscale settings.

+Run the following command. Replace `$jobResourceId` with the resource ID of your Stream Analytics job.

+If you configure the settings successfully, ARM template and parameter files are created in the current directory.

+Then you can deploy the autoscale settings to Azure by following the deployment steps in scenario 1.

+## Get help

+For more information about autoscale settings, run this command in PowerShell:

+If you have any problems with the Stream Analytics CI/CD tool, you can report them in [GitHub](https://github.com/microsoft/vscode-asa/issues).

+After completing this solution, you're able to:

+A toll station is a common phenomenon. You encounter them on many expressways, bridges, and tunnels across the world. Each toll station has multiple toll booths. At manual booths, you stop to pay the toll to an attendant. At automated booths, a sensor on top of each booth scans an RFID card that's affixed to the windshield of your vehicle as you pass the toll booth. It's easy to visualize the passage of vehicles through these toll stations as an event stream over which interesting operations can be performed.

+The entry data stream contains information about cars as they enter toll stations. The exit data events are live streamed into an event hub from a Web App included in the sample app.

+```

+```

+Here's a short description of the columns:

+The exit data stream contains information about cars leaving the toll station. The exit data events are live streamed into an event hub from a Web App included in the sample app.

+Here's a short description of the columns:

+Here's a short description of the columns:

+To complete this solution, you need a Microsoft Azure subscription. If you don't have an Azure account, you can [request a free trial version](https://azure.microsoft.com/pricing/free-trial/).

+6. Specify an **Interval** as a number of seconds. This value is used in the sample web app, for how frequently to send data into an event hub.

+ - One Azure event hub

+1. Starting from the resource group in the previous section, select the Stream Analytics streaming job starting with the name `tollapp` (name contains random characters for uniqueness).

+ - **EntryStream** input is an event hub connection that queues data representing each time a car enters a tollbooth on the highway. A web app that is part of the sample is creating the events, and that data is queued in this event hub. Note that this input is queried in the FROM clause of the streaming query.

+ - **ExitStream** input is an event hub connection that queues data representing each time a car exits a tollbooth on the highway. This streaming input is used in later variations of the query syntax.

+5. In the list of IDs, several docs are shown once the output is available.

+6. Select each ID to review the JSON document. Notice each `tollid`, `windowend time`, and the `count of cars` from that window.

+7. After an additional three minutes, another set of four documents is available, one document per `tollid`.

+For example, this document shows an example car with a certain license plate, the `entrytime` and `exit time`, and the DATEDIFF calculated `durationinminutes` field showing the toll booth duration as two minutes:

+5. **Start** the streaming job to demonstrate the additional scale. Azure Stream Analytics distributes work across more compute resources and achieves better throughput, partitioning the work across resources using the column designated in the PARTITION BY clause.

+This solution introduced you to the Azure Stream Analytics service. It demonstrated how to configure inputs and outputs for the Stream Analytics job. By using the Toll Data scenario, the solution explained common types of problems that arise in the space of data in motion and how they can be solved with simple SQL-like queries in Azure Stream Analytics. The solution described SQL extension constructs for working with temporal data. It showed how to join data streams, how to enrich the data stream with static reference data, and how to scale out a query to achieve higher throughput.

+Although this solution provides a good introduction, it isn't complete by any means. You can find more query patterns using the SAQL language at [Query examples for common Stream Analytics usage patterns](stream-analytics-stream-analytics-query-patterns.md).

+The partition key, or column name, used to partition input data may contain any character that is accepted for [blob names](/rest/api/storageservices/Naming-and-Referencing-Containers--Blobs--and-Metadata). It isn't possible to use nested fields as a partition key unless used in conjunction with aliases, but you can use certain characters to create a hierarchy of files. For example, you can use the following query to create a column that combines data from two other columns to make a unique partition key.

+The partition key must be NVARCHAR(MAX), BIGINT, FLOAT, or BIT (1.2 compatibility level or higher). DateTime, Array, and Records types aren't supported, but could be used as partition keys if they're converted to Strings. For more information, see [Azure Stream Analytics Data types](/stream-analytics-query/data-types-azure-stream-analytics).

+When you use the REST API, the output section of a JSON file used for that request may look like the following image:

+Once the job starts running, the `clients` container may look like the following image:

+Each folder may contain multiple blobs where each blob contains one or more records. In the above example, there's a single blob in a folder labeled "06000000" with the following contents:

+3. Partition keys are case insensitive, so partition keys like `John` and `john` are equivalent. Also, expressions can't be used as partition keys. For example, **{columnA + columnB}** doesn't work.

+4. When an input stream consists of records with a partition key cardinality under 8000, the records are appended to existing blobs, and only create new blobs when necessary. If the cardinality is over 8000, there's no guarantee existing blobs will be written to, and new blobs won't be created for an arbitrary number of records with the same partition key.

+5. If the blob output is [configured as immutable](../storage/blobs/immutable-storage-overview.md), Stream Analytics creates a new blob each time data is sent.

+If you don't wish to use custom DateTime patterns, you can add the {date} and/or {time} token to the Path Prefix to generate a dropdown with built-in DateTime formats.

+This quickstart shows you how to create a Stream Analytics job in the Azure portal. In this quickstart, you define a Stream Analytics job that reads real-time streaming data and filters messages with a temperature greater than 27. Your Stream Analytics job reads data from IoT Hub, transform the data, and write the output data to a container in blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.

+6. Select **Review + create**. Review your IoT Hub information and select **Create**. Your IoT Hub might take a few minutes to create. You can monitor the progress in the **Notifications** pane.

+7. Enter a **Device ID** and select **Save**.

+2. In the **Create storage account** pane, enter a storage account name, location, and resource group. Choose the same location and resource group as the IoT Hub you created. Then select **Review** at the bottom of the page.

+In this section, you configure an IoT Hub device input to the Stream Analytics job. Use the IoT Hub you created in the previous section of the quickstart.

+3. Select **Run**. The output should show the sensor data and messages that are being sent to your IoT Hub.

+1. Select **Run**.

+ > Use a collation with `_UTF8` suffix to ensure that UTF-8 text is properly converted to `VARCHAR` columns. `Latin1_General_100_BIN2_UTF8` provides the best performance in the queries that read data from Parquet files and Azure Cosmos DB containers. For more information on changing collations, refer to [Collation types supported for Synapse SQL](sql/reference-collation-types.md).

+1. Switch the database context from `master` to `DataExplorationDB` using the following command. You can also use the UI control **use database** to switch your current database:

+1. From `DataExplorationDB`, create utility objects such as credentials and data sources.

+1. Optionally, use the newly created `DataExplorationDB` database to create a login for a user in `DataExplorationDB` that will access external data:

+ Next create a database user in `DataExplorationDB` for the above login and grant the `ADMINISTER DATABASE BULK OPERATIONS` permission.

+Data exploration database is just a simple placeholder where you can store your utility objects. Synapse SQL pool enables you to do much more and create a Logical Data Warehouse - a relational layer built on top of Azure data sources. Learn more about [building a logical data warehouse in this tutorial](sql/tutorial-data-analyst.md).

+This article applies to dedicated SQL pools (formerly SQL DW), for more information on dedicated SQL pools in Azure Synapse workspaces, see [Collation types supported for Synapse SQL](../sql/reference-collation-types.md).

+To change the default collation, update to the **Collation** field in the provisioning experience.

+For example, if you wanted to change the default collation to case sensitive, change the collation from `SQL_Latin1_General_CP1_CI_AS` to `SQL_Latin1_General_CP1_CS_AS`.

+## Collation support

+The following table shows which collation types are supported by which service.

+| Collation Type | Serverless SQL Pool | Dedicated SQL Pool - Database & Column Level | Dedicated SQL Pool - External Table (Native Support) | Dedicated SQL Pool - External Table (Hadoop/Polybase) |

+|:--:|:-:|:--:|::|::|

+| Non-UTF-8 Collations | Yes | Yes | Yes | Yes |

+| UTF-8 | Yes | Yes | No | No |

+| Japanese_Bushu_Kakusu_140_* | Yes | Yes | No | No |

+| Japanese_XJIS_140_* | Yes | Yes | No | No |

+| SQL_EBCDIC1141_CP1_CS_AS | No | No | No | No |

+| SQL_EBCDIC277_2_CP1_CS_AS | No | No | No | No |

+When passed 'Collation' as the property parameter, the DatabasePropertyEx function returns the current collation for the database specified. For more information, see [DATABASEPROPERTYEX](/sql/t-sql/functions/databasepropertyex-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).

+## Next steps

+Additional information on best practices for dedicated SQL pool and serverless SQL pool can be found in the following articles:

+- [Best practices for dedicated SQL pool](../sql/best-practices-dedicated-sql-pool.md)

+- [Best practices for serverless SQL pool](../sql/best-practices-serverless-sql-pool.md)

+- If you're filtering results by string column, try to use a `BIN2_UTF8` collation. For more information on changing collations, refer to [Collation types supported for Synapse SQL](reference-collation-types.md).

+ - If the maximum integer column value is 500, use **smallint** because it's the smallest data type that can accommodate this value. For more information, see [integer data type ranges](/sql/t-sql/data-types/int-bigint-smallint-and-tinyint-transact-sql?view=azure-sqldw-latest&preserve-view=true).

+You can use the system stored procedure [sp_describe_first_results_set](/sql/relational-databases/system-stored-procedures/sp-describe-first-result-set-transact-sql?view=sql-server-ver15&preserve-view=true) to check the resulting data types of your query.

+| [File elimination](#file-elimination) (predicate pushdown) | No | Yes in serverless SQL pool. For the string pushdown, you need to use `Latin1_General_100_BIN2_UTF8` collation on the `VARCHAR` columns to enable pushdown. For more information on collations, refer to [Collation types supported for Synapse SQL](reference-collation-types.md).|

+| Storage authentication | Storage Access Key(SAK), Azure Active Directory passthrough, Managed identity, custom application Azure Active Directory identity | [Shared Access Signature(SAS)](develop-storage-files-storage-access-control.md?tabs=shared-access-signature), [Azure Active Directory passthrough](develop-storage-files-storage-access-control.md?tabs=user-identity), [Managed identity](develop-storage-files-storage-access-control.md?tabs=managed-identity), [Custom application Azure AD identity](develop-storage-files-storage-access-control.md?tabs=service-principal). |

+The native external tables in Synapse pools are able to ignore the files placed in the folders that are not relevant for the queries. If your files are stored in a folder hierarchy (for example - `/year=2020/month=03/day=16`) and the values for `year`, `month`, and `day` are exposed as the columns, the queries that contain filters like `year=2020` will read the files only from the subfolders placed within the `year=2020` folder. The files and folders placed in other folders (`year=2021` or `year=2022`) will be ignored in this query. This elimination is known as **partition elimination**.

+This technique is also known as filter predicate pushdown and it can improve the performance of your queries. Filter pushdown is available in the serverless SQL pools on Parquet and Delta formats. To leverage filter pushdown for the string types, use the VARCHAR type with the `Latin1_General_100_BIN2_UTF8` collation. For more information on collations, refer to [Collation types supported for Synapse SQL](reference-collation-types.md).

+External data sources are used to connect to storage accounts. For more information, see [CREATE EXTERNAL DATA SOURCE](/sql/t-sql/statements/create-external-data-source-transact-sql?view=azure-sqldw-latest&preserve-view=true).

+- "1900-01-01" if the column is a date column.

+> Querying Delta Lake format using the serverless SQL pool is **Generally available** functionality. However, querying Spark Delta tables is still in public preview and not production ready. There are known issues that might happen if you query Delta tables created using the Spark pools. See the known issues in [Serverless SQL pool self-help](resources-self-help-sql-on-demand.md#delta-lake).

+> `ALTER DATABASE CURRENT COLLATE Latin1_General_100_BIN2_UTF8;`

+> For more information on collations, see [Collation types supported for Synapse SQL](reference-collation-types.md).

+Make sure that you can access this file. If your file is protected with SAS key or custom Azure identity, you would need to set up [server level credential for sql login](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#server-scoped-credential).

+> `ALTER DATABASE CURRENT COLLATE Latin1_General_100_BIN2_UTF8;`

+> For more information on collations, see [Collation types supported for Synapse SQL](reference-collation-types.md).

+If you use the `Latin1_General_100_BIN2_UTF8` collation you will get an additional performance boost compared to the other collations. The `Latin1_General_100_BIN2_UTF8` collation is compatible with parquet string sorting rules. The SQL pool is able to eliminate some parts of the parquet files that will not contain data needed in the queries (file/column-segment pruning). If you use other collations, all data from the parquet files will be loaded into Synapse SQL and the filtering is happening within the SQL process. The `Latin1_General_100_BIN2_UTF8` collation has additional performance optimization that works only for parquet and Cosmos DB. The downside is that you lose fine-grained comparison rules like case insensitivity.

+If a data source is protected with SAS key or custom identity, you can configure [data source with database scoped credential](develop-storage-files-storage-access-control.md?tabs=shared-access-signature#database-scoped-credential).

+> `ALTER DATABASE CURRENT COLLATE Latin1_General_100_BIN2_UTF8;`

+> You can easily set collation on the colum types, for example:

+> For more information on collations, see [Collation types supported for Synapse SQL](../sql/reference-collation-types.md).

+In the following sections, you can see how to query various types of PARQUET files.

+The following sample shows the automatic schema inference capabilities for Parquet files. It returns the number of rows in September 2018 without specifying a schema.

+This article applies to dedicated SQL pools in Azure Synapse workspaces, for more information on dedicated SQL pools (formerly SQL DW), see [Collation types supported for dedicated SQL pool (formerly SQL DW)](../sql-data-warehouse/sql-data-warehouse-reference-collation-types.md).

+To change the default collation for dedicated SQL pool database, update to the **Collation** field in the provisioning experience. For example, if you wanted to change the default collation to case sensitive, you would change the collation from `SQL_Latin1_General_CP1_CI_AS` to `SQL_Latin1_General_CP1_CS_AS`.

+## Collation support

+The following table shows which collation types are supported by which service.

+| Collation Type | Serverless SQL Pool | Dedicated SQL Pool - Database & Column Level | Dedicated SQL Pool - External Table (Native Support) | Dedicated SQL Pool - External Table (Hadoop/Polybase) |

+|:--:|:-:|:--:|::|::|

+| Non-UTF-8 Collations | Yes | Yes | Yes | Yes |

+| UTF-8 | Yes | Yes | No | No |

+| Japanese_Bushu_Kakusu_140_* | Yes | Yes | No | No |

+| Japanese_XJIS_140_* | Yes | Yes | No | No |

+| SQL_EBCDIC1141_CP1_CS_AS | No | No | No | No |

+| SQL_EBCDIC277_2_CP1_CS_AS | No | No | No | No |

+When passed 'Collation' as the property parameter, the DatabasePropertyEx function returns the current collation for the database specified. For more information, see [DATABASEPROPERTYEX](/sql/t-sql/functions/databasepropertyex-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).

+- [Best practices for dedicated SQL pool](./best-practices-dedicated-sql-pool.md)

+ * Additionally, some Thai language collations are currently not supported by Azure Synapse Link for SQL. These unsupported collations include:

+ * Currently, the collation **Latin1_General_BIN2** is not supported as there is a known issue where the link cannot be stopped nor underlying tables could be removed from replication.