+#Customer intent: As a developer using Azure AD B2C custom policies, I want to understand how to use claim resolvers in my technical profiles, so that I can provide context information about authorization requests and populate claims with dynamic values.

+## SAML key-value parameters

+In a SAML authentication request, any parameter name that's included in the request, but isnΓÇÖt specific to the protocol (such as SAMLRequest) can be mapped to a claim in the user journey. For example, the request may include a custom parameter such as `username`. This applies to both SP-Initiated and IDP-Initiated SAML requests.

+| Claim | Description | Example |

+| -- | -- | --|

+| {SAML-KV:username} | A query string or POST body parameter. | username@domain.com |

+| {SAML-KV:loyalty_number} | A query string or POST body parameter. | 1234 |

+| {SAML-KV:any custom query string} | A query string or POST body parameter. | N/A |

+1. Create an `.env` file in the root folder of the project using `.env.sample` as a guide.

+ B2C_TENANT_NAME=<tenant name>

+ SIGNUPSIGNIN_USER_FLOW=B2C_1_signupsignin1

+ EDITPROFILE_USER_FLOW=B2C_1_profile_editing

+ RESETPASSWORD_USER_FLOW=B2C_1_reset_password

+ |`B2C_TENANT_NAME`| The first part of your Azure AD B2C [tenant name](tenant-management-read-tenant-name.md#get-your-tenant-name) (for example, `contoso`). |

+| [Application Insights event logs](analytics-with-application-insights.md) | Preview | Used to monitor user flows and custom policies in production. |

+1. In **Valid OAuth redirect URIs**, enter `https://your-tenant-name.b2clogin.com/your-tenant-name.onmicrosoft.com/oauth2/authresp`. If you use a [custom domain](custom-domain.md), enter `https://your-domain-name/your-tenant-id.onmicrosoft.com/oauth2/authresp`. Replace `your-tenant-id` with the id of your tenant, and `your-domain-name` with your custom domain.

+&error_description=AADB2C90091%3a+The+user+has+cancelled+entering+self-asserted+information.%0d%0aCorrelation+ID%3a+xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx%0d%0aTimestamp%3a+xxxx-xx-xx+xx%3a23%3a27Z%0d%0a

+ "error": "invalid_grant",

+ "error_description": "AADB2C90080: The provided grant has expired. Please re-authenticate and try again. Current time: xxxxxxxxxx, Grant issued time: xxxxxxxxxx, Grant expiration time: xxxxxxxxxx\r\nCorrelation ID: xxxxxxxx-xxxx-xxxX-xxxx-xxxxxxxxxxxx\r\nTimestamp: xxxx-xx-16 xx:10:52Z\r\n"

+ "error": "invalid_grant",

+ "error_description": "AADB2C90129: The provided grant has been revoked. Please reauthenticate and try again.\r\nCorrelation ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\r\nTimestamp: xxxx-xx-xx xx:xx:xxZ\r\n",

+- Learn more about [Azure AD B2C session](session-behavior.md).

+ public string Version { get; set; }

+ public int Status { get; set; }

+ public string Code { get; set; }

+ public string UserMessage { get; set; }

+ public string DeveloperMessage { get; set; }

+ public string RequestId { get; set; }

+ public string MoreInfo { get; set; }

+#Customer intent: As a developer integrating Azure AD B2C into an application, I want to understand how custom policy user journeys work so that I can design the steps that a users goes through for the relying party application to obtain the desired claims for a user.

+To specify the ordered list of orchestration steps, an **OrchestrationSteps** element is added as part of the policy. This element is required.

+| `Order` | Yes | The order of the orchestration steps. The value of the `Order` attribute starts at `1` through `N`. So, if you've 10 steps and you delete the second step, you need to renumber the steps three to 10 to become two to nine. |

+Learn more about [Improve the reliability of your application by using Azure Advisor - Ensure application gateway fault tolerance)](/azure/advisor/advisor-high-availability-recommendations#ensure-application-gateway-fault-tolerance).

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: http

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+^\* Only customers who have been approved for modified content filtering have full content filtering control and can turn content filters partially or fully off. Content filtering control does not apply to content filters for DALL-E (preview) or GPT-4 Turbo with Vision (preview). Apply for modified content filters using this form: [Azure OpenAI Limited Access Review: Modified Content Filtering (microsoft.com)](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUMlBQNkZMR0lFRldORTdVQzQ0TEI5Q1ExOSQlQCN0PWcu).

+GPT-4 Turbo with Vision accrues charges like other Azure OpenAI chat models. You pay a per-token rate for the prompts and completions, detailed on the [Pricing page](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/). The base charges and additional features are outlined here:

+- See the [completions and embeddings API reference](../reference.md)

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java

+ms.devlang: cpp

+# ms.devlang: cpp, csharp

+ms.devlang: csharp

+# ms.devlang: csharp, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, javascript, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, javascript

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, javascript, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, javascript, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, objective-c, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, python

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, golang, java, javascript, objective-c, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: cpp

+# ms.devlang: cpp, csharp

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java

+| Qatar | Qatar Central | `qatarcentral`⁸ |

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+## Switch compute instance runtime to automatic runtime

+Automatic runtime has following advantages over compute instance runtime:

+- Automatic manage lifecycle of runtime and underlying compute. You don't need to manually create and managed them anymore.

+- Easily customize packages by adding packages in the `requirements.txt` file in the flow folder, instead of creating a custom environment.

+We would recommend you to switch to automatic runtime if you're using compute instance runtime. If you have a compute instance runtime, you can switch it to an automatic runtime (preview) by using the following steps:

+- Prepare your `requirements.txt` file in the flow folder. Make sure that you don't pin the version of `promptflow` and `promptflow-tools` in `requirements.txt`, because we already include them in the runtime base image. Packages specified in `requirements.txt` will be installed when the runtime is started.

+- If you want to keep the automatic runtime (preview) as long running compute like compute instance, you can disable the idle shutdown toggle under automatic runtime (preview) `edit` option.

+ helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version v1.13.0 --set windows.enabled=true

+AKS supports Ubuntu 22.04 and Azure Linux 2.0 as the node operating system (OS) for clusters with Kubernetes 1.25 and higher. Ubuntu 18.04 can also be specified at node pool creation for Kubernetes versions 1.24 and below.

+* Azure CLI version 2.49.0 or later installed. To install or upgrade, see [Install Azure CLI][azure-cli-install].

+* `aks-preview` Azure CLI extension of version 0.5.163 or later installed

+ Title: Plug in CA certificates for Istio-based service mesh add-on on Azure Kubernetes Service (preview)

+description: Plug in CA certificates for Istio-based service mesh add-on on Azure Kubernetes Service (preview)

+# Plug in CA certificates for Istio-based service mesh add-on on Azure Kubernetes Service (preview)

+In the Istio-based service mesh addon for Azure Kubernetes Service (preview), by default the Istio certificate authority (CA) generates a self-signed root certificate and key and uses them to sign the workload certificates. To protect the root CA key, you should use a root CA, which runs on a secure machine offline. You can use the root CA to issue intermediate certificates to the Istio CAs that run in each cluster. An Istio CA can sign workload certificates using the administrator-specified certificate and key, and distribute an administrator-specified root certificate to the workloads as the root of trust. This article addresses how to bring your own certificates and keys for Istio CA in the Istio-based service mesh add-on for Azure Kubernetes Service.

+[  ](./media/istio/istio-byo-ca.png#lightbox)

+This article addresses how you can configure the Istio certificate authority with a root certificate, signing certificate and key provided as inputs using Azure Key Vault to the Istio-based service mesh add-on.

+## Before you begin

+### Verify Azure CLI and aks-preview extension versions

+The add-on requires:

+* Azure CLI version 2.49.0 or later installed. To install or upgrade, see [Install Azure CLI][install-azure-cli].

+* `aks-preview` Azure CLI extension of version 0.5.163 or later installed

+You can run `az --version` to verify above versions.

+To install the aks-preview extension, run the following command:

+```azurecli-interactive

+az extension add --name aks-preview

+```

+Run the following command to update to the latest version of the extension released:

+```azurecli-interactive

+az extension update --name aks-preview

+```

+### Register the _AzureServiceMeshPreview_ feature flag

+Register the `AzureServiceMeshPreview` feature flag by using the [az feature register][az-feature-register] command:

+```azurecli-interactive

+az feature register --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"

+```

+It takes a few minutes for the feature to register. Verify the registration status by using the [az feature show][az-feature-show] command:

+```azurecli-interactive

+az feature show --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"

+```

+When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register][az-provider-register] command:

+```azurecli-interactive

+az provider register --namespace Microsoft.ContainerService

+```

+### Set up Azure Key Vault

+1. You need an [Azure Key Vault resource][akv-quickstart] to supply the certificate and key inputs to the Istio add-on.

+1. You need to generate root certificate, intermediate certificates, intermediate key, and the certificate chain offline. Steps 1-3 from [here][istio-generate-certs] has an example of how to generate these files.

+1. Create secrets in Azure Key Vault using the certificates and key:

+ ```bash

+ az keyvault secret set --vault-name $AKV_NAME --name root-cert --file <path-to-folder/root-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-cert --file <path-to-folder/ca-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-key --file <path-to-folder/ca-key.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name cert-chain --file <path/cert-chain.pem>

+ ```

+1. Enable [Azure Key Vault provider for Secret Store CSI Driver for your cluster][akv-addon]:

+ ```bash

+ az aks enable-addons --addons azure-keyvault-secrets-provider --resource-group $RESOURCE_GROUP --name $CLUSTER

+ ```

+ > [!NOTE]

+ > When rotating certificates, to control how quickly the secrets are synced down to the cluster you can use the `--rotation-poll-interval` parameter of the Azure Key Vault Secrets Provider add-on. For example:

+ > `az aks addon update --resource-group $RESOURCE_GROUP --name $CLUSTER --addon azure-keyvault-secrets-provider --enable-secret-rotation --rotation-poll-interval 20s`

+1. Authorize the user-assigned managed identity of the add-on to have access to the Azure Key Vault resource:

+ ```bash

+ OBJECT_ID=$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER --query 'addonProfiles.azureKeyvaultSecretsProvider.identity.objectId' -o tsv)

+ az keyvault set-policy --name $AKV_NAME --object-id $OBJECT_ID --secret-permissions get list

+ ```

+## Set up Istio-based service mesh addon with plug-in CA certificates

+1. Enable the Istio service mesh addon for your existing AKS cluster while referencing the Azure Key Vault secrets that were created earlier:

+ ```bash

+ az aks mesh enable --resource-group $RESOURCE_GROUP --name $CLUSTER \

+ --root-cert-object-name root-cert \

+ --ca-cert-object-name ca-cert \

+ --ca-key-object-name ca-key \

+ --cert-chain-object-name cert-chain \

+ --key-vault-id /subscriptions/$SUBSCRIPTION/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.KeyVault/vaults/$AKV_NAME

+ ```

+ > [!NOTE]

+ > For existing clusters with Istio addon using self-signed root certificate generated by Istio CA, switching to plugin CA is not supported. You need to [disable the mesh][disable-mesh] on these clusters first and then enable it again using the above command to pass through the plugin CA inputs.

+1. Verify that the `cacerts` gets created on the cluster:

+ ```bash

+ kubectl get secret -n aks-istio-system

+ ```

+ Expected output:

+ ```bash

+ NAME TYPE DATA AGE

+ cacerts opaque 4 13h

+ sh.helm.release.v1.azure-service-mesh-istio-discovery.v380 helm.sh/release.v1 1 2m15s

+ sh.helm.release.v1.azure-service-mesh-istio-discovery.v381 helm.sh/release.v1 1 8s

+ ```

+1. Verify that the Istio control plane picked up the custom certificate authority:

+ ```bash

+ kubectl logs deploy/istiod-asm-1-17 -c discovery -n aks-istio-system | grep -v validationController | grep x509

+ ```

+ Expected output should be similar to:

+ ```bash

+ 2023-11-06T15:49:15.493732Z info x509 cert - Issuer: "CN=Intermediate CA - A1,O=Istio,L=cluster-A1", Subject: "", SN: e191d220af347c7e164ec418d75ed19e, NotBefore: "2023-11-06T15:47:15Z", NotAfter: "2033-11-03T15:49:15Z"

+ 2023-11-06T15:49:15.493764Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Intermediate CA - A1,O=Istio,L=cluster-A1", SN: 885034cba2894f61036f2956fd9d0ed337dc636, NotBefore: "2023-11-04T01:40:02Z", NotAfter: "2033-11-01T01:40:02Z"

+ 2023-11-06T15:49:15.493795Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Root A,O=Istio", SN: 18e2ee4089c5a7363ec306627d21d9bb212bed3e, NotBefore: "2023-11-04T01:38:27Z", NotAfter: "2033-11-01T01:38:27Z"

+ ```

+## Certificate authority rotation

+You may need to periodically rotate the certificate authorities for security or policy reasons. This section walks you through how to handle intermediate CA and root CA rotation scenarios.

+### Intermediate certificate authority rotation

+1. You can rotate the intermediate CA while keeping the root CA the same. Update the secrets in Azure Key Vault resource with the new certificate and key files:

+ ```bash

+ az keyvault secret set --vault-name $AKV_NAME --name root-cert --file <path-to-folder/root-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-cert --file <path-to-folder/ca-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-key --file <path-to-folder/ca-key.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name cert-chain --file <path/cert-chain.pem>

+ ```

+1. Wait for the time duration of `--rotation-poll-interval`. Check if the `cacerts` secret was refreshed on the cluster based on the new intermediate CA that was updated on the Azure Key Vault resource:

+ ```bash

+ kubectl logs deploy/istiod-asm-1-17 -c discovery -n aks-istio-system | grep -v validationController

+ ```

+ Expected output should be similar to:

+ ```bash

+ 2023-11-07T06:16:21.091844Z info Update Istiod cacerts

+ 2023-11-07T06:16:21.091901Z info Using istiod file format for signing ca files

+ 2023-11-07T06:16:21.354423Z info Istiod has detected the newly added intermediate CA and updated its key and certs accordingly

+ 2023-11-07T06:16:21.354910Z info x509 cert - Issuer: "CN=Intermediate CA - A2,O=Istio,L=cluster-A2", Subject: "", SN: b2753c6a23b54d8364e780bf664672ce, NotBefore: "2023-11-07T06:14:21Z", NotAfter: "2033-11-04T06:16:21Z"

+ 2023-11-07T06:16:21.354967Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Intermediate CA - A2,O=Istio,L=cluster-A2", SN: 17f36ace6496ac2df88e15878610a0725bcf8ae9, NotBefore: "2023-11-04T01:40:22Z", NotAfter: "2033-11-01T01:40:22Z"

+ 2023-11-07T06:16:21.355007Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Root A,O=Istio", SN: 18e2ee4089c5a7363ec306627d21d9bb212bed3e, NotBefore: "2023-11-04T01:38:27Z", NotAfter: "2033-11-01T01:38:27Z"

+ 2023-11-07T06:16:21.355012Z info Istiod certificates are reloaded

+ ```

+1. The workloads receive certificates from the Istio control plane that are valid for 24 hours by default. If you don't restart the pods, all the workloads obtain new leaf certificates based on the new intermediate CA in 24 hours. If you want to force all these workloads to obtain new leaf certificates right away from the new intermediate CA, then you need to restart the workloads.

+ ```bash

+ kubectl rollout restart deployment <deployment name> -n <deployment namespace>

+ ```

+### Root certificate authority rotation

+1. You need to update Azure Key Vault secrets with the root certificate file having the concatenation of the old and the new root certificates:

+ ```bash

+ az keyvault secret set --vault-name $AKV_NAME --name root-cert --file <path-to-folder/root-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-cert --file <path-to-folder/ca-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-key --file <path-to-folder/ca-key.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name cert-chain --file <path/cert-chain.pem>

+ ```

+ Contents of `root-cert.pem` follow this format:

+ ```

+ --BEGIN CERTIFICATE--

+ <contents of old root certificate>

+ --END CERTIFICATE--

+ --BEGIN CERTIFICATE--

+ <contents of new root certificate>

+ --END CERTIFICATE--

+ ```

+ The add-on includes a `CronJob` running every ten minutes on the cluster to check for updates to root certificate. If it detects an update, it restarts the Istio control plane (`istiod` deployment) to pick up the updates. You can check its logs to confirm that the root certificate update was detected and that the Istio control plane was restarted:

+ ```bash

+ kubectl logs -n aks-istio-system $(kubectl get pods -n aks-istio-system | grep 'istio-cert-validator-cronjob-' | sort -k8 | tail -n 1 | awk '{print $1}')

+ ```

+ Expected output:

+ ```bash

+ Root certificate update detected. Restarting deployment...

+ deployment.apps/istiod-asm-1-17 restarted

+ Deployment istiod-asm-1-17 restarted.

+ ```

+ After `istiod` was restarted, it should indicate that two certificates were added to the trust domain:

+ ```bash

+ kubectl logs deploy/istiod-asm-1-17 -c discovery -n aks-istio-system

+ ```

+ Expected output:

+ ```bash

+ 2023-11-07T06:42:00.287916Z info Using istiod file format for signing ca files

+ 2023-11-07T06:42:00.287928Z info Use plugged-in cert at etc/cacerts/ca-key.pem

+ 2023-11-07T06:42:00.288254Z info x509 cert - Issuer: "CN=Intermediate CA - A2,O=Istio,L=cluster-A2", Subject: "", SN: 286451ca8ff7bf9e6696f56bef829d42, NotBefore: "2023-11-07T06:40:00Z", NotAfter: "2033-11-04T06:42:00Z"

+ 2023-11-07T06:42:00.288279Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Intermediate CA - A2,O=Istio,L=cluster-A2", SN: 17f36ace6496ac2df88e15878610a0725bcf8ae9, NotBefore: "2023-11-04T01:40:22Z", NotAfter: "2033-11-01T01:40:22Z"

+ 2023-11-07T06:42:00.288298Z info x509 cert - Issuer: "CN=Root A,O=Istio", Subject: "CN=Root A,O=Istio", SN: 18e2ee4089c5a7363ec306627d21d9bb212bed3e, NotBefore: "2023-11-04T01:38:27Z", NotAfter: "2033-11-01T01:38:27Z"

+ 2023-11-07T06:42:00.288303Z info Istiod certificates are reloaded

+ 2023-11-07T06:42:00.288365Z info spiffe Added 2 certs to trust domain cluster.local in peer cert verifier

+ ```

+1. You need to either wait for 24 hours (the default time for leaf certificate validity) or force a restart of all the workloads. This way, all workloads recognize both the old and the new certificate authorities for [mTLS verification][istio-mtls-reference].

+ ```bash

+ kubectl rollout restart deployment <deployment name> -n <deployment namespace>

+ ```

+1. You can now update Azure Key Vault secrets with only the new CA (without the old CA):

+ ```bash

+ az keyvault secret set --vault-name $AKV_NAME --name root-cert --file <path-to-folder/root-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-cert --file <path-to-folder/ca-cert.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name ca-key --file <path-to-folder/ca-key.pem>

+ az keyvault secret set --vault-name $AKV_NAME --name cert-chain --file <path/cert-chain.pem>

+ ```

+ Check the logs of the `CronJob` to confirm detection of root certificate update and the restart of `istiod`:

+ ```bash

+ kubectl logs -n aks-istio-system $(kubectl get pods -n aks-istio-system | grep 'istio-cert-validator-cronjob-' | sort -k8 | tail -n 1 | awk '{print $1}')

+ ```

+ Expected output:

+ ```bash

+ Root certificate update detected. Restarting deployment...

+ deployment.apps/istiod-asm-1-17 restarted

+ Deployment istiod-asm-1-17 restarted.

+ ```

+ After `istiod` was updated, it should only confirm the usage of new root CA:

+ ```bash

+ kubectl logs deploy/istiod-asm-1-17 -c discovery -n aks-istio-system | grep -v validationController

+ ```

+ Expected output:

+ ```bash

+ 2023-11-07T08:01:17.780299Z info x509 cert - Issuer: "CN=Intermediate CA - B1,O=Istio,L=cluster-B1", Subject: "", SN: 1159747c72cc7ac7a54880cd49b8df0a, NotBefore: "2023-11-07T07:59:17Z", NotAfter: "2033-11-04T08:01:17Z"

+ 2023-11-07T08:01:17.780330Z info x509 cert - Issuer: "CN=Root B,O=Istio", Subject: "CN=Intermediate CA - B1,O=Istio,L=cluster-B1", SN: 2aba0c438652a1f9beae4249457023013948c7e2, NotBefore: "2023-11-04T01:42:12Z", NotAfter: "2033-11-01T01:42:12Z"

+ 2023-11-07T08:01:17.780345Z info x509 cert - Issuer: "CN=Root B,O=Istio", Subject: "CN=Root B,O=Istio", SN: 3f9da6ddc4cb03749c3f43243a4b701ce5eb4e96, NotBefore: "2023-11-04T01:41:54Z", NotAfter: "2033-11-01T01:41:54Z"

+ ```

+ From the example outputs shown in this article, you can observe that we moved from Root A (used when enabling the addon) to Root B.

+1. You can either again wait for 24 hours or force a restart of all the workloads. Forcing a restart makes the workloads obtain new leaf certificates from the new root CA immediately.

+ ```bash

+ kubectl rollout restart deployment <deployment name> -n <deployment namespace>

+ ```

+[akv-quickstart]: ../key-vault/general/quick-create-cli.md

+[akv-addon]: ./csi-secrets-store-driver.md

+[install-azure-cli]: /cli/azure/install-azure-cli

+[az-feature-register]: /cli/azure/feature#az-feature-register

+[az-feature-show]: /cli/azure/feature#az-feature-show

+[az-provider-register]: /cli/azure/provider#az-provider-register

+[az-aks-mesh-disable]: /cli/azure/aks/mesh#az-aks-mesh-disable

+[istio-generate-certs]: https://istio.io/latest/docs/tasks/security/cert-management/plugin-ca-cert/#plug-in-certificates-and-key-into-the-cluster

+[istio-mtls-reference]: https://istio.io/latest/docs/concepts/security/#mutual-tls-authentication

+export AKS_OIDC_ISSUER="$(az aks show -n myAKSCluster -g "${RESOURCE_GROUP}" --query "oidcIssuerProfile.issuerUrl" -o tsv)"

+https://eastus.oic.prod-aks.azure.com/00000000-0000-0000-0000-000000000000/11111111-1111-1111-1111-111111111111/

+By default, the Issuer is set to use the base URL `https://{region}.oic.prod-aks.azure.com/{tenant_id}/{uuid}`, where the value for `{region}` matches the location the AKS cluster is deployed in. The value `{uuid}` represents the OIDC key.

+export USER_ASSIGNED_CLIENT_ID="$(az identity show --resource-group "${RESOURCE_GROUP}" --name "${USER_ASSIGNED_IDENTITY_NAME}" --query 'clientId' -o tsv)"

+serviceaccount/workload-identity-sa created

+ name: your-pod

+ containers:

+ - image: <your image>

+ name: <containerName>

+ export KEYVAULT_RESOURCE_GROUP="myResourceGroup"

+ export USER_ASSIGNED_CLIENT_ID="$(az identity show --resource-group "${RESOURCE_GROUP}" --name "${USER_ASSIGNED_IDENTITY_NAME}" --query 'clientId' -o tsv)"

+2. Create a secret in Key Vault:

+ ```azurecli-interactive

+ export KEYVAULT_SECRET_NAME="my-secret"

+ az keyvault secret set --vault-name "${KEYVAULT_NAME}" \

+ --name "${KEYVAULT_SECRET_NAME}" \

+ --value "Hello\!"

+ ```

+3. Export Key Vault URL:

+ ```azurecli-interactive

+ export KEYVAULT_URL="$(az keyvault show -g ${KEYVAULT_RESOURCE_GROUP} -n ${KEYVAULT_NAME} --query properties.vaultUri -o tsv)"

+ ```

+4. Deploy a pod that references the service account and Key Vault URL above:

+ ```yml

+ cat <<EOF | kubectl apply -f -

+ apiVersion: v1

+ kind: Pod

+ metadata:

+ name: quick-start

+ namespace: ${SERVICE_ACCOUNT_NAMESPACE}

+ labels:

+ azure.workload.identity/use: "true"

+ spec:

+ serviceAccountName: ${SERVICE_ACCOUNT_NAME}

+ containers:

+ - image: ghcr.io/azure/azure-workload-identity/msal-go

+ name: oidc

+ env:

+ - name: KEYVAULT_URL

+ value: ${KEYVAULT_URL}

+ - name: SECRET_NAME

+ value: ${KEYVAULT_SECRET_NAME}

+ nodeSelector:

+ kubernetes.io/os: linux

+ EOF

+ ```

+To check whether all properties are injected properly by the webhook, use the [kubectl describe][kubectl-describe] command:

+```bash

+kubectl describe pod quick-start | grep "SECRET_NAME:"

+```

+If successful, the output should be similar to the following:

+```bash

+ SECRET_NAME: ${KEYVAULT_SECRET_NAME}

+```

+To verify that pod is able to get a token and access the resource, use the kubectl logs command:

+```bash

+kubectl logs quick-start

+```

+If successful, the output should be similar to the following:

+```bash

+I0114 10:35:09.795900 1 main.go:63] "successfully got secret" secret="Hello\\!"

+```

+az aks update --resource-group "${RESOURCE_GROUP}" --name myAKSCluster --disable-workload-identity

+ms.devlang: javascript

+# ms.devlang: javascript, devx-track-azurecli

+Your app uses DNS when making calls to dependent resources. Resources could be Azure services such as Key Vault, Storage or Azure SQL, but it could also be web APIs that your app depends on. When you want to make a call to for example *myservice.com*, you're using DNS to resolve the name to an IP. This article describes how App Service is handling name resolution and how it determines what DNS servers to use. The article also describes settings you can use to configure DNS resolution.

+If you aren't integrating your app with a virtual network and custom DNS servers aren't configured, your app uses [Azure DNS](../virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances.md#azure-provided-name-resolution). If you integrate your app with a virtual network, your app uses the DNS configuration of the virtual network. The default for virtual network is also to use Azure DNS. Through the virtual network, it's also possible to link to [Azure DNS private zones](../dns/private-dns-overview.md) and use that for private endpoint resolution or private domain name resolution.

+When your app needs to resolve a domain name using DNS, the app sends a name resolution request to all configured DNS servers. If the first server in the list returns a response within the timeout limit, you get the result returned immediately. If not, the app waits for the other servers to respond within the timeout period and evaluates the DNS server responses in the order you configured the servers. If none of the servers respond within the timeout and you configured retry, you repeat the process.

+## DNS app settings

+App Service has existing app settings to configure DNS servers and name resolution behavior. Site properties override the app settings if both exist. Site properties have the advantage of being auditable with Azure Policy and validated at the time of configuration. We recommend you to use site properties.

+Using the app setting `WEBSITE_DNS_ALT_SERVER`, you appends the specific DNS server to the list of DNS servers configured. The alternative DNS server is appended to both explicitly configured DNS servers and DNS servers inherited from the virtual network.

+App settings also exist for configuring name resolution behavior and are named `WEBSITE_DNS_MAX_CACHE_TIMEOUT`, `WEBSITE_DNS_TIMEOUT` and `WEBSITE_DNS_ATTEMPTS`.

+If you require fine-grained control over name resolution, App Service allows you to modify the default behavior. You can modify retry attempts, retry timeout and cache timeout. Changing behavior like disabling or lowering cache duration can influence performance.

+|dnsMaxCacheTimeout|30|0|0-60|DNS results are cached according to the individual records TTL, but no longer than the defined max cache timeout. Setting cache to zero means caching is disabled.|

+ms.devlang: golang

+# ms.devlang: wordpress

+ms.devlang: csharp

+# ms.devlang: csharp, azurecli

+ms.devlang: javascript

+# ms.devlang: javascript, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp,java,javascript,python

+ms.devlang: javascript

+# ms.devlang: javascript, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, php, python

+The Application Gateway v2 SKU introduces the use of Trusted Root Certificates to allow TLS connections with the backend servers. This provision removes the use of authentication certificates (individual Leaf certificates) that were required in the v1 SKU. The *root certificate* is a Base-64 encoded X.509(.CER) format root certificate from the backend certificate server. It identifies the root certificate authority (CA) that issued the server certificate and the server certificate is then used for the TLS/SSL communication.

+Application Gateway trusts your website's certificate by default if it's signed by a well-known CA (for example, GoDaddy or DigiCert). You don't need to explicitly upload the root certificate in that case. For more information, see [Overview of TLS termination and end to end TLS with Application Gateway](ssl-overview.md). However, if you have a dev/test environment and don't want to purchase a verified CA signed certificate, you can create your own custom Root CA and a leaf certificate signed by that Root CA.

+> Self-generated certificates are not trusted by default, and can be difficult to maintain. Also, they may use outdated hash and cipher suites that may not be strong. For better security, purchase a certificate signed by a well-known certificate authority.

+**You can use the following options to generate your private certificate for backend TLS connections.**

+1. Use the one-click private [**certificate generator tool**](https://appgwbackendcertgenerator.azurewebsites.net/). Using the domain name (Common Name) that you provide, this tool performs the same steps as documented in this article to generate Root and Server certificates. With the generated certificate files, you can immediately upload the Root certificate (.CER) file to the Backend Setting of your gateway and the corresponding certificate chain (.PFX) to the backend server. The password for the PFX file is also supplied in the downloaded ZIP file.

+2. Use OpenSSL commands to customize and generate certificates as per your needs. Continue to follow the instructions in this article if you wish to do this entirely on your own.

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, powershell, python

+> [!WARNING]

+> Arc resource bridge can only use a user account that does not have multifactor authentication enabled.

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, php, python

+ms.devlang: golang

+# ms.devlang: golang, rust

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, powershell, python, java

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python, java

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, powershell, python, java

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: javascript

+# ms.devlang: javascript, typescript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, powershell, python, java

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, powershell, python, java

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python, typescript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python, typescript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, fsharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, fsharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, powershell, python

+You can add a new function to an existing project based on one of the predefined Functions trigger templates. To add a new function trigger, select F1 to open the command palette, and then search for and run the command **Azure Functions: Create Function**. Follow the prompts to choose your trigger type and define the required attributes of the trigger. If your trigger requires an access key or connection string to connect to a service, get it ready before you create the function trigger.

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: javascript

+# ms.devlang: javascript, typescript

+ms.devlang: javascript

+# ms.devlang: javascript, typescript

+ms.devlang: javascript

+# ms.devlang: javascript, typescript

+- [Apache Kafka](#apache-kafka)

+- [Azure Cosmos DB](#azure-cosmos-db)

+- [Azure Event Hubs](#event-hubs)

+- [Azure Queue Storage](#storage-queues)

+- [Azure Service Bus (queue and topics)](#service-bus-queues-and-topics)

+| Event Hubs (Extension v5.x+) | extensions.eventHubs.maxEventBatchSize | 100^* |

+| Event Hubs (Extension v3.x+) | extensions.eventHubs.eventProcessorOptions.maxBatchSize | 10 |

+| Event Hubs (if defined) | extensions.eventHubs.targetUnprocessedEventThreshold | n/a |

+^* The default `maxEventBatchSize` changed in [v6.0.0](https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.EventHubs/6.0.0) of the `Microsoft.Azure.WebJobs.Extensions.EventHubs` package. In earlier versions, this value was 10.

+For some binding extensions, _target executions per instance_ is set using a function attribute:

+| Apache Kafka | `lagThreshold` | 1000 |

+| Azure Cosmos DB | `maxItemsPerInvocation` | 100 |

+| Apache Kafka | 3.9.0 |

+| Azure Cosmos DB | 4.1.0 |

+| Storage Queue | 5.1.0 |

+Target-based scaling introduces faster scaling, and uses defaults for _target executions per instance_. When using Service Bus, Storage queues, or Kafka, you can also enable [dynamic concurrency](functions-concurrency.md#dynamic-concurrency). In this configuration, the _target executions per instance_ value is determined automatically by the dynamic concurrency feature. It starts with limited concurrency and identifies the best setting over time.

+#### [C#](#tab/csharp)

+#### [Java](#tab/java)

+#### [Python](#tab/python)

+For Functions languages that use `function.json`, the `MaxItemsPerInvocation` parameter is defined in the specific binding, as in this Azure Cosmos DB trigger example

+```json

+{

+ "scriptFile": "main.py",

+ "bindings": [

+ {

+ "type": "cosmosDBTrigger",

+ "maxItemsPerInvocation": 100,

+ "connection": "MyCosmosDb",

+ "leaseContainerName": "leases",

+ "containerName": "collectionName",

+ "databaseName": "databaseName",

+ "leaseDatabaseName": "databaseName",

+ "createLeaseContainerIfNotExists": false,

+ "startFromBeginning": false,

+ "name": "input"

+ }

+ ]

+}

+```

+Examples for the Python v2 programming model aren't yet available.

+#### [JavaScript/PowerShell/TypeScript](#tab/node+powershell)

+Examples for the Node.js v4 programming model aren't yet available.

+### Apache Kafka

+The Apache Kafka extension uses a function-level attribute, `LagThreshold`. For Kafka, the number of _desired instances_ is calculated based on the total consumer lag divided by the `LagThreshold` setting. For a given lag, reducing the lag threshold increases the number of desired instances.

+The way you set this function-level attribute depends on your function language. This example sets the threshold to `100`.

+#### [C#](#tab/csharp)

+For a compiled C# function, set `LagThreshold` in your trigger definition, as shown in the following examples for an in-process C# function for a Kafka Event Hubs trigger:

+```C#

+[FunctionName("KafkaTrigger")]

+public static void Run(

+ [KafkaTrigger("BrokerList",

+ "topic",

+ Username = "$ConnectionString",

+ Password = "%EventHubConnectionString%",

+ Protocol = BrokerProtocol.SaslSsl,

+ AuthenticationMode = BrokerAuthenticationMode.Plain,

+ ConsumerGroup = "$Default",

+ LagThreshold = 100)] KafkaEventData<string> kevent, ILogger log)

+{

+ log.LogInformation($"C# Kafka trigger function processed a message: {kevent.Value}");

+}

+```

+#### [Java](#tab/java)

+```java

+public class KafkaTriggerMany {

+ @FunctionName("KafkaTriggerMany")

+ public void runMany(

+ @KafkaTrigger(

+ name = "kafkaTriggerMany",

+ topic = "topic",

+ brokerList="%BrokerList%",

+ consumerGroup="$Default",

+ username = "$ConnectionString",

+ password = "EventHubConnectionString",

+ authenticationMode = BrokerAuthenticationMode.PLAIN,

+ protocol = BrokerProtocol.SASLSSL,

+ LagThreshold = 100,

+ // sslCaLocation = "confluent_cloud_cacert.pem", // Enable this line for windows.

+ cardinality = Cardinality.MANY,

+ dataType = "string"

+ ) String[] kafkaEvents,

+ final ExecutionContext context) {

+ for (String kevent: kafkaEvents) {

+ context.getLogger().info(kevent);

+ }

+```

+#### [Python](#tab/python)

+For Functions languages that use `function.json`, the `LagThreshold` parameter is defined in the specific binding, as in this Kafka Event Hubs trigger example:

+```json

+{

+ "scriptFile": "main.py",

+ "bindings": [

+ {

+ "type": "kafkaTrigger",

+ "name": "kevent",

+ "topic": "topic",

+ "brokerList": "%BrokerList%",

+ "username": "$ConnectionString",

+ "password": "EventHubConnectionString",

+ "consumerGroup" : "functions",

+ "protocol": "saslSsl",

+ "authenticationMode": "plain",

+ "lagThreshold": "100"

+ "FUNCTIONS_RUNTIME_SCALE_MONITORING_ENABLED" : 1,

+ "TARGET_BASED_SCALING_ENABLED" : 1

+ }

+ ]

+}

+```

+The Python v2 programming model isn't currently supported by the Kafka extension.

+#### [JavaScript/PowerShell/TypeScript](#tab/node+powershell)

+For Functions languages that use `function.json`, the `LagThreshold` parameter is defined in the specific binding, as in this Kafka Event Hubs trigger example:

+```json

+{

+ "bindings": [

+ {

+ "type": "kafkaTrigger",

+ "name": "kafkaEvent",

+ "direction": "in",

+ "protocol" : "SASLSSL",

+ "password" : "EventHubConnectionString",

+ "dataType" : "string",

+ "topic" : "topic",

+ "authenticationMode" : "PLAIN",

+ "consumerGroup" : "$Default",

+ "username" : "$ConnectionString",

+ "brokerList" : "%BrokerList%",

+ "sslCaLocation": "confluent_cloud_cacert.pem",

+ "lagThreshold": "100"

+ }

+ ]

+}

+```

+The Node.js v4 programming model isn't currently supported by the Kafka extension.

+ | **Client Id** | Enter the client ID that you want to use for OAuth2 authentication, which you generated earlier. For more information on generating a client ID and a secret, see [Set up OAuth](https://learn.microsoft.com/azure/azure-monitor/alerts/itsmc-connections-servicenow#oauth-setup). |

+The structure of the payload for all alert types except log search V1 alert is [common schema](./alerts-common-schema.md).

+For Log Search Alerts (V1 only), the structure is:

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, vb

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+Get-AzApplicationInsights -SubscriptionId 'Your Subscription ID' | Format-Table -Property Name, IngestionMode, Id, @{label='Type';expression={

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, typescript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, typescript, python

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, typescript, python

+* [Azure Policy documentation](../governance/policy/index.yml)

+You can use inline parameters and a local parameters file in the same deployment operation. For example, you can specify some values in the local parameters file and add other values inline during deployment. If you provide values for a parameter in both the local parameters file and inline, the inline value takes precedence.

+ -ResourceId $sourceResourceGroup.ResourceId `

+ -Parameters @{

+ resources = $resources.ResourceId; # Wrap in an @() array if providing a single resource ID string.

+ targetResourceGroup = $destinationResourceGroup.ResourceId

+ }

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, python

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: php

+###### **January 16, 2023**

+The December Guest OS has released.

+| WA-GUEST-OS-7.36_202312-01 | January 16, 2024 | Post 7.39 |

+|~~WA-GUEST-OS-7.32_202309-01~~| September 25, 2023 | Post 7.36 |

+| WA-GUEST-OS-6.66_202312-01 | January 16, 2024 | Post 6.69 |

+|~~WA-GUEST-OS-6.62_202309-01~~| September 25, 2023 | Post 6.66 |

+| WA-GUEST-OS-5.90_202312-01 | January 16, 2024 | Post 5.93 |

+|~~WA-GUEST-OS-5.86_202309-01~~| September 25, 2023 | Post 5.90 |

+| WA-GUEST-OS-4.126_202312-01 | January 16, 2024 | Post 4.129 |

+|~~WA-GUEST-OS-4.122_202309-01~~| September 25, 2023 | Post 4.126 |

+| WA-GUEST-OS-3.134_202312-01 | January 16, 2024 | Post 3.137 |

+|~~WA-GUEST-OS-3.130_202309-01~~| September 25, 2023 | Post 3.134 |

+| WA-GUEST-OS-2.146_202312-01 | January 16, 2024 | Post 2.149 |

+|~~WA-GUEST-OS-2.142_202309-01~~| September 25, 2023 | Post 2.144 |

+Chrome version 115 for Android introduced a regression when making video calls - the result of this bug is a user making a call on Azure Communication Services with this version of Chrome has no outgoing video in Group and Azure Communication Services-Microsoft Teams calls.

+- This regression is a known issue introduced on [Chromium](https://bugs.chromium.org/p/chromium/issues/detail?id=1469318)

+- As a short term mitigation, instruct users to use Microsoft Edge or Firefox on Android, or avoid using Google Chrome 115/116 on Android

+- Enumerating speakers isn't available: If you're using Firefox, your app can't enumerate or select speakers through the Communication Services device manager. In this scenario, you must select devices via the operating system.

+- Virtual cameras aren't currently supported when making Firefox desktop audio\video calls.

+- No incoming/outgoing audio coming from bluetooth headset. When a user connects bluetooth headset in the middle of Azure Communication Services call, the audio still comes out from the speaker until the user locks and unlocks the phone. We have seen this issue on older iOS versions (15.6, 15.7), and it isn't reproducible on iOS 16.

+The iOS 16 release introduced a bug that can stop the Azure Communication Services audio\video call when using Safari mobile browser. Apple is aware of this issue and is looking for a fix on their side. The impact could be that an Azure Communication Services call might stop working during a call and the only resolution to get it working again is to have the end customer restart their phone.

+- This regression is a known issue introduced on [Chromium](https://bugs.chromium.org/p/chromium/issues/detail?id=1295815)

+The outgoing audio of the ACS call is muted, so other participants can't hear the user who is the PSTN call.

+It's worth noting that the user's incoming audio isn't muted, and this behavior is inherent to the browser.

+There's a related [Chromium](https://bugs.chromium.org/p/chromium/issues/detail?id=1402250) bug that causes this issue, the issue can be mitigated by reconnecting the PeerConnection. We added this workaround since SDK 1.9.1 (stable) and SDK 1.10.0 (beta)

+On Android Chrome, if a user joins Azure Communication Services call several times, the incoming audio can also disappear. The user isn't able to hear the audio from other participants until the page is refreshed. We fixed this issue in SDK 1.10.1-beta.1, and improved the audio resource usage.

+Many specific Android devices fail to start, accept calls, and meetings. The devices that run into this issue, can't recover and fails on every attempt. These are mostly Samsung model A devices, particularly models A326U, A125U and A215U.

+- This regression is a known issue introduced on [Chromium](https://bugs.chromium.org/p/webrtc/issues/detail?id=13223).

+On Android Chrome, if a user is on an Azure Communication Services call and puts the browser into background for one minute. The microphone loses access and the other participants in the call can't hear the audio from the user. Once the user brings the browser to foreground, microphone is available again. Related chromium bugs [here](https://bugs.chromium.org/p/chromium/issues/detail?id=1027446) and [here](https://bugs.chromium.org/p/webrtc/issues/detail?id=10940)

+The problem can occur if a mobile user leaves the Azure Communication Services group call without using the Call.hangUp() API. When a mobile user closes the browser or refreshes the webpage without hang up, other participants in the group call can still see this mobile user on the participant list for about 60 seconds.

+* Sometimes when incoming PSTN is received the tab with the call or meeting hangs. Related WebKit bugs [here](https://bugs.webkit.org/show_bug.cgi?id=233707) and [here](https://bugs.webkit.org/show_bug.cgi?id=233708#c0).

+- A user enables Siri, which captures access to the microphone.

+On iOS, for example, while on an Azure Communication Services call, if a PSTN call comes in, then a microphoneMutedUnexepectedly bad UFD is raised and audio stops flowing in the Azure Communication Services call and the call is marked as muted. Once the PSTN call is over, the user has to unmute the Azure Communication Services call for audio to start flowing again in the Azure Communication Services call. In the case of Android Chrome when a PSTN call comes in, audio stops flowing in the Azure Communication Services call and the Azure Communication Services call isn't marked as muted. In this case, there's no microphoneMutedUnexepectedly UFD event. Once the PSTN call is finished, Android Chrome regains audio automatically and audio starts flowing normally again in the Azure Communication Services call.

+In case camera is on and an interruption occurs, Azure Communication Services call may or may not lose the camera. If lost then camera is marked as off and user has to go turn it back on after the interruption released the camera.

+Occasionally, microphone or camera devices aren't released on time, and that can cause issues with the original call. For example, if the user tries to unmute while watching a YouTube video, or if a PSTN call is active simultaneously.

+Incoming video streams don't stop rendering if the user is on iOS 15.2+ and is using SDK version 1.4.1-beta.1+, the unmute/start video steps are still required to restart outgoing audio and video.

+For iOS 15.4+, audio and video should be able to auto recover on most of the cases. On some edge cases, to unmute, application must call an API to 'unmute' (can be as a result of user action) to recover the outgoing audio.

+### Screen sharing in macOS Ventura Safari (v16.3 and lower)

+Screen sharing doesn't work in macOS Ventura Safari(v16.3 and lower). Known issue from Safari and will be fixed in v16.4+

+If a user is in a call and decides to refresh the page, the Communication Services media service doesn't remove this user immediately from the call. It waits for the user to rejoin. The user is removed from the call after the media service times out.

+If the user was sending video before refreshing, the `videoStreams` collection keeps the previous stream information until the service times out and removes it. In this scenario, the application might decide to observe any new streams added to the collection, and render one with the highest `id`.

+### It isn't possible to render multiple previews from multiple devices on web

+This issue is a known limitation. For more information, see [Calling SDK overview](./voice-video-calling/calling-sdk-features.md).

+Applications can't enumerate or select speaker devices (like Bluetooth) on Safari iOS or iPadOS. This issue is a known limitation of these operating systems.

+If you're using Safari on macOS, your app can't enumerate or select speakers through the Communication Services device manager. In this scenario, you must select devices via the operating system. If you use Chrome on macOS, the app can enumerate or select devices through the Communication Services device manager.

+Bluetooth headsets aren't supported by Safari on iOS. Your Bluetooth device isn't listed in available microphone options, and other participants aren't able to hear you if you try using Bluetooth over Safari.

+This regression is a known operating system limitation. With Safari on macOS and iOS/iPadOS, it isn't possible to enumerate or select speaker devices through Communication Services device manager. This is because Safari doesn't support the enumeration or selection of speakers. In this scenario, use the operating system to update your device selection.

+This problem occurs in the following environments:

+This problem occurs in the following environments:

+If a user turns video on and off quickly while the call is in the *Connecting* state, this action might lead to a problem with the stream acquired for the call. It's best for developers to build their apps in a way that doesn't require video to be turned on and off while the call is in the *Connecting* state. Degraded video performance might occur in the following scenarios:

+In certain environments, you might notice that device permissions are reset after some period of time. On macOS and iOS, Safari doesn't keep permissions for a long time unless there's a stream acquired. The simplest way to work around this limitation is to call the `DeviceManager.askDevicePermission()` API, before calling the device manager's device enumeration APIs. These enumeration APIs include `DeviceManager.getCameras()`, `DeviceManager.getSpeakers()`, and `DeviceManager.getMicrophones()`. If the permissions are there, the user doesn't see anything. If the permissions aren't there, the user is prompted for the permissions again.

+This problem occurs in the following environments:

+If you use `getUserMedia` separately inside the application, the audio stream is lost. Audio stream is lost because a third-party library takes over device access from the Azure Communication Services library.

+This problem occurs in the following environments:

+The cause of this problem might be that acquiring your own stream from the same device has a side effect of running into race conditions. Acquiring streams from other devices might lead the user into insufficient USB/IO bandwidth, and the `sourceUnavailableError` rate skyrockets.

+### Excessive use of certain APIs like mute/unmute results in throttling on Azure Communication Services infrastructure

+Excessive use of mute/unmute is blocked in Azure Communication Services infrastructure. Throttling happens if the participant (or application on behalf of participant) attempts to mute/unmute continuously, every second, more than 15 times in a 30-second rolling window.

+The following limitations are known issues in the Communication Services Call Automation APIs:

+Up to 350 users can join a group call, room or Teams meeting. Only 100 users can join through JS web calling SDK or Teams web client, the remaining needs to join through Android/iOS/Windows calling SDK or Teams desktop/mobile client. Once the call size reaches 100+ participants in the call, only the top 4 most dominant speakers that have their video camera turned on are seen. When the number of people on the call is 100+, the viewable number of incoming renders goes from 3x3 (9 incoming videos) down to 2x2 (4 incoming videos). When the number of users goes below 100, the number of supported incoming videos goes back up to 3x3 (9 incoming videos).

+When utilizing Android API emulators on Android 5.0 (API level 21) and Android 5.1 (API level 22), some crashes are expected.

+> - [South Africa](../numbers/phone-number-management-for-south-africa.md)

+## South Africa telephony offers

+### Phone number leasing charges

+|Number type |Monthly fee |

+|--|--|

+|Toll-Free |USD 22.00/mo |

+### Usage charges

+|Number type |To make calls |To receive calls |

+|-||--|

+|Toll-free |N/A |USD 0.0844/min |

+## Job Router

+When sending or receiving a high volume of requests, you might receive a ```ThrottleLimitExceededException``` error. This error indicates you're hitting the service limitations, and your requests will be dropped until the token of bucket to handle requests is replenished after a certain time.

+Rate Limits for Job Router:

+|Operation|Scope|Timeframe (seconds)| Limit (number of requests) | Timeout in seconds|

+||--|-|-|-|

+|General Requests|Per Resource|10|1000|10|

+### Action to take

+If you need to send a volume of messages that exceeds the rate limits, email us at acs-ccap@microsoft.com.

+The Azure Function can be running either in Azure if you want to test it with some test events or if you want to test the entire flow locally (press `F5` in Visual Studio Code to run it locally). If you want to test the entire flow with an externally triggered webhook, you need to use [ngrok](https://ngrok.com/) to expose your locally running Azure Function

+to the public, allowing it to be triggered by internet sources (as an example from Azure Event WebHooks). Configure ngrok by running the command:

+```

+It is worth remembering that exposing development resources publicly might not be considered as secure. That is why you can also run the entire workflow locally without ngrok by invoking requests to:

+```

+http://localhost:7071/runtime/webhooks/EventGrid?functionName={functionname}

+ms.devlang: csharp

+# ms.devlang: csharp, java, golang

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: python

+# ms.devlang: python, scala

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: golang

+ Title: 'Tutorial: Connect using Spark'

+description: Connect to Azure Cosmos DB for NoSQL using the Spark 3 OLTP connector. Use the connector to query data in your API for NoSQL account.

+zone_pivot_groups: programming-languages-spark-all-minus-sql-r-csharp

+#CustomerIntent: As a data scientist, I want to connect to Azure Cosmos DB for NoSQL using Spark, so that I can perform analytics on my data in Azure Cosmos DB.

+# Tutorial: Connect to Azure Cosmos DB for NoSQL using Spark

+In this tutorial, you use the Azure Cosmos DB Spark connector to read or write data from an Azure Cosmos DB for NoSQL account. This tutorial uses Azure Databricks and a Jupyter notebook to illustrate how to integrate with the API for NoSQL from Spark. This tutorial focuses on Python and Scala even though you can use any language or interface supported by Spark.

+In this tutorial, you learn how to:

+> [!div class="checklist"]

+> - Connect to an API for NoSQL account using Spark and a Jupyter notebook

+> - Create database and container resources

+> - Ingest data to the container

+> - Query data in the container

+> - Perform common operations on items in the container

+## Prerequisites

+- An existing Azure Cosmos DB for NoSQL account.

+ - If you have an existing Azure subscription, [create a new account](how-to-create-account.md?tabs=azure-portal).

+ - No Azure subscription? You can [try Azure Cosmos DB free](../try-free.md) with no credit card required.

+- An existing Azure Databricks workspace.

+## Connect using Spark and Jupyter

+Use your existing Azure Databricks workspace to create a compute cluster ready to use Apache Spark 3.4.x to connect to your Azure Cosmos DB for NoSQL account.

+1. Open your Azure Databricks workspace.

+1. In the workspace interface, create a new **cluster**. Configure the cluster with these settings, at a minimum:

+ | | **Value** |

+ | | |

+ | **Runtime version** | 13.3 LTS (Scala 2.12, Spark 3.4.1) |

+1. Use the workspace interface to search for **Maven** packages from **Maven Central** with a **Group Id** of `com.azure.cosmos.spark`. Install the package specific for Spark 3.4 with an **Artifact Id** prefixed with `azure-cosmos-spark_3-4` to the cluster.

+1. Finally, create a new **notebook**.

+ > [!TIP]

+ > By default, the notebook will be attached to the recently created cluster.

+1. Within the notebook, set OLTP configuration settings for NoSQL account endpoint, database name, and container name.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Set configuration settings

+ config = {

+ "spark.cosmos.accountEndpoint": "<nosql-account-endpoint>",

+ "spark.cosmos.accountKey": "<nosql-account-key>",

+ "spark.cosmos.database": "cosmicworks",

+ "spark.cosmos.container": "products"

+ }

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ # Set configuration settings

+ val config = Map(

+ "spark.cosmos.accountEndpoint" -> "<nosql-account-endpoint>",

+ "spark.cosmos.accountKey" -> "<nosql-account-key>",

+ "spark.cosmos.database" -> "cosmicworks",

+ "spark.cosmos.container" -> "products"

+ )

+ ```

+ ::: zone-end

+## Create a database and container

+Use the Catalog API to manage account resources such as databases and containers. Then, you can use OLTP to manage data within the container resource\[s\].

+1. Configure the Catalog API to manage API for NoSQL resources using Spark.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Configure Catalog Api

+ spark.conf.set("spark.sql.catalog.cosmosCatalog", "com.azure.cosmos.spark.CosmosCatalog")

+ spark.conf.set("spark.sql.catalog.cosmosCatalog.spark.cosmos.accountEndpoint", config["spark.cosmos.accountEndpoint"])

+ spark.conf.set("spark.sql.catalog.cosmosCatalog.spark.cosmos.accountKey", config["spark.cosmos.accountKey"])

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Configure Catalog Api

+ spark.conf.set(s"spark.sql.catalog.cosmosCatalog", "com.azure.cosmos.spark.CosmosCatalog")

+ spark.conf.set(s"spark.sql.catalog.cosmosCatalog.spark.cosmos.accountEndpoint", config("spark.cosmos.accountEndpoint"))

+ spark.conf.set(s"spark.sql.catalog.cosmosCatalog.spark.cosmos.accountKey", config("spark.cosmos.accountKey"))

+ ```

+ ::: zone-end

+1. Create a new database named `cosmicworks` using `CREATE DATABASE IF NOT EXISTS`.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create a database using the Catalog API

+ spark.sql(f"CREATE DATABASE IF NOT EXISTS cosmosCatalog.cosmicworks;")

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create a database using the Catalog API

+ spark.sql(s"CREATE DATABASE IF NOT EXISTS cosmosCatalog.cosmicworks;")

+ ```

+ ::: zone-end

+1. Create a new container named `products` using `CREATE TABLE IF NOT EXISTS`. Ensure that you set the partition key path to `/category` and enable autoscale throughput with a maximum throughput of `1000` request units per second (RU/s).

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create a products container using the Catalog API

+ spark.sql(("CREATE TABLE IF NOT EXISTS cosmosCatalog.cosmicworks.products USING cosmos.oltp TBLPROPERTIES(partitionKeyPath = '/category', autoScaleMaxThroughput = '1000')"))

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create a products container using the Catalog API

+ spark.sql(("CREATE TABLE IF NOT EXISTS cosmosCatalog.cosmicworks.products USING cosmos.oltp TBLPROPERTIES(partitionKeyPath = '/category', autoScaleMaxThroughput = '1000')"))

+ ```

+ ::: zone-end

+1. Create another container named `employees` using a hierarchical partition key configuration with `/organization`, `/department`, and `/team` as the set of partition key paths in that specific order. Also, set the throughput to a manual amount of `400` RU/s

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create an employees container using the Catalog API

+ spark.sql(("CREATE TABLE IF NOT EXISTS cosmosCatalog.cosmicworks.employees USING cosmos.oltp TBLPROPERTIES(partitionKeyPath = '/organization,/department,/team', manualThroughput = '400')"))

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create an employees container using the Catalog API

+ spark.sql(("CREATE TABLE IF NOT EXISTS cosmosCatalog.cosmicworks.employees USING cosmos.oltp TBLPROPERTIES(partitionKeyPath = '/organization,/department,/team', manualThroughput = '400')"))

+ ```

+ ::: zone-end

+1. **Run** the notebook cell\[s\] to validate that your database and containers are created within your API for NoSQL account.

+## Ingest data

+Create a sample dataset and then use OLTP to ingest that data to the API for NoSQL container.

+1. Create a sample data set.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create sample data

+ products = (

+ ("68719518391", "gear-surf-surfboards", "Yamba Surfboard", 12, 850.00, False),

+ ("68719518371", "gear-surf-surfboards", "Kiama Classic Surfboard", 25, 790.00, True)

+ )

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create sample data

+ val products = Seq(

+ ("68719518391", "gear-surf-surfboards", "Yamba Surfboard", 12, 850.00, false),

+ ("68719518371", "gear-surf-surfboards", "Kiama Classic Surfboard", 25, 790.00, true)

+ )

+ ```

+ ::: zone-end

+1. Use `spark.createDataFrame` and the previously saved OLTP configuration to add sample data to the target container.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Ingest sample data

+ spark.createDataFrame(products) \

+ .toDF("id", "category", "name", "quantity", "price", "clearance") \

+ .write \

+ .format("cosmos.oltp") \

+ .options(**config) \

+ .mode("APPEND") \

+ .save()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Ingest sample data

+ spark.createDataFrame(products)

+ .toDF("id", "category", "name", "quantity", "price", "clearance")

+ .write

+ .format("cosmos.oltp")

+ .options(config)

+ .mode("APPEND")

+ .save()

+ ```

+ ::: zone-end

+## Query data

+Load OLTP data into a data frame to perform common queries on the data. You can use various syntaxes filter or query data.

+1. Use `spark.read` to load the OLTP data into a dataframe object. Use the same configuration used earlier in this tutorial. Also, set `spark.cosmos.read.inferSchema.enabled` to true to allow the Spark connector to infer the schema by sampling existing items.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Load data

+ df = spark.read.format("cosmos.oltp") \

+ .options(**config) \

+ .option("spark.cosmos.read.inferSchema.enabled", "true") \

+ .load()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Load data

+ val df = spark.read.format("cosmos.oltp")

+ .options(config)

+ .option("spark.cosmos.read.inferSchema.enabled", "true")

+ .load()

+ ```

+ ::: zone-end

+1. Render the schema of the data loaded in the dataframe using `printSchema`.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Render schema

+ df.printSchema()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Render schema

+ df.printSchema()

+ ```

+ ::: zone-end

+1. Render data rows where the `quantity` column is less than `20`. Use the `where` and `show` functions to perform this query.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Render filtered data

+ df.where("quantity < 20") \

+ .show()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Render filtered data

+ df.where("quantity < 20")

+ .show()

+ ```

+ ::: zone-end

+1. Render the first data row where the `clearance` column is true. Use the `filter` function to perform this query.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Render 1 row of flitered data

+ df.filter(df.clearance == True) \

+ .show(1)

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Render 1 row of flitered data

+ df.filter($"clearance" === true)

+ .show(1)

+ ```

+ ::: zone-end

+1. Render five rows of data with no filter or truncation. Use the `show` function to customize the appearance and number of rows that are rendered.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Render five rows of unfiltered and untruncated data

+ df.show(5, False)

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Render five rows of unfiltered and untruncated data

+ df.show(5, false)

+ ```

+ ::: zone-end

+1. Query your data using this raw NoSQL query string: `SELECT * FROM cosmosCatalog.cosmicworks.products WHERE price > 800`

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Render results of raw query

+ rawQuery = "SELECT * FROM cosmosCatalog.cosmicworks.products WHERE price > 800"

+ rawDf = spark.sql(rawQuery)

+ rawDf.show()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Render results of raw query

+ val rawQuery = s"SELECT * FROM cosmosCatalog.cosmicworks.products WHERE price > 800"

+ val rawDf = spark.sql(rawQuery)

+ rawDf.show()

+ ```

+ ::: zone-end

+## Perform common operations

+When working with API for NoSQL data in Spark, you can perform partial updates or work with data as raw JSON.

+1. To perform a partial update of an item, perform these steps:

+ 1. Copy the existing `config` configuration variable and modify the properties in the new copy. Specifically; configure the write strategy to `ItemPatch`, disable bulk support, set the columns and mapped operations, and finally set the default operation type to `Set`.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Copy and modify configuration

+ configPatch = dict(config)

+ configPatch["spark.cosmos.write.strategy"] = "ItemPatch"

+ configPatch["spark.cosmos.write.bulk.enabled"] = "false"

+ configPatch["spark.cosmos.write.patch.defaultOperationType"] = "Set"

+ configPatch["spark.cosmos.write.patch.columnConfigs"] = "[col(name).op(set)]"

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Copy and modify configuration

+ val configPatch = scala.collection.mutable.Map.empty ++ config

+ configPatch ++= Map(

+ "spark.cosmos.write.strategy" -> "ItemPatch",

+ "spark.cosmos.write.bulk.enabled" -> "false",

+ "spark.cosmos.write.patch.defaultOperationType" -> "Set",

+ "spark.cosmos.write.patch.columnConfigs" -> "[col(name).op(set)]"

+ )

+ ```

+ ::: zone-end

+ 1. Create variables for the item partition key and unique identifier that you intend to target as part of this patch operation.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Specify target item id and partition key

+ targetItemId = "68719518391"

+ targetItemPartitionKey = "gear-surf-surfboards"

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Specify target item id and partition key

+ val targetItemId = "68719518391"

+ val targetItemPartitionKey = "gear-surf-surfboards"

+ ```

+ ::: zone-end

+ 1. Create a set of patch objects to specify the target item and specify fields that should be modified.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create set of patch diffs

+ patchProducts = [{ "id": f"{targetItemId}", "category": f"{targetItemPartitionKey}", "name": "Yamba New Surfboard" }]

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create set of patch diffs

+ val patchProducts = Seq(

+ (targetItemId, targetItemPartitionKey, "Yamba New Surfboard")

+ )

+ ```

+ ::: zone-end

+ 1. Create a data frame using the set of patch objects and use `write` to perform the patch operation.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create data frame

+ spark.createDataFrame(patchProducts) \

+ .write \

+ .format("cosmos.oltp") \

+ .options(**configPatch) \

+ .mode("APPEND") \

+ .save()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create data frame

+ patchProducts

+ .toDF("id", "category", "name")

+ .write

+ .format("cosmos.oltp")

+ .options(configPatch)

+ .mode("APPEND")

+ .save()

+ ```

+ ::: zone-end

+ 1. Run a query to review the results of the patch operation. The item should now be named `Yamba New Surfboard` with no other changes.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create and run query

+ patchQuery = f"SELECT * FROM cosmosCatalog.cosmicworks.products WHERE id = '{targetItemId}' AND category = '{targetItemPartitionKey}'"

+ patchDf = spark.sql(patchQuery)

+ patchDf.show(1)

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create and run query

+ val patchQuery = s"SELECT * FROM cosmosCatalog.cosmicworks.products WHERE id = '$targetItemId' AND category = '$targetItemPartitionKey'"

+ val patchDf = spark.sql(patchQuery)

+ patchDf.show(1)

+ ```

+ ::: zone-end

+1. To work with raw JSON data, perform these steps:

+ 1. Copy the existing `config` configuration variable and modify the properties in the new copy. Specifically; change the target container to `employees` and configure the `contacts` column/field to use raw JSON data.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Copy and modify configuration

+ configRawJson = dict(config)

+ configRawJson["spark.cosmos.container"] = "employees"

+ configRawJson["spark.cosmos.write.patch.columnConfigs"] = "[col(contacts).path(/contacts).op(set).rawJson]"

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Copy and modify configuration

+ val configRawJson = scala.collection.mutable.Map.empty ++ config

+ configRawJson ++= Map(

+ "spark.cosmos.container" -> "employees",

+ "spark.cosmos.write.patch.columnConfigs" -> "[col(contacts).path(/contacts).op(set).rawJson]"

+ )

+ ```

+ ::: zone-end

+ 1. Create a set of employees to ingest into the container.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Create employee data

+ employees = (

+ ("63476388581", "CosmicWorks", "Marketing", "Outside Sales", "Alain Henry", '[ { "type": "phone", "value": "425-555-0117" }, { "email": "alain@adventure-works.com" } ]'),

+ )

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Create employee data

+ val employees = Seq(

+ ("63476388581", "CosmicWorks", "Marketing", "Outside Sales", "Alain Henry", """[ { "type": "phone", "value": "425-555-0117" }, { "email": "alain@adventure-works.com" } ]""")

+ )

+ ```

+ ::: zone-end

+ 1. Create a data frame and use `write` to ingest the employee data.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Ingest data

+ spark.createDataFrame(employees) \

+ .toDF("id", "organization", "department", "team", "name", "contacts") \

+ .write \

+ .format("cosmos.oltp") \

+ .options(**configRawJson) \

+ .mode("APPEND") \

+ .save()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Ingest data

+ spark.createDataFrame(employees)

+ .toDF("id", "organization", "department", "team", "name", "contacts")

+ .write

+ .format("cosmos.oltp")

+ .options(configRawJson)

+ .mode("APPEND")

+ .save()

+ ```

+ ::: zone-end

+ 1. Render the data from the data frame using `show`. Observe that the `contacts` column is raw JSON in the output.

+ ::: zone pivot="programming-language-python"

+ ```python

+ # Read and render data

+ rawJsonDf = spark.read.format("cosmos.oltp") \

+ .options(**configRawJson) \

+ .load()

+ rawJsonDf.show()

+ ```

+ ::: zone-end

+ ::: zone pivot="programming-language-scala"

+ ```scala

+ // Read and render data

+ val rawJsonDf = spark.read.format("cosmos.oltp")

+ .options(configRawJson)

+ .load()

+ rawJsonDf.show()

+ ```

+ ::: zone-end

+## Related content

+- [Apache Spark](https://spark.apache.org/)

+- [Azure Cosmos DB Catalog API](https://github.com/Azure/azure-sdk-for-jav)

+- [Configuration Parameter Reference](https://github.com/Azure/azure-sdk-for-jav)

+- [Sample "New York City Taxi data" notebook](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/Python/NYC-Taxi-Data)

+- [Migrate from Spark 2.4 to Spark 3.*](https://github.com/Azure/azure-sdk-for-jav)

+- Version Compatibility

+ - [Version compatibility for Spark 3.1](https://github.com/Azure/azure-sdk-for-jav#version-compatibility)

+ - [Version compatibility for Spark 3.2](https://github.com/Azure/azure-sdk-for-jav#version-compatibility)

+ - [Version compatibility for Spark 3.3](https://github.com/Azure/azure-sdk-for-jav#version-compatibility)

+ - [Version compatibility for Spark 3.4](https://github.com/Azure/azure-sdk-for-jav#version-compatibility)

+- Release notes

+ - [Release notes for Spark 3.1](https://github.com/Azure/azure-sdk-for-jav)

+ - [Release notes for Spark 3.2](https://github.com/Azure/azure-sdk-for-jav)

+ - [Release notes for Spark 3.3](https://github.com/Azure/azure-sdk-for-jav)

+ - [Release notes for Spark 3.4](https://github.com/Azure/azure-sdk-for-jav)

+- Download links

+ - [Download Azure Cosmos DB Spark connect for Spark 3.1](https://github.com/Azure/azure-sdk-for-jav#download)

+ - [Download Azure Cosmos DB Spark connect for Spark 3.2](https://github.com/Azure/azure-sdk-for-jav#download)

+ - [Download Azure Cosmos DB Spark connect for Spark 3.3](https://github.com/Azure/azure-sdk-for-jav#download)

+ - [Download Azure Cosmos DB Spark connect for Spark 3.4](https://github.com/Azure/azure-sdk-for-jav#download)

+## Next step

+> [!div class="nextstepaction"]

+> [Azure Cosmos DB Spark connector on Maven Central Repository](https://central.sonatype.com/search?q=g:com.azure.cosmos.spark&smo=true)

+ms.devlang: java

+ms.devlang: cpp

+# ms.devlang: cpp, csharp, java, javascript, php, python, ruby

+For more information about enterprise agreements, see [Troubleshoot enterprise cost views](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md).

+For more information, see [Troubleshoot Azure enterprise cost views](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md).

+For more information about troubleshooting disabled costs, see [Troubleshoot Azure enterprise cost views](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md).

+For more information, see [Troubleshoot Azure enterprise cost views](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md).

+The following sections apply to customers who have a Microsoft Online Services Program billing account. Learn how to [check your billing account type](#check-the-type-of-your-account). If your billing account type is Microsoft Online Services Program, payment methods are associated with individual Azure subscriptions. If you get an error after you add the credit card, see [Credit card declined at Azure sign-up](../troubleshoot-billing/troubleshoot-declined-card.md).

+- If you need to create an Azure support request for your EA enrollment, see [How to create an Azure support request for an Enterprise Agreement issue](../troubleshoot-billing/how-to-create-azure-support-request-ea.md).

+ Title: Troubleshoot Azure payment issues

+description: Resolving an issue when updating payment information account in the Azure portal.

+tags: billing

+# Troubleshoot Azure payment issues

+You may experience an issue or error when you try to update the payment information account in the Microsoft Azure portal.

+To resolve your issue, select the subject below which most closely resembles your error.

+## My credit card was declined when I tried to sign up for Azure

+To troubleshoot issues regarding a declined card, see [Troubleshoot a declined card at Azure sign-up](troubleshoot-declined-card.md).

+## Unable to see subscriptions under my account to update the payment method

+You might be using an email ID that differs from the one that is used for the subscriptions.

+To troubleshoot this issue, see [No subscriptions found sign-in error for Azure portal](../troubleshoot-subscription/no-subscriptions-found.md).

+## Unable to use a virtual or prepaid credit as a payment method.

+Virtual or prepaid credit cards aren't accepted as payment for Azure subscriptions.

+For more information, see [Troubleshoot a declined card at Azure sign-up](troubleshoot-declined-card.md).

+## Unable to remove a credit card from a saved billing payment method

+By design, you can't remove a credit card from the active subscription.

+If an existing card has to be deleted, one of the following actions is required:

+- A new card must be added to the subscription so that the old payment instrument can be successfully deleted.

+- You can cancel the subscription to delete the subscription permanently and then remove the card.

+## Unable to delete an old payment method after adding a new payment method

+The new payment instrument might not be associated with the subscription. To help associate the payment instrument with the subscription, see [Add, update, or remove a credit card for Azure](../manage/change-credit-card.md).

+## Unable to delete a payment method because of `Cannot delete payment method` error

+The error occurs because of an outstanding balance. Clear any outstanding balances before you delete the payment method.

+## Unable to make payment for a subscription

+If you receive the error message: `Payment is past due. There is a problem with your payment method` or `We're sorry, the information cannot be saved. Close the browser and try again.`, then there's a pending payment on the card because the card was denied by your financial institution.

+Verify that the credit card has a sufficient balance to make a payment. If it doesn't, use another card to make the payment, or reach out to your financial institution to resolve the issue.

+Check with your bank for the following issues:

+- International transactions aren't enabled.

+- The card has a credit limit, and the balance must be settled.

+- A recurring payment is enabled on the card.

+## Unable to change payment method because of browser issues (browser doesn't respond, doesn't load, and so on)

+Sign out of all active Azure sessions, and then follow the steps in the [Browse InPrivate in Microsoft Edge article](https://support.microsoft.com/help/4026200/microsoft-edge-browse-inprivate) to start an InPrivate session within Microsoft Edge or Internet Explorer.

+In the private session, follow the steps at [How to change a credit card](../manage/change-credit-card.md) to update or change the credit card information.

+You can also try to do the following actions:

+- Refresh your browser

+- Use another browser

+- Delete cached cookies

+## My subscription is still disabled after updating the payment method.

+The issue occurs because of an outstanding balance. Clear any outstanding balances before you delete the payment method.

+## Unable to change payment method because of an XML error response page

+You receive the message if you're using [the Azure portal](https://portal.azure.com/) to add a new credit card.

+To add card details, sign-in to the Azure Account portal by using the account administrator's email address.

+## Why does my invoice appear as unpaid when I've paid it?

+- The invoice number on the remittance wasn't specified.

+- You made one payment for multiple invoices.

+Best practices:

+- Submit one wire transfer payment per invoice.

+- Specify the invoice number on the remittance.

+- Send proof of payment, identification, and remittance details.

+## Other help resources

+Other troubleshooting articles for Azure Billing and Subscriptions

+- [Declined card](troubleshoot-declined-card.md)

+- [Subscription sign-in issues](../troubleshoot-subscription/troubleshoot-sign-in-issue.md)

+- [No subscriptions found](../troubleshoot-subscription/no-subscriptions-found.md)

+- [Enterprise cost view disabled](enterprise-mgmt-grp-troubleshoot-cost-view.md)

+## Contact us for help

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+- [Azure Billing documentation](../index.yml)

+ Title: Troubleshoot Azure EA portal access

+description: This article describes some common issues that can occur with an Azure Enterprise Agreement (EA) in the Azure EA portal.

+# Troubleshoot Azure EA portal access

+This article describes some common issues that can occur with an Azure Enterprise Agreement (EA). The Azure EA portal is used to manage enterprise agreement users and costs. You might come across these issues when you're configuring or updating Azure EA portal access.

+> [!NOTE]

+> We recommend that both direct and indirect EA Azure customers use Cost Management + Billing in the Azure portal to manage their enrollment and billing instead of using the EA portal. For more information about enrollment management in the Azure portal, see [Get started with EA billing in the Azure portal](../manage/ea-direct-portal-get-started.md).

+>

+> As of February 20, 2023 indirect EA customers wonΓÇÖt be able to manage their billing account in the EA portal. Instead, they must use the Azure portal.

+>

+> This change doesnΓÇÖt affect Azure Government EA enrollments. They continue using the EA portal to manage their enrollment.

+## Issues adding a user to an enrollment

+There are different types of authentication levels for enterprise enrollments. When authentication levels are applied incorrectly, you might have issues when you try to sign in to the Azure EA portal.

+You use the Azure EA portal to grant access to users with different authentication levels. An enterprise administrator can update the authentication level to meet security requirements of their organization.

+### Authentication level types

+- Microsoft Account Only - For organizations that want to use, create, and manage users through Microsoft accounts.

+- Work or School Account - For organizations that have set up Microsoft Entra ID with federation to the cloud and all accounts are on a single tenant.

+- Work or School Account Cross Tenant - For organizations that have set up Microsoft Entra ID with federation to the cloud and will have accounts in multiple tenants.

+- Mixed Account - Allows you to add users with Microsoft Account and/or with a Work or School Account.

+The first work or school account added to the enrollment determines the _default_ domain. To add a work or school account with another tenant, you must change the authentication level under the enrollment to cross-tenant authentication.

+To update the Authentication Level:

+1. Sign in to the Azure [EA portal](https://ea.azure.com/) as an Enterprise Administrator.

+2. Select **Manage** on the left navigation panel.

+3. Select the **Enrollment** tab.

+4. Under **Enrollment Details**, select **Auth Level**.

+5. Select the pencil symbol.

+6. Select **Save**.

+

+Microsoft accounts must have an associated ID created at [https://signup.live.com](https://signup.live.com/).

+Work or school accounts are available to organizations that have set up Microsoft Entra ID with federation and where all accounts are on a single tenant. Users can be added with work or school federated user authentication if the company's internal Microsoft Entra ID is federated.

+If your organization doesn't use Microsoft Entra ID federation, you can't use your work or school email address. Instead, register or create a new email address and register it as a Microsoft account.

+## Unable to access the Azure EA portal

+If you get an error message when you try to sign in to the Azure EA portal, use the following the troubleshooting steps:

+- Ensure that you're using the correct Azure EA portal URL, which is https://ea.azure.com.

+- Determine if your access to the Azure EA portal was added as a work or school account or as a Microsoft account.

+ - If you're using your work account, enter your work email and work password. Your work password is provided by your organization. You can check with your IT department about how to reset the password if you have issues with it.

+ - If you're using a Microsoft account, enter your Microsoft account email address and password. If you've forgotten your Microsoft account password, you can reset it at [https://account.live.com/password/reset](https://account.live.com/password/reset).

+- Use an in-private or incognito browser session to sign in so that no cookies or cached information from previous or existing sessions are kept. Clear your browser's cache and use an in-private or incognito window to open https://ea.azure.com.

+- If you get an _Invalid User_ error when using a Microsoft account, it might be because you have multiple Microsoft accounts. The one that you're trying to sign in with isn't the primary email address.

+Or, if you get an _Invalid User_ error, it might be because the wrong account type was used when the user was added to the enrollment. For example, a work or school account instead of a Microsoft account. In this example, you have another EA admin add the correct account or you need to contact [support](https://support.microsoft.com/supportforbusiness/productselection?sapId=cf791efa-485b-95a3-6fad-3daf9cd4027c).

+ - If you need to check the primary alias, go to [https://account.live.com](https://account.live.com). Then, select **Your Info** and then select **Manage how to sign in to Microsoft**. Follow the prompts to verify an alternate email address and obtain a code to access sensitive information. Enter the security code. Select **Set it up later** if you don't want to set up two-factor authentication.

+ - You see the **Manage how to sign in to Microsoft** page where you can view your account aliases. Check that the primary alias is the one that you're using to sign in to the Azure EA portal. If it isn't, you can make it your primary alias. Or, you can use the primary alias for Azure EA portal instead.

+## Next steps

+- Azure EA portal administrators should read [Azure EA portal administration](../manage/ea-portal-administration.md) to learn about common administrative tasks.

+- Read the [Cost Management + Billing FAQ](../cost-management-billing-faq.yml) for questions and answers about common issues for Azure EA Activation.

+ Title: Troubleshoot Azure enterprise cost views

+description: Learn how to resolve any issues you might have with organizational cost views within the Azure portal.

+# Troubleshoot enterprise cost views

+Within enterprise enrollments, there are several settings that could cause users within the enrollment to not see costs. These settings are managed by the enrollment administrator. Or, if the enrollment isn't bought directly through Microsoft, the settings are managed by the partner. This article helps you understand what the settings are and how they impact the enrollment. These settings are independent of the Azure roles.

+> [!NOTE]

+> We recommend that both direct and indirect EA Azure customers use Cost Management + Billing in the Azure portal to manage their enrollment and billing instead of using the EA portal. For more information about enrollment management in the Azure portal, see [Get started with EA billing in the Azure portal](../manage/ea-direct-portal-get-started.md).

+>

+> As of February 20, 2023 indirect EA customers wonΓÇÖt be able to manage their billing account in the EA portal. Instead, they must use the Azure portal.

+>

+> This change doesnΓÇÖt affect Azure Government EA enrollments. They continue using the EA portal to manage their enrollment.

+## Enable access to costs

+Are you seeing a message Unauthorized, or *"Cost views are disabled in your enrollment."* when looking for cost information?

+

+It might be for one of the following reasons:

+1. YouΓÇÖve bought Azure through an enterprise partner, and the partner didn't release pricing yet. Contact your partner to update the pricing setting within the [Enterprise portal](https://ea.azure.com).

+2. If youΓÇÖre an EA Direct customer, there are a couple of possibilities:

+ * You're an Account Owner and your Enrollment Administrator disabled the **AO view charges** setting.

+ * You're a Department Administrator and your Enrollment Administrator disabled the **DA view charges** setting.

+ * Contact your Enrollment Administrator to get access. The Enrollment Admin can now update the settings in [Azure portal](https://portal.azure.com/). Navigate to **Policies** menu to change settings.

+ * The Enrollment Admin can update the settings in the [Enterprise portal](https://ea.azure.com/manage/enrollment).

+ 

+

+

+## Asset is unavailable

+If you get an error message stating **This asset is unavailable** when trying to access a subscription or management group, then you don't have the correct role to view this item.

+

+Ask your Azure subscription or management group administrator for access. For more information, see [Assign Azure roles using the Azure portal](../../role-based-access-control/role-assignments-portal.md).

+## Next steps

+- If you have questions or need help, [create a support request](https://go.microsoft.com/fwlink/?linkid=2083458).

+ Title: How to create an Azure support request for an Enterprise Agreement issue

+description: Enterprise Agreement customers who need assistance can use the Azure portal to find self-service solutions and to create and manage support requests.

+# Create an Azure support request for an Enterprise Agreement issue

+Azure enables you to create and manage support requests, also known as support tickets, for Enterprise Agreements. You can create and manage requests in the [Azure portal](https://portal.azure.com), which is covered in this article. You can also create and manage requests programmatically, using the [Azure support ticket REST API](/rest/api/support), or by using [Azure CLI](/cli/azure/azure-cli-support-request).

+> [!NOTE]

+> The Azure portal URL is specific to the Azure cloud where your organization is deployed.

+>

+>- Azure portal for commercial use is: [https://portal.azure.com](https://portal.azure.com)

+>- Azure portal for Germany is: `https://portal.microsoftazure.de`

+>- Azure portal for the United States government is: [https://portal.azure.us](https://portal.azure.us)

+Azure provides unlimited support for subscription management, which includes billing, quota adjustments, and account transfers. You need a support plan for technical support. For more information, see [Compare support plans](https://azure.microsoft.com/support/plans).

+## Getting started

+You can get to **Help + support** in the Azure portal. It's available from the Azure portal menu, the global header, or the resource menu for a service. Before you can file a support request, you must have appropriate permissions.

+### Azure role-based access control

+To create a support request for an Enterprise Agreement, you must be an Enterprise Administrator or Partner Administrator associated with an enterprise enrollment.

+### Go to Help + support from the global header

+To start a support request from anywhere in the Azure portal:

+1. Select the question mark symbol in the global header, then select **Help + support**.

+ :::image type="content" source="media/how-to-create-azure-support-request-ea/help-support-new-lower.png" alt-text="Screenshot of the Help menu in the Azure portal.":::

+1. Select **Create a support request**. Follow the prompts to provide information about your problem. We'll suggest some possible solutions, gather details about the issue, and help you submit and track the support request.

+ :::image type="content" source="media/how-to-create-azure-support-request-ea/new-support-request-2-lower.png" alt-text="Screenshot of the Help + support page with Create a support request link.":::

+### Go to Help + support from a resource menu

+To start a support request:

+1. From the resource menu, in the **Support + troubleshooting** section, select **New Support Request**.

+ :::image type="content" source="media/how-to-create-azure-support-request-ea/in-context-2-lower.png" alt-text="Screenshot of the New Support Request option in the resource pane.":::

+1. Follow the prompts to provide us with information about the problem you're having. When you start the support request process from a resource, some options are pre-selected for you.

+## Create a support request

+We'll walk you through some steps to gather information about your problem and help you solve it. Each step is described in the following sections.

+### Problem description

+1. Type a summary of your issue and then select **Issue type**.

+1. In the **Issue type** list, select **Enrollment administration** for EA portal related issues.

+ :::image type="content" source="./media/how-to-create-azure-support-request-ea/select-issue-type-enrollment-administration.png" alt-text="Screenshot showing Select Enrollment administration." lightbox="./media/how-to-create-azure-support-request-ea/select-issue-type-enrollment-administration.png" :::

+1. For **Enrollment number**, select the enrollment number.

+ :::image type="content" source="./media/how-to-create-azure-support-request-ea/select-enrollment.png" alt-text="Screenshot showing Select Enrollment number." :::

+1. For **Problem type**, select the issue category that best describes the type of problem that you have.

+ :::image type="content" source="./media/how-to-create-azure-support-request-ea/select-problem-type.png" alt-text="Screenshot showing Select a problem type." :::

+1. For **Problem subtype**, select a problem subcategory.

+After you've provided all of these details, select **Next: Solutions**.

+### Recommended solution

+Based on the information you provided, we'll show you recommended solutions you can use to try to resolve the problem. In some cases, we may even run a quick diagnostic. Solutions are written by Azure engineers and will solve most common problems.

+If you're still unable to resolve the issue, continue creating your support request by selecting **Next: Details**.

+### Other details

+Next, we collect more details about the problem. Providing thorough and detailed information in this step helps us route your support request to the right engineer.

+1. On the Details tab, complete the **Problem details** section so that we have more information about your issue. If possible, tell us when the problem started and any steps to reproduce it. You can upload a file, such as a log file or output from diagnostics. For more information on file uploads, see [File upload guidelines](../../azure-portal/supportability/how-to-manage-azure-support-request.md#file-upload-guidelines).

+1. In the **Share diagnostic information** section, select **Yes** or **No**. Selecting **Yes** allows Azure support to gather [diagnostic information](https://azure.microsoft.com/support/legal/support-diagnostic-information-collection/) from your Azure resources. If you prefer not to share this information, select **No**. In some cases, there will be more options to choose from.

+1. In the **Support method** section, select the severity of the issue. The maximum severity level depends on your [support plan](https://azure.microsoft.com/support/plans).

+1. Provide your preferred contact method, your availability, and your preferred support language.

+1. Next, complete the **Contact info** section so we know how to contact you.

+ :::image type="content" source="./media/how-to-create-azure-support-request-ea/details-tab.png" alt-text="Screenshot showing the Details tab." lightbox="./media/how-to-create-azure-support-request-ea/details-tab.png" :::

+Select **Next: Review + create** when you've completed all of the necessary information.

+### Review + create

+Before you create your request, review all of the details that you'll send to support. You can select **Previous** to return to any tab if you need to make changes. When you're satisfied the support request is complete, select **Create**.

+A support engineer will contact you using the method you indicated. For information about initial response times, see [Support scope and responsiveness](https://azure.microsoft.com/support/plans/response/).

+## Can't create request with Microsoft Account

+If you have a Microsoft Account (MSA) and you aren't able to create an Azure support ticket, use the following steps to file a support case. Microsoft accounts are created for services including Outlook, Windows Live, and Hotmail.

+To create an Azure support ticket, an *organizational account* must have the EA administrator role or Partner administrator role.

+If you have an MSA, have an administrator create an organizational account for you. An enterprise administrator or partner administrator must then add your organizational account as an enterprise administrator or partner administrator. Then you can use your organizational account to file a support request.

+- To add an Enterprise Administrator, see [Create another enterprise administrator](../manage/ea-portal-administration.md#create-another-enterprise-administrator).

+- To add a Partner Administrator, see [Manage partner administrators](../manage/ea-partner-portal-administration.md#manage-partner-administrators).

+## Next steps

+Follow these links to learn more:

+* [How to manage an Azure support request](../../azure-portal/supportability/how-to-manage-azure-support-request.md)

+* [Azure support ticket REST API](/rest/api/support)

+* Engage with us on [Twitter](https://twitter.com/azuresupport)

+* Get help from your peers in the [Microsoft Q&A question page](/answers/products/azure)

+* Learn more in [Azure Support FAQ](https://azure.microsoft.com/support/faq)

+ Title: Troubleshoot viewing your billing account in the Azure portal

+description: This article helps you troubleshoot problems when trying to view your billing account in the Azure portal.

+tags: billing

+# Troubleshoot viewing your billing account in the Azure portal

+A billing account is created when you sign up to use Azure. You use your billing account to manage your invoices, payments, and track costs. You might have access to multiple billing accounts. For example, you might have signed up for Azure for personal use. You could also have access to Azure through your organization's Enterprise Agreement or Microsoft Customer Agreement. For each of these scenarios, you would have a separate billing account. This article helps you troubleshoot problems when trying to view your billing account in the Azure portal.

+You can view your billing accounts one the [Cost Management + Billing](https://portal.azure.com/#blade/Microsoft_Azure_GTM/ModernBillingMenuBlade) page.

+To learn more about billing accounts and identify your billing account type, see [View billing accounts in Azure portal](../manage/view-all-accounts.md).

+If you're unable to see your billing account in the Azure portal, try the following options:

+## Sign in to a different tenant

+Your billing account is associated with a single Microsoft Entra tenant. You won't see your billing account on the Cost Management + Billing page if you're signed in to an incorrect tenant. Use the following steps to switch to another tenant in the Azure portal and view your billing accounts in that tenant.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Select your profile (email address) at the top right of the page.

+1. Select **Switch directory**.

+ 

+1. Select a directory under the **All directories** section.

+ 

+## Sign in with a different email address

+Some users have multiple email addresses to sign in to the [Azure portal](https://portal.azure.com). Not all email addresses have access to a billing account. If you sign in with an email address that has permissions to manage resources but doesn't have permissions to view a billing account, you wouldn't see the billing account on the [Cost Management + Billing](https://portal.azure.com/#blade/Microsoft_Azure_GTM/ModernBillingMenuBlade) page in the Azure portal.

+Sign in to the Azure portal with an email address that has permission to the billing account to access your billing account.

+## Sign in with a different identity

+Some users have two identities with the same email address - a work or school account and a personal account. Typically, only one of their identities has permissions to view a billing account. You might have two identities with a single email address. When you sign in with an identity that doesn't have permission to view a billing account, you won't see the billing account in the [Cost Management + Billing](https://portal.azure.com/#blade/Microsoft_Azure_GTM/ModernBillingMenuBlade) page. Use the following steps to switch your identity:

+1. Sign in to the [Azure portal](https://portal.azure.com) in an InPrivate/Incognito window.

+1. If your email address has two identities, you'll see an option to select a personal account or a work or school account. Select one of the accounts.

+1. If you can't see the billing account in the Cost Management + Billing page in the Azure portal, repeat steps 1 and 2 and select the other identity.

+## Contact us for help

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+Read the following billing and subscription articles to help troubleshoot problems.

+- [Declined card](./troubleshoot-declined-card.md)

+- [Subscription sign in issues](../troubleshoot-subscription/troubleshoot-sign-in-issue.md)

+- [No subscriptions found](../troubleshoot-subscription/no-subscriptions-found.md)

+- [Enterprise cost view disabled](./enterprise-mgmt-grp-troubleshoot-cost-view.md)

+ Title: Troubleshoot cant view invoice in the Azure portal

+description: Resolving an issue when trying to view your invoice in the Azure portal.

+tags: billing

+# Troubleshoot issues while trying to view invoice in the Azure portal

+You may experience issues when you try to view your invoice in the Azure portal. This short guide will discuss some common issues.

+

+## Common issues and solutions

+#### <a name="subnotfound"></a> You see the message ΓÇ£We canΓÇÖt display the invoices for your subscription. This typically happens when you sign in with an email, which doesnΓÇÖt have access to view invoices. Check youΓÇÖve signed in with the correct email address. If you are still seeing the error, see Why you might not see an invoice.ΓÇ¥

+This happens when the identity that you used to sign in does not have access to the subscription.

+To resolve this issue, try one of the following options:

+**Verify that you're signed in with the correct email address:**

+Only the email that has the account administrator role for the subscription can view its invoice. Verify that you've signed in with the correct email address. The email address is displayed in the email that you received when your invoice is generated.

+ 

+**Verify that you're signed in with the correct account:**

+Some customers have two accounts with the same email address - a work or a school account and a personal account. Typically, only one of their accounts has permission to view invoices. You might have two accounts with your email address. If you sign in with the account that doesn't have permission, you would not see the invoice. To identify if you have multiple accounts and use a different account, follow the steps below:

+1. Sign in to the [Azure portal](https://portal.azure.com) in an InPrivate/Incognito window.

+1. If you have multiple accounts with the same email, then you'll be prompted to select either **Work or school account** or **Personal account**. Select one of the accounts then follow the [instructions here to view your invoice](../understand/download-azure-invoice.md#download-your-mosp-azure-subscription-invoice).

+ 

+1. Try other account, if you still can't view the invoice in the Azure portal.

+**Verify that you're signed in to the correct Microsoft Entra tenant:**

+Your billing account and subscription is associated with a Microsoft Entra tenant. If you're signed in to an incorrect tenant, you won't see the invoice for your subscription. Try the following steps to switch tenants in the Azure portal:

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Select your email address from the top-right of the page.

+1. Select Switch directory.

+ 

+1. Select a tenant from the All Directories section. If you don't see All Directories section, you don't have access to multiple tenants.

+ 

+#### <a name="cantsearchinvoice"></a>You couldn't find the invoice that you see on your credit card statement

+You find a charge on your credit card **Microsoft Gxxxxxxxxx**. You can find all other invoices in the portal but not Gxxxxxxxxx. This happens when the invoice belongs to a different subscription or billing profile. Follow the steps below to view the invoice.

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Search for the invoice number in the Azure portal search bar.

+1. Select **View your invoice**.

+ 

+## Contact us for help

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+- [View and download your Azure invoice](../understand/download-azure-invoice.md)

+- [View and download your Azure usage and charges](../understand/download-azure-daily-usage.md)

+- [No subscriptions found sign in error for Azure portal](../troubleshoot-subscription/no-subscriptions-found.md)

+ Title: Troubleshoot Azure CSP billing issues with usage file pivot tables

+description: This article helps you troubleshoot Azure Cloud Solution Provider (CSP) billing issues using pivot tables created from your CSV usage files.

+tags: billing

+# Troubleshoot CSP billing issues with usage file pivot tables

+This article helps you troubleshoot Cloud Solution Provider (CSP) billing issues using pivot tables in your Partner Center reconciliation (usage) files. Azure usage files contain all your Azure usage and consumption information. The information in the file can help you understand:

+- Understand how Azure reservations are getting used and applied

+- Reconcile information in Cost Management with your billed invoice

+- Troubleshoot a cost spike

+- Calculate a refund amount for a service level agreement

+By using the information from your usage files, you can get a better understanding of usage issues and diagnose them. Usage files are generated in comma delimited (CSV) format. Because the usage files might be large CSV files, they're easier to manipulate and view as pivot tables in a spreadsheet application like Excel. Examples in this article use Excel, but you can use any spreadsheet application that you want.

+Only Billing admins and Global admins have access to download reconciliation files. For more information, see [Learn how to read the line items in your Partner Center reconciliation files](/partner-center/use-the-reconciliation-files).

+## Get the data and format it

+Because Azure usage files are in CSV format, you need to prepare the data for use in Excel. Use the following steps to format the data as table.

+1. Download the usage file using the instructions at [Find your bill](/partner-center/read-your-bill#find-your-bill).

+1. Open the file in Excel.

+1. The unformatted data resembles the following example.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/raw-csv-data.png" alt-text="Example showing unformatted data in Excel" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/raw-csv-data.png" :::

+1. Select the first field in the table, **PartnerID**.

+1. Press Ctrl + Shift + Down arrow and then Ctrl + Shift + Right Arrow to select all the information in the table.

+1. In the top menu, select **Insert** > **Table**. In the Create table box, select **My table has headers** and then select **OK**.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/create-table-dialog.png" alt-text="Example showing the Create Table dialog" :::

+1. In top menu, select **Insert** > **Pivot Table** and then select **OK**. The action creates a new sheet in the file and takes you to the pivot table area on the right side of the sheet.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" alt-text="Example showing the PivotTable fields area" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" :::

+The PivotTable Fields area is a drag-and-drop area. Continue to the next section to create the pivot table.

+## Create pivot table to view Azure costs by resources

+In this section, you create a pivot table where you can troubleshoot overall general Azure usage. The example table can help you investigate which service consumes the most resources. Or you can view the resources that incur the most cost and see how a service is getting charged.

+1. In the PivotTable Fields area, drag **Service Name** and **Resource** to the **Rows** area. Put **Resource** below **Service Name**.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/rows-section.png" alt-text="Example showing Service Name and Resource in Rows" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/rows-section.png" :::

+1. Next, put **Post-Tax Total** in the **Values** area. You can also use the Consumed Quantity column instead to get information about consumption units and transactions. For example, GB and Hours. Or, transactions instead of cost in different currencies like USD, EUR, and INR.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" alt-text="Example showing columns added to pivot table fields" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" :::

+1. Now you have a dashboard for generalized consumption investigation. You can filter for a specific service using the filtering options in the pivot table.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" alt-text="Example showing pivot table filter option for row label" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" :::

+ To filter a second level in a pivot table, for example a resource, select a second-level item in the table.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" alt-text="Example showing filter options for Select field" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" :::

+1. For additional filters, you can add **SubscriptionID** and **Customer Company Name** to the **Filters** area and select the desired scope.

+## Create a pivot table to view Azure usage by date

+In this section, you create a pivot table where you can troubleshoot overall general Azure usage by Consumed Quantity and date. It's useful to identify billing spikes by date and service. Or you can view the resources that incur the most cost and see how a service is getting charged.

+Your reconciliation file has two tables. One is at the top (the main table) and there's another table at the bottom of the document. This second table has much of the same information, however it doesn't include pricing or cost details. It does have usage date and consumed quantity.

+1. Use the same steps from the [Get the data and format it](#get-the-data-and-format-it) section to create an Excel table with the information at the bottom of the reconciliation file.

+1. When the table is ready and you have a pivot table sheet, use the same steps from the create-pivot-table-to-view-azure-costs-by-resources section to prepare the dashboard. Instead of using the Post-Tax total, put **Consumed quantity** in the **Values** area.

+1. Add **Usage Date** to the columns section. The pivot table should look like the following example.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/final-pivot-table-fields.png" alt-text="Example showing final pivot table fields" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/final-pivot-table-fields.png" :::

+1. You now have a dashboard that shows the usage per date. You can extend each month by selecting the **+** symbol.

+The dashboard shows the consumed quantity in units such as GB, Hours, and Transfers.

+To view the price per day, you can add **Resource GUID** to the **Rows** area. In the upper table, add the unit price ( **ListPrice** ) for the resource. Multiply **ListPrice** by the **Consumed quantity** to calculate your pre-tax charges. The amounts should match.

+Some resources (services) have scaled pricing by consumed quantity. For example, some resources have a higher price for the first 100 GB consumed and a lower price for the GB used afterward. Keep scaled pricing in mind when you calculate costs manually.

+## Create pivot table to view cost for a specific resource

+A single resource can incur several charges for different services. For example, a virtual machine can incur Compute charges, OS licensing, Bandwidth (Data transfers), RI usage, and storage for snapshots. Whenever you want to review the overall usage for specific resources, the following steps guide you through creating a dashboard to view overall usage with your usage files.

+Reconciliation files don't contain resource-specific details. So, you use the aggregated usage file. Contact [Azure Billing support](https://go.microsoft.com/fwlink/?linkid=2083458) to have them provide you with the aggregated usage file for your subscription. Aggregated files are generated at the subscription level. The unformatted data resembles the following example.

+The file contains the following columns.

+- **UsageStart** and **UsageEnd** - Date for each line item (each unit of usage). For example, each day.

+- **MeteredResourceID** ΓÇô In Azure, it corresponds to the meter ID.

+- **Properties** - Contains the Instance ID (resource name) with other details such as location.

+- **Quantity** - Consumed quantity in the reconciliation file.

+1. Select the first field in the table, **PartnerID**.

+1. Press Ctrl + Shift + Down arrow and then Ctrl + Shift + Right Arrow to select all the information in the table.

+1. In the top menu, select **Insert** > **Table**. In the Create table box, select **My table has headers** and then select **OK**.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/create-table-dialog.png" alt-text="Example showing the Create Table dialog" :::

+1. In top menu, select **Insert** > **Pivot Table** and then select **OK**. The action creates a new sheet in the file and takes you to the pivot table area on the right side of the sheet.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-fields-reconciliation.png" alt-text="Example showing the PivotTable fields area for the reconciliation file" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/pivot-table-fields-reconciliation.png" :::

+1. Next, add **MeteredResourceID** to the **Rows** area and **Quantity** to **Values**. Results show the overall usage information. For additional details, put **UsageEndDateTime** in the **Columns** area.

+ :::image type="content" source="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/overall-usage.png" alt-text="Example showing overall usage information" lightbox="./media/troubleshoot-csp-billing-issues-usage-file-pivot-tables/overall-usage.png" :::

+1. To view an overall report, add **Properties** to **Rows** under **MeteredResourceID**. It shows a complete dashboard for your usage.

+1. To filter by a specific resource, add **Properties** to the **Filters** area and select the desired usage. You can use Search to find a resource name.

+ To view the cost for the resource, find the total consumed quantity and multiply the value by the list price. The list price is specific for each Resource GUID (MeteredResourceID). If a resource is consuming several MeteredResourceIDs, you have to note the total value for each ID.

+## Next steps

+- [Get started with Cost Management for partners](../costs/get-started-partners.md).

+ Title: Troubleshoot Azure MCA billing issues with usage file pivot tables

+description: This article helps you troubleshoot Microsoft Customer Agreement (MCA) billing issues using pivot tables created from your CSV usage files.

+tags: billing

+# Troubleshoot MCA billing issues with usage file pivot tables

+This article helps you troubleshoot Microsoft Customer Agreement (MCA) billing issues using pivot tables in your usage files. Azure usage files contain all your Azure usage and consumption information. The information in the file can help you understand:

+- Understand how Azure reservations are getting used and applied

+- Reconcile information in Cost Management with your billed invoice

+- Troubleshoot a cost spike

+- Calculate a refund amount for a service level agreement

+By using the information from your usage files, you can get a better understanding of usage issues and diagnose them. Usage files are generated in comma delimited (CSV) format. Because the usage files might be large CSV files, they're easier to manipulate and view as pivot tables in a spreadsheet application like Excel. Examples in this article use Excel, but you can use any spreadsheet application that you want.

+Only Billing profile owners, Contributors, Readers, or Invoice Managers have access to download usage files. For more information, see [Download usage for your Microsoft Customer Agreement](../understand/download-azure-daily-usage.md).

+## Get the data and format it

+Because Azure usage files are in CSV format, you need to prepare the data for use in Excel. Use the following steps to format the data as a table.

+1. Download the usage file using the instructions at [Download usage in Azure portal](../understand/download-azure-daily-usage.md).

+1. Open the file in Excel.

+1. The unformatted data resembles the following example.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/raw-csv-data-mca.png" alt-text="Example showing unformatted data" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/raw-csv-data-mca.png" :::

+1. Select the first field in the table, **invoiceID**.

+1. Press Ctrl + Shift + Down arrow and then Ctrl + Shift + Right Arrow to select all the information in the table.

+1. In the top menu, select **Insert** > **Table**. In the Create table box, select **My table has headers** and then select **OK**.

+1. In top menu, select **Insert** > **Pivot Table** and then select **OK**. The action creates a new sheet in the file and takes you to the pivot table area on the right side of the sheet.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" alt-text="Example showing the PivotTable fields area" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" :::

+The PivotTable Fields area is a drag-and-drop area. Continue to the next section to create the pivot table.

+## Create pivot table to view Azure costs by resources

+In this section, you create a pivot table where you can troubleshoot overall general Azure usage. The example table can help you investigate which service consumes the most resources. Or you can view the resources that incur the most cost and how a service is getting charged.

+1. In the PivotTable Fields area, drag **Meter Category** and **Product** to the **Rows** section. Put **Product** below **Meter Category**.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/rows-section.png" alt-text="[Example showing Meter Category and Product in Rows" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/rows-section.png" :::

+1. Next, add the **costInBillingCurrenty** column to the **Values** section. You can also use the **Quantity** column instead to get information about consumption units and transactions. For example, GB and Hours. Or, transactions instead of cost in different currencies like USD, EUR, and INR.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" alt-text="Example showing fields added to pivot table" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" :::

+1. Now you have a dashboard for generalized consumption investigation. You can filter for a specific service using the filtering options in the pivot table.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" alt-text="Example showing pivot table filter option for row label" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" :::

+ To filter a second level in a pivot table, for example a resource, select a second-level item in the table.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" alt-text="Example showing filter options for Select field" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" :::

+1. Drag the **ResourceID** column to the **Rows** area under **Product** to see the cost of each service by resource.

+1. Add the **date** column to the **Columns** area to see daily consumption for the product.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-date.png" alt-text="Example showing where to put date in the columns area" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-date.png" :::

+1. Expand and collapse months with the **+** symbols for each month's column.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" alt-text="Example showing the + symbol" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" :::

+Adding both the **Cost** and **Quantity** columns in the **Values** area is optional. Doing so creates two columns for each data section below each month and day when the Date column is in the Columns section of the pivot table.

+For additional filters, you can add the InvoiceSection, costCenter, SubscriptionID, ResourceGroupName, or Tags to the filters section and select the desired scope.

+## Create pivot table to view cost for a specific resource

+A single resource can incur several charges for different services. For example, a virtual machine can incur Compute charges, OS licensing, Bandwidth (data transfers), RI usage, and storage for snapshots. Whenever you want to review the overall usage for specific resources, the following steps guide you through creating a dashboard to view overall usage with your usage files.

+1. In the right menu, drag **ResourceID** to the **Filter** section in the pivot table menu.

+1. Select the resource that you want to see the cost for. Type in the **Search** box to find a resource name.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/resource-id-search.png" alt-text="Example showing where to search for resourceID" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/resource-id-search.png" :::

+1. Add **meterCategory** and **Product** to the **Rows** area. Put **Product** below **meterCategory**.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-fields-meter-category.png" alt-text="Example showing where to put meterCategory in pivot table fields" lightbox="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-fields-meter-category.png" :::

+1. Next, add the **Extended Cost** column to the **Values** section. You can also use the Consumed Quantity column instead to get information about consumption units and transactions. For example, GB and Hours. Or, transactions instead of cost in different currencies like USD, EUR, and INR. Now you have a dashboard that shows all the services that the resource consumes.

+1. Add the **Date** column to the **Columns** section. It shows the daily consumption.

+1. You can expand and reduce using the **+** icons in each month's column.

+ :::image type="content" source="./media/troubleshoot-customer-agreement-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" alt-text="Example showing the + symbol" :::

+## Next steps

+- [Explore and analyze costs with cost analysis](../costs/quick-acm-cost-analysis.md).

+ Title: Troubleshoot a declined card

+description: Resolve declined credit card problems in the Azure portal.

+# Troubleshoot a declined card

+You may experience an issue or error in which a card is declined at Azure sign-up or after you've started using your Azure subscription.

+To resolve your issue, select one of the following topics that most closely resembles your error.

+## The card is not accepted for your country/region

+When you choose a card, Azure displays the card options that are valid in the country/region that you select. Contact your bank or card issuer to verify that your credit card is enabled for international transactions. For more information about supported countries/regions and currencies, see the [Azure Purchase FAQ](https://azure.microsoft.com/pricing/faq/).

+> [!Note]

+> - American Express credit cards are not currently supported as a payment instrument in India. We have no time frame as to when it may be an accepted form of payment.

+> - Credit cards are accepted and debit cards are accepted by most countries or regions.

+> - Hong Kong Special Administrative Region and Brazil only support credit cards.

+> - India supports debit and credit cards through Visa and Mastercard.

+## You're using a virtual or prepaid card

+Prepaid and virtual cards are not accepted as payment for Azure subscriptions.

+## Your credit information is inaccurate or incomplete

+The name, address, and CVV code that you enter must match exactly what's printed on the card.

+## The card is inactive or blocked

+Contact your bank to make sure that your card is active.

+You may be experiencing other sign-up issues

+For more information about how to troubleshoot Azure sign-up issues, see the following article:

+[You can't sign-up for Azure in the Azure portal](../troubleshoot-subscription/troubleshoot-azure-sign-up.md)

+## You represent a business that doesn't want to pay by card

+If you represent a business, you can use the invoice payment method (wire transfer) to pay for your Azure subscription. After you set up the account to pay by invoice, you can't change to another payment option, unless you have a Microsoft Customer Agreement and signed up for Azure through the Azure website.

+For more information about how to pay by invoice, see [Submit a request to pay Azure subscription by invoice](../manage/pay-by-invoice.md).

+## Your card information is outdated

+For information about how to manage your card information, including changing or removing a card, see [Add, update, or remove a credit for Azure](../manage/change-credit-card.md).

+## Card not authorized for service consumption (threshold billing)

+A billing threshold is a level of spending that, when met, triggers an authorization to the primary payment method associated to your Azure account. If the service consumption surpasses the billing threshold, Microsoft may attempt an authorization on the primary payment method. If the bank approves the authorization, it will be immediately reversed. There will be no settlement record on your bank statement.

+However, if the authorization on the card is declined, you're asked to update the primary payment method in order to continue using the services. For information about how to manage your card information, including changing or removing a card, see [Add, update, or remove a credit card](../manage/change-credit-card.md).

+For more information about threshold billing, see [Troubleshoot threshold billing](troubleshoot-threshold-billing.md).

+## Other help resources

+Other troubleshooting articles for Azure Billing and Subscriptions

+- [Sign-up issues](../troubleshoot-subscription/troubleshoot-azure-sign-up.md)

+- [Subscription sign-in issues](../troubleshoot-subscription/troubleshoot-sign-in-issue.md)

+- [No subscriptions found](../troubleshoot-subscription/no-subscriptions-found.md)

+- [Enterprise cost view disabled](enterprise-mgmt-grp-troubleshoot-cost-view.md)

+## Contact us for help

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+- [Azure Billing documentation](../index.yml)

+ Title: Troubleshoot Azure EA billing issues with usage file pivot tables

+description: This article helps you troubleshoot Enterprise Agreement (EA) billing issues using pivot tables created from your CSV usage files.

+tags: billing

+# Troubleshoot EA billing issues with usage file pivot tables

+This article helps you troubleshoot EA billing issues using pivot tables in your usage files. Azure usage files contain all your Azure usage and consumption information. The information in the file can help you understand:

+- Understand how Azure reservations are getting used and applied

+- Reconcile information in Cost Management with your billed invoice

+- Troubleshoot a cost spike

+- Calculate a refund amount for a service level agreement

+By using the information from your usage files, you can get a better understanding of usage issues and diagnose them. Usage files are generated in comma delimited (CSV) format. Because the usage files might be large CSV files, they're easier to manipulate and view as pivot tables in a spreadsheet application like Excel. Examples in this article use Excel, but you can use any spreadsheet application that you want.

+Only EA admins, Account Owners, and Department Admins have access to download usage files.

+## Get the data and format it

+Because Azure usage files are in CSV format, you need to prepare the data for use in Excel. Use the following steps to format the data as table.

+1. Download the Usage Details Version 2 with All Charges (usage and purchases) file using the instructions at [Download usage for EA customers](../understand/download-azure-daily-usage.md).

+1. Open the file in Excel.

+1. The unformatted data resembles the following example.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/raw-csv-data-ea.png" alt-text="Example showing unformatted data in Excel" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/raw-csv-data-ea.png" :::

+1. Select the first field in the table, the one containing the first column title, **BillingAccountID**.

+1. Press Ctrl + Shift + Down arrow and then Ctrl + Shift + Right Arrow to select all the information in the table.

+1. In the top menu, select **Insert** > **Table**. In the Create table box, select **My table has headers** and then select **OK**.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/create-table-dialog.png" alt-text="Example showing the Create Table dialog" :::

+1. In top menu, select **Insert** > **Pivot Table** and then select **OK**. The action creates a new sheet in the file. It takes you to the pivot table area on the right side of the sheet.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" alt-text="Example showing the PivotTable fields area" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-fields.png" :::

+The PivotTable Fields area is a drag-and-drop area. Continue to the next section to create the pivot table.

+## Create pivot table to view Azure costs by resources

+In this section, you create a pivot table where you can troubleshoot overall general Azure usage. The example table can help you investigate which service consumes the most resources. Or you can view the resources that incur the most cost and how a service is getting charged.

+1. In the PivotTable Fields area, drag **Meter Category** and **Product** to the **Rows** section. Put **Product** below **Meter Category**.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/rows-section.png" alt-text="Example showing Meter Category and Product in Rows" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/rows-section.png" :::

+1. Next, add the **Cost** column to the **Values** section. You can also use the Consumed Quantity column instead to get information about consumption units and transactions. For example, GB and Hours. Or, transactions instead of cost in different currencies like USD, EUR, and INR.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" alt-text="Example showing columns added to pivot table fields" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/add-pivot-table-fields.png" :::

+1. Now you have a dashboard for generalized consumption investigation. You can filter for a specific service using the filtering options in the pivot table.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" alt-text="Example showing pivot table filter option for row label" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-row-label.png" :::

+ To filter a second level in a pivot table, for example a resource, select a second-level item in the table.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" alt-text="Example showing filter options for Select field" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-filter-option-select-field.png" :::

+1. Drag the **ResourceID** column to the **Rows** area under **Product** to see the cost of each service by resource. To view detailed pricing information, view your organization's UnitPrice and search for **Product** in the first column of the price list.

+1. Add the **Date** column to the **Columns** area to see daily consumption for the product.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-date.png" alt-text="Example showing where to put Date in the columns area" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-date.png" :::

+1. Expand and collapse months with the **+** symbols for each month's column.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" alt-text="Example showing the + symbol" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" :::

+ Adding both the **Cost** and **Quantity** columns in the **Values** area is optional. Doing so creates two columns for each data section below each month and day when the Date column is in the Columns section of the pivot table.

+1. For additional filters, you can add the SubscriptionID, Department, ResourceGroup, Tags, or Cost Center columns to the **Filters** area and select the item you want.

+## Create pivot table to view cost for a specific resource

+A single resource can incur several charges for different services. For example, a virtual machine can incur Compute charges, OS licensing, Bandwidth (Data transfers), RI usage, and storage for snapshots. Whenever you want to review the overall usage for specific resources, the following steps guide you through creating a dashboard to view overall usage with your usage files.

+1. In the right menu, drag **ResourceID** to the **Filter** section in the pivot table menu.

+1. Select the resource that you want to see the cost for. Type in the **Search** box to find a resource name.

+1. Add **Meter Category** and **Product** to the Rows section. Put **Product** below **Meter Category**.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-fields-meter-category.png" alt-text="Example showing where to put Meter Category in the pivot table field area" lightbox="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-fields-meter-category.png" :::

+1. Next, add the **Cost** column to the **Values** section. You can also use the Consumed Quantity column instead to get information about consumption units and transactions. For example, GB and Hours. Or, transactions instead of cost in different currencies like USD, EUR, and INR. Now you have a dashboard that shows all the services that the resource consumes.

+1. Add the **Date** column to the **Columns** section. It shows the daily consumption.

+1. You can expand and reduce using the **+** symbols in each month's column.

+ :::image type="content" source="./media/troubleshoot-ea-billing-issues-usage-file-pivot-tables/pivot-table-month-expand-collapse.png" alt-text="Example showing the + symbol" :::

+## Next steps

+- [Explore and analyze costs with cost analysis](../costs/quick-acm-cost-analysis.md).

+ Title: Troubleshoot threshold billing

+description: Resolve threshold billing problems.

+# Troubleshoot threshold billing

+A billing threshold is a level of spending that, when met, triggers an authorization to the primary payment method associated to your Azure account. If the service consumption surpasses the billing threshold, Microsoft may attempt an authorization on the primary payment method. If the bank approves the authorization, it will be immediately reversed. There will be no settlement record on your bank statement.

+However, if the authorization on the card is declined, you're asked to update the primary payment method in order to continue using the services. For information about how to manage your card information, including changing or removing a card, see [Add, update, or remove a credit card](../manage/change-credit-card.md).

+## How am I notified by Microsoft for a threshold billing authorization?

+If the payment authorization is approved by the bank, it will immediately be reversed. You won't receive a notification. However, if the payment authorization is declined, you'll receive an email and an Azure portal notification asking you to update your payment method before your account is disabled.

+## When does Microsoft release withholding funds on my credit card?

+Microsoft immediately reverses all threshold billing authorizations after receiving a bank approval. Microsoft currently only charges the card when the invoice is due. If your bank doesn't release the funds immediately, the card issuer (such as Visa, MasterCard, and American Express) releases the authorization within 30 calendar days.

+## Do free trial accounts that upgrade to Pay-As-You-Go receive billing thresholds?

+Yes, free trial accounts that upgrade to Pay-As-You-Go will receive billing thresholds.

+## Which services (when consumed) count towards my billing threshold?

+All Microsoft services count towards a customer's billing threshold.

+## How do I check my current consumption level?

+Azure customers can view their current usage levels in Cost Management. For more information about viewing your current Azure costs, see [Start using Cost analysis](../costs/quick-acm-cost-analysis.md).

+## When there are multiple payment methods (with multiple billing profiles) linked to a single billing account, which one is authorized?

+Microsoft only authorizes the default payment method on the customer account.

+## Contact us for help

+If you have questions or need help, [create a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+- If your card was declined and you need assistance with troubleshooting, see [Troubleshoot a declined card](troubleshoot-declined-card.md).

+ Title: Message appears when you try to create multiple subscriptions

+description: Provides help for the message you might see when you try to create multiple subscriptions.

+tags: billing

+# Message appears when you try to create multiple subscriptions

+When you try to create multiple Azure subscriptions in a short period of time, you might receive a message stating:

+`Subscription not created. Please try again later.`

+The message is normal and expected.

+> [!IMPORTANT]

+> All existing subscriptions should generate consumption history before you create another one

+The message can appear for customers with the following Azure subscription agreement types:

+- [Microsoft Customer Agreement purchased directly through Azure.com](../manage/create-subscription.md)

+ - You can have a maximum of five subscriptions in a Microsoft Customer Agreement purchased directly through Azure.com.

+ - You can create one subscription per 24 hour period.

+ - The ability to create other subscriptions is determined on an individual basis according to your history with Azure.

+- [Microsoft Online Services Program](https://signup.azure.com/signup?offer=ms-azr-0003p)

+ - A new billing account for a Microsoft Online Services Program can have a maximum of five subscriptions. However, subscriptions transferred to the new billing account don't count against the limit.

+ - The ability to create other Microsoft Online Services Program subscriptions is determined on an individual basis according to your history with Azure.

+## Solution

+Expect a delay before you can create another subscription.

+If you're new to Azure and don't have any consumption usage, read the [Get started guide for Azure developers](../../guides/developer/azure-developer-guide.md) to help you get started with Azure services.

+## Need help? Contact us.

+If you have questions or need help, [create a support request](https://go.microsoft.com/fwlink/?linkid=2083458).

+## Next steps

+- Learn more about [Programmatically creating Azure subscriptions for a Microsoft Customer Agreement with the latest APIs](../manage/programmatically-create-subscription-microsoft-customer-agreement.md).

+ Title: No subscriptions found error - Azure portal sign in

+description: Provides the solution for a problem in which No subscriptions found error occurs during Azure portal sign in.

+tags: billing

+# No subscriptions found sign in error for Azure portal

+You might receive a "No subscriptions found" error message when you try to sign in to the [Azure portal](https://portal.azure.com/). This article provides a solution for this problem.

+## Symptom

+When you try to sign in to the [Azure portal](https://portal.azure.com/), you receive the following error message: "No subscriptions found".

+## Cause

+This problem occurs if you selected at the wrong directory, or if your account doesnΓÇÖt have sufficient permissions.

+## Solution

+### Scenario: Error message is received in the [Azure portal](https://portal.azure.com)

+To fix this issue:

+* Make sure that the correct Azure directory is selected by selecting your account at the top right.

+ 

+* If the right Azure directory is selected but you still receive the error message, [assign the Owner role to your account](../../role-based-access-control/role-assignments-portal.md).

+## Need help? Contact us.

+If you have questions or need help, [create a support request](https://go.microsoft.com/fwlink/?linkid=2083458).

+ Title: Troubleshoot issues when you sign up for a new account in the Azure portal

+description: Resolving an issue when trying to sign up for a new account in the Microsoft Azure portal.

+tags: billing

+# Troubleshoot issues when you sign up for a new account in the Azure portal

+You might experience an issue when you try to sign up for a new account in the Microsoft Azure portal. This short guide walks you through the sign-up process and discusses some common issues at each step.

+> [!NOTE]

+> If you already have an existing account and are looking for guidance to troubleshoot sign-in issues, see [Troubleshoot Azure subscription sign-in issues](troubleshoot-sign-in-issue.md).

+## Before you begin

+Before beginning sign-up, verify the following information:

+- The information for your Azure profile (including contact email address, street address, and telephone number) is correct.

+- Your credit card information is correct.

+- You don't already have a Microsoft account that has the same information.

+## Guided walkthrough of Azure sign-up

+The Azure sign-up experience consists of four sections:

+- About you

+- Identity verification by phone

+- Identity verification by card

+- Agreement

+This walkthrough provides examples of the correct information to sign up for an Azure account. Each section also contains some common issues and how to resolve them.

+## About you

+When you initially sign up for Azure, you have to provide some information about yourself, including:

+- Your country/region

+- First name

+- Last name

+- Email address

+- Phone number

+- Credit card information

+

+### Common issues and solutions

+#### You see the message ΓÇ£We cannot proceed with sign-up due to an issue with your account. Please contact billing supportΓÇ¥

+To resolve this error, follow these steps:

+1. Sign in to the [Microsoft account center](https://account.microsoft.com/).

+1. At the top of the page, select **Your info**.

+1. Verify that your billing and shipping details are completed and valid.

+1. When you sign up for the Azure subscription, verify that the billing address for the credit card registration matches your bank records.

+If you continue to receive the message, try to sign up by using a different browser.

+How about InPrivate browsing?

+#### Free trial isn't available

+Have you used an Azure subscription in the past? The Azure Terms of Use agreement limits free trial activation only for a user that's new to Azure. If you have had any other type of Azure subscription, you can't activate a free trial. Consider signing up for a [Pay-As-You-Go subscription](https://azure.microsoft.com/offers/ms-azr-0003p/).

+#### You see the message 'You are not eligible for an Azure subscription'

+To resolve this issue, double-check whether the following items are true:

+- The information that you provided for your Azure account profile (including contact email address, street address, and telephone number) is correct.

+- The credit card information is correct.

+- You don't already have a Microsoft account that uses the same information.

+#### You see the message 'Your current account type is not supported'

+This issue can occur if the account is registered in an [unmanaged Microsoft Entra directory](../../active-directory/enterprise-users/directory-self-service-signup.md), and it isn't in your organization's Microsoft Entra directory. To resolve this issue, sign up the Azure account by using another account, or take over the unmanaged AD directory. For more information, see [Take over an unmanaged directory as administrator in Microsoft Entra ID](../../active-directory/enterprise-users/domains-admin-takeover.md).

+The issue can also occur if the account was created using the Microsoft 365 Developer Program. Microsoft doesn't allow purchasing other paid services using your Microsoft 365 Developer Program subscription. For more information, see [Does the subscription also include a subscription to Azure?](/office/developer-program/microsoft-365-developer-program-faq#does-the-subscription-also-include-a-subscription-to-azure-)

+## Identity verification by phone

+

+

+When you get the text message or telephone call, enter the code that you receive in the text box.

+### Common issues and solutions

+#### No verification text message or phone call

+Although the sign-up verification process is typically quick, it might take up to four minutes for a verification code to be delivered.

+Here are some other tips:

+- You can use any phone number for verification as long as it meets the requirements. The phone number that you enter for verification isn't stored as a contact number for the account.

+ - A Voice-over-IP (VoiP) phone number can't be used for the phone verification process.

+ - Check that your phone can receive calls or SMS messages from a United States-based telephone number.

+- Double-check the phone number that you enter, including the country code that you select in the drop-down menu.

+- If your phone doesn't receive text messages (SMS), try the **Call me** option.

+## Identity verification by card

+

+

+### Common issues and solutions

+#### Credit card declined or not accepted

+Virtual or prepaid credit cards aren't accepted as payment for Azure subscriptions. To see what else might cause your card to be declined, see [Troubleshoot a declined card at Azure sign-up](../troubleshoot-billing/troubleshoot-declined-card.md).

+#### Credit card form doesn't support my billing address

+Your billing address must be in the country/region that you select in the **About you** section. Verify that you have selected the correct country/region.

+#### Progress bar hangs in identity verification by card section

+To complete the identity verification by card, third-party cookies must be allowed for your browser.

+Use the following steps to update your browser's cookie settings.

+1. Update the cookie settings.

+ - If you're using **Chrome**:

+ - Select **Settings** > **Show advanced settings** > **Privacy** > **Content settings**. Clear **Block third-party cookies and site data**.

+ - If you're using **Microsoft Edge**:

+ - Select **Settings** > **View advanced settings** > **Cookies** > **Don't block cookies**.

+1. Refresh the Azure sign-up page and check whether the problem is resolved.

+1. If the refresh didn't resolve the issue, then exit and restart the browser, and try again.

+### I saw a charge on my free trial account

+You might see a small, temporary verification hold on your credit card account after you sign up. This hold is removed within three to five days. If you're worried about managing costs, read more about [Analyzing unexpected charges](../understand/analyze-unexpected-charges.md).

+## Agreement

+Complete the Agreement.

+## Other issues

+### Can't activate Azure benefit plan like Visual Studio or Microsoft Cloud Partner Program

+Check that you're using the correct sign-in credentials. Then, check the benefit program and verify that you're eligible.

+- Visual Studio

+ - Verify your eligibility status on your [Visual Studio account page](https://my.visualstudio.com/Benefits).

+ - If you can't verify your status, contact [Visual Studio Subscription Support](https://visualstudio.microsoft.com/subscriptions/support/).

+- Microsoft for Startups

+ - Sign in to the [Microsoft for Startups portal](https://startups.microsoft.com/#start-two) to verify your eligibility status for Microsoft for Startups.

+ - If you can't verify your status, you can get help by creating a [Microsoft for Startups support request](https://support.microsoft.com/supportrequestform/354fe60a-ba6d-92ad-208a-6a41387aa9d8).

+- Cloud Partner Program

+ - Sign in to the [Cloud Partner Program portal](https://mspartner.microsoft.com/Pages/Locale.aspx) to verify your eligibility status. If you have the appropriate [Cloud Platform Competencies](https://mspartner.microsoft.com/pages/membership/cloud-platform-competency.aspx), you might be eligible for other benefits.

+ - If you can't verify your status, contact [Cloud Partner Program Support](https://mspartner.microsoft.com/Pages/Support/Premium/contact-support.aspx).

+### Can't activate new Azure In Open subscription

+To create an Azure In Open subscription, you must have a valid Online Service Activation (OSA) key that has at least one Azure In Open token associated with it. If you don't have an OSA key, contact one of the Microsoft Partners that are listed in [Microsoft Pinpoint](https://pinpoint.microsoft.com/).

+## Other help resources

+Other troubleshooting articles for Azure Billing and Subscriptions

+- [Declined card](../troubleshoot-billing/troubleshoot-declined-card.md)

+- [Subscription sign-in issues](troubleshoot-sign-in-issue.md)

+- [No subscriptions found](./no-subscriptions-found.md)

+- [Enterprise cost view disabled](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md)

+## Contact us for help

+- Get answers in [Azure forums](https://azure.microsoft.com/support/forums/).

+- Connect with [@AzureSupport](https://twitter.com/AzureSupport)- answers, support, experts.

+- If you have a support plan, [open a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).

+## Next steps

+- Read the [Cost Management and Billing documentation](../index.yml)

+ Title: Troubleshoot Not available due to conflict error

+description: Provides the solutions for a problem where you can't select a management group for a reservation or a savings plan.

+# Troubleshoot Not available due to conflict error

+You might see a `Not available due to conflict` error message when you try select a management group for a savings plan or reservation in to the [Azure portal](https://portal.azure.com/). This article provides solutions for the problem.

+## Symptom

+When you try to buy a reservation or savings plan in to the [Azure portal](https://portal.azure.com/) and you select a scope, you see might see a `Not available due to conflicts error`.

+## Cause

+This issue can occur when a management group is selected as the scope. An active benefit (savings plan, reservation, or centrally managed Azure Hybrid Benefit) is already applied at a parent or child scope.

+## Solutions

+To resolve this issue with overlapping benefits, you can do one of the following actions:

+- Select another scope.

+- Change the scope of the existing benefit (savings plan, reservation or centrally managed Azure Hybrid Benefit) to prevent the overlap.

+ - For more information about how to change the scope for a reservation, see [Change the savings plan scope](../reservations/manage-reserved-vm-instance.md#change-the-reservation-scope).

+ - For more information about how to change the scope for a savings plan, see [Change the savings plan scope](../savings-plan/manage-savings-plan.md#change-the-savings-plan-scope).

+## Need help? Contact us.

+If you have questions or need help, [create a support request](https://go.microsoft.com/fwlink/?linkid=2083458).

+ Title: Troubleshoot Azure subscription sign-in issues

+description: Helps to resolve the issues in which you can't sign in to the Azure portal.

+tags: billing

+# Troubleshoot Azure subscription sign-in issues

+This guide helps to resolve the issues in which you can't sign in to the Azure portal.

+> [!NOTE]

+> If you are having issues signing up for a new Azure account, see [Troubleshoot Azure subscription sign-up issues](./troubleshoot-azure-sign-up.md).

+## Page hangs in the loading status

+If your internet browser page hangs, try each of the following steps until you can get to the Azure portal.

+- Refresh the page.

+- Use a different internet browser.

+- Use the private browsing mode for your browser:

+ - **Edge:** Open **Settings** (the three dots by your profile picture), select **New InPrivate window**, and then browse and sign in to the [Azure portal](https://portal.azure.com).

+ - **Chrome:** Choose **Incognito** mode.

+ - **Safari:** Choose **File**, then **New Private Window**.

+- Clear the cache and delete Internet cookies:

+ - **Edge:** Open **Settings** and select **Privacy and Services**. Follow the steps under **Clear Browsing Data**. Verify that the check boxes for **Browsing history**, **Download history**, and **Cached images and files** are selected, and then select **Delete**.

+ - **Chrome:** Choose **Settings** and select **Clear browsing data** under **Privacy and Security**.

+## You are automatically signed in as a different user

+This issue can occur if you use more than one user account in an internet browser.

+To resolve the issue, try one of the following methods:

+- Clear the cache and delete Internet cookies.

+ - **Edge:** Open **Settings** and select **Privacy and Services**. Follow the steps under **Clear Browsing Data**. Verify that the check boxes for **Browsing history**, **Download history**, **Cookies**, and **Cached images and files** are selected, and then select **Delete**.

+ - **Chrome:** Choose **Settings** and select **Clear browsing data** under **Privacy and Security**.

+- Reset your browser settings to defaults.

+- Use the private browsing mode for your browser.

+ - **Edge:** Open **Settings** (the three dots by your profile picture), select **New InPrivate window**, and then browse and sign in to the [Azure portal](https://portal.azure.com).

+ - **Chrome:** Choose **Incognito** mode.

+ - **Safari:** Choose **File**, then **New Private Window**.

+## I can sign in, but I see the error, No subscriptions found

+This problem occurs if you selected at the wrong directory, or if your account doesn't have sufficient permissions.

+**Scenario:** You receive the error signing into the [Azure portal](https://portal.azure.com).

+To fix this issue:

+- Verify that the correct Azure directory is selected by selecting your account at the top-right corner.

+- If the correct Azure directory is selected, but you still receive the error message, have your account [added as an Owner](../manage/add-change-subscription-administrator.md).

+## Additional help resources

+Other troubleshooting articles for Azure Billing and Subscriptions

+- [Declined card](../troubleshoot-billing/troubleshoot-declined-card.md)

+- [Subscription sign-up issues](./troubleshoot-azure-sign-up.md)

+- [No subscriptions found](./no-subscriptions-found.md)

+- [Enterprise cost view disabled](../troubleshoot-billing/enterprise-mgmt-grp-troubleshoot-cost-view.md)

+- [Azure Billing documentation](../index.yml)

+## Contact us for help

+If you have questions or need help but can't sign in to the Azure portal, [create a support request](https://support.microsoft.com/oas/?prid=15470).

+- Unsupported regions: Jio India West, Korea South, South Africa West.

+- Regions that are supported by Defender for Storage but not by malware scanning. Learn more about [availability for Defender for Storage.](/azure/defender-for-cloud/defender-for-storage-introduction)

+ - **Owner** or **Security Admin** or **Resource Policy Contributor** to create an exemption

+ - To create a rule, you need permissions to edit policies in Azure Policy. [Learn more](../governance/policy/overview.md#azure-rbac-permissions-in-azure-policy).

+

+- Some recommendations included in Microsoft cloud security benchmark do not support exemptions, a list of those recommendations can be found [here](/azure/defender-for-cloud/faq-general)

+- Recommendations included in multible policy initiatives must [all be exempted](/azure/defender-for-cloud/faq-general)

+1. To filter the findings by a specific VM, open the "Affected resources" section and select the VM that interests you. Or you can select a VM from the resource health view, and view all relevant recommendations for that resource.

+If you have an organizational need to ignore a finding, rather than remediate it, you can optionally disable it. Disabled findings don't affect your secure score or generate unwanted noise.

+When a finding matches the criteria you defined in your disable rules, it doesn't appear in the list of findings. Typical scenarios include:

+To export vulnerability assessment results, you need to use [Azure Resource Graph](https://azure.microsoft.com/features/resource-graph/) (ARG). This tool provides instant access to resource information across your cloud environments with robust filtering, grouping, and sorting capabilities. It's a quick and efficient way to query information across Azure subscriptions programmatically or from within the Azure portal.

+|SAM259213: “Scan was throttled by the service."| The scan request has temporarily exceeded the service’s rate limit. This is a measure we take to manage server load and ensure optimal performance for all users. For more information, see the [malware scanning limitations](/azure/defender-for-cloud/defender-for-storage-malware-scan) documentation.|To avoid this issue in the future, please ensure your scan requests stay within the service’s rate limit. If your needs exceed the current rate limit, consider distributing your scan requests more evenly over time. |No|

+Your dev center and projects typically exist in the same region as your main office or IT management center.

+- Microsoft Intune subscription.

+- Permission to add and manage groups in Microsoft Intune.

+To ensure that resources are available for customers, Microsoft Dev Box has a limit on the number of each type of resource that can be used in a subscription. This limit is called a quota. There are different types of quotas related to Dev Box that you might see in the Developer portal and Azure portal, such as quota for Dev Box vCPU for box creation as well as resource limits for Dev Centers, network connections, and Dev Box Definitions.

+Understanding quota limits that affect your Dev Box resources helps you to plan for future use. You can check the [default quota level](/azure/azure-resource-manager/management/azure-subscription-service-limits?branch=main#microsoft-dev-box-limits) for each resource, view your current usage, and determine how much quota remains in each region. By monitoring the rate at which your quota is used, you can plan and prepare to [request a quota limit increase](how-to-request-quota-increase.md) before you reach the quota limit for the resource.

+To help you understand where and how you're using your quota, Azure provides the **Usage + Quotas** page in the Azure portal. Each subscription has its own **Usage + quotas** page that covers all the various services in the subscription.

+For example, if dev box users encounter a vCPU quota error such as, *QuotaExceeded*, during dev box creation there might be a need to increase this quota. A great place to start is to determine the current quota available.

+1. You can see the **Quota name**, the **Region**, the **Subscription** the quota is assigned to, the **Current Usage**, and whether or not the limit is **Adjustable**.

+ Title: Dev center (DevCenter) diagnostic logs reference

+description: Schema reference for dev center (DevCenter) diagnostic logs. Review the list of Azure Storage and Azure Monitor Logs properties included in monitoring data.

+# Microsoft dev center monitoring data reference

+This article provides a reference for log and metric data collected for a Microsoft Dev Box dev center. You can use the collected data to analyze the performance and availability of resources within your dev center. For details about how to collect and analyze monitoring data for your dev center, see [Configure Azure diagnostic logs for a dev center](how-to-configure-dev-box-azure-diagnostic-logs.md).

+The following table lists the properties of resource logs in a dev center. The resource logs are collected into Azure Monitor Logs or Azure Storage. In Azure Monitor, logs are collected in the **DevCenterDiagnosticLogs** table under the resource provider name of `MICROSOFT.DEVCENTER`.

+| **resourceId** | **ResourceId** | The dev center resource for which logs are enabled. |

+| **operationName** | **OperationName** | Name of the operation. If the event represents an Azure role-based access control (RBAC) operation, specify the Azure RBAC operation name (for example, `Microsoft.DevCenter/projects/users/devboxes/write`). This name is typically modeled in the form of an Azure Resource Manager operation, even if it's not a documented Resource Manager operation: (`Microsoft.<providerName>/<resourceType>/<subtype>/<Write/Read/Delete/Action>`). |

+| **TargetResourceId** | **ResourceId** | The subresource that pertains to the request. Depending on the operation performed, this value might point to a `devbox` or `environment`. |

+| **resultType** | **OperationResult** | Indicates whether the operation failed or succeeded. |

+| **correlationId** | **CorrelationId** | The unique correlation ID for the operation that can be shared with the app team to support further investigation. |

+For a list of all Azure Monitor log categories and links to associated schemas, see [Common and service-specific schemas for Azure resource logs](../azure-monitor/essentials/resource-logs-schema.md).

+A dev center uses Kusto tables from Azure Monitor Logs. You can query these tables with Log Analytics. For a list of Kusto tables that a dev center uses, see the [Azure Monitor Logs table reference organized by resource type](/azure/azure-monitor/reference/tables/tables-resourcetype#dev-centers).

+## Related content

+- [Configure Azure diagnostic logs for a dev center](how-to-configure-dev-box-azure-diagnostic-logs.md)

+- [Monitor Azure resources with Azure Monitor](../azure-monitor/essentials/monitor-azure-resource.md)

+ Title: "Tutorial: Limit the number of dev boxes in a project to help control costs"

+description: Each dev box incurs compute and storage costs. This tutorial shows you how to set a limit on the number of dev boxes developers can create in a project.

+In the developer portal, a Dev Box User can see their existing dev boxes and their total number of allocations for each project. When they reach their allocation limit for a project, they can't create a new dev box for that project.

+> * View dev box limits in the developer portal

+- Project Admin role permissions to that project

+The dev box limit is the number of dev boxes each developer can create in a project. For example, if you set the limit to 3, each developer in your team can create three dev boxes.

+1. In the search box, enter **projects**. In the list of results, select **Projects**.

+1. On the **Limits** page, toggle the **Enable dev box limit** setting to **Yes**.

+ :::image type="content" source="media/tutorial-dev-box-limits/enable-dev-box-limits.png" alt-text="Screenshot showing the dev box limits options for a project, with Yes highlighted." lightbox="media/tutorial-dev-box-limits/enable-dev-box-limits.png":::

+ :::image type="content" source="media/tutorial-dev-box-limits/dev-box-limit-number.png" alt-text="Screenshot showing dev box limits for a project enabled, with dev boxes per developer highlighted." lightbox="media/tutorial-dev-box-limits/dev-box-limit-number.png":::

+> [!TIP]

+> To prevent developers creating more dev boxes in a project, set the dev box limit to 0. This action doesn't delete existing dev boxes, but it prevents creation of new dev boxes in the project.

+In the developer portal, select a project to see the number of existing dev boxes and the total number of dev boxes you can create in that project.

+If all of your available dev boxes in a project are in use, you see an error message and you can't create a new dev box:

+*Your project administrator has set a limit of \<number> dev boxes per user in \<project name>. Please delete a dev box in this project, or contact your administrator to increase your limit.*

+1. In the search box, enter **projects**. In the list of results, select **Projects**.

+1. Select the project associated with the limit that you want to remove.

+1. On the **Limits** page, change the **Enable dev box limit** setting to **No**.

+- [Use the Azure CLI to configure dev box limits](/cli/azure/devcenter/admin/project)

+The solution enables the speed of creating virtual machines from custom images while eliminating extra ongoing maintenance costs. With this solution, you can automatically create custom images, distribute them to other DevTest Labs, and retire the old images. All the Azure PowerShell scripts are freely available and located here: [https://aka.ms/dtlimagefactory](https://aka.ms/dtlimagefactory).

+<!-- Comment:Now supported by SHIR 5.37 onwards--

+ 3) Or As an alternative, Migrate the database schema from source to target by using [PowerShell](/powershell/module/az.datamigration/new-azdatamigrationsqlserverschema) or the [SQL Server dacpac extension](/azure-data-studio/extensions/sql-server-dacpac-extension) or the [SQL Database Projects](/azure-data-studio/extensions/sql-database-project-extension) extension in Azure Data Studio.-->

+ > 1) Currently, only Azure SQL Managed Instance targets are supported.

+> 2) And Encrypted Backups are not supported.

+### Find redirect-uri

+The `redirect-uri` of your app, where your app sends and receives the authentication responses. It must exactly match one of the redirect URIs that you registered in the portal, except that it must be URL encoded.

+1. Go to **App registrations**.

+1. Under the **Manage** section, select **Authentication**.

+1. Fetch the `redirect-uri` (or reply URL) for your app to receive responses from Microsoft Entra ID.

+ :::image type="content" source="media/how-to-generate-auth-token/redirect-uri.png" alt-text="Screenshot that shows redirect-uri.":::

+### Find the adme-url for your Azure Data Manager for Energy instance

+### Find data-partition-id

+### Get the authorization-code

+1. Keep this `authorization-code` handy for future use.

+ &code={authorization-code}

+ &redirect_uri={redirect-uri}

+ curl --location --request POST "https://<adme-url>/api/entitlements/v2/groups/" \

+curl --location --request PUT 'https://<adme-url>/api/storage/v2/records/' \

+curl --location 'https://<adme-url>/api/storage/v2/records/opendes:master-data--Well:999736019023' \

+curl --location --request PATCH 'https://<adme-url>/api/storage/v2/records/' \

+ curl --location --request GET "https://<adme-url>/api/entitlements/v2/groups/" \

+ curl --location --request POST 'https://<adme-url>/api/entitlements/v2/groups/<group-name>@<data-partition-id>.dataservices.energy/members' \

+ curl --location --request GET 'https://<adme-url>/api/entitlements/v2/members/<OBJECT_ID>/groups?type=none' \

+ curl --location --request DELETE 'https://<adme-url>/api/entitlements/v2/members/<OBJECT_ID>' \

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+## Resource logs

+MQTT broker in Azure Event Grid captures diagnostic logs for the following categories:

+- [Failed MQTT connections](#failed-mqtt-connections)

+- [Successful MQTT connections](#successful-mqtt-connections)

+- [MQTT disconnections](#mqtt-disconnections)

+- [Failed MQTT published messages](#failed-mqtt-published-messages)

+- [Failed MQTT subscription operations](#failed-mqtt-subscription-operations)

+This section provides schema and examples for these logs.

+### Common properties

+The following properties are common for all the resource logs from MQTT broker.

+| Property name | Type | Description |

+| -- | - | -- |

+| `time` | `DateTime` | Timestamp (UTC) when the log was generated. |

+| `resourceId` | `String` | Resource ID of the Event Grid namespace. For example: `/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNAMESPACE`. |

+| `location` | `String` | Location of the namespace |

+| `operationName` | `String` | Name of the operation. For example: `Microsoft.EventGrid/topicspaces/connect`, `Microsoft.EventGrid/topicspaces/disconnect`, `Microsoft.EventGrid/topicspaces/publish`, `Microsoft.EventGrid/topicspaces/subscribe`, `Microsoft.EventGrid/topicspaces/unsubscribe`. |

+| `category` | `String` | Category or type of the operation. For example: `FailedMQTTConnections`, `SuccessfulMQTTConnections`, `MQTTDisconnections`, `FailedMQTTPublishedMessages`, `FailedMQTTSubscriptionOperations`. |

+| `resultType` | `String` | Result of the operation. For example: `Failed`, `Succeeded`. |

+| `resultSignature` | `String` | Result of the failed operation. For example: `QuotaExceeded`, `ClientAuthenticationError`, `AuthorizationError`. This property isn't included for the successful events like `SuccessfulMQTTConnections`. |

+| `resultDescription` | `String` | More description about the result of the failed operation. This property isn't included for the successful events like `SuccessfulMQTTConnections`. |

+| `AuthenticationAuthority` | `String` | Type of authority used to authenticate your MQTT client. It's set to one of the following values: `Local` for clients registered in Event Grid's local registry, or `AAD` for clients using Microsoft Entra for authentication. |

+| `authenticationType` | `String` | Type of authentication used by the client. It's set to one of the following values: `CertificateThumbprintMatch`, `AccessToken`, or `CACertificate`. |

+| `clientIdentitySource` | `String` | Source of the clientΓÇÖs identity. It's `JWT` when you use Microsoft Entra ID authentication. |

+| `authenticationAuthority` | `String` | Authority of the client's identity. It's set to one of the following values: `local` for the clients in Event Grid namespace's local registry, `AAD` for AAD clients. |

+| `clientIdentity` | `String` | Value of the clientΓÇÖs identity. It's the name of the local registry or object ID for Microsoft Entra ID clients.|

+### Failed MQTT connections

+This log includes an entry for every failed `MQTT CONNECT` operation by the client. This log can be used to diagnose connectivity issues.

+Here's a sample Failed MQTT connection log entry.

+```json

+[

+ {

+ "time": "2023-11-06T22:45:02.6829930Z",

+ "resourceId": "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNS",

+ "location": "eastus",

+ "operationName": "Microsoft.EventGrid/topicspaces/connect",

+ "category": "FailedMqttConnections",

+ "resultType": "Failed",

+ "resultSignature": "AuthenticationError",

+ "resultDescription": "Client could not be found",

+ "identity": {

+ "authenticationType": "CertificateThumbprintMatch",

+ "clientIdentitySource": "UserName",

+ "authenticationAuthority": "Local",

+ "clientIdentity": "testclient-1"

+ },

+ "properties": {

+ "sessionName": "testclient1",

+ "protocol": "MQTT5",

+ "traceId": "pwu5p3uuvzbyzpe4vyygij3it4"

+ }

+ }

+]

+```

+Here are the properties and their descriptions.

+| Property | Type | Description |

+| -- | - | -- |

+| `sessionName` | `String` | Name of the session provided by the client in the `MQTT CONNECT` packetΓÇÖs clientId field. |

+| `protocol` | `String` | Protocol used by the client to connect. Possible values are: MQTT3, MQTT3-WS, MQTT5, MQTT5-WS. |

+| `traceId` | `Int` | Generated trace ID. |

+### Successful MQTT connections

+This log includes an entry for every successful `MQTT CONNECT` operation by the client. This log can be used for auditing purposes.

+```json

+[

+ {

+ "time": "2023-11-07T01:22:05.2804980Z",

+ "resourceId": "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNS",

+ "location": "eastus",

+ "operationName": "Microsoft.EventGrid/topicspaces/connect",

+ "category": "SuccessfulMqttConnections",

+ "resultType": "Succeeded",

+ "identity": {

+ "authenticationType": "CertificateThumbprintMatch",

+ "clientIdentitySource": "UserName",

+ "authenticationAuthority": "Local",

+ "clientIdentity": "client1"

+ },

+ "properties": {

+ "sessionName": "client1",

+ "protocol": "MQTT5"

+ }

+ }

+]

+```

+Here are the properties and their descriptions.

+| Property | Type | Description |

+| -- | - | -- |

+| `sessionName` | `String` | Name of the session provided by the client in the `MQTT CONNECT` packetΓÇÖs clientId field. |

+| `protocol` | `String` | Protocol used by the client to connect. Possible values are: MQTT3, MQTT3-WS, MQTT5, MQTT5-WS. |

+### MQTT disconnections

+This log includes an entry for every MQTT client disconnection from an Event Grid namespace. This log can be used to diagnose connectivity issues.

+```json

+[

+ {

+ "time": "2023-11-07T01:29:22.4591610Z",

+ "resourceId": "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNS",

+ "location": "eastus",

+ "operationName": "Microsoft.EventGrid/topicspaces/disconnect",

+ "category": "MqttDisconnections",

+ "resultType": "Failed",

+ "resultSignature": "ClientError",

+ "resultDescription": "Timed out per negotiated Keep Alive",

+ "identity": {

+ "clientIdentity": "client1"

+ },

+ "properties": {

+ "sessionName": "client1",

+ "protocol": "MQTT5"

+ }

+ }

+]

+```

+Here are the properties and their descriptions.

+| Property | Type | Description |

+| -- | - | -- |

+| `sessionName` | `String` | Name of the session provided by the client in the `MQTT CONNECT` packetΓÇÖs clientId field. |

+| `protocol` | `String` | Protocol used by the client to connect. Possible values are: MQTT3, MQTT3-WS, MQTT5, MQTT5-WS. |

+### Failed MQTT published messages

+This log includes an entry for every MQTT message that failed to be published to or delivered by an Event Grid namespace. This log can be used to diagnose publishing issues and message loss.

+```json

+[

+ {

+ "time": "2023-11-07T01:22:48.2811790Z",

+ "resourceId": "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNS",

+ "location": "eastus",

+ "operationName": "Microsoft.EventGrid/topicspaces/publish",

+ "category": "FailedMqttPublishedMessages",

+ "resultType": "Failed",

+ "resultSignature": "AuthorizationError",

+ "resultDescription": "Topic name 'testtopic/small4.0' does not match any topicspaces",

+ "identity": { "clientIdentity": "client1" },

+ "properties": {

+ "sessionName": "client1",

+ "protocol": "MQTT5",

+ "traceId": "ako65yewjjhzbdp3lxny7557fu",

+ "qos": 1,

+ "topicName": "testtopic/small4.0",

+ "operationCount": 1

+ }

+ }

+]

+```

+Here are the columns of the `EventGridNamespaceFailedMqttPublishedMessages` Log Analytics table and their descriptions.

+| Column name | Type | Description |

+| -- | - | -- |

+| `sessionName` | `String` | Name of the session provided by the client in the MQTT CONNECT packetΓÇÖs clientId field. |

+| `protocol` | `String` | Protocol used by the client to publish. Possible values are: MQTT3, MQTT3-WS, MQTT5, MQTT5-WS. |

+| `traceId` | `Int` | Generated trace ID. |

+| `qos` | `Int` | Quality of service used by the client to publish. Possible values are: 0 or 1. |

+| `topicName` | `String` | MQTT Topic Name used by the client to publish. |

+| `operationCount` | `Int` | Count of MQTT message that failed to be published to or delivered by an Event Grid namespace with the same resultDescription. |

+### Failed MQTT subscription operations

+This log includes an entry for every MQTT subscribe operation by an MQTT client. A log entry is added for each topic filter within the same subscribe/unsubscribe packet that has the same error. This log can be used to diagnose subscription issues and message loss.

+```json

+[

+ {

+ "time": "2023-11-07T01:22:39.0339970Z",

+ "resourceId": "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/RESOURCEGROUPS/MYRG/PROVIDERS/MICROSOFT.EVENTGRID/NAMESPACE/MYNS",

+ "location": "eastus",

+ "operationName": "Microsoft.EventGrid/topicspaces/subscribe",

+ "category": "FailedMqttSubscriptionOperations",

+ "resultType": "Failed",

+ "resultSignature": "AuthorizationError",

+ "resultDescription": "Topic filter 'testtopic/#' does not match any topicspaces",

+ "identity": { "clientIdentity": "client1" },

+ "properties": {

+ "sessionName": "client1",

+ "protocol": "MQTT5",

+ "traceId": "gnz3cgqpozg4tbm5anvsvopafi",

+ "topicFilters": ["testtopic/#"]

+ }

+ }

+]

+```

+Here are the columns of the `EventGridNamespaceFailedMqttSubscriptions` Log Analytics table and their descriptions.

+| Column name | Type | Description |

+| -- | - | -- |

+| `sessionName` | `String` | Name of the session provided by the client in the MQTT CONNECT packetΓÇÖs clientId field. |

+| `protocol` | `String` | Protocol used by the client to publish. Possible values are: MQTT3, MQTT3-WS, MQTT5, MQTT5-WS. |

+| `traceId` | `Int` | Generated trace ID. |

+| `topicFilters` | Array of strings | List of topic names within the same packet that have the same error. |

+- [Monitor push delivery reference](monitor-push-reference.md).

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, php

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: csharp

+# ms.devlang: csharp, java

+ms.devlang: spark-scala

+ms.devlang: spark-scala

+In addition to DICOM data, a small file to enable [health checks](#health-check) will be written to this location.

+The DICOM service is granted access to the data like any other service or application accessing data in a storage account. Access can be revoked at any time without affecting your organization's ability to access the data. The DICOM service needs the ability to read, write, and delete files in the provided file system. This can be provided by granting the [Storage Blob Data Contributor](/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) role to the system-assigned or user-assigned managed identity attached to the DICOM service.

+## Health check

+The DICOM service writes a small file to the data lake every 30 seconds, following the [Data Contract](#data-contracts) to ensure it maintains access. Making any changes to files stored under the `healthCheck` sub-directory might result in incorrect status of the health check.

+If there is an issue with access, status and details are displayed by [Azure Resource Health](../../service-health/overview.md). Azure Resource Health specifies if any action is required to restore access, for example reinstating a role to the DICOM service's identity.

+ms.devlang: csharp

+ms.devlang: csharp

+# ms.devlang: csharp, java, nodejs

+ms.devlang: csharp

+# ms.devlang: csharp, java, nodejs

+> [!NOTE]

+> CentOS Linux 7 will reach [end of life (EOL) on June 30, 2024](https://www.redhat.com/topics/linux/centos-linux-eol). In July 2024, CentOS 7 will be removed from IoT Edge *Tier 2* supported platform. If you take no action, CentOS 7 based IoT Edge devices continue to work but ongoing security patches and bug fixes in the host packages for CentOS 7 won't be available after June 30, 2024. To continue to receive support and security updates, we recommend that you update your host OS to a *Tier 1* supported platform. To learn more about CentOS 7 EOL, see [What to know about CentOS Linux EOL](https://www.redhat.com/topics/linux/centos-linux-eol) article.

+# Send and receive messages between Azure IoT MQ and Event Hubs or Kafka

+| trustedCaCertificateConfigMap | The name of the config map that contains the CA certificate for verifying the server's identity. This field isn't required for Event Hubs communication, as Event Hubs uses well-known CAs that are trusted by default. However, you can use this field if you want to use a custom CA certificate. | No |

+When specifying a trusted CA is required, create a ConfigMap containing the public potion of the CA in PEM format, and specify the name in the `trustedCaCertificateConfigMap` property.

+| sasl | The configuration for SASL authentication. Specify the `saslType`, which can be *plain*, *scramSha256*, or *scramSha512*, and `token` to reference the Kubernetes `secretName` or Azure Key Vault `keyVault` secret containing the password. | Yes, if using SASL authentication |

+| x509 | The configuration for X509 authentication. Specify the `secretName` or `keyVault` field. The `secretName` field is the name of the secret that contains the client certificate and the client key in PEM format, stored as a TLS secret. | Yes, if using X509 authentication |

+To learn how to use Azure Key Vault and the `keyVault` to manage secrets for Azure IoT MQ instead of Kubernetes secrets, see [Manage secrets using Azure Key Vault or Kubernetes secrets](../manage-mqtt-connectivity/howto-manage-secrets.md).

+##### Authenticate to Event Hubs

+To connect to Event Hubs using a connection string and Kubernetes secret, use `plain` SASL type and `$ConnectionString` as the username and the full connection string as the password. First create the Kubernetes secret:

+kubectl create secret generic cs-secret -n azure-iot-operations \

+Then, reference the secret in the configuration:

+```yaml

+authentication:

+ enabled: true

+ authType:

+ sasl:

+ saslType: plain

+ token:

+ secretName: cs-secret

+```

+To use Azure Key Vault instead of Kubernetes secrets, create an Azure Key Vault secret with the connection string `Endpoint=sb://..`, reference it with `vaultSecret`, and specify the username as `"$ConnectionString"` in the configuration.

+```yaml

+authentication:

+ enabled: true

+ authType:

+ sasl:

+ saslType: plain

+ token:

+ keyVault:

+ username: "$ConnectionString"

+ vault:

+ name: my-key-vault

+ directoryId: <AKV directory ID>

+ credentials:

+ servicePrincipalLocalSecretName: aio-akv-sp

+ vaultSecret:

+ name: my-cs # Endpoint=sb://..

+ # version: 939ecc2...

+```

+To use managed identity, specify it as the only method under authentication. You also need to assign a role to the managed identity that grants permission to send and receive messages from Event Hubs, such as Azure Event Hubs Data Owner or Azure Event Hubs Data Sender/Receiver. To learn more, see [Authenticate an application with Microsoft Entra ID to access Event Hubs resources](/azure/event-hubs/authenticate-application#built-in-roles-for-azure-event-hubs).

+```yaml

+authentication:

+ enabled: true

+ authType:

+ systemAssignedManagedIdentity:

+ audience: https://<NAMESPACE>.servicebus.windows.net

+```

+##### X.509

+kubectl create secret tls my-tls-secret -n azure-iot-operations \

+Then specify the `secretName` in configuration.

+ x509:

+ secretName: my-tls-secret

+```

+To use Azure Key Vault instead, make sure the [certificate and private key are properly imported](../../key-vault/certificates/tutorial-import-certificate.md) and then specify the reference with `vaultCert`.

+```yaml

+authentication:

+ enabled: true

+ authType:

+ x509:

+ keyVault:

+ vault:

+ name: my-key-vault

+ directoryId: <AKV directory ID>

+ credentials:

+ servicePrincipalLocalSecretName: aio-akv-sp

+ vaultCert:

+ name: my-cert

+ # version: 939ecc2...

+ ## If presenting full chain also

+ # vaultCaChainSecret:

+ # name: my-chain

+```

+Or, if presenting the full chain is required, upload the full chain cert and key to AKV as a PFX file and use the `vaultCaChainSecret` field instead.

+```yaml

+# ...

+keyVault:

+ vaultCaChainSecret:

+ name: my-cert

+ # version: 939ecc2...

+| vaultCaChainSecret | Yes, when using certificate chain | Specifies the certificate chain in the Azure Key Vault. |

+| vaultCaChainSecret.name | Yes | Specifies the name of the certificate chain. |

+| vaultCaChainSecret.version | No | Specifies the version of the certificate chain. |

+| username | No | Used only for Event Hubs Kafka connector, see [Send and receive messages between Azure IoT MQ and Event Hubs or Kafka](../connect-to-cloud/howto-configure-kafka.md). |

+- `vaultCaChainSecret`: Use this field when you need to present a full certificate chain, with all extra intermediate or root certificates, to the remote server. For example, you can use this field for configuring a *MqttBridgeConnector* resource with the `remoteBrokerConnection` field. To use this field, import X.509 certificates without private keys in PEM format as a multi-line regular secret (not certificate-type) to Key Vault. This field should be used in addition to `vaultCert` that has the client certificate and private key.

+ # version: 939ecc2...

+ # version: 939ecc2...

+ vaultCaChainSecret:

+ # version: 939ecc2...

+Key Vault doesn't currently provide a way to back up an entire key vault in a single operation and keys, secrets and certitificates must be backup indvidually.

+Also consider the following issues:

+ Title: Compare load test runs

+# Compare load test runs in Azure Load Testing

+In this article, you learn how you can compare test runs in Azure Load Testing. You can view trends across the last 10 test runs, or you can select and compare up to five individual test runs. Optionally, you can mark a test run as a baseline to compare against.

+To identify regressions over time, you can use the client-side metrics trends of the last 10 test runs, such as the response time, error rate, and more. In combination with [CI/CD integration](./quickstart-add-load-test-cicd.md), the trends data might help you identify which application build introduced a performance issue.

+When you want to compare the client-side metrics trends against a specific reference test run, you can mark that test run as your baseline. For example, before you implement performance optimizations in your application, you might first create a baseline load test run, and then validate the effects of your optimizations against your baseline.

+To compare both client-side and server-side metrics, you can select up to five test runs, and compare them in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each test run. By also comparing server-side application metrics in the dashboard, you can identify which application component was the root cause for a sudden performance degradation.

+- An Azure load testing resource, which has a test with multiple test runs. To create a load testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).

+## Compare multiple load test runs

+To compare test runs in Azure Load Testing, you first have to select up to five runs within a load test. You can only compare runs that belong to the same load test. After you select the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.

+1. Go to your load testing resource, and then select **Tests** in the left pane.

+ > [!TIP]

+ > You can also use the filters to find your load test.

+1. Select two or more test runs, and then select **Compare** to compare test runs.

+ :::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button in the Azure portal." lightbox="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png":::

+1. On the dashboard, each test run is shown as an overlay in the different graphs.

+ The dashboard enables you to compare both client-side metrics and server-side metrics. You can view the color-coding for each test run in the **Test run details** section.

+ > [!NOTE]

+ :::image type="content" source="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png" alt-text="Screenshot of the load testing dashboard in the Azure portal, comparing two test runs." lightbox="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png":::

+## View metrics trends across load test runs

+To view metrics trends across test runs in Azure Load Testing, you need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.

+1. Select the **Trends** tab to view the metrics trends for the load test.

+ The graphs show the trends for total requests, response time, error percentage, and throughput for the 10 most recent test runs.

+## Compare load test runs against a baseline

+You can mark a test run as a baseline to compare the client-side metrics of the recent test runs with the metrics of the baseline.

+1. On the **Trends** tab, select **Mark baseline**.

+1. From the list of test runs, select the checkbox for the test run that you want to mark as baseline, and then select **Mark baseline**.

+1. On the **Trends** tab, you can now view the baseline test run in the table and charts.

+ The baseline value is shown as a horizontal line in the charts. In the table view, an extra row with the baseline test run details is shown.

+ In the table, an arrow icon indicates whether the metric is trending favorably or unfavorably as compared to the baseline metric value.

+## Related content

+ Title: Monitor server-side metrics

+description: Learn how to capture and monitor server-side application metrics when running a load test with Azure Load Testing. Add Azure app components and resource metrics to your load test configuration.

+In this article, you learn how to capture and monitor server-side application metrics when running a load test with Azure Load Testing. When you run a load test for an Azure-hosted application, Azure Load Testing collects resource metrics for your application components and presents them in the load testing dashboard.

+To capture metrics during your load test, you update the load test configuration and [add the Azure app components](#add-azure-app-components-to-a-load-test) that make up your application. The service automatically selects the most relevant resource metrics for these app components, depending on the type of component. Optionally, you can [update the list of server-side metrics](#configure-resource-metrics-for-a-load-test) for each Azure component.

+- An Azure load testing resource. To create a load testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).

+## Add Azure app components to a load test

+To monitor resource metrics for an Azure-hosted application, you need to specify the list of Azure application components in your load test configuration. Azure Load Testing automatically captures a set of relevant resource metrics for each selected component. During the load test and after the test finishes, you can view the server-side metrics in the load testing dashboard.

+1. In the [Azure portal](https://portal.azure.com), go to your Azure load testing resource.

+1. On the test details page, select **Configure**, and then select **App Components** to add or remove Azure resources to monitor during the load test.

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/configure-app-components.png" alt-text="Screenshot that shows the 'App Components' button for displaying app components to configure for a load test." lightbox="media/how-to-monitor-server-side-metrics/configure-app-components.png":::

+1. On the **Configure App Components** page, select or clear the checkboxes for the Azure resources you want to add or remove, and then select **Apply**.

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/modify-app-components.png" alt-text="Screenshot that shows how to add or remove app components from a load test configuration." lightbox="media/how-to-monitor-server-side-metrics/modify-app-components.png":::

+ When you run the load test, Azure Load Testing displays the default resource metrics for the selected app components in the test run dashboard.

+You can change the list of resource metrics for each app component at any time.

+## Configure resource metrics for a load test

+When you add app components to your load test configuration, Azure Load Testing adds the most relevant resource metrics for these components. You can add or remove resource metrics for each of the app components in your load test.

+Use the following steps to view and update the list of resource metrics for a load test:

+1. On the test details page, select **Configure**, and then select **Metrics** to select the specific resource metrics to capture during the load test.

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/configure-metrics.png" alt-text="Screenshot that shows the 'Metrics' button to configure metrics for a load test." lightbox="media/how-to-monitor-server-side-metrics/configure-metrics.png":::

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/modify-metrics.png" alt-text="Screenshot that shows a list of resource metrics to configure for a load test." lightbox="media/how-to-monitor-server-side-metrics/modify-metrics.png":::

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/run-load-test.png" alt-text="Screenshot that shows the 'Run' button for running the load test from the test details page." lightbox="media/how-to-monitor-server-side-metrics/run-load-test.png":::

+ :::image type="content" source="media/how-to-monitor-server-side-metrics/dashboard-updated-metrics.png" alt-text="Screenshot that shows the updated server-side metrics on the test result dashboard." lightbox="media/how-to-monitor-server-side-metrics/dashboard-updated-metrics.png":::

+> [!NOTE]

+> When you update the load test configuration of a load test, all future test runs use the updated configuration. You can also update app components and metrics on the load testing dashboard. In this case, the configuration changes only apply to the current test run.

+## Related content

+- [View metrics trends and compare load test results to identify performance regressions](./how-to-compare-multiple-test-runs.md).

+ Title: Use secrets & environment variables

+# Use secrets and environment variables in Azure Load Testing

+In this article, you learn how to pass secrets and environments as parameters to a load test in Azure Load Testing. You can use parameters to change the behavior of a load test without having to edit the Apache JMeter script. For example, to test a web application, specify the endpoint URL as a parameter to reuse your test script across multiple environments. You can also use parameters to avoid that you have to hard code sensitive information in the JMeter test script.

+You can specify parameters in the load test configuration when you create a new test or update an existing test. If you run a load test in your CI/CD workflow, you define parameters in the load test configuration file or in the CI/CD workflow definition.

+## Related content

+- Use secrets to [load test secured endpoints](./how-to-test-secured-endpoints.md).

+### IP-based connections to SAP Message Server (load-balanced configuration)

+If you specify an IP address to connect to an SAP Message Server, for example, a load balancer, the connection might still fail with an error message similar to **"hostname SAPDBSERVER01.example.com unknown"**. The message server instructs the SAP connector to use a hostname for the connection to the backend SAP Application Server, or the server behind the load balancer. If DNS can't resolve the hostname, the connection fails.

+For this problem, the following workarounds or solutions exist:

+- Make sure that the client making the connection, such as the computer with the on-premises data gateway for the SAP connector or the ISE connector host for the ISE-based SAP connector, can resolve the hostnames returned by the message server.

+- In the transaction named **RZ11**, change or add the SAP setting named **ms/lg_with_hostname=0**.

+#### Problem context or background

+SAP upgraded their .NET connector (NCo) to version 3.1, which changed the way that the connector requests connections to backend servers from message servers. The connector now uses a new API for application server resolution by the message server unless you force the connector to use the previous API through the setting named **ms/lg_with_hostname=0`**. For more information, see [SAP KB Article 3305039 - SMLG IP Address setting not considered during Logon Group login](https://me.sap.com/notes/3305039/E).

+Support for CLI v2 will end on September 30, 2025.

+We encourage you to migrate your code for both CLI and SDK v1 to CLI and SDK v2. For more information, see [Upgrade to v2](how-to-migrate-from-v1.md).

+Azure Machine Learning CLI v1 has been deprecated. Support for the v1 extension will end on September 30, 2025. You will be able to install and use the v1 extension until that date.

+We recommend that you transition to the `ml`, or v2, extension before September 30, 2025.

+ms.devlang: azurecli

+# ms.devlang: azurecli, cliv2

+> [!NOTE]

+> If you are using UAI workspace please make sure to add the Network Contributor role to your identity. For more information, see [User-assigned managed identity](how-to-identity-based-service-authentication.md).

+> For more information about the supported input file types and details about how MLflow model works see [Considerations when deploying to batch inference](#considerations-when-deploying-to-batch-inference).

+ > [!IMPORTANT]

+ > Configure `timeout` in your deployment based on how long it takes for your model to run inference on a single batch. The bigger the batch size the longer this value has to be. Remeber that `mini_batch_size` indicates the number of files in a batch, not the number of samples. When working with tabular data, each file may contain multiple rows which will increase the time it takes for the batch endpoint to process each file. Use high values on those cases to avoid time out errors.

+Azure Machine Learning supports deploying MLflow models to batch endpoints without indicating a scoring script. This represents a convenient way to deploy models that require processing of big amounts of data in a batch-fashion. Azure Machine Learning uses information in the MLflow model specification to orchestrate the inference process.

+Batch Endpoints distribute work at the file level, for both structured and unstructured data. As a consequence, only [URI file](reference-yaml-data.md) and [URI folders](reference-yaml-data.md) are supported for this feature. Each worker processes batches of `Mini batch size` files at a time. For tabular data, batch endpoints don't take into account the number of rows inside of each file when distributing the work.

+Batch deployments will call the `predict` function of the MLflow model once per file. For CSV files containing multiple rows, this may impose a memory pressure in the underlying compute and may increase the time it takes for the model to score a single file (specially for expensive models like large language models). If you encounter several out-of-memory exceptions or time-out entries in logs, consider splitting the data in smaller files with less rows or implement batching at the row level inside of the model/scoring script.

+> If you choose to indicate a scoring script for an MLflow model deployment, you will also have to specify the environment where the deployment will run.

+- `args` - the command line arguments that should be passed to the job entry point Python script. See the YAML specification file provided here for an example.

+- `entry` - the entry point for the job. It should be a dictionary that defines the file entry point.

+- `args` - the command line arguments that should be passed to the job entry point Python script. See the sample code provided here for an example.

+- Understand what is an [Azure Machine Learning pipeline](concept-ml-pipelines.md)

+## Switch compute instance runtime to automatic runtime (preview)

+Automatic runtime (preview) has following advantages over compute instance runtime:

+- Automatic manage lifecycle of runtime and underlying compute. You don't need to manually create and managed them anymore.

+- Easily customize packages by adding packages in the `requirements.txt` file in the flow folder, instead of creating a custom environment.

+We would recommend you to switch to automatic runtime (preview) if you're using compute instance runtime. If you have a compute instance runtime, you can switch it to an automatic runtime (preview) by using the following steps:

+- Prepare your `requirements.txt` file in the flow folder. Make sure that you don't pin the version of `promptflow` and `promptflow-tools` in `requirements.txt`, because we already include them in the runtime base image. Automatic runtime (preview) will install the packages in `requirements.txt` file when it starts.

+- If you create custom environment to create compute instance runtime, you can also use get the image from environment detail page, and specify it in `flow.dag.yaml` file in the flow folder. To learn more, see [Change the base image for automatic runtime (preview)](#change-the-base-image-for-automatic-runtime-preview). Make sure you have `acr pull` permission for the image.

+- If you want to keep the automatic runtime (preview) as long running compute like compute instance, you can disable the idle shutdown toggle under automatic runtime (preview) edit option.

+2. Add workspace MSI as `Storage File Data Privileged Contributor` to storage account linked with workspace.

+- AI studio don't support bring your own virtual network, it only support workspace managed virtual network.

+description: This article addresses frequent questions prompt flow usage.

+This article addresses frequent questions about prompt flow usage.

+There are possible reasons for this issue:

+- If you disabled public access to storage account, then you need have access to storage account either add you IP to the storage Firewall or add access studio through the virtual network which have private endpoint to the storage account.

+ :::image type="content" source="../media/faq/storage-account-networking-firewall.png" alt-text="Screenshot that shows firewall setting on storage account." lightbox = "../media/faq/storage-account-networking-firewall.png":::

+- There are some cases, the account key in data store is out of sync with the storage account, you can try to update the account key in data store detail page to fix this.

+ :::image type="content" source="../media/faq/datastore-with-wrong-account-key.png" alt-text="Screenshot that shows datastore with wrong account key." lightbox = "../media/faq/datastore-with-wrong-account-key.png":::

+

+- If you are using AI studio, the storage account need set CORS to allow AI studio access the storage account, otherwise, you will see the flow missing issue. You can add following CORS settings to the storage account to fix this issue.

+ - Go to storage account page, select `Resource sharing (CORS)` under `settings`, and select to `File service` tab.

+ - Allowed origins: `https://mlworkspace.azure.ai,https://ml.azure.com,https://*.ml.azure.com,https://ai.azure.com,https://*.ai.azure.com,https://mlworkspacecanary.azure.ai,https://mlworkspace.azureml-test.net`

+ - Allowed methods: `DELETE, GET, HEAD, POST, OPTIONS, PUT`

+ :::image type="content" source="../media/faq/resource-sharing-setting-storage-account.png" alt-text="Screenshot that shows data store with wrong account key." lightbox = "../media/faq/resource-sharing-setting-storage-account.png":::

+## Flow run related issues

+### How to find the raw inputs and outputs of in LLM tool for further investigation?

+In prompt flow, on flow page with successful run and run detail page, you can find the raw inputs and outputs of LLM tool in the output section. Click the `view full output` button to view full output.

+`Trace` section includes each request and response to the LLM tool. You can check the raw message sent to the LLM model and the raw response from the LLM model.

+## How to fix 409 error in from Azure OpenAI?

+You may encounter 409 error from Azure OpenAI, it means you have reached the rate limit of Azure OpenAI. You can check the error message in the output section of LLM node. Learn more about [Azure OpenAI rate limit](../../../ai-services/openai/quotas-limits.md).

+| `entry` | object | **Required.** The entry point for the component. It could define a `file`. | | |

+| `args` | string | The command line arguments that should be passed to the component entry point Python script. These arguments may contain the paths of input data and the location to write the output, for example `"--input_data ${{inputs.<input_name>}} --output_path ${{outputs.<output_name>}}"` | | |

+| `entry` | object | **Required.** The entry point for the job. It could define a `file`. | | |

+| `args` | string | The command line arguments that should be passed to the job entry point Python script. These arguments may contain the input data paths, the location to write the output, for example `"--input_data ${{inputs.<input_name>}} --output_path ${{outputs.<output_name>}}"` | | |

+| `resources.runtime_version` | string | The Spark runtime version. | `3.2`, `3.3` | |

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, python

+ > Each Azure subscription can benefit from one and only one free Grafana Enterprise trial. The free trial lets you try the Grafana Enterprise plan for one month.

+ > - If you select a free trial and enable recurring billing, you will start getting charged after the end of your first month. Disable recurring billing if you just want to test Grafana Enterprise.

+ > - If you delete a Grafana Enterprise free trial resource, you will not be able to create another Grafana Enterprise free trial. Free trial is for one-time use only.

+ > Each Azure subscription can benefit from one and only one free Grafana Enterprise trial. The free trial lets you try the Grafana Enterprise plan for one month.

+ > - If you select a free trial and enable recurring billing, you will start getting charged after the end of your first month. Disable recurring billing if you just want to test Grafana Enterprise.

+ > - If you delete a Grafana Enterprise free trial resource, you will not be able to create another Grafana Enterprise free trial. Free trial is for one-time use only.

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, php, python, ruby

+- Private DNS integration config cannot be changed once deployed.

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, php, python, ruby

+> [!NOTE]

+> Between 2024/1/12 04:00 UTC and 2024/1/15 07:00 UTC, we strategically paused Azure MySQL maintenance to proactively address a detected issue that could lead to maintenance interruptions. We're happy to report that maintenance operations are fully restored. For those impacted, you're welcome to utilize our flexible maintenance feature to conveniently reschedule your maintenance times as needed.

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, php, python, ruby

+NSG flow logging is billed on the volume of logs produced. High traffic volume can result in large flow-log volume which increases the associated costs.

+NSG flow log pricing doesn't include the underlying costs of storage. Using the retention policy feature with NSG flow logs means incurring separate storage costs for extended periods of time.

+If you want to retain data forever and don't want to apply a retention policy, set retention days to 0. For more information, see [Network Watcher Pricing](https://azure.microsoft.com/pricing/details/network-watcher/) and [Azure Storage Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/).

+NSG flow logs are charged per gigabyte of *Network flow logs collected* and come with a free tier of 5 GB/month per subscription. If traffic analytics is enabled with NSG flow logs, traffic analytics pricing applies at per gigabyte processing rates. Traffic analytics isn't offered with a free tier of pricing. For more information, see [Network Watcher pricing](https://azure.microsoft.com/pricing/details/network-watcher/).

+Storage of logs is charged separately. For more information, see [Azure Blob Storage pricing](https://azure.microsoft.com/pricing/details/storage/blobs/).

+If you still have questions, [contact support](https://portal.azure.com/?#blade/Microsoft_Azure_Support/HelpAndSupportBlade).

+For more information about Support plans, see [Azure Support plans](https://azure.microsoft.com/support/plans/response/).

+If you still have questions, [contact support](https://portal.azure.com/?#blade/Microsoft_Azure_Support/HelpAndSupportBlade).

+For more information about Support plans, see [Azure Support plans](https://azure.microsoft.com/support/plans/response/).

+For more information about Support plans, see [Azure Support plans](https://azure.microsoft.com/support/plans/response/).

+For more information about Support plans, see [Azure Support plans](https://azure.microsoft.com/support/plans/response/).

+Before you proceed, please make sure that your subscription is marketplace purchase enabled. For more information, visit [Purchase validation checks](/marketplace/purchase-validation-checks).

+|**Active Connections** |`active_connections` |Count |Total number of connections to database Server, covering all states of connections, including active, idle, and others as seen in `pg_stat_activity` view. For a detailed view focusing solely on active connections, use "Sessions By State" metric. |Yes |

+|**Sessions By State** |`sessions_by_state` |Count|Overall state of the backend. |State|No|

+|**Backends** |`numbackends` |Count|Number of backends that are connected to this database. |DatabaseName|No |

+ Title: Manage read replicas - Azure portal, CLI, REST API - Azure Database for PostgreSQL - Flexible Server

+description: Learn how to manage read replicas Azure Database for PostgreSQL - Flexible Server from the Azure portal, CLI and REST API.

+# Create and manage read replicas in Azure Database for PostgreSQL - Flexible Server from the Azure portal, CLI or REST API

+In this article, you learn how to create and manage read replicas in Azure Database for PostgreSQL from the Azure portal, CLI and REST API. To learn more about read replicas, see the [overview](concepts-read-replicas.md).

+ mkdir -p ${HOME}/Azure_SAP_Automated_Deployment; cd $_

+ cp -Rp samples/Terraform/WORKSPACES ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES

+ az role assignment create --assignee ${appId} \

+ --role "User Access Administrator" \

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES

+ dns_label = "lab.sdaf.contoso.net"

+For example, choose **West Europe** as the deployment location, with the four-character name `WEEU`, as previously described. The sample deployer configuration file `LAB-WEEU-DEP05-INFRASTRUCTURE.tfvars` is in the `${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/DEPLOYER/LAB-WEEU-DEP05-INFRASTRUCTURE` folder.

+The sample SAP library configuration file `LAB-WEEU-SAP_LIBRARY.tfvars` is in the `${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/LIBRARY/LAB-WEEU-SAP_LIBRARY` folder.

+export region_code="WEEU"

+export DEPLOYMENT_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/sap-automation"

+export CONFIG_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES"

+export SAP_AUTOMATION_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/sap-automation"

+az login --service-principal -u "${ARM_CLIENT_ID}" -p="${ARM_CLIENT_SECRET}" --tenant "${ARM_TENANT_ID}"

+To copy the control plane configuration files to the deployer VM, you can use the `sync_deployer.sh` script. Sign in to the deployer VM and update the following command to use your terraform state storage account name. Then, run the following script:

+terraform_state_storage_account=labweeutfstate###

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES

+../sap-automation/deploy/scripts/sync_deployer.sh --storageaccountname $terraform_state_storage_account --state_subscription $ARM_SUBSCRIPTION_ID

+export region_code="WEEU"

+terraform_state_storage_account=labweeutfstate###

+ vault_name="LABWEEUDEP05user###"

+export DEPLOYMENT_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/sap-automation"

+export CONFIG_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES"

+export SAP_AUTOMATION_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/sap-automation"

+ library_parameter_file="${CONFIG_REPO_PATH}/LIBRARY/${env_code}-${region_code}-SAP_LIBRARY/${env_code}-${region_code}-SAP_LIBRARY.tfvars"

+az logout

+az login --service-principal -u "${ARM_CLIENT_ID}" -p="${ARM_CLIENT_SECRET}" --tenant "${ARM_TENANT_ID}"

+ --storageaccountname "${terraform_state_storage_account}" \

+export region_code="WEEU"

+1. If necessary, register the Service Principal, for this tutorial this step is not needed.

+ The first time an environment is instantiated, a Service Principal must be registered. In this tutorial, the control plane is in the `LAB` environment and the workload zone is also in `LAB`. Therefore, a Service Principal must be registered for the `LAB` environment.

+ export env_code="LAB"

+ export region_code="WEEU"

+ ```

+ ```bash

+

+ cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/LANDSCAPE/LAB-WEEU-SAP04-INFRASTRUCTURE

+```

+```bash

+export deployer_env_code="LAB"

+export sap_env_code="LAB"

+export region_code="WEEU"

+export deployer_vnet_code="DEP05"

+export vnet_code="SAP04"

+export tfstate_storage_account="<storageaccountName>"

+export key_vault="<vaultName>"

+export sap_env_code="LAB"

+export region_code="WEEU"

+export vnet_code="SAP04"

+export SID="L00"

+${DEPLOYMENT_REPO_PATH}/deploy/scripts/installer.sh \

+ sap_username=<sap-username>

+ cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES

+ bom_base_name: S42022SPS00_v0001ms

+ ${HOME}/Azure_SAP_Automated_Deployment/sap-automation/deploy/ansible/download_menu.sh

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/SYSTEM/LAB-WEEU-SAP04-L00/

+Make sure you have the following files in the current folders: `sap-parameters.yaml` and `L00_host.yaml`.

+${HOME}/Azure_SAP_Automated_Deployment/sap-automation/deploy/ansible/configuration_menu.sh

+### Playbook: Base Operating System configuration

+This playbook performs the generic OS configuration setup on all the machines, which includes configuration of software repositories, packages, and services.

+### Playbook: SAP specific Operating System configuration

+This playbook performs the SAP OS configuration setup on all the machines. The steps include creation of volume groups and file systems and configuration of software repositories, packages, and services.

+### Playbook: BOM Processing

+### Playbook: Database Instance installation

+This playbook installs the database instances.

+### Playbook: Database Load

+### Playbook: Database High Availability Setup

+This playbook configures the Database High availability, for HANA it entails HANA system replication and Pacemaker for the HANA database.

+### Playbook: Primary Application Server installation

+### Playbook: Application Server installations

+### Playbook: Web Dispatcher installations

+This playbook installs the web dispatchers.

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/SYSTEM/${sap_env_code}-${region_code}-${sap_vnet_code}-X00

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/LANDSCAPE/${sap_env_code}-${region_code}-${sap_vnet_code}-INFRASTRUCTURE

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES/

+export DEPLOYMENT_REPO_PATH="${HOME}/Azure_SAP_Automated_Deployment/sap-automation"

+cd ${HOME}/Azure_SAP_Automated_Deployment/WORKSPACES

+tar xzvf node_exporter-<xxx>.linux-amd64.tar.gz

+## Overview

+SAP HANA supports a third system replication site in two modes.

+Requirements for a third HSR site are different between HANA scale-up and HANA scale-out.

+Requirements for a third HSR site are different between HANA scale-up and HANA scale-out.

+1. Deploy Azure resources for the third node. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.

+ Steps required for the third site are similar to [virtual machines for HANA scale-up cluster](./sap-hana-high-availability.md#prepare-the-infrastructure). Third site will use Azure infrastructure, operating system and HANA version matching the existing Pacemaker cluster, with the following exceptions:

+2. Install SAP HANA on third node.

+

+3. With SAP HANA on third site installed and running, register the third site with the primary site.

+1. Deploy Azure resources for the third site. Depending on your requirements, you can use a different Azure region for disaster recovery purposes.

+ Steps required for the HANA scale-out on third site are mirroring steps to deploy the [HANA scale-out cluster](./sap-hana-high-availability-scale-out-hsr-suse.md#prepare-the-infrastructure). Third site will use Azure infrastructure, operating system and HANA installation steps for SITE1 of the scale-out cluster, with the following exceptions:

+ You must use the same HANA SID and HANA installation number for third site.

+2. With SAP HANA scale-out on third site installed and running, register the third site with the primary site.

+

+- [High-availability architecture and scenarios for SAP NetWeaver](./sap-hana-availability-across-regions.md)

+## Prepare the infrastructure

+Azure Marketplace contains images qualified for SAP HANA with the High Availability add-on, which you can use to deploy new VMs by using various versions of Red Hat.

+### Deploy Linux VMs manually via the Azure portal

+This document assumes that you've already deployed a resource group, an [Azure virtual network](../../virtual-network/virtual-networks-overview.md), and a subnet.

+Deploy VMs for SAP HANA. Choose a suitable RHEL image that's supported for the HANA system. You can deploy a VM in any one of the availability options: virtual machine scale set, availability zone, or availability set.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+#### [Azure portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+#### [PowerShell](#tab/lb-powershell)

+> Don't enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps could cause the health probes to fail. Set the parameter **net.ipv4.tcp_timestamps** to **0**. For more information, see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP Note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+> [!NOTE]

+While designing the infrastructure for SAP HANA on Azure with Azure NetApp Files, be aware of the recommendations in [NFS v4.1 volumes on Azure NetApp Files for SAP HANA](./hana-vm-operations-netapp.md#sizing-for-hana-database-on-azure-netapp-files).

+The configuration in this article is presented with simple Azure NetApp Files Volumes.

+> For production systems, where performance is a key, we recommend to evaluate and consider using [Azure NetApp Files application volume group for SAP HANA](hana-vm-operations-netapp.md#deployment-through-azure-netapp-files-application-volume-group-for-sap-hana-avg).

+## Prepare the infrastructure

+The resource agent for SAP HANA is included in SUSE Linux Enterprise Server for SAP Applications. An image for SUSE Linux Enterprise Server for SAP Applications 12 or 15 is available in Azure Marketplace. You can use the image to deploy new VMs.

+### Deploy Linux VMs manually via Azure portal

+Deploy virtual machines for SAP HANA. Choose a suitable SLES image that is supported for HANA system. You can deploy VM in any one of the availability options - virtual machine scale set, availability zone or availability set.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+#### [Azure Portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+#### [PowerShell](#tab/lb-powershell)

+>

+> - Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter `net.ipv4.tcp_timestamps` to `0`. For details see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+> - To prevent saptune from changing the manually set `net.ipv4.tcp_timestamps` value from `0` back to `1`, update saptune version to 3.1.1 or higher. For more details, see [saptune 3.1.1 ΓÇô Do I Need to Update?](https://www.suse.com/c/saptune-3-1-1-do-i-need-to-update/).

+## Prepare the infrastructure

+Deploy VMs for SAP HANA. Choose a suitable RHEL image that's supported for the HANA system. You can deploy a VM in any one of the availability options: virtual machine scale set, availability zone, or availability set.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+#### [Azure portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+#### [PowerShell](#tab/lb-powershell)

+> Don't enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps could cause the health probes to fail. Set the parameter `net.ipv4.tcp_timestamps` to `0`. For more information, see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP Note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+# High availability of SAP HANA scale-out system on Red Hat Enterprise Linux

+This article describes how to deploy a highly available SAP HANA system in a scale-out configuration. Specifically, the configuration uses HANA system replication (HSR) and Pacemaker on Azure Red Hat Enterprise Linux virtual machines (VMs). The shared file systems in the presented architecture are NFS mounted and are provided by [Azure NetApp Files](../../azure-netapp-files/azure-netapp-files-introduction.md) or [NFS share on Azure Files](../../storage/files/files-nfs-protocol.md).

+* [Azure NetApp Files documentation][anf-azure-doc].

+For recommended SAP HANA storage configurations, see [SAP HANA Azure VMs storage configurations](./hana-vm-operations-storage.md).

+The preceding diagram shows three subnets represented within one Azure virtual network, following the SAP HANA network recommendations:

+1. Deploy the Azure VMs. For this configuration, deploy seven virtual machines:

+ * Three virtual machines to serve as HANA DB nodes for HANA replication site 1: **hana-s1-db1**, **hana-s1-db2** and **hana-s1-db3**.

+ * Three virtual machines to serve as HANA DB nodes for HANA replication site 2: **hana-s2-db1**, **hana-s2-db2** and **hana-s2-db3**.

+ * A small virtual machine to serve as majority maker: **hana-s-mm**.

+ For the majority maker node, you can deploy a small VM, because this VM doesn't run any of the SAP HANA resources. The majority maker VM is used in the cluster configuration to achieve and odd number of cluster nodes in a split-brain scenario. The majority maker VM only needs one virtual network interface in the `client` subnet in this example.

+2. Create six network interfaces, one for each HANA DB virtual machine, in the `inter` virtual network subnet (in this example, **hana-s1-db1-inter**, **hana-s1-db2-inter**, **hana-s1-db3-inter**, **hana-s2-db1-inter**, **hana-s2-db2-inter**, and **hana-s2-db3-inter**).

+3. Create six network interfaces, one for each HANA DB virtual machine, in the `hsr` virtual network subnet (in this example, **hana-s1-db1-hsr**, **hana-s1-db2-hsr**, **hana-s1-db3-hsr**, **hana-s2-db1-hsr**, **hana-s2-db2-hsr**, and **hana-s2-db3-hsr**).

+4. Attach the newly created virtual network interfaces to the corresponding virtual machines:

+ 2. On the left pane, select **Virtual Machines**. Filter on the virtual machine name (for example, **hana-s1-db1**), and then select the virtual machine.

+ 3. On the **Overview** pane, select **Stop** to deallocate the virtual machine.

+ 4. Select **Networking**, and then attach the network interface. In the **Attach network interface** dropdown list, select the already created network interfaces for the `inter` and `hsr` subnets.

+ 5. Select **Save**.

+ 6. Repeat steps b through e for the remaining virtual machines (in our example, **hana-s1-db2**, **hana-s1-db3**, **hana-s2-db1**, **hana-s2-db2** and **hana-s2-db3**)

+ 7. Leave the virtual machines in the stopped state for now.

+5. Enable [accelerated networking](../../virtual-network/create-vm-accelerated-networking-cli.md) for the additional network interfaces for the `inter` and `hsr` subnets by doing the following:

+ 1. Open [Azure Cloud Shell](https://azure.microsoft.com/features/cloud-shell/) in the [Azure portal](https://portal.azure.com/#home).

+ 2. Run the following commands to enable accelerated networking for the additional network interfaces, which are attached to the `inter` and `hsr` subnets.

+ ```azurecli

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db1-inter --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db2-inter --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db3-inter --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db1-inter --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db2-inter --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db3-inter --accelerated-networking true

+

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db1-hsr --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db2-hsr --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s1-db3-hsr --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db1-hsr --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db2-hsr --accelerated-networking true

+ az network nic update --id /subscriptions/your subscription/resourceGroups/your resource group/providers/Microsoft.Network/networkInterfaces/hana-s2-db3-hsr --accelerated-networking true

+ ```

+6. Start the HANA DB virtual machines.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+> [!NOTE]

+>

+> * For HANA scale out, select the NIC for the `client` subnet when adding the virtual machines in the backend pool.

+> * The full set of command in Azure CLI and PowerShell adds the VMs with primary NIC in the backend pool.

+#### [Azure Portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+The full set of Azure CLI codes display the setup of the load balancer, which include two VMs in the backend pool. Depending on the number of VMs in your HANA scale-out, you could add more VMs in the backend pool.

+#### [PowerShell](#tab/lb-powershell)

+The full set of PowerShell code display the setup of the load balancer, which include two VMs in the backend pool. Depending on the number of VMs in your HANA scale-out, you could add more VMs in the backend pool.

+> [!IMPORTANT]

+> Floating IP isn't supported on a NIC secondary IP configuration in load-balancing scenarios. For details, see [Azure Load Balancer limitations](../../load-balancer/load-balancer-multivip-overview.md#limitations). If you need an additional IP address for the VM, deploy a second NIC.

+> [!NOTE]

+> When you're using the standard load balancer, you should be aware of the following limitation. When you place VMs without public IP addresses in the back-end pool of an internal load balancer, there's no outbound internet connectivity. To allow routing to public end points, you need to perform additional configuration. For more information, see [Public endpoint connectivity for Virtual Machines using Azure Standard Load Balancer in SAP high-availability scenarios](./high-availability-guide-standard-load-balancer-outbound-connections.md).

+> [!IMPORTANT]

+> Don't enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps causes the health probes to fail. Set the parameter `net.ipv4.tcp_timestamps` to `0`. For details, see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+#### Deploy the Azure NetApp Files infrastructure

+In this example, you use the following Azure NetApp Files volumes:

+* **[A]**: Applicable to all nodes

+* **[AH]**: Applicable to all HANA DB nodes

+* **[M]**: Applicable to the majority maker node

+* **[AH1]**: Applicable to all HANA DB nodes on SITE 1

+* **[AH2]**: Applicable to all HANA DB nodes on SITE 2

+* **[1]**: Applicable only to HANA DB node 1, SITE 1

+* **[2]**: Applicable only to HANA DB node 1, SITE 2

+ # Client subnet

+ 10.23.0.11 hana-s1-db1

+ 10.23.0.12 hana-s1-db1

+ 10.23.0.13 hana-s1-db2

+ 10.23.0.14 hana-s2-db1

+ 10.23.0.15 hana-s2-db2

+ 10.23.0.16 hana-s2-db3

+ 10.23.0.17 hana-s-mm

+ # Internode subnet

+ 10.23.1.138 hana-s1-db1-inter

+ 10.23.1.139 hana-s1-db2-inter

+ 10.23.1.140 hana-s1-db3-inter

+ 10.23.1.141 hana-s2-db1-inter

+ 10.23.1.142 hana-s2-db2-inter

+ 10.23.1.143 hana-s2-db3-inter

+ # HSR subnet

+ 10.23.1.202 hana-s1-db1-hsr

+ 10.23.1.203 hana-s1-db2-hsr

+ 10.23.1.204 hana-s1-db3-hsr

+ 10.23.1.205 hana-s2-db1-hsr

+ 10.23.1.206 hana-s2-db2-hsr

+ 10.23.1.207 hana-s2-db3-hsr

+ ```bash

+

+

+ ```

+ ```bash

+ yum install nfs-utils

+ ```

+1. **[AH]** Red Hat for HANA configuration.

+ * [2292690 - SAP HANA DB: Recommended OS settings for RHEL 7](https://launchpad.support.sap.com/#/notes/2292690)

+ * [2777782 - SAP HANA DB: Recommended OS settings for RHEL 8](https://launchpad.support.sap.com/#/notes/2777782)

+ * [2455582 - Linux: Running SAP applications compiled with GCC 6.x](https://launchpad.support.sap.com/#/notes/2455582)

+ * [2593824 - Linux: Running SAP applications compiled with GCC 7.x](https://launchpad.support.sap.com/#/notes/2593824)

+ * [2886607 - Linux: Running SAP applications compiled with GCC 9.x](https://launchpad.support.sap.com/#/notes/2886607)

+3. **[AH]** Create mount points for the HANA database volumes.

+4. **[AH]** Verify the NFS domain setting. Make sure that the domain is configured as the default Azure NetApp Files domain: `defaultv4iddomain.com`. Make sure the mapping is set to `nobody`.

+5. **[AH]** Verify `nfs4_disable_idmapping`. It should be set to `Y`. To create the directory structure where `nfs4_disable_idmapping` is located, run the mount command. You won't be able to manually create the directory under */sys/modules*, because access is reserved for the kernel or drivers.

+6. **[AH1]** Mount the shared Azure NetApp Files volumes on the SITE1 HANA DB VMs.

+7. **[AH2]** Mount the shared Azure NetApp Files volumes on the SITE2 HANA DB VMs.

+8. **[AH]** Verify that the corresponding `/hana/shared/` file systems are mounted on all HANA DB VMs, with NFS protocol version **NFSv4**.

+In the presented configuration, you deploy file systems `/hana/data` and `/hana/log` on a managed disk, and you attach these file systems locally to each HANA DB VM. Run the following steps to create the local data and log volumes on each HANA DB virtual machine.

+2. **[AH]** Create physical volumes for all of the disks that you want to use:

+3. **[AH]** Create a volume group for the data files. Use one volume group for the log files and one for the shared directory of SAP HANA:

+4. **[AH]** Create the logical volumes. A *linear* volume is created when you use `lvcreate` without the `-i` switch. We suggest that you create a *striped* volume for better I/O performance. Align the stripe sizes to the values documented in [SAP HANA VM storage configurations](./hana-vm-operations-storage.md). The `-i` argument should be the number of the underlying physical volumes and the `-I` argument is the stripe size. In this article, two physical volumes are used for the data volume, so the `-i` switch argument is set to `2`. The stripe size for the data volume is `256 KiB`. One physical volume is used for the log volume, so you don't need to use explicit `-i` or `-I` switches for the log volume commands.

+5. **[AH]** Create the mount directories and copy the UUID of all of the logical volumes:

+6. **[AH]** Create `fstab` entries for the logical volumes and mount:

+1. **[AH]** Before the HANA installation, set the root password. You can disable the root password after the installation has been completed. Run as `root` command `passwd` to set the password.

+2. **[1,2]** Change the permissions on `/hana/shared`.

+3. **[1]** Verify that you can sign in **hana-s1-db2** and **hana-s1-db3** via secure shell (SSH), without being prompted for a password. If that isn't the case, exchange `ssh` keys, as documented in [Using key-based authentication](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/s2-ssh-configuration-keypairs).

+4. **[2]** Verify that you can sign in **hana-s2-db2** and **hana-s2-db3** via SSH, without being prompted for a password. If that isn't the case, exchange `ssh` keys, as documented in [Using key-based authentication](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/s2-ssh-configuration-keypairs).

+5. **[AH]** Install additional packages, which are required for HANA 2.0 SP4. For more information, see SAP Note [2593824](https://launchpad.support.sap.com/#/notes/2593824) for RHEL 7.

+6. **[A]** Disable the firewall temporarily, so that it doesn't interfere with the HANA installation. You can re-enable it after the HANA installation is done.

+1. **[1]** Install SAP HANA by following the instructions in the [SAP HANA 2.0 installation and update guide](https://help.sap.com/viewer/2c1988d620e04368aa4103bf26f17727/2.0.04/en-US/7eb0167eb35e4e2885415205b8383584.html). The following instructions show the SAP HANA installation on the first node on SITE 1.

+ * For **Choose an action**, enter **1** (for install).

+ * For **Additional components for installation**, enter **2, 3**.

+ * For the installation path, press Enter (defaults to */hana/shared*).

+ * For **Local Host Name**, press Enter to accept the default.

+ * For **Do you want to add hosts to the system?**, enter **n**.

+ * For **SAP HANA System ID**, enter **HN1**.

+ * For **Instance number** [00], enter **03**.

+ * For **Local Host Worker Group** [default], press Enter to accept the default.

+ * For **Select System Usage / Enter index [4]**, enter **4** (for custom).

+ * For **Location of Data Volumes** [/hana/data/HN1], press Enter to accept the default.

+ * For **Location of Log Volumes** [/hana/log/HN1], press Enter to accept the default.

+ * For **Restrict maximum memory allocation?** [n], enter **n**.

+ * For **Certificate Host Name For Host hana-s1-db1** [hana-s1-db1], press Enter to accept the default.

+ * For **SAP Host Agent User (sapadm) Password**, enter the password.

+ * For **Confirm SAP Host Agent User (sapadm) Password**, enter the password.

+ * For **System Administrator (hn1adm) Password**, enter the password.

+ * For **System Administrator Home Directory** [/usr/sap/HN1/home], press Enter to accept the default.

+ * For **System Administrator Login Shell** [/bin/sh], press Enter to accept the default.

+ * For **System Administrator User ID** [1001], press Enter to accept the default.

+ * For **Enter ID of User Group (sapsys)** [79], press Enter to accept the default.

+ * For **System Database User (system) Password**, enter the system's password.

+ * For **Confirm System Database User (system) Password**, enter system's password.

+ * For **Restart system after machine reboot?** [n], enter **n**.

+ * For **Do you want to continue (y/n)**, validate the summary and if everything looks good, enter **y**.

+2. **[2]** Repeat the preceding step to install SAP HANA on the first node on SITE 2.

+3. **[1,2]** Verify *global.ini*.

+4. **[1,2]** Prepare *global.ini* for installation in non-shared environment, as described in SAP note [2080991](https://launchpad.support.sap.com/#/notes/0002080991).

+5. **[1,2]** Restart SAP HANA to activate the changes.

+6. **[1,2]** Verify that the client interface uses the IP addresses from the `client` subnet for communication.

+7. **[AH]** Change permissions on the data and log directories to avoid a HANA installation error.

+8. **[1]** Install the secondary HANA nodes. The example instructions in this step are for SITE 1.

+ 1. Start the resident `hdblcm` program as `root`.

+ 2. At the prompt, enter the following values:

+ * For **Choose an action**, enter **2** (for add hosts).

+ * For **Enter comma separated host names to add**, enter hana-s1-db2, hana-s1-db3.

+ * For **Additional components for installation**, enter **2, 3**.

+ * For **Enter Root User Name [root]**, press Enter to accept the default.

+ * For **Select roles for host 'hana-s1-db2' [1]**, select 1 (for worker).

+ * For **Enter Host Failover Group for host 'hana-s1-db2' [default]**, press Enter to accept the default.

+ * For **Enter Storage Partition Number for host 'hana-s1-db2' [\<\<assign automatically\>\>]**, press Enter to accept the default.

+ * For **Enter Worker Group for host 'hana-s1-db2' [default]**, press Enter to accept the default.

+ * For **Select roles for host 'hana-s1-db3' [1]**, select 1 (for worker).

+ * For **Enter Host Failover Group for host 'hana-s1-db3' [default]**, press Enter to accept the default.

+ * For **Enter Storage Partition Number for host 'hana-s1-db3' [\<\<assign automatically\>\>]**, press Enter to accept the default.

+ * For **Enter Worker Group for host 'hana-s1-db3' [default]**, press Enter to accept the default.

+ * For **System Administrator (hn1adm) Password**, enter the password.

+ * For **Enter SAP Host Agent User (sapadm) Password**, enter the password.

+ * For **Confirm SAP Host Agent User (sapadm) Password**, enter the password.

+ * For **Certificate Host Name For Host hana-s1-db2** [hana-s1-db2], press Enter to accept the default.

+ * For **Certificate Host Name For Host hana-s1-db3** [hana-s1-db3], press Enter to accept the default.

+ * For **Do you want to continue (y/n)**, validate the summary and if everything looks good, enter **y**.

+9. **[2]** Repeat the preceding step to install the secondary SAP HANA nodes on SITE 2.

+2. **[2]** Configure system replication on SITE 2:

+3. **[1]** Check the replication status and wait until all databases are in sync.

+

+ # | Database | Host | Port | Service Name | Volume ID | Site ID | Site Name | Secondary | Secondary | Secondary | Secondary | Secondary | Replication | Replication | Replication |

+ # | | | | | | | | Host | Port | Site ID | Site Name | Active Status | Mode | Status | Status Details |

+ # | -- | - | -- | | | - | | - | | | | - | -- | -- | -- |

+ # | HN1 | hana-s1-db3 | 30303 | indexserver | 5 | 1 | HANA_S1 | hana-s2-db3 | 30303 | 2 | HANA_S2 | YES | SYNC | ACTIVE | |

+ # | SYSTEMDB | hana-s1-db1 | 30301 | nameserver | 1 | 1 | HANA_S1 | hana-s2-db1 | 30301 | 2 | HANA_S2 | YES | SYNC | ACTIVE | |

+ # | HN1 | hana-s1-db1 | 30307 | xsengine | 2 | 1 | HANA_S1 | hana-s2-db1 | 30307 | 2 | HANA_S2 | YES | SYNC | ACTIVE | |

+ # | HN1 | hana-s1-db1 | 30303 | indexserver | 3 | 1 | HANA_S1 | hana-s2-db1 | 30303 | 2 | HANA_S2 | YES | SYNC | ACTIVE | |

+ # | HN1 | hana-s1-db2 | 30303 | indexserver | 4 | 1 | HANA_S1 | hana-s2-db2 | 30303 | 2 | HANA_S2 | YES | SYNC | ACTIVE | |

+ #

+ # status system replication site "2": ACTIVE

+ # overall system replication status: ACTIVE

+ #

+ # Local System Replication State

+ #

+ # mode: PRIMARY

+ # site id: 1

+ # site name: HANA_S1

+4. **[1,2]** Change the HANA configuration so that communication for HANA system replication is directed though the HANA system replication virtual network interfaces.

+ 2. Edit *global.ini* to add the host mapping for HANA system replication. Use the IP addresses from the `hsr` subnet.

+ 3. Start HANA on both sites.

+ ```bash

+ sudo -u hn1adm /usr/sap/hostctrl/exe/sapcontrol -nr 03 -function StartSystem HDB

+ ```

+ For more information, see [Host name resolution for system replication](https://help.sap.com/viewer/eb3777d5495d46c5b2fa773206bbfb46/1.0.12/en-US/c0cba1cb2ba34ec89f45b48b2157ec7b.html).

+5. **[AH]** Re-enable the firewall and open the necessary ports.

+ 2. Open the necessary firewall ports. You will need to adjust the ports for your HANA instance number.

+> Don't set `quorum expected-votes` to 2. This isn't a two-node cluster. Make sure that the cluster property `concurrent-fencing` is enabled, so that node fencing is deserialized.

+2. **[AH]** Unmount file system `/hana/shared`, which was temporarily mounted for the installation on all HANA DB VMs. Before you can unmount it, you need to stop any processes and sessions that are using the file system.

+3. **[1]** Create the file system cluster resources for `/hana/shared` in the disabled state. You use `--disabled` because you have to define the location constraints before the mounts are enabled.

+ You chose to deploy /han).

+ * In this example, the '/hana/shared' file system is deployed on Azure NetApp Files and mounted over NFSv4.1. Follow the steps in this section, only if you're using NFS on Azure NetApp Files.

+

+ # /hana/shared file system for site 2

+

+ # clone the /hana/shared file system resources for both site1 and site2

+ pcs resource clone fs_hana_shared_s1 meta clone-node-max=1 interleave=true

+ pcs resource clone fs_hana_shared_s2 meta clone-node-max=1 interleave=true

+ * In this example, the '/hana/shared' file system is deployed on NFS on Azure Files. Follow the steps in this section, only if you're using NFS on Azure Files.

+

+ # /hana/shared file system for site 2

+

+ # clone the /hana/shared file system resources for both site1 and site2

+ The `OCF_CHECK_LEVEL=20` attribute is added to the monitor operation, so that monitor operations perform a read/write test on the file system. Without this attribute, the monitor operation only verifies that the file system is mounted. This can be a problem because when connectivity is lost, the file system might remain mounted, despite being inaccessible.

+ The `on-fail=fence` attribute is also added to the monitor operation. With this option, if the monitor operation fails on a node, that node is immediately fenced. Without this option, the default behavior is to stop all resources that depend on the failed resource, then restart the failed resource, and then start all the resources that depend on the failed resource. Not only can this behavior take a long time when an SAP HANA resource depends on the failed resource, but it also can fail altogether. The SAP HANA resource can't stop successfully, if the NFS share holding the HANA binaries is inaccessible.

+ The timeouts in the above configurations may need to be adapted to the specific SAP setup.

+4. **[1]** Configure and verify the node attributes. All SAP HANA DB nodes on replication site 1 are assigned attribute `S1`, and all SAP HANA DB nodes on replication site 2 are assigned attribute `S2`.

+ # HANA replication site 2

+ # To verify the attribute assignment to nodes execute

+5. **[1]** Configure the constraints that determine where the NFS file systems will be mounted, and enable the file system resources.

+

+6. **[AH]** Verify that the Azure NetApp Files volumes are mounted under `/hana/shared`, on all HANA DB VMs on both sites.

+ * Example, if using Azure NetApp Files:

+ * Example, if using Azure Files NFS:

+7. **[1]** Configure and clone the attribute resources, and configure the constraints, as follows:

+ # Clone the attribute resources

+ # Configure the constraints, which will set the attribute values

+8. **[1]** Place Pacemaker in maintenance mode, in preparation for the creation of the HANA cluster resources.

+1. **[A]** Install the HANA scale-out resource agent on all cluster nodes, including the majority maker.

+2. **[1,2]** Install the HANA system replication hook on one HANA DB node on each system replication site. SAP HANA should still be down.

+ 1. Prepare the hook as `root`.

+ mkdir -p /hana/shared/myHooks

+ cp /usr/share/SAPHanaSR-ScaleOut/SAPHanaSR.py /hana/shared/myHooks

+ chown -R hn1adm:sapsys /hana/shared/myHooks

+ 2. Adjust `global.ini`.

+

+3. **[AH]** The cluster requires sudoers configuration on the cluster node for <sid\>adm. In this example, you achieve this by creating a new file. Run the commands as `root`.

+4. **[1,2]** Start SAP HANA on both replication sites. Run as <sid\>adm.

+5. **[1]** Verify the hook installation. Run as <sid\>adm on the active HANA system replication site.

+ # 2020-07-21 22:04:32.364379 ha_dr_SAPHanaSR SFAIL

+ # 2020-07-21 22:04:46.905661 ha_dr_SAPHanaSR SFAIL

+6. **[1]** Create the HANA cluster resources. Run the following commands as `root`.

+ 2. Next, create the HANA topology resource.

+ If you're building a RHEL **7.x** cluster, use the following commands:

+

+ If you're building a RHEL >= **8.x** cluster, use the following commands:

+

+ 3. Create the HANA instance resource.

+ If you're building a RHEL **7.x** cluster, use the following commands:

+

+ If you're building a RHEL >= **8.x** cluster, use the following commands:

+

+ > It's a good idea to set `AUTOMATED_REGISTER` to `false`, while you're performing failover tests, to prevent a failed primary instance to automatically register as secondary. After testing, as a best practice, set `AUTOMATED_REGISTER` to `true`, so that after takeover, system replication can resume automatically.

+ 4. Create the virtual IP and associated resources.

+ 5. Create the cluster constraints.

+ If you're building a RHEL **7.x** cluster, use the following commands:

+

+ If you're building a RHEL >= **8.x** cluster, use the following commands:

+

+7. **[1]** Place the cluster out of maintenance mode. Make sure that the cluster status is `ok`, and that all of the resources are started.

+To proceed with provisioning your second virtual IP, make sure you have configured Azure Load Balancer as described in [Configure Azure Load Balancer](#configure-azure-load-balancer).

+1. Create a second front-end IP pool:

+```bash

+You can configure the second virtual IP and the additional constraints with the following commands. If the secondary instance is down, the secondary virtual IP will be switched to the primary.

+```bash

+```bash

+* When cluster resource **SAPHana_HN1_HDB03** moves to the secondary site (**S2**), the second virtual IP will move to the other site, **hana-s1-db1**. If you have configured `AUTOMATED_REGISTER="false"`, and HANA system replication isn't registered automatically, then the second virtual IP will run on **hana-s2-db1**.

+* When you're testing server crash, the second virtual IP resources (**secvip_HN1_03**) and the Azure Load Balancer port resource (**secnc_HN1_03**) run on the primary server, alongside the primary virtual IP resources. While the secondary server is down, the applications that are connected to the read-enabled HANA database will connect to the primary HANA database. This behavior is expected. It allows applications that are connected to the read-enabled HANA database to operate while a secondary server is unavailable.

+* During failover and fallback, the existing connections for applications that are using the second virtual IP to connect to the HANA database might be interrupted.

+## Test SAP HANA failover

+ 1. Verify that there are no failed cluster actions.

+

+ 2. Verify that SAP HANA system replication is in sync.

+

+

+

+2. Verify the cluster configuration for a failure scenario, when a node loses access to the NFS share (`/hana/shared`).

+ **Expected result**: When you block the access to the `/hana/shared` NFS mounted file system on one of the primary site VMs, the monitoring operation that performs read/write operation on file system, will fail, as it is not able to access the file system and will trigger HANA resource failover. The same result is expected when your HANA node loses access to the NFS share.

+

+

+ The HANA VM that lost access to `/hana/shared` should restart or stop, depending on the cluster configuration. The cluster resources are migrated to the other HANA system replication site.

+ If the cluster hasn't started on the VM that was restarted, start the cluster by running the following:

+ When the cluster starts, file system `/hana/shared` is automatically mounted. If you set `AUTOMATED_REGISTER="false"`, you will need to configure SAP HANA system replication on the secondary site. In this case, you can run these commands to reconfigure SAP HANA as secondary.

+ The state of the resources, after the test:

+For recommended SAP HANA storage configurations, see [SAP HANA Azure VMs storage configurations](./hana-vm-operations-storage.md).

+## Prepare the infrastructure

+ >

+ > * Make sure that the OS you select is SAP-certified for SAP HANA on the specific VM types you're using. For a list of SAP HANA certified VM types and OS releases for those types, go to the [SAP HANA certified IaaS platforms](https://www.sap.com/dmc/exp/2014-09-02-hana-hardware/enEN/#/solutions?filters=v:deCertified;ve:24;iaas;v:125;v:105;v:99;v:120) site. Click into the details of the listed VM type to get the complete list of SAP HANA-supported OS releases for that type.

+ > * If you choose to deploy `/hana/shared` on NFS on Azure Files, we recommend to deploy on SLES 15 SP2 and above.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+> [!NOTE]

+>

+> * For HANA scale out, select the NIC for the `client` subnet when adding the virtual machines in the backend pool.

+> * The full set of command in Azure CLI and PowerShell adds the VMs with primary NIC in the backend pool.

+#### [Azure Portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+The full set of Azure CLI codes display the setup of the load balancer, which include two VMs in the backend pool. Depending on the number of VMs in your HANA scale-out, you could add more VMs in the backend pool.

+#### [PowerShell](#tab/lb-powershell)

+The full set of PowerShell code display the setup of the load balancer, which include two VMs in the backend pool. Depending on the number of VMs in your HANA scale-out, you could add more VMs in the backend pool.

+> [!IMPORTANT]

+> Floating IP is not supported on a NIC secondary IP configuration in load-balancing scenarios. For details see [Azure Load balancer Limitations](../../load-balancer/load-balancer-multivip-overview.md#limitations). If you need additional IP address for the VM, deploy a second NIC.

+> [!NOTE]

+> When VMs without public IP addresses are placed in the backend pool of internal (no public IP address) Standard Azure load balancer, there will be no outbound internet connectivity, unless additional configuration is performed to allow routing to public end points. For details on how to achieve outbound connectivity see [Public endpoint connectivity for Virtual Machines using Azure Standard Load Balancer in SAP high-availability scenarios](./high-availability-guide-standard-load-balancer-outbound-connections.md).

+> [!IMPORTANT]

+>

+> * Do not enable TCP timestamps on Azure VMs placed behind Azure Load Balancer. Enabling TCP timestamps will cause the health probes to fail. Set parameter `net.ipv4.tcp_timestamps` to `0`. For details see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) and SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+> * To prevent saptune from changing the manually set `net.ipv4.tcp_timestamps` value from `0` back to `1`, update saptune version to 3.1.1 or higher. For more details, see [saptune 3.1.1 ΓÇô Do I Need to Update?](https://www.suse.com/c/saptune-3-1-1-do-i-need-to-update/).

+[sles-for-sap-bp]:https://documentation.suse.com/sbp-supported.html

+- [SUSE Linux Enterprise Server for SAP Applications best practices guides][sles-for-sap-bp]:

+ - Setting up an SAP HANA SR Performance Optimized Infrastructure (SLES for SAP Applications). The guide contains all the required information to set up SAP HANA system replication for on-premises development. Use this guide as a baseline.

+ - Setting up an SAP HANA SR Cost Optimized Infrastructure (SLES for SAP Applications).

+## Prepare the infrastructure

+Deploy virtual machines for SAP HANA. Choose a suitable SLES image that is supported for HANA system. You can deploy VM in any one of the availability options - virtual machine scale set, availability zone, or availability set.

+### Configure Azure load balancer

+During VM configuration, you have an option to create or select exiting load balancer in networking section. Follow below steps, to setup standard load balancer for high availability setup of HANA database.

+#### [Azure Portal](#tab/lb-portal)

+#### [Azure CLI](#tab/lb-azurecli)

+#### [PowerShell](#tab/lb-powershell)

+>

+> - Don't enable TCP timestamps on Azure VMs that are placed behind Azure Load Balancer. Enabling TCP timestamps causes the health probes to fail. Set parameter `net.ipv4.tcp_timestamps` to `0`. For details see [Load Balancer health probes](../../load-balancer/load-balancer-custom-probe-overview.md) or SAP note [2382421](https://launchpad.support.sap.com/#/notes/2382421).

+> - To prevent saptune from changing the manually set `net.ipv4.tcp_timestamps` value from `0` back to `1`, update saptune version to 3.1.1 or higher. For more details, see [saptune 3.1.1 ΓÇô Do I Need to Update?](https://www.suse.com/c/saptune-3-1-1-do-i-need-to-update/).

+When configuring a fencing device, it's recommended to configure [`pcmk_delay_max`](https://www.suse.com/support/kb/doc/?id=000019110) property. So, in the event of split-brain scenario, the cluster introduces a random delay up to the `pcmk_delay_max` value, to the fencing action on each node. The node with the shortest delay will be selected for fencing.

+Hybrid search is predicated on having a search index that contains fields of various [data types](/rest/api/searchservice/supported-data-types), including plain text and numbers, geo coordinates for geospatial search, and vectors for a mathematical representation of a chunk of text. You can use almost all query capabilities in Azure AI Search with a vector query, except for client-side interactions such as autocomplete and suggestions.

+| DocumentsProcessedCount | Count | Total of the number of documents successfully processed in an indexing operation by an indexer. |

+Retrieval Augmentation Generation (RAG) is an architecture that augments the capabilities of a Large Language Model (LLM) like ChatGPT by adding an information retrieval system that provides grounding data. Adding an information retrieval system gives you control over grounding data used by an LLM when it formulates a response. For an enterprise solution, RAG architecture means that you can constrain generative AI to *your enterprise content* sourced from vectorized documents and images, and other data formats if you have embedding models for that content.

+| image | vectors ¹ | Images can be vectorized externally for a mathematical representation of image content and then [indexed as vector fields](vector-search-how-to-create-index.md) in your index. You can use an open source model like [OpenAI CLIP](https://github.com/openai/CLIP/blob/main/README.md) to vectorize text and images in the same embedding space.|

+<!-- | audio | vectors ¹ | Vectorized audio content can be [indexed as vector fields](vector-search-how-to-create-index.md) in your index. Vectorization of audio content often requires intermediate processing that converts audio to text, and then text to vecctors. [Azure AI Speech](/azure/ai-services/speech-service/overview) and [OpenAI Whisper](https://platform.openai.com/docs/guides/speech-to-text) are two examples for this scenario. |

+| video | vectors ¹ | Vectorized video content can be [indexed as vector fields](vector-search-how-to-create-index.md) in your index. Similar to audio, vectorization of video content also requires extra processing, such as breaking up the video into frames or smaller chunks for vectorization. | -->

+>To use this preview, [request access](https://aka.ms/azure-cognitive-search/indexer-preview). Any access request is automatically accepted after submission. After access is enabled, use a [preview REST API (2023-10-01-Preview or later)](search-api-preview.md) to index your content.

+| Microsoft.CognitiveServices/accounts (preview) ⁵| `openai_account` |

+⁵ The `Microsoft.CognitiveServices/accounts` resource type is used for indexer connections to Azure OpenAI when implementing [integrated Vectorization](vector-search-integrated-vectorization.md).

+Multi-query vector search sends multiple queries across multiple vector fields in your search index. A common example of this query request is when using models such as [CLIP](https://openai.com/research/clip) for a multimodal vector search where the same model can vectorize image and text content.

+The following query example looks for similarity in both `myImageVector` and `myTextVector`, but sends in two different query embeddings respectively, each executing in parallel. This query produces a result that's scored using [Reciprocal Rank Fusion (RRF)](hybrid-search-ranking.md).

+How you get embeddings from your source content depends on your approach and whether you can use preview features. You can vectorize or generate embeddings using models from OpenAI, Azure OpenAI, and any number of providers, over a wide range of source content including text, images, and other content types supported by the models. You can then push pre-vectorized content to [vector fields](vector-search-how-to-create-index.md) in a search index. That's the generally available approach. If you can use preview features, Azure AI Search provides [integrated data chunking and vectorization](vector-search-integrated-vectorization.md) in an indexer pipeline. You still provide the resources (endpoints and connection information), but Azure AI Search makes all of the calls and handles the transitions.

+Vector search is a method of information retrieval where documents and queries are represented as vectors instead of plain text. In vector search, machine learning models generate the vector representations of source inputs, which can be text, images, or other content. Having a mathematic representation of content provides a common basis for search scenarios. If everything is a vector, a query can find a match in vector space, even if the associated original content is in different media or language than the query.

+When searchable content is represented as vectors, a query can find close matches in similar content. The embedding model used for vector generation knows which words and concepts are similar, and it places the resulting vectors close together in the embedding space. For example, vectorized source documents about "clouds" and "fog" are more likely to show up in a query about "mist" because they're semantically similar, even if they aren't a lexical match.

+ms.devlang: csharp

+# ms.devlang: csharp,java,javascript,python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: java

+# ms.devlang: java, azurecli

+> JDK tools aren't included in the path for the *source code* deployment type. Run `export PATH="$PATH:/layers/tanzu-buildpacks_microsoft-openjdk/jdk/bin"` before running any JDK commands.

+| Provides the Microsoft OpenJDK. | Configures the JVM version. The default JDK version is 11. Currently supported: JDK 8, 11, 17, and 21. | `BP_JVM_VERSION` | `--build-env BP_JVM_VERSION=11.*` |

+| Configure the .NET Core runtime version. | Supports *Net6.0*, *Net7.0*, and *Net8.0*. <br> You can configure through a *runtimeconfig.json* or MSBuild Project file. <br> The default runtime is *6.0.\**. | N/A | N/A |

+| Specify a Python version. | Supports *3.8.\**, *3.9.\**, *3.10.\**, *3.11.\**, *3.12.\**. The default value is *3.10.\**<br> You can specify the version via the `BP_CPYTHON_VERSION` environment variable during build. | `BP_CPYTHON_VERSION` | `--build-env BP_CPYTHON_VERSION=3.8.*` |

+| Specify a Go version. | Supports *1.20.\**, *1.21.\**. The default value is *1.20.\**.<br> The Go version is automatically detected from the appΓÇÖs *go.mod* file. You can override this version by setting the `BP_GO_VERSION` environment variable at build time. | `BP_GO_VERSION` | `--build-env BP_GO_VERSION=1.20.*` |

+| Specify a Node version. | Supports *16.\**, *18.\**, *19.\**, *20.\**. The default value is *20.\**. <br>You can specify the Node version via an *.nvmrc* or *.node-version* file at the application directory root. `BP_NODE_VERSION` overrides the settings. | `BP_NODE_VERSION` | `--build-env BP_NODE_VERSION=19.*` |

+| Integrate with Bellsoft OpenJDK. | Configures the JDK version. Currently supported: JDK 8, 11, 17, and 20. | `BP_JVM_VERSION` | `--build-env BP_JVM_VERSION=17` |

+### [Azure portal + Maven plugin](#tab/Azure-portal-maven-plugin-ent)

+- An Azure subscription. If you don't have an Azure subscription, create an [Azure free account](https://azure.microsoft.com/free/) before you begin.

+- One of the following roles:

+ - Global Administrator or Privileged Role Administrator, for granting consent for apps requesting any permission, for any API.

+ - Cloud Application Administrator or Application Administrator, for granting consent for apps requesting any permission for any API, except Microsoft Graph app roles (application permissions).

+ - A custom directory role that includes the [permission to grant permissions to applications](/entra/identity/role-based-access-control/custom-consent-permissions), for the permissions required by the application.

+ For more information, see [Grant tenant-wide admin consent to an application](/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal).

+- If you're deploying an Azure Spring Apps Enterprise plan instance for the first time in the target subscription, see the [Requirements](./how-to-enterprise-marketplace-offer.md#requirements) section of [Enterprise plan in Azure Marketplace](./how-to-enterprise-marketplace-offer.md).

+### [Azure CLI](#tab/Azure-CLI)

+- An Azure subscription, If you don't have an Azure subscription, create an [Azure free account](https://azure.microsoft.com/free/) before you begin.

+- One of the following roles:

+ - Global Administrator or Privileged Role Administrator, for granting consent for apps requesting any permission, for any API.

+ - Cloud Application Administrator or Application Administrator, for granting consent for apps requesting any permission for any API, except Microsoft Graph app roles (application permissions).

+ - A custom directory role that includes the [permission to grant permissions to applications](/entra/identity/role-based-access-control/custom-consent-permissions), for the permissions required by the application.

+ For more information, see [Grant tenant-wide admin consent to an application](/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal).

+- If you're deploying an Azure Spring Apps Enterprise plan instance for the first time in the target subscription, see the [Requirements](./how-to-enterprise-marketplace-offer.md#requirements) section of [Enterprise plan in Azure Marketplace](./how-to-enterprise-marketplace-offer.md).

+- [Git](https://git-scm.com/downloads).

+- [Java Development Kit (JDK)](/java/azure/jdk/), version 17.

+- A Microsoft Entra tenant. For instructions on creating one, see [Quickstart: Create a new tenant in Microsoft Entra ID](../active-directory/fundamentals/create-new-tenant.md).

+- [Azure CLI](/cli/azure/install-azure-cli) version 2.53.1 or higher.

+- An Azure subscription, If you don't have an Azure subscription, create an [Azure free account](https://azure.microsoft.com/free/) before you begin.

+- One of the following roles:

+ - Global Administrator or Privileged Role Administrator, for granting consent for apps requesting any permission, for any API.

+ - Cloud Application Administrator or Application Administrator, for granting consent for apps requesting any permission for any API, except Microsoft Graph app roles (application permissions).

+ - A custom directory role that includes the [permission to grant permissions to applications](/entra/identity/role-based-access-control/custom-consent-permissions), for the permissions required by the application.

+ For more information, see [Grant tenant-wide admin consent to an application](/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal).

+- An Azure subscription, If you don't have an Azure subscription, create an [Azure free account](https://azure.microsoft.com/free/) before you begin.

+- One of the following roles:

+ - Global Administrator or Privileged Role Administrator, for granting consent for apps requesting any permission, for any API.

+ - Cloud Application Administrator or Application Administrator, for granting consent for apps requesting any permission for any API, except Microsoft Graph app roles (application permissions).

+ - A custom directory role that includes the [permission to grant permissions to applications](/entra/identity/role-based-access-control/custom-consent-permissions), for the permissions required by the application.

+ For more information, see [grant admin consent](/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal#prerequisites).

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+ms.devlang: powershell

+# ms.devlang: powershell, azurecli

+This condition allows users to read blobs with a [blob index tag](storage-blob-index-how-to.md) key of Project and a value of Cascade. Attempts to access blobs without this key-value tag isn't allowed.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+> [!NOTE]

+> If you try to perform an action in the assigned role that is *not* the action restricted by the condition, `!(ActionMatches)` evaluates to true and the overall condition evaluates to true. This result allows the action to be performed.

+>

+> If you try to perform the action restricted by the condition, `!(ActionMatches)` evaluates to false, so the expression is evaluated. If the expression evaluates to true, the overall condition evaluates to true and allows the action to be performed. Otherwise, the action is not allowed to be performed.

+>

+> In this example, the condition restricts the read action except when the suboperation is `Blob.List`. This means that a List Blobs operation is allowed, but all other read actions are further evaluated against the expression that checks for the blob index tag.

+>

+>To learn more about how conditions are formatted and evaluated, see [Conditions format](../../role-based-access-control/conditions-format.md).

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+To add the condition using the code editor, copy the condition code sample and paste it into the code editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+The condition can be added to a role assignment using either the Azure portal or Azure PowerShell. The portal has two tools for building ABAC conditions - the visual editor and the code editor. You can switch between the two editors in the Azure portal to see your conditions in different views. Switch between the **Visual editor** tab and the **Code editor** tabs to view the examples for your preferred portal editor.

+ms.devlang: csharp

+# ms.devlang: csharp, python

+ms.devlang: csharp

+# ms.devlang: csharp, java, javascript, python

+ms.devlang: typescript

+ms.devlang: csharp

+# ms.devlang: csharp, javascript, python

+When looking at Storage Analytics data, you can use the tables in the [Storage Analytics Logged Operations and Status Messages](/rest/api/storageservices/storage-analytics-logged-operations-and-status-messages) topic to determine what requests are billable. Then you can compare your log data to the status messages to see if you were charged for a particular request. You can also use the tables in the previous topic to investigate availability for a storage service or individual API operation.

+After you have enabled the desired endpoints and granted access in your network rules, you can connect to the appropriate Elastic SAN volumes using the iSCSI protocol. To learn how to configure client connections, see the articles on how to connect to [Linux](elastic-san-connect-linux.md), [Windows](elastic-san-connect-windows.md), or [Azure Kubernetes Service cluster](elastic-san-connect-aks.md).

+iSCSI sessions can periodically disconnect and reconnect over the course of the day. These disconnects and reconnects are part of regular maintenance or the result of network fluctuations. You shouldn't experience any performance degradation as a result of these disconnects and reconnects, and the connections should re-establish by themselves. If a connection doesn't re-establish itself, or you're experiencing performance degradation, raise a support ticket.

+ms.devlang: csharp

+# ms.devlang: csharp, powershell, azurecli

+ms.devlang: csharp

+# ms.devlang: csharp, golang, java, javascript, python

+1. From the **Category** dropdown, select **Update Manager**. Select **Configure periodic checking for missing system updates on Azure virtual machines** for Azure machines.

+1. From the **Category** dropdown, select **Update Manager**. Select **Configure periodic checking for missing system updates on Azure Arc-enabled servers** for Azure Arc-enabled machines.

+1. From the **Category** dropdown, select **Update Manager**. Select **Machines should be configured to periodically check for missing system updates**.

+# Create and assign an autoscale scaling plan for Azure Virtual Desktop

+ Title: What's new in Azure Disk Storage

+description: Learn about new features and enhancements in Azure Disk Storage.

+# What's new for Azure Disk Storage

+Azure Disk Storage regularly receives updates for new features and enhancements. This article provides information about what's new in Azure Disk Storage.

+## Update summary

+- [What's new in 2024](#whats-new-in-2024)

+ - [Quarter 1 (January, February, March)](#quarter-1-january-february-march)

+ - [Generally available: Trusted launch support for Ultra Disks and Premium SSD v2](#generally-available-trusted-launch-support-for-ultra-disks-and-premium-ssd-v2)

+ - [Expanded regional availability for Ultra Disks](#expanded-regional-availability-for-ultra-disks)

+ - [Expanded regional availability for zone-redundant storage disks](#expanded-regional-availability-for-zone-redundant-storage-disks)

+- [What's new in 2023](#whats-new-in-2023)

+ - [Quarter 4 (October, November, December)](#quarter-4-october-november-december)

+ - [Encryption at host GA for Premium SSD v2 and Ultra Disks](#encryption-at-host-ga-for-premium-ssd-v2-and-ultra-disks)

+ - [New latency metrics (preview)](#new-latency-metrics-preview)

+ - [Expanded regional availability for Premium SSD v2](#expanded-regional-availability-for-premium-ssd-v2)

+ - [Expanded regional availability for ZRS disks](#expanded-regional-availability-for-zrs-disks)

+ - [Quarter 3 (July, August, September)](#quarter-3-july-august-september)

+ - [Expanded regional availability for ZRS disks (1)](#expanded-regional-availability-for-zrs-disks-1)

+ - [Expanded regional availability for Premium SSD v2](#expanded-regional-availability-for-premium-ssd-v2-1)

+ - [General Availability - Incremental Snapshots for Premium SSD v2 and Ultra Disks](#general-availabilityincremental-snapshots-for-premium-ssd-v2-and-ultra-disks)

+ - [Quarter 2 (April, May, June)](#quarter-2-april-may-june)

+ - [Expanded regional availability for Premium SSD v2 (2)](#expanded-regional-availability-for-premium-ssd-v2-2)

+ - [Expanded regional availability for ZRS disks (2)](#expanded-regional-availability-for-zrs-disks-2)

+ - [Azure Backup support (preview) for Premium SSD v2](#azure-backup-support-preview-for-premium-ssd-v2)

+ - [Quarter 1 (January, February, March)](#quarter-1-january-february-march-1)

+ - [Expanded regional availability for Premium SSD v2 (3)](#expanded-regional-availability-for-premium-ssd-v2-3)

+ - [Preview - Performance plus](#previewperformance-plus)

+ - [Expanded regional availability for Ultra Disks](#expanded-regional-availability-for-ultra-disks-1)

+ - [More transactions at no extra cost - Standard SSDs](#more-transactions-at-no-extra-coststandard-ssds)

+## What's new in 2024

+### Quarter 1 (January, February, March)

+#### Generally available: Trusted launch support for Ultra Disks and Premium SSD v2

+Trusted launch VMs added support for Ultra Disks and Premium SSD v2, allowing you to combine the foundational compute security of Trusted Launch with the high throughput, high IOPS, and low latency of Ultra Disks and Premium SSD v2. For more information, see [Trusted launch for Azure virtual machines](trusted-launch.md) or the [Azure Update](https://azure.microsoft.com/updates/premium-ssd-v2-and-ultra-disks-support-with-trusted-launch-vm/).

+#### Expanded regional availability for Ultra Disks

+Ultra Disks were made available in the UK West and Poland Central regions.

+#### Expanded regional availability for zone-redundant storage disks

+Zone-redundant storage (ZRS) disks were made available in West US 3 and Germany Central regions.

+## What's new in 2023

+### Quarter 4 (October, November, December)

+#### Encryption at host GA for Premium SSD v2 and Ultra Disks

+Encryption at host was previously only available for Standard HDDs, Standard SSDs, and Premium SSDs. Encryption at host is now also available as a GA offering for Premium SSD v2 and Ultra Disks. For more information on encryption at host, see [Encryption at host - End-to-end encryption for your VM data](disk-encryption.md#encryption-at-hostend-to-end-encryption-for-your-vm-data).

+There are some additional restrictions for Premium SSD v2 and Ultra Disks that enable encryption at host. For more information, see [Restrictions](disk-encryption.md#restrictions-1).

+#### New latency metrics (preview)

+Metrics dedicated to monitoring latency are now available as a preview feature. To learn more, see either the [metrics article](disks-metrics.md#disk-io-throughput-queue-depth-and-latency-metrics) or the [Azure Update](https://azure.microsoft.com/updates/latency-metrics-for-azure-disks-and-performance-metrics-for-temporary-disks-on-azure-virtual-machines/).

+#### Expanded regional availability for Premium SSD v2

+Premium SSD v2 disks were made available in Poland Central, China North 3, and US Gov Virginia. For more information, see the [Azure Update](https://azure.microsoft.com/updates/generally-available-azure-premium-ssd-v2-disk-storage-is-now-available-in-more-regions-pcu/).

+#### Expanded regional availability for ZRS disks

+ZRS disks were made available in the Norway East and UAE North regions. For more information, see the [Azure Update](https://azure.microsoft.com/updates/generally-available-zone-redundant-storage-for-azure-disks-is-now-available-in-norway-east-uae-north-regions/).

+### Quarter 3 (July, August, September)

+#### Expanded regional availability for ZRS disks

+In quarter 3, ZRS disks were made available in the China North 3, East Asia, India Central, Switzerland North, South Africa North, and Sweden Central regions.

+#### Expanded regional availability for Premium SSD v2

+In Quarter 3, Premium SSD v2 were made available in the Australia East, Brazil South, Canada Central, Central India, Central US, East Asia, France Central, Japan East, Korea Central, Norway East, South Africa North, Sweden Central, Switzerland North, and UAE North regions.

+#### General Availability - Incremental Snapshots for Premium SSD v2 and Ultra Disks

+Incremental snapshots for Premium SSD v2 and Ultra Disks were made available as a general availability (GA) feature. For more information, see either the [documentation](disks-incremental-snapshots.md#incremental-snapshots-of-premium-ssd-v2-and-ultra-disks) or the [Azure Update](https://azure.microsoft.com/updates/general-availability-incremental-snapshots-for-premium-ssd-v2-disk-and-ultra-disk-storage-3/).

+### Quarter 2 (April, May, June)

+#### Expanded regional availability for Premium SSD v2

+In quarter 2, Premium SSD v2 disks were made available in the Southeast Asia, UK South, South Central US, and West US 3 regions.

+#### Expanded regional availability for ZRS disks

+In quarter 2, ZRS disks were made available in the Australia East, Brazil South, Japan East, Korea Central, Qatar Central, UK South, East US, East US 2, South Central US, and Southeast Asia regions.

+#### Azure Backup support (preview) for Premium SSD v2

+Azure Backup added preview support for Azure virtual machines using Premium SSD v2 disks in the East US and West Europe regions. For more information, see the [Azure Update](https://azure.microsoft.com/updates/premium-ssd-v2-backup-support/).

+### Quarter 1 (January, February, March)

+#### Expanded regional availability for Premium SSD v2

+In quarter 1, Premium SSD v2 disks were made available in the East US 2, North Europe, and West US 2 regions.

+#### Preview - Performance plus

+Azure Disk Storage added a new preview feature, performance plus. Performance plus enhances the IOPS and throughput performance for Premium SSDs, Standard SSDs, and Standard HDDs that are 513 GiB and larger. For details, see [Increase IOPS and throughput limits for Azure Premium SSDs and Standard SSD/HDDs](disks-enable-performance.md)

+#### Expanded regional availability for Ultra Disks

+In quarter 1, Ultra Disks were made available in the Brazil Southeast, China North 3, Korea South, South Africa North, Switzerland North, and UAE North regions.

+#### More transactions at no extra cost - Standard SSDs

+In quarter 1, we added an hourly limit to the number of transactions that can occur a billable cost. Any transactions beyond that limit don't occur a cost. For information, see the [blog post](https://aka.ms/billedcapsblog) or [Standard SSD transactions](disks-types.md#standard-ssd-transactions).

+## Next steps

+- [Azure managed disk types](disks-types.md)

+- [Introduction to Azure managed disks](managed-disks-overview.md)

+ Title: What is Azure Virtual Network encryption?

+description: Learn about Azure Virtual network encryption. Virtual network encryption allows you to seamlessly encrypt and decrypt traffic between Azure Virtual Machines.

+# customer intent: As a network administrator, I want to learn about encryption in Azure Virtual Network so that I can secure my network traffic.

+# What is Azure Virtual Network encryption?

+> Azure Virtual Network encryption is currently GA in the following regions: **UK South**, **Swiss North**, and **West Central US**. Azure Virtual Network encryption is in public preview in the remaining regions listed later in the article.

+ | D-series V5 | **[Dv5 and Dsv5-series](/azure/virtual-machines/dv5-dsv5-series)**, **[Ddv5 and Ddsv5-series](/azure/virtual-machines/ddv5-ddsv5-series)** |

+ | E-series V5 | **[Ev4 and Esv4-series](/azure/virtual-machines/ev5-esv5-series)**, **[Edv4 and Edsv4-series](/azure/virtual-machines/edv5-edsv5-series)** |

+ | LSv3 | **[LSv3-series](/azure/virtual-machines/lsv3-series)** |

+ | M-series | **[Mv2-series](/azure/virtual-machines/mv2-series)**, **[Msv3 and Mdsv3 Medium Memory Series](/azure/virtual-machines/msv3-mdsv3-medium-series)** |

+

+- Traffic to unsupported Virtual Machines is unencrypted. Use Virtual Network Flow Logs to confirm flow encryption between virtual machines. For more information, see [Virtual network flow logs](../network-watcher/vnet-flow-logs-overview.md).

+- The start/stop of existing virtual machines is required after enabling encryption in a virtual network.

+General Availability (GA) of Azure Virtual Network encryption is available in the following regions:

+- UK South

+- Swiss North

+Azure Virtual Network encryption is available in the following regions during the public preview:

+- East US 2 EUAP

+- Central US EUAP

+To sign up to obtain, access to the public preview, see [Virtual Network Encryption - Public Preview Sign Up](https://aka.ms/vnet-encryption-sign-up).

+(\*\*) The Basic SKU is considered a legacy SKU. The Basic SKU has certain feature and performance limitations and should not be used for production purposes. Verify that the feature that you need is supported before you use the Basic SKU. The Basic SKU doesn't support IPv6 and can only be configured using PowerShell or Azure CLI. Additionally, the Basic SKU doesn't support RADIUS authentication.

+(\*\*) The Basic SKU is considered a legacy SKU. The Basic SKU has certain feature and performance limitations and should not be used for production purposes. Verify that the feature that you need is supported before you use the Basic SKU. The Basic SKU doesn't support IPv6 and can only be configured using PowerShell or Azure CLI. Additionally, the Basic SKU doesn't support RADIUS authentication.

+This article walks you through the necessary steps to configure VPN clients for point-to-site (P2S) virtual network connections that use certificate authentication. These steps continue on from previous articles where the [VPN Gateway point-to-site](vpn-gateway-howto-point-to-site-resource-manager-portal.md) server settings are configured.

+There are multiple sets of steps in this article, depending on the tunnel type you selected for your P2S configuration, and the VPN client that is used to connect.

+This article assumes that you have already created and configured your VPN gateway for P2S certificate authentication. See [Configure server settings for P2S VPN Gateway connections - certificate authentication](vpn-gateway-howto-point-to-site-resource-manager-portal.md) for steps.

+Before beginning the workflow, verify that you're on the correct article. The following table shows the configuration articles available for Azure VPN Gateway P2S VPN clients. Steps differ, depending on the authentication type, tunnel type, and the client OS.

+## Workflow

+In this article, we start with generating VPN client configuration files and client certificates:

+1. [Generate files to configure the VPN client](#1-generate-vpn-client-configuration-files)

+1. [Generate certificates for the VPN client](#2-generate-client-certificates)

+After the steps in these sections are completed, continue on to [3. Configure the VPN client](#3-configure-the-vpn-client). The steps you use to configure your VPN client depend on the tunnel type for your P2S VPN gateway, and the VPN client on the client computer.

+* **IKEv2 and SSTP - native VPN client steps** - If your P2S VPN gateway is configured to use IKEv2/SSTP and certificate authentication, you can connect to your VNet using the native VPN client that's part of your Windows operating system. This configuration doesn't require additional client software. See [IKEv2 and SSTP - native VPN client](#ike).

+* **OpenVPN** - If your P2S VPN gateway is configured to use an OpenVPN tunnel and certificate authentication, you have the option of using either the [Azure VPN Client](#openvpn), or the [OpenVPN client](#azurevpn).

+In many cases, you can install the client certificate directly on the client computer by double-clicking. However, for certain OpenVPN client configurations, you might need to extract information from the client certificate in order to complete the configuration.

+## <a name="ike"></a>Native VPN client steps - IKEv2/SSTP

+If your P2S VPN gateway is configured to use IKEv2/SSTP and certificate authentication, you can connect to your VNet using the native VPN client that's part of your Windows operating system. This configuration doesn't require additional client software.

+1. Connect to your VPN. Go to the **VPN** settings and locate the VPN connection that you created. It's the same name as your virtual network. Select **Connect**. A pop-up message might appear. Select **Continue** to use elevated privileges.

+## <a name="azurevpn"></a>Azure VPN Client steps - OpenVPN

+If your P2S VPN gateway is configured to use an OpenVPN tunnel type and certificate authentication, you can connect using the Azure VPN Client.

+The following steps help you download, install, and configure the Azure VPN Client to connect to your VNet. Note that these steps apply to certificate authentication. If you're using OpenVPN with Microsoft Entra authentication, see the [Microsoft Entra ID](openvpn-azure-ad-client.md) configuration article instead.

+* If you're using Microsoft Entra authentication, you might not have an AzureVPN folder. See the [Microsoft Entra ID](openvpn-azure-ad-client.md) configuration article instead.

+## <a name="openvpn"></a>OpenVPN Client steps - OpenVPN

+If your P2S VPN gateway is configured to use an OpenVPN tunnel type and certificate authentication, you can connect using an OpenVPN client. The following steps help you configure the **OpenVPN &reg; Protocol** client and connect to your VNet.

+* If you're using Microsoft Entra authentication, you might not have an OpenVPN folder. See the [Microsoft Entra ID](openvpn-azure-ad-client.md) configuration article instead.

+## Create a gateway subnet

+The virtual network gateway requires a specific subnet named **GatewaySubnet**. The gateway subnet is part of IP address range for your virtual network and contains the IP addresses that the virtual network gateway resources and services use. Specify a gateway subnet that is /27 or larger.

+## Create a gateway subnet

+## <a name="VNetGateway"></a>Create a VPN gateway

+In this step, you create the virtual network gateway for your VNet. Creating a gateway can often take 45 minutes or more, depending on the selected gateway SKU.